BertViz: A Tool for Visualizing Multi-Head Self-Attention in the BERT model

We introduce BertViz, an open-source tool for visualizing self-attention in the BERT language representation model. BertViz extends earlier work by visualizing attention at three different levels of granularity: the attention-head level, the model level, and the neuron level. We describe how each of these views can help to interpret the BERT model, and we demonstrate a debugging use case.