BertViz: A Tool for Visualizing Multi-Head Self-Attention in the BERT model

Details

New Orleans, LA USA. Date of Talk: 2019-05-06

Speakers

Jesse Vig
Event

BertViz: A Tool for Visualizing Multi-Head Self-Attention in the BERT model

We introduce BertViz, an open-source tool for visualizing self-attention in the BERT language representation model. BertViz extends earlier work by visualizing attention at three different levels of granularity: the attention-head level, the model level, and the neuron level. We describe how each of these views can help to interpret the BERT model, and we demonstrate a debugging use case.

Additional information

Focus Areas

Our work is centered around a series of Focus Areas that we believe are the future of science and technology.

FIND OUT MORE
Licensing & Commercialization Opportunities

We’re continually developing new technologies, many of which are available for¬†Commercialization.

FIND OUT MORE
News

PARC scientists and staffers are active members and contributors to the science and technology communities.

FIND OUT MORE