Individual results
View docsView in-depth performance of a single language model on a single test suite.
Region-by-region surprisal
Sample item for Subject-Verb Number Agreement (with prepositional phrase)
The first item of the test suite is shown below for quick reference. Please visit the page for Subject-Verb Number Agreement (with prepositional phrase) to see the full list of items.
| Item |
Condition
|
intro | np_subject | prep | the | prep_np | matrix_v | continuation |
|---|---|---|---|---|---|---|---|---|
| Item | Condition | intro | np_subject | prep | the | prep_np | matrix_v | continuation |
| 1 | match_sing | The | author | next to | the | senators | is | good |
| 1 | mismatch_sing | The | author | next to | the | senators | are | good |
| 1 | match_plural | The | authors | next to | the | senator | are | good |
| 1 | mismatch_plural | The | authors | next to | the | senator | is | good |
Prediction performance for GPT-2 on Subject-Verb Number Agreement (with prepositional phrase)
| Accuracy |
Formula
|
Description |
|---|---|---|
| Accuracy | Prediction | Description |
| 78.95% | (594,match_sing/6,matrix_v) < (596,mismatch_sing/6,matrix_v) | No description provided. |
| 100.00% | (595,match_plural/6,matrix_v) < (593,mismatch_plural/6,matrix_v) | No description provided. |