Individual results
View docsView in-depth performance of a single language model on a single test suite.
Region-by-region surprisal
Sample item for Subordination (with object relative clause)
The first item of the test suite is shown below for quick reference. Please visit the page for Subordination (with object relative clause) to see the full list of items.
| Item |
Condition
|
Subordinate clause 1 | subj_modifier | Subordinate clause 2 | obj_modifier | Main clause |
|---|---|---|---|---|---|---|
| Item | Condition | Subordinate clause 1 | subj_modifier | Subordinate clause 2 | obj_modifier | Main clause |
| 1 | sub_no-matrix | As the doctor | who the administrator had recently hired | studied the book | that colleagues had written on cancer therapy | . |
| 1 | no-sub_no-matrix | The doctor | who the administrator had recently hired | studied the book | that colleagues had written on cancer therapy | . |
| 1 | sub_matrix | As the doctor | who the administrator had recently hired | studied the book | that colleagues had written on cancer therapy | , the nurse walked into the room . |
| 1 | no-sub_matrix | The doctor | who the administrator had recently hired | studied the book | that colleagues had written on cancer therapy | , the nurse walked into the room . |
Prediction performance for GPT-2 XL on Subordination (with object relative clause)
| Accuracy |
Formula
|
Description |
|---|---|---|
| Accuracy | Prediction | Description |
| 86.96% | (556,sub_no-matrix/5,Main clause) > (558,no-sub_no-matrix/5,Main clause) | No description provided. |
| 100.00% | (557,sub_matrix/5,Main clause) < (555,no-sub_matrix/5,Main clause) | No description provided. |