Individual results

View docs

View in-depth performance of a single language model on a single test suite.

Region-by-region surprisal
Sample item for Subordination (with object relative clause)
Item
Condition
Subordinate clause 1 subj_modifier Subordinate clause 2 obj_modifier Main clause
Item Condition Subordinate clause 1 subj_modifier Subordinate clause 2 obj_modifier Main clause
1 sub_no-matrix As the doctor who the administrator had recently hired studied the book that colleagues had written on cancer therapy .
1 no-sub_no-matrix The doctor who the administrator had recently hired studied the book that colleagues had written on cancer therapy .
1 sub_matrix As the doctor who the administrator had recently hired studied the book that colleagues had written on cancer therapy , the nurse walked into the room .
1 no-sub_matrix The doctor who the administrator had recently hired studied the book that colleagues had written on cancer therapy , the nurse walked into the room .
Prediction performance for GPT-2 XL on Subordination (with object relative clause)
Accuracy
Formula
Description
AccuracyPredictionDescription
86.96% (556,sub_no-matrix/5,Main clause) > (558,no-sub_no-matrix/5,Main clause) No description provided.
100.00% (557,sub_matrix/5,Main clause) < (555,no-sub_matrix/5,Main clause) No description provided.