Individual results

View docs

View in-depth performance of a single language model on a single test suite.

Region-by-region surprisal
Sample item for Subject-Verb Number Agreement (with prepositional phrase)
Item
Condition
intro np_subject prep the prep_np matrix_v continuation
Item Condition intro np_subject prep the prep_np matrix_v continuation
1 match_sing The author next to the senators is good
1 mismatch_sing The author next to the senators are good
1 match_plural The authors next to the senator are good
1 mismatch_plural The authors next to the senator is good
Prediction performance for GPT-2 XL on Subject-Verb Number Agreement (with prepositional phrase)
Accuracy
Formula
Description
AccuracyPredictionDescription
78.95% (594,match_sing/6,matrix_v) < (596,mismatch_sing/6,matrix_v) No description provided.
100.00% (595,match_plural/6,matrix_v) < (593,mismatch_plural/6,matrix_v) No description provided.