Individual results

View docs

View in-depth performance of a single language model on a single test suite.

Region-by-region surprisal
Sample item for Reflexive Number Agreement (feminine; with prepositional phrase)
Item
Condition
intro np_subject prep the prep_np matrix_v reflexive
Item Condition intro np_subject prep the prep_np matrix_v reflexive
1 match_sing The author next to the senators hurt herself
1 mismatch_sing The author next to the senators hurt themselves
1 match_plural The authors next to the senator hurt themselves
1 mismatch_plural The authors next to the senator hurt herself
Prediction performance for GPT-2 XL on Reflexive Number Agreement (feminine; with prepositional phrase)
Accuracy
Formula
Description
AccuracyPredictionDescription
57.89% (564,match_sing/7,reflexive) < (566,mismatch_sing/7,reflexive) No description provided.
94.74% (565,match_plural/7,reflexive) < (563,mismatch_plural/7,reflexive) No description provided.