Individual results

View docs

View in-depth performance of a single language model on a single test suite.

Region-by-region surprisal
Sample item for Reflexive Number Agreement (feminine; with object relative clause)
Item
Condition
intro np_subject that the embed_np embed_vp matrix_v reflexive
Item Condition intro np_subject that the embed_np embed_vp matrix_v reflexive
1 match_sing The author that the senators liked hurt herself
1 mismatch_sing The author that the senators liked hurt themselves
1 match_plural The authors that the senator liked hurt themselves
1 mismatch_plural The authors that the senator liked hurt herself
Prediction performance for GPT-2 XL on Reflexive Number Agreement (feminine; with object relative clause)
Accuracy
Formula
Description
AccuracyPredictionDescription
47.37% (614,match_sing/8,reflexive) < (616,mismatch_sing/8,reflexive) No description provided.
100.00% (615,match_plural/8,reflexive) < (613,mismatch_plural/8,reflexive) No description provided.