Individual results

View docs

View in-depth performance of a single language model on a single test suite.

Region-by-region surprisal
Sample item for Subject-Verb Number Agreement (with object relative clause)
Item
Condition
intro np_subject that the embed_np embed_vp matrix_v continuation
Item Condition intro np_subject that the embed_np embed_vp matrix_v continuation
1 match_sing The author that the senators hurt is good
1 mismatch_sing The author that the senators hurt are good
1 match_plural The authors that the senator hurt are good
1 mismatch_plural The authors that the senator hurt is good
Prediction performance for Ordered Neurons on Subject-Verb Number Agreement (with object relative clause)
Accuracy
Formula
Description
AccuracyPredictionDescription
47.37% (656,match_sing/7,matrix_v) < (658,mismatch_sing/7,matrix_v) No description provided.
21.05% (657,match_plural/7,matrix_v) < (655,mismatch_plural/7,matrix_v) No description provided.