Individual results

View docs

View in-depth performance of a single language model on a single test suite.

Region-by-region surprisal
Sample item for NP/Z Garden-path Ambiguity (Overt Object)
Item
Condition
Start Transitive Verb Comma NP/Z Verb Continuation
Item Condition Start Transitive Verb Comma NP/Z Verb Continuation
1 no-obj_no-comma As the criminal shot the woman yelled at the top of her lungs
1 obj_no-comma As the criminal shot his gun the woman yelled at the top of her lungs
1 no-obj_comma As the criminal shot , the woman yelled at the top of her lungs
1 obj_comma As the criminal shot his gun , the woman yelled at the top of her lungs
Prediction performance for Vanilla LSTM on NP/Z Garden-path Ambiguity (Overt Object)
Accuracy
Formula
Description
AccuracyPredictionDescription
95.83% (654,no-obj_no-comma/5,Verb) > (653,no-obj_comma/5,Verb) No description provided.
91.67% (654,no-obj_no-comma/5,Verb) > (652,obj_no-comma/5,Verb) No description provided.
95.83% ((654,no-obj_no-comma/5,Verb) - (653,no-obj_comma/5,Verb) ) > ((652,obj_no-comma/5,Verb) - (651,obj_comma/5,Verb) ) We expect that the difference in surprisal between the Verb region in the no-obj_no-comma condition minus the no-obj_comma condition will be greater than the difference in the Verb region between the obj_comma and the no-obj_comma. This is because the comma unambiguously introduces a matrix clause.