SyntaxGym
The Gym
Home
Test suites
Browse test suites
Create new test suite
Upload new test suite
Language models
Browse models
Add new model
Visualizations
Gallery
Individual results
Summary results
Resources
About
Documentation
This is a beta release of SyntaxGym. Please send questions and comments to
contact@syntaxgym.org
.
Log in
Register
Individual results
View docs
View in-depth performance of a single language model on a single test suite.
Region-by-region surprisal
BERT
TinyLSTM
GPT-2
GPT-2 XL
JRNN
Ordered Neurons
RNNG
Transformer XL
Vanilla LSTM
Subject-Verb Number Agreement (with object relative clause)
Subject-Verb Number Agreement (with prepositional phrase)
Subject-Verb Number Agreement (with subject relative clause)
Center Embedding
Center Embedding (with modifier)
Main-verb/Reduced-relative Garden-path Disambiguation
Main-verb/Reduced-relative Garden-path Disambiguation (with modifier)
NP/Z Garden-path Ambiguity (Verb Transitivity)
NP/Z Garden-path Ambiguity with Modifier (Verb Transitivity)
NP/Z Garden-path Ambiguity (Overt Object)
NP/Z Garden-path Ambiguity with Modifier (Overt Object)
Subordination
Subordination (with object relative clause)
Subordination (with prepositional phrase)
Subordination (with subject relative clause)
Negative Polarity Licensing (any; with object relative clause)
Negative Polarity Licensing (ever; with object relative clause)
Negative Polarity Licensing (any; with subject relative clause)
Negative Polarity Licensing (ever; with subject relative clause)
Reflexive Number Agreement (feminine; with object relative clause)
Reflexive Number Agreement (masculine; with object relative clause)
Reflexive Number Agreement (feminine; with prepositional phrase)
Reflexive Number Agreement (masculine; with prepositional phrase)
Reflexive Number Agreement (feminine; with subject relative clause)
Reflexive Number Agreement (masculine; with subject relative clause)
Cleft Structure
Cleft Structure (with modifier)
Filler-Gap Dependencies (3 sentential embeddings)
Filler-Gap Dependencies (4 sentential embeddings)
Filler-Gap Dependencies (hierarchy)
Filler-Gap Dependencies (object extraction)
Filler-Gap Dependencies (extraction from prepositional phrase)
Filler-Gap Dependencies (subject extraction)
← Back to visualization home
