SyntaxGym
The Gym
Home
Test suites
Browse test suites
Create new test suite
Upload new test suite
Language models
Browse models
Add new model
Visualizations
Gallery
Individual results
Summary results
Resources
About
Documentation
This is a beta release of SyntaxGym. Please send questions and comments to
contact@syntaxgym.org
.
Log in
Register
Viewing model
RNNG
View on Docker
Author
Dyer et al.
Test suites completed
89% (35/39)
Average performance
74.22%
Description
No description has been provided for this model.
Performance for
RNNG
by tag
Tag
Accuracy
Suites in tag
Tag
Accuracy
Suites in tag
Center Embedding
78.57%
Center Embedding
,
Center Embedding (with modifier)
Long-Distance Dependencies
53.92%
Cleft Structure
,
Cleft Structure (with modifier)
,
Filler-Gap Dependencies (hierarchy)
,
Filler-Gap Dependencies (hierarchy)
,
Filler-Gap Dependencies (object extraction)
,
Filler-Gap Dependencies (extraction from prepositional phrase)
,
Filler-Gap Dependencies (3 sentential embeddings)
,
Filler-Gap Dependencies (3 sentential embeddings)
,
Filler-Gap Dependencies (subject extraction)
,
Filler-Gap Dependencies (subject extraction)
,
Filler-Gap Dependencies (4 sentential embeddings)
,
Filler-Gap Dependencies (4 sentential embeddings)
,
Filler-Gap Dependencies (object extraction)
,
Filler-Gap Dependencies (extraction from prepositional phrase)
Garden-Path Effects
90.84%
Main-verb/Reduced-relative Garden-path Disambiguation (with modifier)
,
NP/Z Garden-path Ambiguity (Verb Transitivity)
,
NP/Z Garden-path Ambiguity (Verb Transitivity)
,
NP/Z Garden-path Ambiguity (Overt Object)
,
NP/Z Garden-path Ambiguity with Modifier (Overt Object)
,
NP/Z Garden-path Ambiguity with Modifier (Overt Object)
,
NP/Z Garden-path Ambiguity with Modifier (Overt Object)
,
Main-verb/Reduced-relative Garden-path Disambiguation
,
Main-verb/Reduced-relative Garden-path Disambiguation
,
Main-verb/Reduced-relative Garden-path Disambiguation
,
Main-verb/Reduced-relative Garden-path Disambiguation (with modifier)
,
Main-verb/Reduced-relative Garden-path Disambiguation (with modifier)
,
NP/Z Garden-path Ambiguity (Verb Transitivity)
,
NP/Z Garden-path Ambiguity with Modifier (Verb Transitivity)
,
NP/Z Garden-path Ambiguity with Modifier (Verb Transitivity)
,
NP/Z Garden-path Ambiguity with Modifier (Verb Transitivity)
,
NP/Z Garden-path Ambiguity (Overt Object)
,
NP/Z Garden-path Ambiguity (Overt Object)
Licensing
62.94%
Negative Polarity Licensing (any; with object relative clause)
,
Negative Polarity Licensing (any; with object relative clause)
,
Negative Polarity Licensing (any; with object relative clause)
,
Negative Polarity Licensing (ever; with object relative clause)
,
Negative Polarity Licensing (ever; with object relative clause)
,
Negative Polarity Licensing (ever; with object relative clause)
,
Negative Polarity Licensing (any; with subject relative clause)
,
Negative Polarity Licensing (any; with subject relative clause)
,
Negative Polarity Licensing (any; with subject relative clause)
,
Negative Polarity Licensing (ever; with subject relative clause)
,
Negative Polarity Licensing (ever; with subject relative clause)
,
Reflexive Number Agreement (feminine; with object relative clause)
,
Reflexive Number Agreement (feminine; with prepositional phrase)
,
Reflexive Number Agreement (feminine; with prepositional phrase)
,
Reflexive Number Agreement (masculine; with prepositional phrase)
,
Reflexive Number Agreement (masculine; with object relative clause)
,
Reflexive Number Agreement (masculine; with object relative clause)
,
Reflexive Number Agreement (feminine; with subject relative clause)
,
Reflexive Number Agreement (feminine; with subject relative clause)
,
Negative Polarity Licensing (ever; with subject relative clause)
,
Reflexive Number Agreement (feminine; with object relative clause)
,
Reflexive Number Agreement (masculine; with prepositional phrase)
,
Reflexive Number Agreement (masculine; with subject relative clause)
,
Reflexive Number Agreement (masculine; with subject relative clause)
Agreement
85.09%
Subject-Verb Number Agreement (with prepositional phrase)
,
Subject-Verb Number Agreement (with prepositional phrase)
,
Subject-Verb Number Agreement (with subject relative clause)
,
Subject-Verb Number Agreement (with object relative clause)
,
Subject-Verb Number Agreement (with object relative clause)
,
Subject-Verb Number Agreement (with subject relative clause)
Gross Syntactic State
98.37%
Subordination
,
Subordination
,
Subordination (with object relative clause)
,
Subordination (with object relative clause)
,
Subordination (with prepositional phrase)
,
Subordination (with prepositional phrase)
,
Subordination (with subject relative clause)
,
Subordination (with subject relative clause)
← Back to models