This is a beta release of SyntaxGym. Please send questions and comments to contact@syntaxgym.org.

Individual results

View in-depth performance of a single language model on a single test suite.

Region-by-region surprisal

Tip: hover over bars to see surprisal values. Surprisals are averaged over items. Error bars show 95% confidence intervals.
More info: view full details for model TinyLSTM or test suite ORC with gap and no gap.

Sample item for ORC with gap and no gap

The first item of the test suite is shown below for quick reference. Please visit the page for ORC with gap and no gap to see the full list of items.

Item	Condition	sujet1	verbe1	cmp1	that	sujet2	aux	verbe2	gap/noGap	test	cmp2
Item	Condition	sujet1	verbe1	cmp1	that	sujet2	aux	verbe2	gap/noGap	test	cmp2
1	ORCpl_gap	Roland	smiles	o the offenders	that	the policeman	has been	investigating		secretly	for a month.
1	ORCsg_gap	Roland	smiles	o the offender	that	the policeman	has been	investigating		secretly	for a month.
1	ORCpl_noGap	Roland	smiles	o the offenders	that	the policeman	has been	investigating	him	secretly	for a month.
1	ORCsg_noGap	Roland	smiles	o the offender	that	the policeman	has been	investigating	him	secretly	for a month.

More info: view full details for test suite ORC with gap and no gap.

Prediction performance for TinyLSTM on ORC with gap and no gap

Accuracy	Formula	Description
Accuracy	Prediction	Description
62.50%	((688,ORCpl_gap/9,test) < (690,ORCpl_noGap/9,test))	No description provided.
54.17%	((689,ORCsg_gap/9,test) < (691,ORCsg_noGap/9,test))	No description provided.

Tip: hover over region or condition names to highlight the corresponding entries in the Sample Item above. Predictions are evaluated on region-level surprisal values.
Need help? Take a look at the documentation.

← Back to visualization home