Viewing test suite Subordination (with prepositional phrase)

JSON CSV
Reference
"Futrell R. Wilcox E. Morita T. Qian P. Ballesteros M. & Levy R. (2019). Neural language models as psycholinguistic subjects: Representations of syntactic state."
Number of items
23
Tags
Gross Syntactic State
Models evaluated
77% (7/9)
Description
Beginning a sentence with "As", "When", "Before", "After", or "Because", implies that an immediately following clause is not the main clause of the sentence, as would have otherwise been the case, but instead is a subordinate that must be followed by the main clause. Ending the sentence without a main clause is problematic. Conversely, following an initial clause with a second clause (without linking it to the initial clause with "and", "but", "despite", or a similar coordinator or subordinator) is unexpected and odd.
Items for subordination_pp-pp
Item
Condition
Subordinate clause 1 subj_modifier Subordinate clause 2 obj_modifier Main clause
Item Condition Subordinate clause 1 subj_modifier Subordinate clause 2 obj_modifier Main clause
1 sub_no-matrix As the doctor in a long white lab jacket studied the book about several recent advances in cancer therapy .
1 no-sub_no-matrix The doctor in a long white lab jacket studied the book about several recent advances in cancer therapy .
1 sub_matrix As the doctor in a long white lab jacket studied the book about several recent advances in cancer therapy , the nurse walked into the room .
1 no-sub_matrix The doctor in a long white lab jacket studied the book about several recent advances in cancer therapy , the nurse walked into the room .
2 sub_no-matrix After the man with a very expensive gun shot the bird on the run for its life .
2 no-sub_no-matrix The man with a very expensive gun shot the bird on the run for its life .
2 sub_matrix After the man with a very expensive gun shot the bird on the run for its life , he loaded his gun .
2 no-sub_matrix The man with a very expensive gun shot the bird on the run for its life , he loaded his gun .
3 sub_no-matrix Because the students in the teacher 's third grade class did not like the material on the evolution of the human skeleton .
3 no-sub_no-matrix The students in the teacher 's third grade class did not like the material on the evolution of the human skeleton .
3 sub_matrix Because the students in the teacher 's third grade class did not like the material on the evolution of the human skeleton , they complained to the principal .
3 no-sub_matrix The students in the teacher 's third grade class did not like the material on the evolution of the human skeleton , they complained to the principal .
4 sub_no-matrix When the general in the green uniform and black shoes entered the camp in the middle of the busy camp .
4 no-sub_no-matrix The general in the green uniform and black shoes entered the camp in the middle of the busy camp .
4 sub_matrix When the general in the green uniform and black shoes entered the camp in the middle of the busy camp , the soldiers cheered .
4 no-sub_matrix The general in the green uniform and black shoes entered the camp in the middle of the busy camp , the soldiers cheered .
5 sub_no-matrix After the delivery man with the friendly face delivered the newspaper from a city far away .
5 no-sub_no-matrix The delivery man with the friendly face delivered the newspaper from a city far away .
5 sub_matrix After the delivery man with the friendly face delivered the newspaper from a city far away , the dog looked at the door .
5 no-sub_matrix The delivery man with the friendly face delivered the newspaper from a city far away , the dog looked at the door .
6 sub_no-matrix While the lawyers from the city up north printed the documents on very expensive recycled paper .
6 no-sub_no-matrix The lawyers from the city up north printed the documents on very expensive recycled paper .
6 sub_matrix While the lawyers from the city up north printed the documents on very expensive recycled paper , the aids made coffee .
6 no-sub_matrix The lawyers from the city up north printed the documents on very expensive recycled paper , the aids made coffee .
7 sub_no-matrix Because the editor with higher standards than anyone liked the draft about the failed local politicians .
7 no-sub_no-matrix The editor with higher standards than anyone liked the draft about the failed local politicians .
7 sub_matrix Because the editor with higher standards than anyone liked the draft about the failed local politicians , he published it the next month .
7 no-sub_matrix The editor with higher standards than anyone liked the draft about the failed local politicians , he published it the next month .
8 sub_no-matrix Since the nurse from the neighboring hospital down the street lost her documents from the expensive jewelry store down the street .
8 no-sub_no-matrix The nurse from the neighboring hospital down the street lost her documents from the expensive jewelry store down the street .
8 sub_matrix Since the nurse from the neighboring hospital down the street lost her documents from the expensive jewelry store down the street , she administered the wrong medicine .
8 no-sub_matrix The nurse from the neighboring hospital down the street lost her documents from the expensive jewelry store down the street , she administered the wrong medicine .
9 sub_no-matrix Because the scientist in blue pants and gloves broke the computer in the basement of the office building .
9 no-sub_no-matrix The scientist in blue pants and gloves broke the computer in the basement of the office building .
9 sub_matrix Because the scientist in blue pants and gloves broke the computer in the basement of the office building , the administrator wrote everything by hand .
9 no-sub_matrix The scientist in blue pants and gloves broke the computer in the basement of the office building , the administrator wrote everything by hand .
10 sub_no-matrix After the minister in the dark suit and white tie praised the new building on the town 's main square .
10 no-sub_no-matrix The minister in the dark suit and white tie praised the new building on the town 's main square .
10 sub_matrix After the minister in the dark suit and white tie praised the new building on the town 's main square , it started to rain .
10 no-sub_matrix The minister in the dark suit and white tie praised the new building on the town 's main square , it started to rain .
11 sub_no-matrix Because the architect from the rival firm across town read the plans from the local housing authority .
11 no-sub_no-matrix The architect from the rival firm across town read the plans from the local housing authority .
11 sub_matrix Because the architect from the rival firm across town read the plans from the local housing authority , the house was built correctly .
11 no-sub_matrix The architect from the rival firm across town read the plans from the local housing authority , the house was built correctly .
12 sub_no-matrix Before the actor with the loud singing voice delivered the speech on the true meaning of life .
12 no-sub_no-matrix The actor with the loud singing voice delivered the speech on the true meaning of life .
12 sub_matrix Before the actor with the loud singing voice delivered the speech on the true meaning of life , she took a deep breath .
12 no-sub_matrix The actor with the loud singing voice delivered the speech on the true meaning of life , she took a deep breath .
13 sub_no-matrix While the pianist with the long fingers practiced the music with a difficult time signature .
13 no-sub_no-matrix The pianist with the long fingers practiced the music with a difficult time signature .
13 sub_matrix While the pianist with the long fingers practiced the music with a difficult time signature , the conductor stretched his arms .
13 no-sub_matrix The pianist with the long fingers practiced the music with a difficult time signature , the conductor stretched his arms .
14 sub_no-matrix Although the students in the third year class enjoyed the novel about life during the civil war .
14 no-sub_no-matrix The students in the third year class enjoyed the novel about life during the civil war .
14 sub_matrix Although the students in the third year class enjoyed the novel about life during the civil war , the teacher considered it poorly written .
14 no-sub_matrix The students in the third year class enjoyed the novel about life during the civil war , the teacher considered it poorly written .
15 sub_no-matrix While the maid with an abrasive and negative attitude read the magazine on home repair and backyard maintenance .
15 no-sub_no-matrix The maid with an abrasive and negative attitude read the magazine on home repair and backyard maintenance .
15 sub_matrix While the maid with an abrasive and negative attitude read the magazine on home repair and backyard maintenance , the customer waited at her table .
15 no-sub_matrix The maid with an abrasive and negative attitude read the magazine on home repair and backyard maintenance , the customer waited at her table .
16 sub_no-matrix Before the assistant from the crowded office down the hall wrote the letter about potential corruption and bribery .
16 no-sub_no-matrix The assistant from the crowded office down the hall wrote the letter about potential corruption and bribery .
16 sub_matrix Before the assistant from the crowded office down the hall wrote the letter about potential corruption and bribery , she had a cup of coffee .
16 no-sub_matrix The assistant from the crowded office down the hall wrote the letter about potential corruption and bribery , she had a cup of coffee .
17 sub_no-matrix After the diplomat from the friendly country in Africa presented the file on international peace and cooperation .
17 no-sub_no-matrix The diplomat from the friendly country in Africa presented the file on international peace and cooperation .
17 sub_matrix After the diplomat from the friendly country in Africa presented the file on international peace and cooperation , the ambassador gave a speech .
17 no-sub_matrix The diplomat from the friendly country in Africa presented the file on international peace and cooperation , the ambassador gave a speech .
18 sub_no-matrix While the officials in dark suits and bright ties reviewed the contract for increased salaries and bonuses .
18 no-sub_no-matrix The officials in dark suits and bright ties reviewed the contract for increased salaries and bonuses .
18 sub_matrix While the officials in dark suits and bright ties reviewed the contract for increased salaries and bonuses , the players waited nervously .
18 no-sub_matrix The officials in dark suits and bright ties reviewed the contract for increased salaries and bonuses , the players waited nervously .
19 sub_no-matrix Because the clerk in the New England town lost the plans for the broken water system .
19 no-sub_no-matrix The clerk in the New England town lost the plans for the broken water system .
19 sub_matrix Because the clerk in the New England town lost the plans for the broken water system , the detective looked elsewhere .
19 no-sub_matrix The clerk in the New England town lost the plans for the broken water system , the detective looked elsewhere .
20 sub_no-matrix While the tourists from the village far away read the guide on local customs and religious practices .
20 no-sub_no-matrix The tourists from the village far away read the guide on local customs and religious practices .
20 sub_matrix While the tourists from the village far away read the guide on local customs and religious practices , the children played in the sand .
20 no-sub_matrix The tourists from the village far away read the guide on local customs and religious practices , the children played in the sand .
21 sub_no-matrix When the politicians with white hair expensive suits passed the new law on tax reform and student debt .
21 no-sub_no-matrix The politicians with white hair expensive suits passed the new law on tax reform and student debt .
21 sub_matrix When the politicians with white hair expensive suits passed the new law on tax reform and student debt , the protestors blocked traffic outside .
21 no-sub_matrix The politicians with white hair expensive suits passed the new law on tax reform and student debt , the protestors blocked traffic outside .
22 sub_no-matrix When the police from the small town in Alabama found new evidence about the alleged art theft .
22 no-sub_no-matrix The police from the small town in Alabama found new evidence about the alleged art theft .
22 sub_matrix When the police from the small town in Alabama found new evidence about the alleged art theft , the judge declared a mistrial .
22 no-sub_matrix The police from the small town in Alabama found new evidence about the alleged art theft , the judge declared a mistrial .
23 sub_no-matrix Before the students from foreign countries in Europe handed in their papers on the modern art world .
23 no-sub_no-matrix The students from foreign countries in Europe handed in their papers on the modern art world .
23 sub_matrix Before the students from foreign countries in Europe handed in their papers on the modern art world , they revised them all night long .
23 no-sub_matrix The students from foreign countries in Europe handed in their papers on the modern art world , they revised them all night long .
Need help? Take a look at the documentation.
Predictions for subordination_pp-pp
Formula
Description
Formula Description
(630,sub_no-matrix/5,Main clause) > (632,no-sub_no-matrix/5,Main clause) No description provided.
(631,sub_matrix/5,Main clause) < (629,no-sub_matrix/5,Main clause) No description provided.
Tip: hover over region or condition names to highlight the corresponding entries in the Items table above. Predictions are evaluated on region-level surprisal values.
Need help? Take a look at the documentation.
Results for subordination_pp-pp
Raw results (CSV) Raw results (JSON) Prediction results (CSV)
Model Prediction 1 accuracy Prediction 2 accuracy
Model Prediction 1 accuracy Prediction 2 accuracy
GPT-2 91.30% 100.00% Visualize results
GPT-2 XL 95.65% 100.00% Visualize results
JRNN 52.17% 95.65% Visualize results
Ordered Neurons 100.00% 100.00% Visualize results
RNNG 100.00% 100.00% Visualize results
Transformer XL 100.00% 95.65% Visualize results
Vanilla LSTM 73.91% 100.00% Visualize results
← Back to test suites