Viewing test suite Negative Polarity Licensing (any; with object relative clause)

JSON CSV
Reference
No reference provided.
Number of items
38
Tags
Licensing
Models evaluated
88% (8/9)
Description
The words any and ever, in their most common uses, are "negative polarity items' (NPIs): they can only be used in an appropriate syntactic-semantic environment---to a first approximation, in the scope of negation. For example, the determiner "no" can license NPIs, but its NP has to structurally command the NPI. Some sentences with NPIs may be ungrammatical, even if they include a negative determiner earlier in the sentence, because "no" is embedded inside a modifier of the main-clause subject and thus does not command the NPI.
Items for npi_orc_any
Item
Condition
Licensor np compl rc_dp rc_subj rc_verb matrix_v npi continuation
Item Condition Licensor np compl rc_dp rc_subj rc_verb matrix_v npi continuation
1 neg_pos No author that the senators liked has had any success
1 neg_neg No author that no senators liked has had any success
1 pos_pos The author that the senators liked has had any success
1 pos_neg The author that no senators liked has had any success
2 neg_pos No pilot that the consultants praised has shown any courage
2 neg_neg No pilot that no consultants praised has shown any courage
2 pos_pos The pilot that the consultants praised has shown any courage
2 pos_neg The pilot that no consultants praised has shown any courage
3 neg_pos No surgeon that the guards hated has known any setback
3 neg_neg No surgeon that no guards hated has known any setback
3 pos_pos The surgeon that the guards hated has known any setback
3 pos_neg The surgeon that no guards hated has known any setback
4 neg_pos No farmer that the clerks discussed has had any luck
4 neg_neg No farmer that no clerks discussed has had any luck
4 pos_pos The farmer that the clerks discussed has had any luck
4 pos_neg The farmer that no clerks discussed has had any luck
5 neg_pos No manager that the architects loved has completed any project
5 neg_neg No manager that no architects loved has completed any project
5 pos_pos The manager that the architects loved has completed any project
5 pos_neg The manager that no architects loved has completed any project
6 neg_pos No customer that the managers liked has spent any money
6 neg_neg No customer that no managers liked has spent any money
6 pos_pos The customer that the managers liked has spent any money
6 pos_neg The customer that no managers liked has spent any money
7 neg_pos No officer that the journalists praised has had any violations
7 neg_neg No officer that no journalists praised has had any violations
7 pos_pos The officer that the journalists praised has had any violations
7 pos_neg The officer that no journalists praised has had any violations
8 neg_pos No teacher that the ministers hated has failed any student
8 neg_neg No teacher that no ministers hated has failed any student
8 pos_pos The teacher that the ministers hated has failed any student
8 pos_neg The teacher that no ministers hated has failed any student
9 neg_pos No senator that the taxi drivers admired has passed any bills
9 neg_neg No senator that no taxi drivers admired has passed any bills
9 pos_pos The senator that the taxi drivers admired has passed any bills
9 pos_neg The senator that no taxi drivers admired has passed any bills
10 neg_pos No consultant that the secretaries loved has passed up any project
10 neg_neg No consultant that no secretaries loved has passed up any project
10 pos_pos The consultant that the secretaries loved has passed up any project
10 pos_neg The consultant that no secretaries loved has passed up any project
11 neg_pos No guard that the executives liked has seen any crimes
11 neg_neg No guard that no executives liked has seen any crimes
11 pos_pos The guard that the executives liked has seen any crimes
11 pos_neg The guard that no executives liked has seen any crimes
12 neg_pos No clerk that the authors praised has burned any dishes
12 neg_neg No clerk that no authors praised has burned any dishes
12 pos_pos The clerk that the authors praised has burned any dishes
12 pos_neg The clerk that no authors praised has burned any dishes
13 neg_pos No architect that the pilots hated has had any fame
13 neg_neg No architect that no pilots hated has had any fame
13 pos_pos The architect that the pilots hated has had any fame
13 pos_neg The architect that no pilots hated has had any fame
14 neg_pos No athlete that the doctors helped has broken any equipment
14 neg_neg No athlete that no doctors helped has broken any equipment
14 pos_pos The athlete that the doctors helped has broken any equipment
14 pos_neg The athlete that no doctors helped has broken any equipment
15 neg_pos No dancer that the farmers loved has broken any bones
15 neg_neg No dancer that no farmers loved has broken any bones
15 pos_pos The dancer that the farmers loved has broken any bones
15 pos_neg The dancer that no farmers loved has broken any bones
16 neg_pos No minister that the managers liked has missed any services
16 neg_neg No minister that no managers liked has missed any services
16 pos_pos The minister that the managers liked has missed any services
16 pos_neg The minister that no managers liked has missed any services
17 neg_pos No taxi driver that the customers praised has crashed any car
17 neg_neg No taxi driver that no customers praised has crashed any car
17 pos_pos The taxi driver that the customers praised has crashed any car
17 pos_neg The taxi driver that no customers praised has crashed any car
18 neg_pos No secretary that the officers hated has missed any phone calls
18 neg_neg No secretary that no officers hated has missed any phone calls
18 pos_pos The secretary that the officers hated has missed any phone calls
18 pos_neg The secretary that no officers hated has missed any phone calls
19 neg_pos No executive that the teachers knew has fired any worker
19 neg_neg No executive that no teachers knew has fired any worker
19 pos_pos The executive that the teachers knew has fired any worker
19 pos_neg The executive that no teachers knew has fired any worker
20 neg_pos No authors that the officer loved have read any poetry
20 neg_neg No authors that no officer loved have read any poetry
20 pos_pos The authors that the officer loved have read any poetry
20 pos_neg The authors that no officer loved have read any poetry
21 neg_pos No pilots that the teacher liked have landed any plane
21 neg_neg No pilots that no teacher liked have landed any plane
21 pos_pos The pilots that the teacher liked have landed any plane
21 pos_neg The pilots that no teacher liked have landed any plane
22 neg_pos No doctors that the senator praised have failed any surgery
22 neg_neg No doctors that no senator praised have failed any surgery
22 pos_pos The doctors that the senator praised have failed any surgery
22 pos_neg The doctors that no senator praised have failed any surgery
23 neg_pos No farmers that the consultant hated have planted any beans
23 neg_neg No farmers that no consultant hated have planted any beans
23 pos_pos The farmers that the consultant hated have planted any beans
23 pos_neg The farmers that no consultant hated have planted any beans
24 neg_pos No managers that the guard respected have had any luck
24 neg_neg No managers that no guard respected have had any luck
24 pos_pos The managers that the guard respected have had any luck
24 pos_neg The managers that no guard respected have had any luck
25 neg_pos No customers that the clerk loved have purchased any apparel
25 neg_neg No customers that no clerk loved have purchased any apparel
25 pos_pos The customers that the clerk loved have purchased any apparel
25 pos_neg The customers that no clerk loved have purchased any apparel
26 neg_pos No officers that the architect liked have arrested any criminal
26 neg_neg No officers that no architect liked have arrested any criminal
26 pos_pos The officers that the architect liked have arrested any criminal
26 pos_neg The officers that no architect liked have arrested any criminal
27 neg_pos No teachers that the athlete praised have failed any students
27 neg_neg No teachers that no athlete praised have failed any students
27 pos_pos The teachers that the athlete praised have failed any students
27 pos_neg The teachers that no athlete praised have failed any students
28 neg_pos No senators that the dancer hated have passed any bills
28 neg_neg No senators that no dancer hated have passed any bills
28 pos_pos The senators that the dancer hated have passed any bills
28 pos_neg The senators that no dancer hated have passed any bills
29 neg_pos No consultants that the minister impressed have missed any projects
29 neg_neg No consultants that no minister impressed have missed any projects
29 pos_pos The consultants that the minister impressed have missed any projects
29 pos_neg The consultants that no minister impressed have missed any projects
30 neg_pos No guards that the taxi driver loved have caught any thieves
30 neg_neg No guards that no taxi driver loved have caught any thieves
30 pos_pos The guards that the taxi driver loved have caught any thieves
30 pos_neg The guards that no taxi driver loved have caught any thieves
31 neg_pos No clerks that the secretary liked have burned any dishes
31 neg_neg No clerks that no secretary liked have burned any dishes
31 pos_pos The clerks that the secretary liked have burned any dishes
31 pos_neg The clerks that no secretary liked have burned any dishes
32 neg_pos No architects that the executive praised have passed any projects
32 neg_neg No architects that no executive praised have passed any projects
32 pos_pos The architects that the executive praised have passed any projects
32 pos_neg The architects that no executive praised have passed any projects
33 neg_pos No athletes that the pilot hated have broken any equipment
33 neg_neg No athletes that no pilot hated have broken any equipment
33 pos_pos The athletes that the pilot hated have broken any equipment
33 pos_neg The athletes that no pilot hated have broken any equipment
34 neg_pos No journalists that the surgeon contacted have broken any bones
34 neg_neg No journalists that no surgeon contacted have broken any bones
34 pos_pos The journalists that the surgeon contacted have broken any bones
34 pos_neg The journalists that no surgeon contacted have broken any bones
35 neg_pos No ministers that the farmer loved have advocated any evil
35 neg_neg No ministers that no farmer loved have advocated any evil
35 pos_pos The ministers that the farmer loved have advocated any evil
35 pos_neg The ministers that no farmer loved have advocated any evil
36 neg_pos No taxi drivers that the manager liked have crashed any cars
36 neg_neg No taxi drivers that no manager liked have crashed any cars
36 pos_pos The taxi drivers that the manager liked have crashed any cars
36 pos_neg The taxi drivers that no manager liked have crashed any cars
37 neg_pos No secretaries that the customer praised have missed any appointments
37 neg_neg No secretaries that no customer praised have missed any appointments
37 pos_pos The secretaries that the customer praised have missed any appointments
37 pos_neg The secretaries that no customer praised have missed any appointments
38 neg_pos No executives that the officer hated have refused any compensation
38 neg_neg No executives that no officer hated have refused any compensation
38 pos_pos The executives that the officer hated have refused any compensation
38 pos_neg The executives that no officer hated have refused any compensation
Need help? Take a look at the documentation.
Predictions for npi_orc_any
Formula
Description
Formula Description
(579,neg_pos/8,npi) < (577,pos_pos/8,npi) No description provided.
(578,neg_neg/8,npi) < (580,pos_neg/8,npi) No description provided.
(579,neg_pos/8,npi) < (580,pos_neg/8,npi) No description provided.
Tip: hover over region or condition names to highlight the corresponding entries in the Items table above. Predictions are evaluated on region-level surprisal values.
Need help? Take a look at the documentation.
Results for npi_orc_any
Raw results (CSV) Raw results (JSON) Prediction results (CSV)
Model Prediction 1 accuracy Prediction 2 accuracy Prediction 3 accuracy
Model Prediction 1 accuracy Prediction 2 accuracy Prediction 3 accuracy
TinyLSTM 100.00% 100.00% 0.00% Visualize results
GPT-2 100.00% 97.37% 97.37% Visualize results
GPT-2 XL 100.00% 100.00% 100.00% Visualize results
JRNN 100.00% 97.37% 81.58% Visualize results
Ordered Neurons 100.00% 100.00% 0.00% Visualize results
RNNG 100.00% 92.11% 23.68% Visualize results
Transformer XL 100.00% 92.11% 26.32% Visualize results
Vanilla LSTM 100.00% 100.00% 0.00% Visualize results
← Back to test suites