Viewing tag Agreement
Number of test suites
3
Average performance
66.45%
Description
No description has been provided for this tag.
Test suites tagged with Agreement
-
Subject-Verb Number Agreement (with object relative clause)
- This task tests a language model for how well it predicts the number marking on English finite present-tense verbs (whether it should be the third-person singular form, or the non-third-person-singular form, generally referred to as the plural form for simplicity, although technically this is the form for first- and second-person singular as well). In controlled, targeted versions of this test, multiple NP precede the verb: the verb's actual subject, as well as a distractor NP with number that is different from that of the subject. A successful language model should place higher probability on the verbform matching that of the subject, not the distractor. We have three versions of this test suite with different types of intervening material.
-
Subject-Verb Number Agreement (with prepositional phrase)
- This task tests a language model for how well it predicts the number marking on English finite present-tense verbs (whether it should be the third-person singular form, or the non-third-person-singular form, generally referred to as the plural form for simplicity, although technically this is the form for first- and second-person singular as well). In controlled, targeted versions of this test, multiple NP precede the verb: the verb's actual subject, as well as a distractor NP with number that is different from that of the subject. A successful language model should place higher probability on the verbform matching that of the subject, not the distractor. We have three versions of this test suite with different types of intervening material.
-
Subject-Verb Number Agreement (with subject relative clause)
- This task tests a language model for how well it predicts the number marking on English finite present-tense verbs (whether it should be the third-person singular form, or the non-third-person-singular form, generally referred to as the plural form for simplicity, although technically this is the form for first- and second-person singular as well). In controlled, targeted versions of this test, multiple NP precede the verb: the verb's actual subject, as well as a distractor NP with number that is different from that of the subject. A successful language model should place higher probability on the verbform matching that of the subject, not the distractor. We have three versions of this test suite with different types of intervening material.
Results for Agreement
| Test suite | Average accuracy |
|---|---|
| Test suite | Average accuracy |
| Subject-Verb Number Agreement (with object relative clause) | 58.88% |
| Subject-Verb Number Agreement (with prepositional phrase) | 71.71% |
| Subject-Verb Number Agreement (with subject relative clause) | 68.75% |