Viewing tag Center Embedding
Number of test suites
2
Average performance
77.90%
Description
No description has been provided for this tag.
Test suites tagged with Center Embedding
-
Center Embedding
- Center embedding, the ability to embed a phrase in the middle of another phrase of the same type, is a hallmark feature of natural language syntax. Center-embedding creates nested dependencies, which could pose a challenge for some language models. To succeed in generating expectations about how sentences will continue in the context of multiple center embedding, a model must maintain a representation not only of what words appear in the preceding context but also of the order of those words, and must predict that upcoming words occur in the appropriate order. In this test suite we use verb transitivity and subject/verb plausibility to test model capabilities in this respect.
-
Center Embedding (with modifier)
- Center embedding, the ability to embed a phrase in the middle of another phrase of the same type, is a hallmark feature of natural language syntax. Center-embedding creates nested dependencies, which could pose a challenge for some language models. To succeed in generating expectations about how sentences will continue in the context of multiple center embedding, a model must maintain a representation not only of what words appear in the preceding context but also of the order of those words, and must predict that upcoming words occur in the appropriate order. In this test suite we use verb transitivity and subject/verb plausibility to test model capabilities in this respect.
Results for Center Embedding
| Test suite | Average accuracy |
|---|---|
| Test suite | Average accuracy |
| Center Embedding | 85.27% |
| Center Embedding (with modifier) | 70.54% |