TY - GEN
T1 - Embedding lexical features via low-rank tensors
AU - Yu, Mo
AU - Dredze, Mark
AU - Arora, Raman
AU - Gormley, Matthew R.
N1 - Funding Information:
A major portion of this work was done when MY was visiting MD and RA at JHU. This research was supported in part by NSF grant IIS-1546482.
Publisher Copyright:
©2016 Association for Computational Linguistics.
PY - 2016
Y1 - 2016
N2 - Modern NLP models rely heavily on engineered features, which often combine word and contextual information into complex lexical features. Such combination results in large numbers of features, which can lead to overfitting. We present a new model that represents complex lexical features - comprised of parts for words, contextual information and labels - in a tensor that captures conjunction information among these parts. We apply low-rank tensor approximations to the corresponding parameter tensors to reduce the parameter space and improve prediction speed. Furthermore, we investigate two methods for handling features that include n-grams of mixed lengths. Our model achieves state-of-the-art results on tasks in relation extraction, PP-attachment, and preposition disambiguation.
AB - Modern NLP models rely heavily on engineered features, which often combine word and contextual information into complex lexical features. Such combination results in large numbers of features, which can lead to overfitting. We present a new model that represents complex lexical features - comprised of parts for words, contextual information and labels - in a tensor that captures conjunction information among these parts. We apply low-rank tensor approximations to the corresponding parameter tensors to reduce the parameter space and improve prediction speed. Furthermore, we investigate two methods for handling features that include n-grams of mixed lengths. Our model achieves state-of-the-art results on tasks in relation extraction, PP-attachment, and preposition disambiguation.
UR - https://www.scopus.com/pages/publications/84994165653
UR - https://www.scopus.com/pages/publications/84994165653#tab=citedBy
U2 - 10.18653/v1/n16-1117
DO - 10.18653/v1/n16-1117
M3 - Conference contribution
AN - SCOPUS:84994165653
T3 - 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference
SP - 1019
EP - 1029
BT - 2016 Conference of the North American Chapter of the Association for Computational Linguistics
PB - Association for Computational Linguistics (ACL)
T2 - 15th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016
Y2 - 12 June 2016 through 17 June 2016
ER -