Annotating named entities in Twitter data with crowdsourcing

Tim Finin; Will Murnane; Anand Karandikar; Nicholas Keller; Justin Martineau; Mark Dredze

Annotating named entities in Twitter data with crowdsourcing

Tim Finin, Will Murnane, Anand Karandikar, Nicholas Keller, Justin Martineau, Mark Dredze

Research output: Contribution to conference › Paper › peer-review

Abstract

We describe our experience using both Amazon Mechanical Turk (MTurk) and CrowdFlower to collect simple named entity annotations for Twitter status updates. Unlike most genres that have traditionally been the focus of named entity experiments, Twitter is far more informal and abbreviated. The collected annotations and annotation techniques will provide a first step towards the full study of named entity recognition in domains like Facebook and Twitter. We also briefly describe how to use MTurk to collect judgements on the quality of “word clouds.

Original language	English (US)
Pages	80-88
Number of pages	9
State	Published - 2010
Externally published	Yes
Event	2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Mturk 2010 at the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2010 - Los Angeles, United States Duration: Jun 6 2010 → …

Conference

Conference	2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Mturk 2010 at the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2010
Country/Territory	United States
City	Los Angeles
Period	6/6/10 → …

ASJC Scopus subject areas

Language and Linguistics
Linguistics and Language

Cite this

Finin, T., Murnane, W., Karandikar, A., Keller, N., Martineau, J., & Dredze, M. (2010). Annotating named entities in Twitter data with crowdsourcing. 80-88. Paper presented at 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Mturk 2010 at the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2010, Los Angeles, United States.

Annotating named entities in Twitter data with crowdsourcing. / Finin, Tim; Murnane, Will; Karandikar, Anand et al.
2010. 80-88 Paper presented at 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Mturk 2010 at the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2010, Los Angeles, United States.

Research output: Contribution to conference › Paper › peer-review

Finin, T, Murnane, W, Karandikar, A, Keller, N, Martineau, J & Dredze, M 2010, 'Annotating named entities in Twitter data with crowdsourcing', Paper presented at 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Mturk 2010 at the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2010, Los Angeles, United States, 6/6/10 pp. 80-88.

Finin T, Murnane W, Karandikar A, Keller N, Martineau J, Dredze M. Annotating named entities in Twitter data with crowdsourcing. 2010. Paper presented at 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Mturk 2010 at the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2010, Los Angeles, United States.

Finin, Tim ; Murnane, Will ; Karandikar, Anand et al. / Annotating named entities in Twitter data with crowdsourcing. Paper presented at 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Mturk 2010 at the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2010, Los Angeles, United States.9 p.

@conference{cd6559baa3904e8980240b477b887817,

title = "Annotating named entities in Twitter data with crowdsourcing",

abstract = "We describe our experience using both Amazon Mechanical Turk (MTurk) and CrowdFlower to collect simple named entity annotations for Twitter status updates. Unlike most genres that have traditionally been the focus of named entity experiments, Twitter is far more informal and abbreviated. The collected annotations and annotation techniques will provide a first step towards the full study of named entity recognition in domains like Facebook and Twitter. We also briefly describe how to use MTurk to collect judgements on the quality of “word clouds.",

author = "Tim Finin and Will Murnane and Anand Karandikar and Nicholas Keller and Justin Martineau and Mark Dredze",

note = "Publisher Copyright: {\textcopyright} 2010 Association for Computational Linguistics; 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Mturk 2010 at the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2010 ; Conference date: 06-06-2010",

year = "2010",

language = "English (US)",

pages = "80--88",

}

TY - CONF

T1 - Annotating named entities in Twitter data with crowdsourcing

AU - Finin, Tim

AU - Murnane, Will

AU - Karandikar, Anand

AU - Keller, Nicholas

AU - Martineau, Justin

AU - Dredze, Mark

PY - 2010

Y1 - 2010

N2 - We describe our experience using both Amazon Mechanical Turk (MTurk) and CrowdFlower to collect simple named entity annotations for Twitter status updates. Unlike most genres that have traditionally been the focus of named entity experiments, Twitter is far more informal and abbreviated. The collected annotations and annotation techniques will provide a first step towards the full study of named entity recognition in domains like Facebook and Twitter. We also briefly describe how to use MTurk to collect judgements on the quality of “word clouds.

AB - We describe our experience using both Amazon Mechanical Turk (MTurk) and CrowdFlower to collect simple named entity annotations for Twitter status updates. Unlike most genres that have traditionally been the focus of named entity experiments, Twitter is far more informal and abbreviated. The collected annotations and annotation techniques will provide a first step towards the full study of named entity recognition in domains like Facebook and Twitter. We also briefly describe how to use MTurk to collect judgements on the quality of “word clouds.

UR - http://www.scopus.com/inward/record.url?scp=85094865799&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85094865799&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85094865799

SP - 80

EP - 88

T2 - 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Mturk 2010 at the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2010

Y2 - 6 June 2010

ER -

Annotating named entities in Twitter data with crowdsourcing

Abstract

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this