A tool for automatic scoring of spelling performance

Charalambos Themistocleous, Kyriaki Neophytou, Brenda Rapp, Kyrana Tsapkini

Research output: Contribution to journalArticlepeer-review

Abstract

Purpose: The evaluation of spelling performance in aphasia reveals deficits in written language and can facilitate the design of targeted writing treatments. Nevertheless, manual scoring of spelling performance is time-consuming, laborious, and error prone. We propose a novel method based on the use of distance metrics to automatically score spelling. This study compares six automatic distance metrics to identify the metric that best corresponds to the gold standard— manual scoring—using data from manually obtained spelling scores from individuals with primary progressive aphasia. Method: Three thousand five hundred forty word and nonword spelling productions from 42 individuals with primary progressive aphasia were scored manually. The gold standard—the manual scores—were compared to scores from six automated distance metrics: sequence matcher ratio, Damerau–Levenshtein distance, normalized Damerau– Levenshtein distance, Jaccard distance, Masi distance, and Jaro–Winkler similarity distance. We evaluated each distance metric based on its correlation with the manual spelling score. Results: All automatic distance scores had high correlation with the manual method for both words and nonwords. The normalized Damerau–Levenshtein distance provided the highest correlation with the manual scoring for both words (rs = .99) and nonwords (rs = .95). Conclusions: The high correlation between the automated and manual methods suggests that automatic spelling scoring constitutes a quick and objective approach that can reliably substitute the existing manual and time-consuming spelling scoring process, an important asset for both researchers and clinicians.

Original languageEnglish (US)
Pages (from-to)4179-4192
Number of pages14
JournalJournal of Speech, Language, and Hearing Research
Volume63
Issue number12
DOIs
StatePublished - Dec 2020

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language
  • Speech and Hearing

Fingerprint

Dive into the research topics of 'A tool for automatic scoring of spelling performance'. Together they form a unique fingerprint.

Cite this