Test-retest reliability of freesurfer measurements within and between sites: Effects of visual approval process

Zafer Iscan, Tony B. Jin, Alexandria Kendrick, Bryan Szeglin, Hanzhang Lu, Madhukar Trivedi, Maurizio Fava, Patrick J. Mcgrath, Myrna Weissman, Benji T. Kurian, Phillip Adams, Sarah Weyandt, Marisa Toups, Thomas Carmody, Melvin Mcinnis, Cristina Cusin, Crystal Cooper, Maria A. Oquendo, Ramin V. Parsey, Christine Delorenzo

    Research output: Contribution to journalArticlepeer-review

    80 Scopus citations


    In the last decade, many studies have used automated processes to analyze magnetic resonance imaging (MRI) data such as cortical thickness, which is one indicator of neuronal health. Due to the convenience of image processing software (e.g., FreeSurfer), standard practice is to rely on automated results without performing visual inspection of intermediate processing. In this work, structural MRIs of 40 healthy controls who were scanned twice were used to determine the test-retest reliability of FreeSurfer-derived cortical measures in four groups of subjects-those 25 that passed visual inspection (approved), those 15 that failed visual inspection (disapproved), a combined group, and a subset of 10 subjects (Travel) whose test and retest scans occurred at different sites. Test-retest correlation (TRC), intraclass correlation coefficient (ICC), and percent difference (PD) were used to measure the reliability in the Destrieux and Desikan-Killiany (DK) atlases. In the approved subjects, reliability of cortical thickness/surface area/volume (DK atlas only) were: TRC (0.82/0.88/0.88), ICC (0.81/0.87/0.88), PD (0.86/1.19/1.39), which represent a significant improvement over these measures when disapproved subjects are included. Travel subjects' results show that cortical thickness reliability is more sensitive to site differences than the cortical surface area and volume. To determine the effect of visual inspection on sample size required for studies of MRI-derived cortical thickness, the number of subjects required to show group differences was calculated. Significant differences observed across imaging sites, between visually approved/disapproved subjects, and across regions with different sizes suggest that these measures should be used with caution.

    Original languageEnglish (US)
    Pages (from-to)3472-3485
    Number of pages14
    JournalHuman Brain Mapping
    Issue number9
    StatePublished - Sep 1 2015


    • Cerebral cortical surface area
    • Cerebral cortical thickness
    • Cerebral cortical volume
    • FreeSurfer
    • Multisite MRI
    • Test-retest reliability

    ASJC Scopus subject areas

    • Anatomy
    • Radiological and Ultrasound Technology
    • Radiology Nuclear Medicine and imaging
    • Neurology
    • Clinical Neurology


    Dive into the research topics of 'Test-retest reliability of freesurfer measurements within and between sites: Effects of visual approval process'. Together they form a unique fingerprint.

    Cite this