TY - JOUR
T1 - A Framework for Clinical Validation of Automatic Contour Propagation
T2 - Standardizing Geometric and Dosimetric Evaluation
AU - Frederick, Amy
AU - Roumeliotis, Michael
AU - Grendarova, Petra
AU - Craighead, Peter
AU - Abedin, Tasnima
AU - Watt, Elizabeth
AU - Olivotto, Ivo A.
AU - Meyer, Tyler
AU - Quirk, Sarah
N1 - Funding Information:
Disclosures: Ms Frederick received a postgraduate scholarship from the Natural Sciences and Engineering Research Council of Canada.
Publisher Copyright:
© 2019 American Society for Radiation Oncology
PY - 2019/11
Y1 - 2019/11
N2 - Purpose: The objective of this work was to outline and demonstrate a standardized framework for evaluating automatically propagated contour quality against expert contours. A 2-pronged approach is used to evaluate contour quality: a geometric evaluation to identify geometric and spatial discrepancies between propagated and expert contours, and a comprehensive dosimetric comparison to provide clinical context for the results. Methods and Materials: The standardized framework requires a primary image, with reference contours and a radiation therapy treatment plan, and a secondary image. Reference contours are automatically propagated onto the secondary image anatomy and compared with expert contours obtained in an interobserver study. The standardized framework outlines geometric and dosimetric evaluation methodologies for determining indistinguishability between propagated and expert contours in a cohort analysis. Propagated contours are geometrically compared with expert contours in terms of the Dice similarity coefficient and the mean distance to agreement. Statistical analysis is performed on the central tendency and variability of Dice similarity coefficient and mean distance to agreement values over the patient cohort. Dosimetric evaluation involves computing the mean and 95% confidence intervals for the differences in cumulative dose-volume histograms for propagated and expert contours. A case study in accelerated partial breast irradiation was shown to demonstrate the framework. Results: The standardized framework was applied to a case study of 24 patient data sets with 3 radiation oncologists providing the expert contours. Cohort analysis indicated that propagated contours were geometrically indistinguishable and dosimetrically distinguishable from expert contours. Conclusions: The recommended framework standardizes the comparison of geometric and dosimetric parameters to demonstrate indistinguishability of propagated contours from expert contours. Adoption of this framework is vital for consistent and comprehensive validation of automatic contour propagation for use in large-scale cohort analyses.
AB - Purpose: The objective of this work was to outline and demonstrate a standardized framework for evaluating automatically propagated contour quality against expert contours. A 2-pronged approach is used to evaluate contour quality: a geometric evaluation to identify geometric and spatial discrepancies between propagated and expert contours, and a comprehensive dosimetric comparison to provide clinical context for the results. Methods and Materials: The standardized framework requires a primary image, with reference contours and a radiation therapy treatment plan, and a secondary image. Reference contours are automatically propagated onto the secondary image anatomy and compared with expert contours obtained in an interobserver study. The standardized framework outlines geometric and dosimetric evaluation methodologies for determining indistinguishability between propagated and expert contours in a cohort analysis. Propagated contours are geometrically compared with expert contours in terms of the Dice similarity coefficient and the mean distance to agreement. Statistical analysis is performed on the central tendency and variability of Dice similarity coefficient and mean distance to agreement values over the patient cohort. Dosimetric evaluation involves computing the mean and 95% confidence intervals for the differences in cumulative dose-volume histograms for propagated and expert contours. A case study in accelerated partial breast irradiation was shown to demonstrate the framework. Results: The standardized framework was applied to a case study of 24 patient data sets with 3 radiation oncologists providing the expert contours. Cohort analysis indicated that propagated contours were geometrically indistinguishable and dosimetrically distinguishable from expert contours. Conclusions: The recommended framework standardizes the comparison of geometric and dosimetric parameters to demonstrate indistinguishability of propagated contours from expert contours. Adoption of this framework is vital for consistent and comprehensive validation of automatic contour propagation for use in large-scale cohort analyses.
UR - http://www.scopus.com/inward/record.url?scp=85070533681&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85070533681&partnerID=8YFLogxK
U2 - 10.1016/j.prro.2019.06.017
DO - 10.1016/j.prro.2019.06.017
M3 - Article
C2 - 31279940
AN - SCOPUS:85070533681
SN - 1879-8500
VL - 9
SP - 448
EP - 455
JO - Practical Radiation Oncology
JF - Practical Radiation Oncology
IS - 6
ER -