Error measures for objective assessment of scene segmentation algorithms

Scene segmentation is an important element in pattern recognition problems. Previous efforts to evaluate and compare scene segmentation procedures have been largely subjective. Quantitative error measures would facilitate objective comparison of scene segmentation algorithms. A theoretical discussion leading to a new generalized quantitative error measure, G₂, based on comparison of both pixel class proportions and spatial distributions of 'true' and test segmentation, is presented. This error measure was tested on 14 manual segmentations and 40 gynecologic cytology specimens segmented with five different scene segmentation techniques. Results indicate that G₂ seems to have the desirable properties of correlation with human observation, categorization of error allowing for weighting, invariance with picture size and ease of computation necessary for a useful scene segmentation error measure.