Novel approaches to measuring knowledge among frontline health workers in India: Are phone surveys a reliable option?

Background In 2017, India was home to nearly 20% of maternal and child deaths occurring globally. Accredited social health activists (ASHAs) act as the frontline for health services delivery in India, providing a range of reproductive, maternal, newborn, child health, and nutrition (RMNCH&N) services. Empirical evidence on ASHAs' knowledge is limited, yet is a critical determinant of the quality of health services provided. We assessed the determinants of RMNCH&N knowledge among ASHAs and examined the reliability of alternative modalities of survey delivery, including face-to-face and caller attended telephone interviews (phone surveys) in 4 districts of Madhya Pradesh, India. Methods We carried out face-to-face surveys among a random cross-sectional sample of ASHAs (n = 1,552), and administered a follow-up test-retest survey within 2 weeks of the initial survey to a subsample of ASHAs (n = 173). We interviewed a separate sub-sample of ASHAs 2 weeks of the face-to-face interview over the phone (n = 155). Analyses included bivariate analyses, multivariable linear regression, and prevalence and bias adjusted kappa analyses. Findings The average ASHA knowledge score was 64% and ranged across sub-domains from 71% for essential newborn care, 71% for WASH/ diarrhea, 64% for infant feeding, 61% for family planning, and 60% for maternal health. Leading determinants of knowledge included geographic location, age <30 years of age, education, experience as an ASHA, completion of seven or more client visits weekly, phone ownership and use as a communication tool for work, as well as the ability to navigate interactive voice response prompts (a measure of digital literacy). Efforts to develop a phone survey tool for measuring knowledge suggest that findings on inter-rater and inter-modal reliability were similar. Reliability was higher for shorter, widely known questions, including those about timing of exclusive breastfeeding or number of tetanus shots during pregnancy. Questions with lower reliability included those on sensitive topics such as family planning; questions with multiple response options; or which were difficult for the enumerator to convey. Conclusions Overall results highlight important gaps in the knowledge of ASHAs. Findings on the reliability of phone surveys led to the development of a tool, which can be widely used for the routine, low cost measurement of ASHA RMNCH&N knowledge in India.