Design and testing of a tool for evaluating the quality of diabetesconsumer-information web sites

Joshua J. Seidman, Donald Steinwachs, Haya R. Rubin

Research output: Contribution to journalArticlepeer-review

38 Scopus citations


Background: Most existing tools for measuring the quality of Internet health information focus almost exclusively on structural criteria or other proxies for quality information rather than evaluating actual accuracy and comprehensiveness. Objective: This research sought to develop a new performance-measurement tool for evaluating the quality of Internet health information, test the validity and reliability of the tool, and assess the variability in diabetes Web site quality. Methods: An objective, systematic tool was developed to evaluate Internet diabetes information based on a quality-of-care measurement framework. The principal investigator developed an abstraction tool and trained an external reviewer on its use. The tool included 7 structural measures and 34 performance measures created by using evidence-based practice guidelines and experts' judgments of accuracy and comprehensiveness. Results: Substantial variation existed in all categories, with overall scores following a normal distribution and ranging from 15% to 95% (mean was 50% and median was 51%). Lin's concordance correlation coefficient to assess agreement between raters produced a rho of 0.761 (Pearson's r of 0.769), suggesting moderate to high agreement. The average agreement between raters for the performance measures was 0.80. Conclusions: Diabetes Web site quality varies widely. Alpha testing of this new tool suggests that it could become a reliable and valid method for evaluating the quality of Internet health sites. Such an instrument could help lay people distinguish between beneficial and misleading information.

Original languageEnglish (US)
Pages (from-to)136-156
Number of pages21
JournalJournal of medical Internet research
Issue number4
StatePublished - 2003
Externally publishedYes


  • Diabetes
  • Guidelines
  • Information management/standards
  • Internet/standards
  • Medical informatics/standards
  • Quality of health care

ASJC Scopus subject areas

  • Health Informatics


Dive into the research topics of 'Design and testing of a tool for evaluating the quality of diabetesconsumer-information web sites'. Together they form a unique fingerprint.

Cite this