Using prediction polling to harness collective intelligence for disease forecasting

Tara Kirk Sell; Kelsey Lane Warmbrod; Crystal Watson; Marc Trotochaud; Elena Martin; Sanjana J. Ravi; Maurice Balick; Emile Servan-Schreiber

doi:10.1186/s12889-021-12083-y

Using prediction polling to harness collective intelligence for disease forecasting

Tara Kirk Sell, Kelsey Lane Warmbrod, Crystal Watson, Marc Trotochaud, Elena Martin, Sanjana J. Ravi, Maurice Balick, Emile Servan-Schreiber

Bloomberg School of Public Health

Research output: Contribution to journal › Article › peer-review

Abstract

Background: The global spread of COVID-19 has shown that reliable forecasting of public health related outcomes is important but lacking. Methods: We report the results of the first large-scale, long-term experiment in crowd-forecasting of infectious-disease outbreaks, where a total of 562 volunteer participants competed over 15 months to make forecasts on 61 questions with a total of 217 possible answers regarding 19 diseases. Results: Consistent with the “wisdom of crowds” phenomenon, we found that crowd forecasts aggregated using best-practice adaptive algorithms are well-calibrated, accurate, timely, and outperform all individual forecasters. Conclusions: Crowd forecasting efforts in public health may be a useful addition to traditional disease surveillance, modeling, and other approaches to evidence-based decision making for infectious disease outbreaks.

Original language	English (US)
Article number	2132
Journal	BMC public health
Volume	21
Issue number	1
DOIs	https://doi.org/10.1186/s12889-021-12083-y
State	Published - Dec 2021

Keywords

COVID-19
Crowd-sourced
Ebola
Epidemic prediction
Forecasting
Infectious disease
Influenza

ASJC Scopus subject areas

Public Health, Environmental and Occupational Health

Access to Document

10.1186/s12889-021-12083-y

Cite this

@article{1edf0ff6aed742d1a189a9ec17d1d455,

title = "Using prediction polling to harness collective intelligence for disease forecasting",

abstract = "Background: The global spread of COVID-19 has shown that reliable forecasting of public health related outcomes is important but lacking. Methods: We report the results of the first large-scale, long-term experiment in crowd-forecasting of infectious-disease outbreaks, where a total of 562 volunteer participants competed over 15 months to make forecasts on 61 questions with a total of 217 possible answers regarding 19 diseases. Results: Consistent with the “wisdom of crowds” phenomenon, we found that crowd forecasts aggregated using best-practice adaptive algorithms are well-calibrated, accurate, timely, and outperform all individual forecasters. Conclusions: Crowd forecasting efforts in public health may be a useful addition to traditional disease surveillance, modeling, and other approaches to evidence-based decision making for infectious disease outbreaks.",

keywords = "COVID-19, Crowd-sourced, Ebola, Epidemic prediction, Forecasting, Infectious disease, Influenza",

author = "Sell, {Tara Kirk} and Warmbrod, {Kelsey Lane} and Crystal Watson and Marc Trotochaud and Elena Martin and Ravi, {Sanjana J.} and Maurice Balick and Emile Servan-Schreiber",

note = "Funding Information: This research was funded by the Open Philanthropy Project. The funder had no role in designing the study; collecting, analyzing, or interpreting the data; or, writing the manuscript. Publisher Copyright: {\textcopyright} 2021, The Author(s).",

year = "2021",

month = dec,

doi = "10.1186/s12889-021-12083-y",

language = "English (US)",

volume = "21",

journal = "BMC public health",

issn = "1471-2458",

publisher = "BioMed Central",

number = "1",

}

TY - JOUR

T1 - Using prediction polling to harness collective intelligence for disease forecasting

AU - Sell, Tara Kirk

AU - Warmbrod, Kelsey Lane

AU - Watson, Crystal

AU - Trotochaud, Marc

AU - Martin, Elena

AU - Ravi, Sanjana J.

AU - Balick, Maurice

AU - Servan-Schreiber, Emile

N1 - Funding Information: This research was funded by the Open Philanthropy Project. The funder had no role in designing the study; collecting, analyzing, or interpreting the data; or, writing the manuscript. Publisher Copyright: © 2021, The Author(s).

PY - 2021/12

Y1 - 2021/12

N2 - Background: The global spread of COVID-19 has shown that reliable forecasting of public health related outcomes is important but lacking. Methods: We report the results of the first large-scale, long-term experiment in crowd-forecasting of infectious-disease outbreaks, where a total of 562 volunteer participants competed over 15 months to make forecasts on 61 questions with a total of 217 possible answers regarding 19 diseases. Results: Consistent with the “wisdom of crowds” phenomenon, we found that crowd forecasts aggregated using best-practice adaptive algorithms are well-calibrated, accurate, timely, and outperform all individual forecasters. Conclusions: Crowd forecasting efforts in public health may be a useful addition to traditional disease surveillance, modeling, and other approaches to evidence-based decision making for infectious disease outbreaks.

AB - Background: The global spread of COVID-19 has shown that reliable forecasting of public health related outcomes is important but lacking. Methods: We report the results of the first large-scale, long-term experiment in crowd-forecasting of infectious-disease outbreaks, where a total of 562 volunteer participants competed over 15 months to make forecasts on 61 questions with a total of 217 possible answers regarding 19 diseases. Results: Consistent with the “wisdom of crowds” phenomenon, we found that crowd forecasts aggregated using best-practice adaptive algorithms are well-calibrated, accurate, timely, and outperform all individual forecasters. Conclusions: Crowd forecasting efforts in public health may be a useful addition to traditional disease surveillance, modeling, and other approaches to evidence-based decision making for infectious disease outbreaks.

KW - COVID-19

KW - Crowd-sourced

KW - Ebola

KW - Epidemic prediction

KW - Forecasting

KW - Infectious disease

KW - Influenza

UR - http://www.scopus.com/inward/record.url?scp=85119489772&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85119489772&partnerID=8YFLogxK

U2 - 10.1186/s12889-021-12083-y

DO - 10.1186/s12889-021-12083-y

M3 - Article

C2 - 34801014

AN - SCOPUS:85119489772

SN - 1471-2458

VL - 21

JO - BMC public health

JF - BMC public health

IS - 1

M1 - 2132

ER -

Using prediction polling to harness collective intelligence for disease forecasting

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this