|
Over the past decade, crowd-sourcing complex image analysis tasks to a human crowd has emerged as an alternative to
energy-inefficient and difficult-to-implement computational approaches. Following this trend, we have developed a
mathematical framework for statistically combining human crowd-sourcing of biomedical image analysis and diagnosis
through games. Using a web-based smart game (BioGames), we demonstrated this platform’s effectiveness for telediagnosis
of malaria from microscopic images of individual red blood cells (RBCs). After public release in early 2012
(http://biogames.ee.ucla.edu), more than 3000 gamers (experts and non-experts) used this BioGames platform to
diagnose over 2800 distinct RBC images, marking them as positive (infected) or negative (non-infected). Furthermore,
we asked expert diagnosticians to tag the same set of cells with labels of positive, negative, or questionable (insufficient
information for a reliable diagnosis) and statistically combined their decisions to generate a gold standard malaria image
library. Our framework utilized minimally trained gamers’ diagnoses to generate a set of statistical labels with an
accuracy that is within 98% of our gold standard image library, demonstrating the “wisdom of the crowd”. Using the
same image library, we have recently launched a web-based malaria training and educational game allowing
diagnosticians to compare their performance with their peers. After diagnosing a set of ~500 cells per game,
diagnosticians can compare their quantified scores against a leaderboard and view their misdiagnosed cells. Using this
platform, we aim to expand our gold standard library with new RBC images and provide a quantified digital tool for
measuring and improving diagnostician training globally.
|
