Quanti.us: a tool for rapid, flexible, crowd-based annotation of images

• DOI: 10.1038/s41592-018-0069-0
• Publication Date: 2018-07-19T14:03:44Z
• Authors: Alex J. Hughes; Joseph D. Mornin; Sujoy K. Biswas; Lauren E. Beck; David P. Bauer; Arjun Raj; Simone Bianco; Zev J. Gartner
• Keywords: 0301 basic medicine; Technology; Image Processing; Bioengineering; Medical and Health Sciences; Imaging; Machine Learning; 03 medical and health sciences; Computer-Assisted; Imaging, Three-Dimensional; Machine Learning and Artificial Intelligence; Image Processing, Computer-Assisted; Animals; Humans; Internet; Computational Biology; Biological Sciences; 1.4 Methodologies and measurements; 004; Biological sciences; Networking and Information Technology R&D (NITRD); Three-Dimensional; Crowdsourcing; Algorithms; Software; Developmental Biology

We describe Quanti.us , a crowd-based image-annotation platform that provides an accurate alternative to computational algorithms for difficult image-analysis problems. We used Quanti.us for a variety of medium-throughput image-analysis tasks and achieved 10-50× savings in analysis time compared with that required for the same task by a single expert annotator. We show equivalent deep learning performance for Quanti.us-derived and expert-derived annotations, which should allow scalable integration with tailored machine learning algorithms.