LISH works to better understand the variability of doctors’ diagnoses in healthcare, the drivers behind such variability, the impact it has on developing and benchmarking AI systems for healthcare, as well as the methodologies for attaining high quality datasets for training AI algorithms to achieve superhuman performance. In addition, we apply such techniques in combination with crowdsourcing to develop AI solutions to medical imaging problems. We also seek to better understand how human-AI interactions impact decision making in healthcare.
Dental Image Recognition System
In collaboration with Charite-Berlin Hospital, we are studying the drivers of variability in doctor performance when diagnosing ailments in dental x-ray images, and how multiple human-labelings of the same data can yield more reliable diagnoses of ailments. These studies aim to provide new insights on improving clinical care and a better understanding biases and shortcomings in evaluating diagnostic imaging. Further, we want to understand if multiple-expert labelling can be used a proxy for clinical ground-truth which should have implications of how to test AI algorithms in medical imaging in general.
Lung Cancer Tumor Characterization Image Recognition Algorithm & System
Lung cancer is the leading cause of cancer death in the United States. Successful treatment depends on a radiation oncologist’s ability to accurately measure the tumor’s shape and responsiveness to interventions. Furthermore, manual delineation of tumors is time consuming and prone to inconsistency or bias. The goal of this project was to produce, through a series of competitions on Topcoder in 2016 and 2017 (see the problem statement for Part 1; problem statement for Part 2), an automatic tumor delineation algorithm that parallels the accuracy of an average expert while exceeding an expert in terms of speed and consistency.
In Part 1, contestants were tasked with locating and contouring the tumor on images. Thirty-one competitors submitted 244 solutions and competed for $35,000 in prizes. The top solution was able to locate about 70% of the tumors. In Part 2, contestants were asked to improve the contouring performance of their solutions, given a point within the tumor as additional data. Eleven competitors submitted 164 solutions and competed for $15,000. Part 2 yielded a 40% improvement in performance from Part 1. For more information, see here.