Physician burnout is estimated to affect more than 50% of physicians, with clerical tasks, time pressures, and inefficient work environments as three of the largest contributing factors . Microsoft, as part of its Healthcare NExT Initiative, is partnering with the University of Pittsburgh Medical Center (UPMC) to combat physician burnout in a new initiative entitled Project EmpowerMD. The first project of this collaboration is an Intelligent Scribe, a computerized audio processing program that not only transcribes patient-physician interactions, but also provides the physician with decision support tools and suggestions derived from machine learning on proper diagnosis and clinical management—a virtual AI consultant.
How does it work?
The Intelligent Scribe relies on voice detection of the patient-physician interaction, extracting key words and phrases from both the patient and the physician into the physician’s note and analyzing them using a variety of Microsoft’s cloud computing services, including Custom Speech Services and Language Understanding Intelligent Services . It uses these words and phrases to make suggestions to the physician regarding further diagnostic questioning and even clinical next steps. Physicians are able to interact with the technology by accepting or rejecting suggestions, and the machine learning algorithm incorporates these actions into the algorithm to “learn” from them.
See below for a video released by Microsoft describing the technology:
Given the rapidly expanding body of medical knowledge and the various forms that this information takes (e.g. clinical trials, reviews, public databases, clinical guidelines, etc.), physicians will require these AI decision support tools more and more to be able to deliver the most up-to-date, evidenced-based care possible while still maintaining meaningful relationships with patients. Furthermore, the Intelligent Scribe platform will allow the integration of actual behaviors of numerous physicians during real patient interactions, creating a dynamic program of best practices in the field.
Not only could this program help reduce administrative documentation burden for physicians and improve the patient-physician interaction, but it also has the potential to improve patient outcomes by helping physicians synthesize enormous amounts of clinical and scientific data in its decision support capabilities. With potential healthcare savings from artificial intelligence initiatives estimated well into the billions of dollars annually , there is clearly an economic argument for continued investment in this area as well. Indeed, investors seem to agree with Microsoft’s expansion of these artificial intelligence and cloud computing initiatives .
Artificial intelligence in medicine is here to stay. In an effort to move broadly and swiftly in this space, Microsoft, as part of the Healthcare NExT Initiative, is employing artificial intelligence and machine learning in a whole host of other healthcare contexts, including cancer genomics, health information security and compliance, global health, and medical diagnostics, trying to establish itself as a trustworthy leader in healthcare artificial intelligence and cloud computing, according to Peter Lee, the Corporate Vice President of Microsoft Healthcare . The company’s long-term vision for the initiative is “a learning system that incorporates data from longitudinal medical records, medical devices, genomics, population health, research papers, and more” . In order to keep up with fast-approaching rivals in this space, such as Amazon, Google, and Apple, in addition to the numerous healthcare AI startups popping up, Microsoft will need to continue establishing and fostering relationships with academic leaders like UPMC while ramping up its own internal research and development in this area. Furthermore, in addition to the provider side, Microsoft should consider partnerships with health insurers, who are currently sitting on vast quantities of claims data from which artificial intelligence and machine learning could derive key health insights.
Questions still remain about the efficiency and accuracy of these and other digital scribe initiatives. What will the medicolegal implications of these emerging technologies be, and how will Microsoft and others handle backlash associated with the unavoidable medical errors resulting from this technology? How will Microsoft unlock the potential of this technology while ensuring security and confidentiality in compliance with the Health Insurance Portability and Accountability Act (HIPPA)? The company is keenly aware of this and other data security issues and appears to be addressing them . And finally, what other healthcare domains should Microsoft explore with this technology?
Patients and physicians are eagerly awaiting answers to these questions, hoping that the responses will provide value in the form of more personalized clinical encounters, reduced physician burnout, and improved patient outcomes.
Photo from Steelcase. https://www.steelcase.com/research/articles/topics/healthcare/exam-rooms-that-empower-people/, accessed November 2018.
 Rothenberger, D. “Physician Burnout and Well-Being.” Diseases of the Colon & Rectum, 60(6) (2017), pp.567-576.
 Microsoft. “Project EmpowerMD: Medical conversations to medical intelligence.” https://www.microsoft.com/en-us/research/project/empowermd/, accessed November 2018.
 Kalis, B., Collier, M., and Fu, R. “10 Promising AI Applications in Health Care.” Harvard Business Review, May 10, 2018. [https://hbr.org/2018/05/10-promising-ai-applications-in-health-care], accessed November 2018.
 Ranks, T. “3 Fascinating Companies Using AI to Forge New Advances in Healthcare.” TheStreet, May 18, 2018. [https://www.thestreet.com/investing/three-intriguing-companies-using-artificial-intelligence-14591053], accessed November 2018.
 Lee, P. “Microsoft’s focus on transforming healthcare: Intelligent health through AI and the cloud.” Official Microsoft Blog, February 28, 2018. [https://blogs.microsoft.com/blog/2018/02/28/microsofts-focus-transforming-healthcare-intelligent-health-ai-cloud/], accessed November 2018.
 Pohar, D. “Four paths to voice innovation through healthcare provider and technology partnerships.” Microsoft Research, February 28, 2018. [https://www.microsoft.com/en-us/research/project/empowermd/articles/four-paths-voice-innovation-healthcare-provider-technology-partnerships/], accessed November 2018.