Predicting Psychiatric Diseases Using AutoAI: A Performance Analysis Based on Health Insurance Billing Data

Digital transformation enables a vast growth of health data. Because of that, scholars and professionals considered AI to enhance quality of care significantly. Machine learning (ML) algorithms for improvement have been studied extensively, but automatic artificial intelligence (autoAI/autoML) has been widely neglected. AutoAI aims to automate the complete AI lifecycle to save data scientists from doing low-level coding tasks. Additionally, autoAI has the potential to democratize AI by empowering non-IT users to build AI algorithms. In this paper, we analyze the suitability of autoAI for mental health screening to detect psychiatric diseases. A sooner diagnosis can lead to cost savings for healthcare systems and decrease patients’ suffering. We evaluate AutoAI using the open-source machine learning library auto-sklearn, as well as the commercial Watson Studio’s AutoAI platform to predict depression, post-traumatic stress disorder, and psychiatric disorders in general. We use health insurance billing data from 83,986 patients with a total of 687,697 ICD-10 coded diseases. The results of our research are as follows: (i) on average, an accuracy of 0.6 (F 1 –score 0.58) with a precision of 0.61 and recall of 0.56 was achieved using auto-sklearn. (ii) The evaluation metrics for Watson Studio’s autoAI were 0.59 accuracy, 0.57 F 1 –score, a precision of 0.6, and a recall of 0.55. We conclude that the prediction quality of autoAI in psychiatry still lacks behind traditional ML approaches by about 24% and is therefore not ready for production use yet.