Panel dis­cus­sion on eval­u­ation in med­ical in­form­at­ics

The panel’s primary aim was to give participants an overview of evaluation methods currently used in different domains of medical informatics and related fields and to make them aware of the shortcomings of these methods. The second aim was to inform participants about the research that is currently being conducted to improve evaluation methods in medical informatics with a holistic view. The diverse backgrounds of the panelists, ranging from medicine and public health to medical informatics and economics, showcased the interdisciplinary nature of medical informatics.

After a brief introduction of the panel topic and the panelists by moderator Felix Holl from HNU’s Digihealth Institute, Marina Fotteler held the first presentation, sharing results from her work on collecting and synthesizing the existing evidence for ambient assisted living (AAL) technologies. She also highlighted recommendations for new evaluation approaches these technologies.

Felix Holl gave an overview of the current state of the evaluation of mHealth applications. After highlighting current shortcomings, he introduced the participants to the current developments around the German Digital Care Act. He focused on how the evaluation of mobile health applications is currently being implemented as part of the act.

Stefan Müller-Mielitz, Chief Executive Officer (CEO) of the IEKF GmbH and a close collaborator of the DigiHealth Institute, presented the work of the eHealth working group of the German Health Economic Society (dggö) on the development of a 12-point road map for the economic analysis of eHealth projects.

As the last presenter, Walter Swoboda, Director of the DigiHealth Institute, highlighted the distinction between a classical evaluation of new treatments and drugs compared to the assessment of digital technologies in medicine. He described several different aspects and shared an example from his tenure as the Chief Information Officer (CIO) at the University of Munich Medical Center, where a holistic evaluation had been performed before the introduction of a digital documentation system for vital signs. In closing, he echoed his viewpoint and the need for a holistic evaluation framework for eHealth applications.

Following the panel presentations, participants were asked to pose questions to the panelists and to share their viewpoints. There was consensus, both among the panelists and participants (from the Netherlands, Malaysia, and Germany) that the current evaluation methods used for digital solutions in medicine are inadequate and neglect several important aspects. Felix Holl facilitated the panel discussion with a set of guiding questions both to the participants and panelists. 

As a next step forward, the results from the panel discussion will be submitted to a scientific journal. The participants agreed that the development of new evaluation methods for eHealth should be placed on the research agenda. Prof. Swoboda concluded that “the most important thing is that we, as a scientific community, have to try to establish a new, holistic evaluation standard that is broadly used and validated.”

The panel had initially been submitted to and accepted at the “Medical Informatics Europe 2020” conference, organized by the European Federation for Medical Informatics (EFMI), and was originally scheduled for April 2020 in Geneva, Switzerland. Unfortunately, the conference was later canceled due to the COVID-19 pandemic. However, the panel was then held online as part of the EFMI webinar series in July 2020 in conjunction with the International Conference on Informatics, Management, and Technology in Healthcare (ICIMTH) 2020.