Integrated Citizen Centered Digital Health and Social Care

Ulla-Mari Kinnunen

Over recent years there has been major investment in research infrastructure to harness the potential of routinely collected health data. In 2013, The Farr Institute for Health Informatics Research was established in the UK, undertaking health informatics research to enhance patient and public health by the analysis of data from multiple sources and unleashing the value of vast sources of clinical, biological, population and environmental data for public benefit. The Medical Informatics Europe (MIE) conference is already established as a key event in the calendar of the European Federation of Medical Informatics (EFMI); The Farr Institute has been establishing a conference series. For 2017, the decision was made to combine the power and established reputational excellence of EFMI with the emerging and innovative research of The Farr Institute community to create ‘Informatics for Health 2017’, a joint conference that creates a scientific forum allowing these two communities to share ...

The Informatics for Health congress, 24-26 April 2017, in Manchester, UK, brought together the Medical Informatics Europe (MIE) conference and the Farr Institute International Conference. This special issue of the Journal of Innovation in Health Informatics contains 113 presentation abstracts and 149 poster abstracts from the congress. The twin programmes of "Big Data" and "Digital Health" are not always joined up by coherent policy and investment priorities. Substantial global investment in health IT and data science has led to sound progress but highly variable outcomes. Society needs an approach that brings together the science and the practice of health informatics. The goal is multi-level Learning Health Systems that consume and intelligently act upon both patient data and organizational intervention outcomes. Informatics for Health demonstrated the art of the possible, seen in the breadth and depth of our contributions. We call upon policy makers, research ...

Background: The International Academy of Health Sciences Informatics (IAHSI) is the Academy of the International Medical Informatics Association (IMIA). As an international forum for peers in biomedical and health informatics, the Academy shall play an important role in exchanging knowledge, providing education and training, and producing policy documents. Objectives: A major priority of the Academy’s activities in its inaugural phase was to define its strategy and focus areas in accordance with its objectives and to prioritize the Academy’s work, which can then be transferred to respective taskforces. Method: This document reflects the major outcomes of intensive discussions that occurred during 2019. It was presented at the Academy’s 3rd Plenary on August 25th, 2019, in Lyon, France. Results: Regardless of the ‘living nature’ of the strategy and focus areas document, it was concluded during the Plenary that the first version, which will be used as a base for decisions on the Acade...

INTEGRATED CITIZEN CENTERED DIGITAL HEALTH AND SOCIAL CARE Studies in Health Technology and Informatics International health informatics is driven by developments in biomedical technologies and medical informatics research that are advancing in parallel and form one integrated world of information and communication media and result in massive amounts of health data. These components include genomics and precision medicine, machine learning, translational informatics, intelligent systems for clinicians and patients, mobile health applications, datadriven telecommunication and rehabilitative technology, sensors, intelligent home technology, EHR and patient-controlled data, and Internet of Things. Studies in Health Technology and Informatics (HTI) series was started in 1990 in collaboration with EU programmes that preceded the Horizon 2020 to promote biomedical and health informatics research. It has developed into a highly visible global platform for the dissemination of original research in this field, containing more than 250 volumes of high-quality works from all over the world. The international Editorial Board selects publications with relevance and quality for the field. All contributions to the volumes in the series are peer reviewed. Volumes in the HTI series are submitted for indexing by MEDLINE/PubMed; Web of Science: Conference Proceedings Citation Index – Science (CPCI-S) and Book Citation Index – Science (BKCI-S); Google Scholar; Scopus; EMCare. Series Editors: B. Blobel, O. Bodenreider, E. Borycki, M. Braunstein, C. Bühler, J.P. Christensen, R. Cooper, R. Cornet, J. Dewen, O. Le Dour, P.C. Dykes, A. Famili, M. González-Sancho, E.J.S. Hovenga, J.W. Jutai, Z. Kolitsi, C.U. Lehmann, J. Mantas, V. Maojo, A. Moen, J.F.M. Molenbroek, G. de Moor, M.A. Musen, P.F. Niederer, C. Nøhr, A. Pedotti, N. Peek, O. Rienhoff, G. Riva, W. Rouse, K. Saranto, M.J. Scherer, S. Schürer, E.R. Siegel, C. Safran, N. Sarkar, T. Solomonides, E. Tam, J. Tenenbaum, B. Wiederhold, P. Wilson and L.H.W. van der Woude Volume 275 Recently published in this series Vol. 274 J. Mantas, R. Šendelj, I. Ognjanović, P. Knaup, E. Ammenwerth and O. Varga (Eds.), Health Information Management: Empowering Public Health Vol. 273 B. Blobel, L. Lhotska, P. Pharow and F. Sousa (Eds.), pHealth 2020 – Proceedings of the 17th International Conference on Wearable Micro and Nano Technologies for Personalized Health – 14–16 September 2020, Prague, Czech Republic Vol. 272 J. Mantas, A. Hasman, M.S. Househ, P. Gallos and E. Zoulias (Eds.), The Importance of Health Informatics in Public Health during a Pandemic Vol. 271 G. Schreier, D. Hayn and A. Eggerth (Eds.), dHealth 2020 – Biomedical Informatics for Health and Care – Proceedings of the 14th Health Informatics Meets Digital Health Conference ISSN 0926-9630 (print) ISSN 1879-8365 (online) Integrated Citizen Centered Digital Health and Social Care Citizens as Data Producers and Service co-Creators Proceedings of the EFMI 2020 Special Topic Conference Edited by Alpo Värri Faculty of Medicine and Health Technology, Tampere University, Finland Jaime Delgado Department of Computer Architecture, Universitat Politècnica de Catalunya, Spain Parisis Gallos Health Informatics Laboratory, National and Kapodistrian University of Athens, Greece Maria Hägglund Department of Women’s and Children’s Health, Uppsala University, Sweden Kristiina Häyrinen Finnish Social and Health Informatics Association, Finland Ulla-Mari Kinnunen Department of Health and Social Management, University of Eastern Finland, Finland Louise B. Pape-Haugaard Department of Health Science and Technology, Aalborg University, Denmark Laura-Maria Peltonen Department of Nursing Science, University of Turku, Finland Kaija Saranto Department of Health and Social Management, University of Eastern Finland, Finland and Philip Scott Centre for Healthcare Modelling & Informatics, University of Portsmouth, United Kingdom Amsterdam  Berlin  Washington, DC © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This book is published online with Open Access and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). ISBN 978-1-64368-144-3 (print) ISBN 978-1-64368-145-0 (online) Library of Congress Control Number: 2020948489 doi: 10.3233/SHTI275 Publisher IOS Press BV Nieuwe Hemweg 6B 1013 BG Amsterdam Netherlands fax: +31 20 687 0019 e-mail: order@iospress.nl For book sales in the USA and Canada: IOS Press, Inc. 6751 Tepper Drive Clifton, VA 20124 USA Tel.: +1 703 830 6300 Fax: +1 703 830 2300 sales@iospress.com LEGAL NOTICE The publisher is not responsible for the use which might be made of the following information. PRINTED IN THE NETHERLANDS v Preface This volume presents the proceedings of the EFMI Special Topic Conference 2020 organized in November 2020 as the first virtual EFMI conference. This conference focused on citizen centered aspects of health informatics. The conference invited papers particularly from the following topics:         Tools and technologies to support citizen centered digital services Capacity building to enhance the development and use of digital services Confidentiality, data integrity and data protection to guarantee trustworthy services Citizen safety in digital services Effectiveness and impacts of citizen digital and integrated health and social services Evaluation approaches and methods for digital services Usability, usefulness and user acceptance of digital services Guidelines for successful implementation of digital services for citizen This volume shows what kind of a collection of papers received the highest marks in the peer review process. The most popular track among the authors was the track Tools and technologies to support citizen centered digital services. The papers in this track cover a wide area of applications. Surprisingly, quite few authors addressed the second main theme of the conference, Citizens as data producers and service co-creators. This may indicate that the progress in this area is not yet as fast as expected. Usability was, however, addressed by several authors. Privacy and security are – and given the developing security threat landscape, will be an important topic, as well. Some of the papers are related to the COVID-19 epidemic – the phenomenon of year 2020. The local organization committee which became a part of the scientific program committee (SPC), had representatives from the Finnish Social and Health Informatics Association, University of Eastern Finland, University of Turku and Tampere University. The other SPC members were representatives from the EFMI working groups Citizen and Health Data, Education, Assessment of Health Information Systems, and Security, Safety and Ethics. The SPC consisted of the following people: Jaime Delgado, Parisis Gallos, Maria Hägglund, Kristiina Häyrinen (vice chair), Ulla-Mari Kinnunen, Louise Pape-Haugaard, Laura-Maria Peltonen, Kaija Saranto, Philip Scott, and Alpo Värri (chair). On behalf of the scientific program committee I would like to warmly thank all the authors who submitted their papers to the conference. Many thanks also to the reviewers whose voluntary work contributed to the quality of the conference, not forgetting the scientific program committee itself that put the whole conference together in its 20+ meetings and individual work. Alpo Värri Chair of Scientific Programme Committee Tampere, October 2020 This page intentionally left blank vii EFMI STC 2020 Scientific Programme Committee Alpo Värri, Jaime Delgado, Parisis Gallos, Maria Hägglund, Kristiina Häyrinen, UllaMari Kinnunen, Louise B. Pape-Haugaard, Laura-Maria Peltonen, Kaija Saranto, and Philip Scott. Reviewers Leila Ahmadian Yasser Alsafadi Mansoor Baig Alireza Banaye Yazdipour Mohamed Ben Said Oana Sorina Chirila Tony Cornford Mihaela Crișan - Vida Riitta Danielsson-Ojala Jaime Delgado Kerstin Denecke Persephone Doupi Mariusz Duplaga Peter Elkin Uwe Engelmann Mohammad Faysel Mircea Focsa Gerard Freriks Jan Gaebel Parisis Gallos Mauro Giacomini Denise Giles Natalia Grabar Kemal Hakan Gulkesen Brahim Hadji Maria Hägglund Angelika Händel Lyn Hanmer Kristiina Häyrinen Jacob Hofdijk Sanja Ivankovic Henry Joutsijoki Oladimeji Kazeem Kampol Khemthong Ulla-Mari Kinnunen Anne Kuusisto Antoine Lamer Pashalina Lialiou Pia Liljamo Sarah Lim Choi Keung Silvia Llorente Ljerka Luić Diana Lungeanu Nestor Adolfo Mamani Macedo Carlos Molina Eustache Muteba Ayumba Juha Mykkänen Pantelis Natsiavas Ernest Nwadinobi Pirkko Nykänen Roxana Ologeanu-Taddei Samir Omanovic Andrej Orel Louise Pape-Haugaard Carlos Parra Tiago Pedrosa Laura-Maria Peltonen Monika Pobiruchin Lucian Prodan Giuseppe Rauch Alejandro Rodríguez González Andreas Schuler Mary Sharp Abbas Sheikhtaheri Berglind Smaradottir Martin Staemmler Milton Stern Oscar Tamburis Philipp Urbauer Alpo Värri Irina Vasilyeva Jan-Patrick Weiß Alfred Winter This page intentionally left blank ix Contents Preface Alpo Värri EFMI STC 2020 Scientific Programme Committee and Reviewers Visualization of Guideline-Based Decision Support for the Management of Pressure Ulcers in Nursing Homes Abir Abdellatif, Jacques Bouaud, Joël Belmin and Brigitte Seroussi Exploring the Social Drivers of Health During a Pandemic: Leveraging Knowledge Graphs and Population Trends in COVID-19 Joao H. Bettencourt-Silva, Natasha Mulligan, Charles Jochim, Nagesh Yadav, Walter Sedlazek, Vanessa Lopez and Martin Gleize Integrating Patient-Generated Health Data in an Electronic Medical Record: Stakeholders’ Perspectives Katherine Blondon and Frederic Ehrler Integrating Healthcare Data for Enhanced Citizen-Centred Care and Analytics Juliana K.F. Bowles, Juan Mendoza-Santana, Andreas F. Vermeulen, Thais Webber and Euan Blackledge Implementing an Urban Public Health Observatory for (Near) Real-Time Surveillance for the COVID-19 Pandemic Whitney S. Brakefield, Nariman Ammar, Olufunto Olusanya, Esra Ozdenerol, Fridtjof Thomas, Altha J. Stewart, Karen C. Johnson, Robert L. Davis, David L. Schwartz and Arash Shaban-Nejad v vii 1 6 12 17 22 Dashboard Visualization of Information for Emergency Medical Services Oliver M. Christen, Yannic Mösching, Patrik Müller, Kerstin Denecke and Stephan Nüssli 27 Latent COVID-19 Clusters in Patients with Chronic Respiratory Conditions Wanting Cui, Manuel Cabrera and Joseph Finkelstein 32 Security and Privacy when Applying FAIR Principles to Genomic Information Jaime Delgado and Silvia Llorente 37 SLEEPexpert App – A Mobile Application to Support Insomnia Treatment for Patients with Severe Psychiatric Disorders Kerstin Denecke, Carlotta L. Schneider, Elisabeth Hertenstein and Christoph Nissen Measures of Decision Aid Quality Are Preference-Sensitive and Interest-Conflicted – 1: Normative Measures Jack Dowie, Mette Kjer Kaltoft and Vije Kumar Rajput 42 47 x Measures of Decision Aid Quality Are Preference-Sensitive and Interest-Conflicted – 2: Empirical Measures Jack Dowie, Mette Kjer Kaltoft and Vije Kumar Rajput Acceptance Study on the Usage of Health-Enabling Technologies in Therapy and Diagnostics for People with Mental Disorders Bastian Droegemueller, Corinna Mielke, Reinhold Haux and Alexander Diehl From Personalised Predictions to Targeted Advice: Improving Self-Management in Rheumatoid Arthritis Ali Fahmi, Hamit Soyel, William Marsh, Paul Curzon, Amy MacBrayne and Frances Humby What is Digital Health? Review of Definitions Farhad Fatehi, Mahnaz Samadbeik and Azar Kazemi Usability of Remote Assessment of Exercise Capacity for Pulmonary Telerehabilitation Program Joseph Finkelstein, In cheol Jeong, Mackenzie Doerstling, Yichao Shen, Chenhao Wei and Herbert Karpatkin Personalization Dimensions for MHealth to Improve Behavior Change: A Scoping Review Laëtitia Gosetto, Frédéric Ehrler and Gilles Falquet 52 57 62 67 72 77 Mobile Access and Adoption of the Swedish National Patient Portal Maria Hägglund, Charlotte Blease and Isabella Scandurra 82 Chronic Disease Self Management Using a Social Networking PHR/UHR Jeremy S. Kagan 87 A Secure Protocol for Managing and Sharing Personal Healthcare Data Athanasios Kiourtis, Argyro Mavrogiorgou, Sofia-Anna Menesidou, Panagiotis Gouvas and Dimosthenis Kyriazis 92 Technology Supporting Nursing at Homecare – Seems to Be Lacking Eija Kivekäs, Santtu Mikkonen, Samuli Koponen and Kaija Saranto 97 Health Professionals’ Perceptions and Reactions to ICT-Related Patient Safety Incidents Jouni Kouvo, Samuli Koponen, Hanna Kuusisto and Kaija Saranto From Atomic Guideline-Based Recommendations to Complete Therapeutic Care Plans: A Knowledge-Based Approach Applied to Breast Cancer Management Hicham Kouz, Jacques Bouaud, Gilles Guézennec and Brigitte Séroussi Automatic Exploitation of YouTube Data: A Study of Videos Published by a French YouTuber During COVID-19 Quarantine in France Gery Laurent, Benjamin Guinhouya, Marielle Whatelet and Antoine Lamer 102 107 112 xi Reuse of Clinical COVID-19 Patient Data: Pre-Processing for Future Classification Elena Lazarova, Sara Mora, Antonio Di Biagio, Antonio Vena and Mauro Giacomini Development and Validation of Standardized Pain Management Documentation Pia Liljamo and Ulla-Mari Kinnunen Patient-Centered Development of a Digital Care Pathway for Arrhythmia Patients Pia Liljamo, Hanna Säilynoja, Kirsi Tuomikoski, Anja Henner and Kirsi Koivunen 117 122 127 Word-Final Phoneme Segmentation Using Cross-Correlation Emilian-Erman Mahmut, Stelian Nicola and Vasile Stoicu-Tivadar 132 A Decentralized Framework for Biostatistics and Privacy Concerns Paul Mangold, Alexandre Filiot, Mouhamed Moussa, Vincent Sobanski, Gregoire Ficheur, Paul Andrey and Antoine Lamer 137 Do You Know Who Is Talking to Your Wearable Smartband? Andrei Kazlouski, Thomas Marchioro, Harry Manifavas and Evangelos Markatos 142 The Effect of Chronic Diseases on the Use of Health Technology and Digital Services in the Elderly Population in Finland Jukka Mielonen, Ulla-Mari Kinnunen, Kaija Saranto, Anssi Kemppi and Hanna Kuusisto 147 Personalized Predictive Models for Identifying Clinical Deterioration Using LSTM in Emergency Departments Amin Naemi, Thomas Schmidt, Marjan Mansourvar and Uffe Kock Wiil 152 Electronic Health Record System-Related Patient Safety Incidents – How to Classify Them? Sari Palojoki, Riikka Vuokko, Anne Vakkuri and Kaija Saranto 157 Semantic Clustering to Augment Qualitative Content Analysis in Exploring Reasons for Emergency Department Transfer Delays Laura-Maria Peltonen, Sanna Salanterä and Hans Moen 162 Health Data Privacy: Research Fronts, Hot Topics and Future Directions Javad Pool, Farhad Fatehi, Farkhondeh Hassandoust and Saeed Akhlaghpour 167 Multicriteria Decision Support Would Avoid Overdiagnosis and Overtreatment Vije Kumar Rajput, Jack Dowie and Mette Kjer Kaltoft 172 Creating Synthetic Patients to Address Interoperability Issues: A Case Study with the Management of Breast Cancer Patients Akram Redjdal, Jacques Bouaud, Gilles Guézennec, Joseph Gligorov and Brigitte Seroussi 177 xii The New Smart-Meds: Redesign of a Gamified App to Improve Medication Adherence Using a Mixed Methods Design Arnaud Ricci, Laetitia Gosetto, Katherine Blondon and Frédéric Ehrler New Scopes for Practice – Interdisciplinary Webinars for Emergency Medicine and Biomedical Informatics – Health Informatics Kaija Saranto, Catherine Chronaki, Luis Garcia-Castrillo Riesgo, Louise B. Pape-Haugaard and John Mantas Description of Data Breaches Notifications in France and Lessons Learned for the Healthcare Stakeholders Marie Simon and Vincent Looten User-Centred Design with a Remote Approach: Experiences from the Chronic Pain Project Berglind F. Smaradottir, Johan Gustav Bellika, Aina Fredeng and Asbjørn J. Fagerlund Analysis of ISO/TS 21526 Towards the Extension of a Standardized Query API Hannes Ulrich, Ann-Kristin Kock-Schoppenhauer, Cora Drenkhahn, Matthias Löbe and Josef Ingenerf 182 187 192 197 202 Effects of User Participation in the Development of Health Information Systems on Their Evaluation Within Occupational Health Services Anna Vahteristo and Virpi Jylhä 207 Typology-Based Analysis of Covid-19 Mobile Applications: Implications for Patient Empowerment Riikka Vuokko, Kaija Saranto and Sari Palojoki 212 The Master Study in Telemedicine and E-health at the University of Tromsø, Norway, 2005–2018 Rolf Wynn and Gunnar Ellingsen 217 Health Informatics Solutions in Response to COVID-19: Preliminary Insights from an International Survey Seyedeh-Samin Barakati, Maxim Topaz, Laura-Maria Peltonen, James Mitchell, Dari Alhuwail, Tracie Risling and Charlene Ronquillo 222 Continuity of Health, Citizen Empowerment as Key Driver Jacob Hofdijk and Felix Cillessen 224 Digital Allergy Card: Design and Users’ Perceptions Rhode Ghislaine Nguewo Ngassam, Linnea Ung, Roxana Ologeanu-Taddei, Pascal Demoly, Jorick Lartigau and Anca M. Chiriac 226 Physical Activity in Cardiac Rehabilitation: Towards Citizen-Centered Digital Evidence-Based Interventions Johanna Gutenberg, Stefan Tino Kulnik, Rada Hussein, Thomas Stütz, Josef Niebauer and Rik Crutzen Citizens’ Opinions About a Digital Health Insurance Record George Kostikidis, Parisis Gallos, Ioannis S. Triantafyllou and Vassilis Plagianakos 228 230 xiii Semi-Automated Method to Generate Simulated Clinical Data from OpenEHR Platform – Think!EHR Abdul Mateen Rajput 232 Standardizing the Unit of Measurements in LOINC-Coded Laboratory Tests Can Significantly Improve Semantic Interoperability Abdul Mateen Rajput, Sarah Ballout and Cora Drenkhahn 234 The Drug Addicts’ Usage of Information and Communication Technologies Christos Mimigiannis, Parisis Gallos and John Mantas 236 Subject Index 239 Author Index 243 This page intentionally left blank Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200683 1 Visualization of Guideline-Based Decision Support for the Management of Pressure Ulcers in Nursing Homes Abir ABDELLATIFa,b,c,1, Jacques BOUAUDd,a, Joël BELMINe,b and Brigitte SEROUSSIa,f a Sorbonne Université, Université Sorbonne Paris Nord, Inserm, UMR S_1142, LIMICS, Paris, France b AP-HP, Hôpital Charles-Foix, Ivry-sur-Seine, France c Teranga Software, Paris, France d AP-HP, DRCI, Paris, France e Sorbonne Université, Paris, France f AP-HP, Hôpital Tenon, Paris, France Abstract. Though a preventable risk, the management of pressure ulcers (PUs) in nursing homes is not satisfactory due to inadequate prevention and complex care plans. PUs early detection and wound assessment require to know the patient condition and risk factors and to have a good knowledge of best practices. We built a guideline-based clinical decision support system (CDSS) for the prevention, the assessment, and the management of PUs. Clinical practice guidelines have been modeled as decision trees and formalized as IF-THEN rules to be triggered by electronic health record (EHR) data. From PU assessment yielded by the CDSS, we propose a synthetic visualization of PU current and previous stages as a gauge that illustrates the different stages of PU continuous evolution. This allows to display PU current and previous stages to inform health care professionals of PU updated assessment and support their evaluation of previously delivered care efficiency. The CDSS will be integrated in NETSoins nursing homes EHR where gauges for several health problems constitute a patient dashboard. Keywords. Clinical decision support system, clinical practice guidelines, information display, nursing homes, pressure ulcer, geriatrics 1. Introduction Pressure ulcers (PUs) are injuries to the skin and underlying tissues. These ulcers are painful and significantly reduce a person's quality of life. PUs are expensive to manage and impact negatively the achievement of cost-effective and efficient care delivery. In most countries, PUs are thus considered as key clinical care quality indicators in care facilities. Yet, physically limited or bedridden elderly in NHs frequently suffer from PUs, and best practices for prevention and treatment of PUs may not be systematically applied 1 Corresponding Author, Abir Abdellatif, Hôpital Charles-Foix, 7 avenue de la République, 94200 Ivrysur-Seine, France. E-mail: abir.abdellatif@gmail.com 2 A. Abdellatif et al. / Guideline-Based Decision Support for the Management of Pressure Ulcers [1]. Incidence studies report PU figures between 6.2% and 8.8% in hospitalized geriatric patients, and even higher figures (between 11.9% and 23.2%) in NHs [2]. Clinical practice guidelines (CPGs) are narrative recommendations about the care of patients with specific conditions. They have the potential to reduce unwarranted practice variation and improve healthcare quality and safety. Guideline-based clinical decision support systems (CDSSs) that provide patient-specific recommended care protocols have shown to positively impact patient clinical outcomes, e.g., reduction of PU incidence [3], and to support nurses in improving PU assessment documentation [4]. However, few CDSS are routinely used in NHs. One reason is that NHs have been slow to adopt health information technology tools, e.g., computerized medical records. Another reason could be that currently developed CDSSs do not meet NH users’ expectations. For instance, alert fatigue and alert overrides may question the classic display of recommendations. In a previous work [5], we have proposed to implement gauges gathered in dashboards to represent guideline-based recommendations for the management of NH resident malnutrition. This paper presents the application of this method to the prevention and management of PUs. This work has been conducted within the NETSmart project that aims at developing guideline-based CDS modules to enrich the NETSoins EHR developed by Teranga Software in France2. 2. Material and Methods 2.1. Pressure ulcer risk assessment, prevention, and management We chose to implement the national CPGs developed by the French agency for healthcare quality “Haute Autorité de Santé” for the prevention and the management of pressure ulcers. We used Shiffman’s method [6] to identify both decision and action variables and translate guideline knowledge into a decision tree, finally rewritten as human-readable IF-THEN decision rules. The assessment and management of PUs requires a multidisciplinary approach. Nurses are the main actors in pressure ulcer care management. However, physicians are also involved in the therapeutic management and follow-up, especially when classic PU therapeutic protocols are not efficient. PU management differs according to the existence of a PU wound and PU risk factors. The wound is clinically established and documented by nurses. CPGs recommend to consider ten risk factors: immobilization with limited ability of reposition > 3 hours, return from intensive care < 24 hours, malnutrition, dehydration, urinary or fecal incontinence, antecedent of PU, hypotension, perfusion, consciousness disorders, nervous system diseases. When there is a wound, both the wound and PU risk factors should be assessed: the treatment includes the therapeutic management of the wound (including the management of pain, flushing, exudate, and necrotic tissues) and monitoring of risk factors. When there is no PU wound, then risk factors should be assessed and controlled, the skin should be inspected daily to identify the presence of erythema. PU management is organized in two main steps: PU risk assessment and prevention protocol, wound management and follow-up. Figure 1 displays the guideline-based decision tree of PU prevention, management, and follow up. 2 This research is partially funded by ANRT CIFRE Grant n° 2018/0307 for AA. A. Abdellatif et al. / Guideline-Based Decision Support for the Management of Pressure Ulcers 3 Figure 1. Decision tree for PU assessment, prevention, and management (Id means no modification).  PU risk assessment and prevention: this applies when there is no PU wound. Daily, the CDSS evaluate all PU risk factors from EHR data. If there is no risk factor then PU risk is computed as low. If there is at least one PU risk factor, then PU assessment scales should be completed by nurses (i.e. Norton or Braden). According to the PU scale values, PU risk is computed as high (Braden < 18 / Norton < 16) or low (Braden  18 / Norton  16). When PU risk = low, the level of the risk is displayed in the system interface, and no particular preventive action is recommended. When PU risk = high, then PU prevention protocol (including repositioning every two hours, pressure reduction, symptom assessment, hygiene, malnutrition management) is displayed along with the level of the risk. Recommendations to monitor detected PU risk factors are provided.  PU wound management and follow-up: this applies when there is at least one PU wound. In this case, nurses have to document the wound (wound stage, depth, location) and it is recommended that they take a picture of the wound to be included in the EHR. Once the stage is established, both the prevention and the stage-specific therapeutic protocols are triggered. The therapeutic protocol is provided according to the patient symptoms recorded in his/her EHR (e.g., when the patient suffers from pain, analgesics are recommended). Reassessment of PU wound and risk factors is performed on a daily basis. According to the wound evolution (healing, stable, aggravation), CPGs provide follow-up recommendations including repositioning, fluid therapy, or alerting the physician in case of aggravation despite the implementation of the therapeutic protocol and the correction of risk factors. 4 A. Abdellatif et al. / Guideline-Based Decision Support for the Management of Pressure Ulcers 2.2. CDSS visualization of alerts and recommendations As already done with malnutrition [5], we propose to display PU assessment by using a gauge (see Figure 2). When there is no PU wound, the gauge uses icons to indicate the risk of PU (low risk in green and high risk in red). When there is a PU wound, the display relies on a graphical drawing to illustrate the different stages of PU continuous evolution, from stage I, characterized by superficial reddening of the skin to stage IV where PUs are the deepest, extending into muscle, tendon, ligament, cartilage, or bone. Figure 2. Display of PU assessment with the stepwise evolution from low risk to stage IV. A pilot evaluation of gauges used as graphical user interfaces to present patient data and the display of recommendations has been made using three focus groups on a sample of care practitioners. Each focus groups lasted 30 minutes and was made of three steps: (i) presentation of the CDSS functionalities and graphical interfaces on a simulated clinical case, (ii) open discussion to let participants share their opinions on advantages and drawbacks of the presented interfaces, (iii) qualitative user assessment through questionnaires derived from the USE questionnaire3. 3. Results To illustrate the CDSS processing, we considered the case of a 74-year-old patient with moderate Alzheimer disease. At NH admission in March 2020, weight=75 kg, height=1.80 m, BMI=23 kg/m2, Alb=40 g/l, and MNA=22. Three weeks later, on April 3rd, 2020, new measures (weight=70.5 kg, Braden score=8 for a reddening of the skin located at the heel) triggered (i) a moderate malnutrition alert, and (2) a high risk PU alert, along with the recommendations to manage the two health problems. About two weeks later, on April 15th, heel pressure ulcer rapidly worsened to stage I in a context of broncho-pneumopathy associated with a depression episode, and malnutrition status became severe. On April 18th, we observed a worsening of the wound with loss of superficial skin staged as a stage II PU. The two leftmost gauges in the EHR user interface displayed in Figure 3 represent malnutrition status and PU status in their current (black cursor) and previous (grey cursor) states thus providing information about the evolution. The display of EHR interfaces as gauges and dashboards was assessed by a sample of 16 care practitioners (six geriatricians, six nurses, two care assistants, one dietician, and one psychometrician). Most of them considered this visualization modality was useful or very useful (94%), easy or very easy to use (63%), easy or very easy to learn (88%), and they were globally 88% to think they could be satisfied with it. 3 https://garyperlman.com/quest/quest.cgi?form=USE A. Abdellatif et al. / Guideline-Based Decision Support for the Management of Pressure Ulcers 5 Figure 3. Dashboard visualization included in the NETSoins EHR interface. 4. Discussion and Conclusions We are developing a CDSS to assist healthcare professionals in improving quality of care in NHs by supporting the management of common critical conditions like PUs, malnutrition, osteoporosis, and polymedication. Because of PU assessment difficulties and of the need for well-coordinated and guideline-based care plans, we proposed a DS module to evaluate patient PU risk or severity to provide management and follow-up recommendations, similar to the malnutrition DS module [5]. The extra-value of PU stage representation is to provide a graphical illustration. It is currently the user’s responsibility to match the patient’s PU to the appropriate stage illustration, and AI techniques would automatically classify PU stage from the skin picture currently taken and included within the resident EHR. Such graphical proposal has been assessed in focus groups with regular users of NH EHR systems and well accepted despite some training guidance seems necessary (only 63% found the visualization easy or very easy to use). However further work is needed to test the CDSS under real conditions. References [1] [2] [3] [4] [5] [6] White EM, Aiken LH, McHugh MD. Registered Nurse Burnout, Job Dissatisfaction, and Missed Care in Nursing Homes. J Am Geriatr Soc. 2019;67(10):2065-2071. Beeckman D, Clays E, Van Hecke A, Vanderwee K, Schoonhoven L, Verhaeghe S. A multi-faceted tailored strategy to implement an electronic clinical decision support system for pressure ulcer prevention in nursing homes: a two-armed randomized controlled trial. Int J Nurs Stud. 2013;50(4):475-486. Olsho LE, Spector WD, Williams CS, et al. Evaluation of AHRQ's on-time pressure ulcer prevention program: a facilitator-assisted clinical decision support intervention for nursing homes. Med Care. 2014;52(3):258-266. Fossum M, Ehnfors M, Svensson E, Hansen LM, Ehrenberg A. Effects of a computerized decision support system on care planning for pressure ulcers and malnutrition in nursing homes: an intervention study. Int J Med Inform. 2013;82(10):911-921. Abdellatif A, Bouaud J, Belmin J, Seroussi B. Guideline-Based Decision Support System for Nursing Homes: A Case Study with the Management of Malnutrition. Stud Health Technol Inform. 2020;272:296299. Shiffman RN, Michel G, Essaihi A, Thornquist E. Bridging the guideline implementation gap: a systematic, document-centered approach to guideline implementation. J Am Med Inform Assoc. 2004;11(5):418-426. 6 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200684 Exploring the Social Drivers of Health During a Pandemic: Leveraging Knowledge Graphs and Population Trends in COVID-19 Joao H. BETTENCOURT-SILVAa,1, Natasha MULLIGANa, Charles JOCHIMa, Nagesh YADAVb, Walter SEDLAZEKb, Vanessa LOPEZa and Martin GLEIZE a a IBM Research Europe, Dublin, Ireland b IBM Watson Health Abstract. Social determinants of health (SDoH) are the factors which lie outside of the traditional health system, such as employment or access to nutritious foods, that influence health outcomes. Some efforts have focused on identifying vulnerable populations during the COVID-19 pandemic, however, both the short- and longterm social impacts of the pandemic on individuals and populations are not well understood. This paper presents a pipeline to discover health outcomes and related social factors based on trending SDoH at population-level using Google Trends. A knowledge graph was built from a corpus of research literature (PubMed) and the social determinants that trended high at the start of the pandemic were examined. This paper reports on related social and health concepts which may be impacted by the COVID-19 outbreak and may be important to monitor as the pandemic evolves. The proposed pipeline should have wider applicability in surfacing related social or clinical characteristics of interest, outbreak surveillance, or to mine relations between social and health concepts that can, in turn, help inform and support citizencentred services. Keywords. Social determinants of health, Knowledge Graphs, Natural Language Processing, Relation Extraction, Population Trends, COVID-19 risk factors 1. Introduction The World Health Organisation (WHO) defines the Social Determinants of Health (SDoH) as the circumstances in which people grow, live and work that affect their health [1]. Examples of SDoH include socioeconomic status, education or unemployment and addressing them is important to improve health and to reduce longstanding disparities [2]. In recent years, there has been a growing number of government initiatives that tackle SDoH, including nutritional programs addressing food insecurity (i.e. availability and access to healthy foods) or transportation programs boosting access to employment [2]. However, further work is needed to measure the impact of SDoH dimensions and to identify gaps and inconsistencies from data. For example, electronic health record systems have not traditionally been designed to capture SDoH related data and healthcare terminologies such as ICD-10 or SNOMED-CT may not extensively cover social concepts [3]. The COVID-19 pandemic is magnifying disparities across the SDoH and can disproportionately affect low-income, food-insecure households that struggle to meet basic needs [4]. Furthermore, certain social environments or vulnerabilities may increase 1 Corresponding Author, JH Bettencourt, IBM Research, Dublin, Ireland; E-mail: jbettencourt@ie.ibm.com. J.H. Bettencourt-Silva et al. / Exploring the Social Drivers of Health During a Pandemic 7 the susceptibility of contracting COVID-19 as well as the risk of developing complications or poorer outcomes. For example, overcrowding and housing insecurity has been shown to lead to increased COVID-19 transmission rates [5]. Therefore, identifying SDoH dimensions that characterise vulnerable populations is of utmost importance, especially during a pandemic. Social media has been used to track trends and disseminate health information during viral epidemics. Examples include understanding sentiment during COVID-19 [6], and visualising health-related spatial social media data [7]. The latter used Twitter to study the link between healthy/unhealthy food tweets and locations with limited access to affordable and nutritious food. Google Trends has also been used to investigate symptom searches during the COVID-19 outbreak and results showed a strong correlation between the frequency of searches for smell-related symptoms information and the onset of COVID-19 infection in several countries [8]. This paper focuses on the use of natural language processing (NLP) techniques to investigate COVID-19 and its collateral impacts through the SDoH. We propose a pipeline to (1) monitor population-level trends for arising social determinants and (2) utilize a knowledge graph (KG) built from research literature (PubMed) to surface concepts related to those SDoH which may also be valuable to monitor. Previous work in relation extraction has used semi-automatic methods to discover lexico-syntactic patterns of causal relations [9] and KGs have been built from PubMed for COVID-19 [10], yet, to our knowledge, no works have been published describing how such graphs may be used to help monitoring population trends of social-related aspects during public health crises such as COVID-19. 2. Methods This section describes the steps taken to develop the pipeline and their respective components. Figure 1 shows an overview of the proposed pipeline. A typical use-case begins by monitoring population trends for a predefined set of keywords. In this paper, a well-established set of SDoH keywords was monitored using Google Trends and this is described in detail in section 2.1. Specific SDoH keywords are then identified by performing a statistical analysis of population data (e.g. keywords trending higher in a particular time period compared to historical data). Such keywords become seeded terms to be found as nodes in a knowledge graph (KG) of related concepts. Finding the nodes connected to the seeded terms by traversing the KG yields additional nodes with insights of potentially relevant concepts to be investigated further. The knowledge graph in this paper was built by first mining co-occurring concepts from the literature (section 2.2) and then extracting relations between those concepts (section 2.3) using a trained classifier. A graph database was subsequently used to store, query and visualise the mined concepts (section 2.4). Figure 1. Overview of the pipeline. 8 J.H. Bettencourt-Silva et al. / Exploring the Social Drivers of Health During a Pandemic Figure 2. Chart showing Google Trends’ interest across 10 SDoH dimensions between February-June 2020 and a summary of the average interest at the start of the pandemic (February-April 2020) compared with previous years. 2.1. Monitoring Trends with Google Analytics In order to identify relevant SDoH trends, a list of ten seeded terms (e.g. Unemployment, Social exclusion) based on the WHO’s definition of social determinants of health [1] was selected, each corresponding to a SDoH dimension. This list was then mapped to the exact (n=6) or nearest Google Trends’ Topic (n=4, e.g. early life mapped to Childhood) so that trends data could be collected for the past 5-year period (the longest period available for collection) and in English language. The collected trends data was then used to monitor the interest of SDoH dimensions from Google with particular attention to the months when the COVID-19 pandemic flared beyond China (defined in this paper as the time period between February to April 2020). Figure 2 shows the trends during this time period and also reveals the averages for the same months over the past four years for all ten SDoH dimensions. From the list of ten Google Topics, Unemployment and Food Insecurity were the two that peaked the most during the start of the pandemic and also saw their highest 5-year peaks in the same period. These two concepts were selected for the case study presented in this paper to illustrate the developed pipeline. Other methods, techniques and data sources may be used in this step for trend surveillance. 2.2. Indexing Evidence from PubMed A natural way to mine relationships between socio-medical concepts is to look for their co-occurrence in published literature [11]. We indexed the full 2019 MEDLINE PubMedBaseline1, which notably includes the abstracts of articles. We used MetaMap [12] to tokenise and identify UMLS concepts in the sentences of the abstracts and indexed each single sentence so that it could be retrieved using multiple annotation layers, like words and phrases, UMLS semantic types, or UMLS concepts. We then queried the index for any pair of a SDoH identified in Section 2.1, and another of the same SDoH or 1 https://www.nlm.nih.gov/databases/download/pubmed_medline.html J.H. Bettencourt-Silva et al. / Exploring the Social Drivers of Health During a Pandemic 9 a medical concept (listed in UMLS). UMLS is a very extensive ontology [13] and the concepts identified by MetaMap vary widely in nature, so we restricted the medical concepts to only those of the following UMLS semantic types2: Disease or Syndrome, Individual Behavior, Mental or Behavioral Dysfunction. These concepts seemed to be the most relevant to our use case, which is to identify potential socio-medical issues in the context of COVID-19. We additionally filtered out of the results the sentences containing three concepts or more, which we believed would prove too difficult to use to extract accurate pairwise relations. In total, these queries yielded 20,244 sentences. 2.3. Relation Extraction Given co-occurring concepts of interest and their sentence context, we then predicted the relation between them within sentences. Previous approaches either used co-occurrence counting [11] or syntactic rules [14] to predict a Bayesian probabilistic relation between biomedical concepts. In this work, we captured more fine-grained relations, using a supervised sentence classification model. 2.3.1. Ground Truth Annotation We sampled 550 of the context sentences and manually annotated them with 5 labels according to the statement made on the relation between the concepts in the article: positive if the concepts were found to be in positive correlation, negative for a negative correlation, complex for a more complex relation not easily classified as the first two (e.g. a relation conditioned on a specific characteristic of the population), nocor if the authors did not find a correlation, and n/a for sentences not expressing any sort of statement on the relation at all. In terms of balance, each label made up respectively 39, 3, 19, 1, 37 percents of the dataset. Four independent annotators labeled 50 sentences as the pilot, with a Fleiss’ kappa of 0.732, then pairs of annotators labelled the remaining 500 sentences. 2.3.2. Sentence Classification Experiments We fine-tuned a transformer BERT-base-uncased model [15] with a dense last layer on the train portion of our dataset and evaluated it on the test, with an 80/20 split. Most learning parameters were kept as default, batch size was set to 8 and we ran 2 epochs. Given the class imbalance found in our annotations (positive, complex and n/a make up 96% of the instances), we also performed the same experiment on a 2-class restriction of the problem, turning positive/negative/complex into positive, and nocor and n/a into n/a, to model a binary “relation”-“no relation” classification. For each setup we saw a performance accuracy of 63% (5-class) and 83% (2-class) compared with baselines for predicting the most frequent label in the train set of 41% (5-class) and 57% (2-class). We report in each case a higher accuracy for the trained classifier than the basic baseline, and also logically see a higher accuracy for the 2-class classification compared to the 5class. We then used the 2-class classifier to validate an edge between co-occurring concepts in order to ultimately build a graph. When the same pair of concepts occurs in multiple sentence contexts, we validate their relation using a majority vote of all the predictions. 2 https://www.nlm.nih.gov/research/umls/META3_current_semantic_types.html 10 J.H. Bettencourt-Silva et al. / Exploring the Social Drivers of Health During a Pandemic 2.4. Graph Database A graph database (Apache Tinkerpop stack) was used to store and query the co-occurring concepts and relations. A property graph was modeled in GraphML language and visualised using Graphexp connected to a Tinkerpop Gremlin server. A first version of the pipeline was built on a smaller subset of nodes and their related concepts (top-5 relative co-occurrences). 3. Results and Discussion In this paper, a pipeline was built to identify and extract related social and health concepts of relevance to COVID-19. The analysis of trending SDoH dimensions at the start of the pandemic identified Unemployment and Food Insecurity. Relative frequencies were computed for all concepts that co-occurred with Unemployment (n=16,314) and Food Insecurity (n=7,876). A sub-graph (Figure 3) showing the two SDoH dimension concepts and their most relevant neighbours based on relative frequency was produced. Figure 3 illustrates disease concepts associated with Unemployment such as Tuberculosis and mental health disorders. Similarly, health conditions related to Food Insecurity include Malnutrition, Diabetes and Anemia. It is also reassuring that only a small number of noisy nodes are seen in this sub-graph (e.g. Likely). Noise can be controlled by selecting semantic types and it is likely to increase as thresholds for selecting neighbours are relaxed. Most interesting are the nodes connected to both SDoH dimensions (e.g. Obesity or Depression). It can be argued that any of these concepts should be closely monitored and analysed in the time period following the start of the pandemic. For example, a simple analysis of Google Trends (Worldwide) from May to June 2020 revealed peaks for Obesity (Google Trend class: medical condition) and Coping (topic) in May 2020 and for Anxiety (emotional disorder) in June. These examples show the largest interest recorded in the past 5-years. Further work is needed to analyse this data, inspect other geographical levels, and understand the causes for the sudden rise in these concepts. However, these first results indicate that a pipeline such as the one presented in this paper may be a useful first step to extract structured knowledge that can be used, for example, to help identify upcoming trends that may affect services and populations. Figure 3. Visualisation of a sub-graph showing two SDoH dimensions (Unemployment and Food Insecurity) and their top-5 related concepts (nodes) based on selected UMLS Semantic Types. J.H. Bettencourt-Silva et al. / Exploring the Social Drivers of Health During a Pandemic 11 4. Conclusions We present a pipeline for mining relations between health and social concepts from published literature based on trending SDoH dimensions at the start of the COVID-19 pandemic. Future work will explore ways to extend our Knowledge Graph with additional social concepts, to learn better relation type labels and weights for edges, link social concepts to other ontologies and. Further work is also needed to continue analysing population data. References [1] Wilkinson RG, Marmot M, for Europe WHORO, Project WHC, for Health WIC, Society. Social Determinants of Health: The Solid Facts. Academic Search Complete. World Health Organization; 2003. [2] Artiga S, Hinton E. Beyond health care: the role of social determinants in promoting health and health equity. Health. 2018;20(10):1–13. [3] Bettencourt-Silva J, Mulligan N, et al. Discovering New Social Determinants of Health Concepts from Unstructured Data: Framework and Evaluation. Stud Health Technol Inform. 2020 Jun;270:173–177. [4] Wolfson JA, Leung CW. Food Insecurity and COVID-19: Disparities in Early Effects for US Adults. Nutrients. 2020;12(6):1648. [5] Deziel NC, Allen JG, et al. The COVID-19 pandemic: a moment for exposure science. Journal of Exposure Science & Environmental Epidemiology. 2020; p. 1–3. [6] Medford R, Saleh S, et al. An ”Info-demic”: Leveraging High-Volume Twitter Data to Understand Public Sentiment for the COVID-19 Outbreak. medRxiv; 2020. [7] Widener MJ, Li W. Using geolocated Twitter data to monitor the prevalence of healthy and unhealthy food references across the US. Applied Geography. 2014;54:189-197. [8] Walker A, Hopkins C, Surda P. Use of Google Trends to investigate loss-of-smell-related searches during the COVID-19 outbreak. Int Forum Allergy Rhinol. 2020. [9] Girju R, Moldovan DI, et al. Text mining for causal relations. In: Proceedings of the Fifteenth International Florida Artificial Intelligence Research Society Conference, May 14-16, 2002, Florida, USA; 2002. [10] Oniani D, Jiang G, Liu H, Shen F. Constructing Co-occurrence Network Embed-dings to Assist Association Extraction for COVID-19 and Other Coronavirus In-fectious Diseases. J of the Am Med Informatics Ass. 2020. [11] Theobald M, Shah N, Shrager J. Extraction of Conditional Probabilities of the Relationships Between Drugs, Diseases, and Genes from PubMed Guided by Relationships in PharmGKB. In: 2009 AMIA Summit on Translational Bioinformatics. American Medical Informatics Association. AMIA; 2009. p. 124–128. [12] Aronson A, Lang FM. An Overview of MetaMap: Historical Perspective and Recent Advances. Journal of the American Medical Informatics Association : JAMIA. 2010. 05;17:229–36. [13] Bodenreider O. The unified medical language system (UMLS): integrating biomedical terminology. Nucleic acids research. 2004;32(suppl1):D267–D270. [14] Trovati M, Hayes J, Palmieri F, Bessis N. Automated extraction of fragments of Bayesian networks from textual sources. Applied Soft Computing. 2017;60:508–519. [15] Devlin J, Chang M, et al. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the ACL, pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. 12 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200685 Integrating Patient-Generated Health Data in an Electronic Medical Record: Stakeholders’ Perspectives Katherine BLONDONa,b,1 and Frederic EHRLERa a University Hospitals of Geneva, Switzerland b University of Geneva, Swtizerland Abstract. Patient-generated health data (PGHD), when shared with the provider, provides potential as an approach to improve quality of care. Based on interviews and a focus group with stakeholders involved in PGHD integration in the electronic medical record (EMR), we explore the benefits, barriers and possible risks. We propose solutions to address liability concerns, such as clarifying patient and provider expectations for the analyses of PGHD and emphasize considerations for future steps, which include the need to screen PGHD for patient safety. Keywords. Patient-generated health data (PGHD), patient-reported outcomes, electronic medical record, data visualization 1. Introduction Patient-generated health data (PGHD) are defined as “health-related data created, recorded, or gathered by or from patients (or family members or other caregivers) to help address a health concern” by the Office of the National Coordinator for Health Information Technology (ONC) [1]. PGHD can play an important role in understanding a patient’s health when they are away from healthcare providers: it is well known that certain clinical parameters such as blood pressure measurements can vary in the presence of a healthcare provider, an effect known as the white coat effect. Adapting treatments solely based on the blood pressure measurements at a doctor’s practice may lead to overtreatment, with a risk for hypotensive or other side effects. Patients can measure their own blood pressure values at home nowadays, and then share this information with their doctor to help adapt the medication. When we consider that patients spend more than 99% of their time away from healthcare providers, the value of PGHD to help adapt healthcare management may seem obvious, yet many elements hinder the use of these data in current care [2]. There are several types of PGHD reported in prior literature [3]. One type of PGHD concerns clinical parameters such as blood pressure and glucose levels that patients collect on their own: it may be self-driven, or recommended by a care-provider. It is often quantitative, with datasets that can have occasional points or a very large number of data points (i.e. glucose monitoring). Another type of PGHD are patient-reported outcomes 1 Corresponding Author, Katherine Blondon, Medical Directorate, University Hospitals of Geneva 1205 Geneva, Switzerland; E-mail: Katherine.blondon@hcuge.ch. K. Blondon and F. Ehrler / Integrating Patient-Generated Health Data in an EMR 13 (PROs), which are often questionnaires that patients fill out, once or several times. They can be for screening or monitoring, used for patient care or for institutional reporting (ex: perceived pain assessments) often collected through the use of questionnaires that are filled out by patients. It is clear that PGHD can be a source of data for several different goals: PROs are commonly used by healthcare professionals to collected data for clinical studies, or to monitor cohorts. PGHD can allow patients to share and compare experiences among themselves, or as mentioned above, to improve the care that they receive. They can also help institutions improve the quality of care provided at a population level. With the advent of mobile and connected devices, the amount of PGHD has increased tremendously [4]. Although prior studies underline the potential of PGHD in helping improve the care patients receive, very little PGHD is integrated in electronic medical records (EMRs) [5]. Healthcare professionals’ use of PGHD is hindered by several factors. In this paper, we report our findings from encounters with the stakeholders involved in the integration of PGHD for patient care [6]. We propose a synthesis of the issues to overcome, and suggest solutions that need to be considered when integrating PGHD into EMRs. 2. Methods We identified and encountered the stakeholders for the integration of PGHD in the EMRs, including physicians, patients, computer scientists, and a legal services representative (purposive sampling). We used a semi-structured approach with two scenarios to illustrate the different types of PGHD, to help contextualize and to drive the discussion about barriers, risks and possible solutions in integrating PGHD into a patient’s care. We conducted both individual interviews and a focus group with physicians, due to the stakeholders’ limited availability during the COVID crisis. We conducted a thematic analysis of our interviews and report our findings and proposed solutions [7]. The first scenario was about the use of questionnaires, which can be one-shot or repeated to detect changes over time. These questionnaires can be a clinical score with a quantitative result, or a survey with multiple choice questions or free text responses. After discussing general principles, we proposed the example of a questionnaire including questions about suicidal thoughts: a patient fills this questionnaire in after office hours, indicating distress and high risk of suicide, and even attempts suicide during the night. We collected responses to this scenario, enquiring about initial reactions, preventive measures and legal considerations. The second scenario illustrated the use of quantitative, patient-measured parameters, such as glucose, weight or blood pressure results. These parameters typically contain a large number of data points, and can potentially require a rapid clinical response (e.g., low blood sugar result). In this scenario, we considered a patient with diabetes, who shares his glucose levels with his healthcare team. Over the course of three days, the patient presents low glucose values at the end of the day, and on the fourth, he has an accident at about that same time of day. He had an appointment with his doctor the next day. We collected responses to this scenario, discussing accountability, preventive measures and legal considerations. As we aimed to focus on patient-generated and patient-reported data, we did not include other types of data where the patient does not intervene in data sharing after 14 K. Blondon and F. Ehrler / Integrating Patient-Generated Health Data in an EMR giving her initial consent (e.g., passive data such as EKG tracings from pace-makers, are collected automatically via a medical device, and are transmitted to an app or cloud). 3. Results Apart from four physicians who took part in a focus group, we encountered the participants during individual interviews. Two patients, an informaticist, a legal representative, and two physicians (hospital and ambulatory care) were included. Two ambulatory care physicians, one hospitalist, and a hospital and ambulatory care physician took part in the focus group. The care-providers all immediately recognized the potential benefits of including PGHD in the EMR to improve patient care, as they provide additional data points to help understand health issues or to adapt treatments (e.g., glucose or blood pressure). This is particularly the case for quantifiable measures such as glucose results or blood pressure measurements. It was important for the clinicians that PGHD be easily distinguishable from clinician-collected data: dates are not sufficient to identify PGHD, as it can also be generated during hospital stays. For example, some individuals with diabetes continue checking their own blood glucose levels, even when they are hospitalized. The source of data is important because of the reliability that clinicians attribute to the results. Some blood pressure measurement devices (wrist devices, for example) that patients use may produce less reliable values than the hospital devices. Healthcare providers were also interested in sending questionnaires to the patients, either to be completed at home or even in the waiting-room before a visit. These results can serve two purposes: quality assessment and improvement at an institutional level, and quality assessment and adaptation of treatment at the patient’s individual level. For the first purpose, integration in the EMR chart is low priority, since it is the pooled results that are analyzed. For the second, integrating PGHD in the EMR is very important for patient care, and may be a key element for clinicians to adopt the use of PGHD. Our scenarios raised several questions about shared PGHDs. In both situations, clinicians expressed concerns about the accountability for uploaded data. Physicians cannot always be checking if a patient uploads data in the EMR, nor do they have time to analyze all the data, even with the support of a care-provider team. For the legal representative, all tests initiated by the healthcare provider must be followed up on, with the adequate action if needed. When these tests are conducted by the patient, the provider must provide the patient sufficient guidance to understand and react to the results if needed: responses can be to contact a healthcare provider, or selfmanagement measures (e.g., taking sugar for hypoglycemia). As soon as PGHD are shared by the patient with his provider, a clear understanding must be given to the patient about the expected action and accountability from the healthcare team. For example, a statement that answers are not seen by the provider outside of office hours would be particularly important for suicide issues. Another approach to decrease liability is to limit when the patient can upload data. In some cases, such as blood glucose results, data upload could occur prior to the next visit, and a window of 48 hours was suggested during the focus group. Although this limited upload is beneficial to preserve provider liability, it could decrease the sharing of PGHD by patients. Interestingly, our patient participants K. Blondon and F. Ehrler / Integrating Patient-Generated Health Data in an EMR 15 interpreted this message positively, because it implied that providers would be looking at the data. Several barriers can be identified when attempting to use PGHD to improve healthcare. First, patients need to be willing to share their data. Willingness to share data is affected by perceived benefits and expectations: one patient explained that when doctors do not look at a patient’s glucose values during a visit, for example, patients will rapidly lose interest in testing their glucose levels, and will report them even less. This feedback loop can play an important in a patient’s self-management. Although patients understand that some questionnaires may be needed for quality improvement at the institutional level, they expect individual responses to have an impact on their care, whenever possible. Second, there may be interoperability issues, with data mainly accessible in proprietary software: although this in itself may not be a barrier for one health professional and one type of data for one patient, if each patient uses different apps to collect data, and patients differ in their choices of apps, health professionals do not want to log in to each of these to access a patients’ PGHD. Beyond the time needed to log into the various apps, the scattering of the data in several sources makes it hard for the doctor to get an overview of a patient’s health status. Finally, even if all the PGHD were collected in a single place for a provider to review, providers raised other concerns: they would still need to navigate back and forth between the EMR and the PGHD platform, and the amounts of data may be too vast for them to review appropriately. Therefore, PGHD should ideally be visualized in the places where similar data is collected by clinicians. 4. Discussion The interviews and focus group point out some key elements in using PGHD in improving patient care. Beside the often cited interoperability and confidentiality issues, it is important to clarify the expected benefits and perceived risks for both the patient and provider users [4,6,8]. For patients, the feedback loop from the provider is a driver for continuing to self-test or to answer a questionnaire. Patients can accept a time-frame to upload data if it helps ensure that their data is analyzed. For example, this could be during the 48h prior to the next planned visit: in fact, a reminder to upload PGHD could even be included in the visit reminder text messages that are growing in popularity. Another feedback mechanism that could help motivate patients to fill out questionnaires and share PGHD would be to provide patients with visualisations of the results, both at an individual level (changes in monitored PGHD for example) as well as for the institutional level (i.e., how one individual’s perspectives compare to the general population) [9]. For providers, the potential benefits of PGHD seemed obvious. One of the main concerns however was the required effort to access PGHD: integration of PGHD into the provider’s electronic tool is essential, as it removes a barrier for clinicians to using PGHD. Ideally, PGHD should be visualized with other similar data in the EMR (i.e., glucose values together), but differentiated from the other data in the EMR. Filters can be useful for clinicians to view the data with or without PGHD, for example. A major concern about PGHD for clinicians is data overload, and subsequent accountability [10]. Besides setting a timeframe for upload, certain data may require developing analytical tools [11]: providers do not wish to be alerted to every low blood 16 K. Blondon and F. Ehrler / Integrating Patient-Generated Health Data in an EMR glucose level, but may want to rapidly see the frequency of occurrence of this type of event, as well as if there is a pattern for recurrence. In the case of our scenario, the ideal system could have detected the pattern by the third occurrence, and notified the provider. Decreasing the morning insulin dose could have prevented the accident. In terms of liability, clinicians are accountable for dealing with abnormal results, and therefore need to be explicit about when they will review the uploaded data [12]. Patient expectations need to be aligned to avoid incidents(e.g., suicidal thoughts). Therefore, integrating PGHD will need to consider how to develop tools to screen PGHD to detect both abnormal responses, as well as abnormal patterns, for both structured and unstructured data. 5. Conclusion We discussed the possibilities and challenges of using PGHD in patient care with major stakeholders through two scenarios with PGHD. Besides the importance of fully integrating PGHD in the EMR to facilitate its use for care, clinicians underlined the importance of comparing and distinguishing PGHD from other data. In terms of liability, it is essential to define PGHD upload conditions and analyses by the care-providers. Furthermore, future tools need to be developed to help screen for PGHD anomalies, including patterns of abnormality, in both structured and unstructured data, to improve the safety of patient care. The patients’ positive appreciation of sharing conditions and clarified expectations in our study needs to be re-assessed in a larger population. References [1] Shapiro M, Johnston D, Wald J, Mon D. Patient-Generated Health Data - White Paper. Prep Off Policy Planning, Off Natl Coord Heal Inf Technol Res Triangle Park NC RTI Int. 2012;(April):35. [2] Cohen DJ, Keller SR, Hayes GR, Dorr DA, Ash JS, Sittig DF. Integrating Patient-Generated Health Data Into Clinical Care Settings or Clinical Decision-Making: Lessons Learned From Project HealthDesign. JMIR Hum Factors. 2016;3(2):e26. [3] Nittas V, Lun P, Ehrler F, Puhan MA, Mütsch M. Electronic Patient-Generated Health Data to Facilitate Disease Prevention and Health Promotion: Scoping Review. J Med Internet Res. 2019;21(10). [4] Lavallee DC, Lee JR, Austin E, Bloch R, Lawrence SO, McCall D, et al. mHealth and patient generated health data: stakeholder perspectives on opportunities and barriers for transforming healthcare. mHealth. 2020;6(January):8–8. [5] Demiris G, Iribarren SJ, Sward K, Lee S, Yang R. Patient generated health data use in clinical practice: A systematic review. Nurs Outlook. 2019;67(4):311–30. [6] Genes N, Violante S, Cetrangol C, Rogers L, Schadt EE, Chan Y-FY. From smartphone to EHR: a case report on integrating patient-generated health data. npj Digit Med. 2018;1(1). [7] Wilson V. Focus Groups: ? Br Educ Res J. 1997 Apr;23(2):209–24. [8] Papautsky EL, Panganiban AR. Implications of Trust on Patient Generated Health Data. Proc Hum Factors Ergon Soc Annu Meet. 2018 Sep 27;62(1):550–550. [9] DiClemente CC, Marinilli AS, Singh M, Bellino LE. The Role of Feedback in the Process of Health Behavior Changetle. Am J Health Behav. 2001;25(3):217–27. [10] Chung AE, Basch EM. Potential and challenges of patient-generated health data for high-quality cancer care. J Oncol Pract. 2015;11(3):195–7. [11] Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. npj Digit Med. 2020;3(1):1–10. [12] Pine KH, Bossen C, Chen Y, Ellingsen G, Grisot M, Mazmanian M, et al. Data Work in Healthcare. In: Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing. New York, NY, USA: ACM; 2018. p. 433–9. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200686 17 Integrating Healthcare Data for Enhanced Citizen-Centred Care and Analytics Juliana K. F. BOWLESa,1, Juan MENDOZA-SANTANAa, Andreas F. VERMEULENa, Thais WEBBERa, and Euan BLACKLEDGEb a School of Computer Science, University of St Andrews, UK b Sopra Steria, Edinburgh, UK Abstract. The potential of healthcare systems worldwide is expanding as new medical devices and data sources are regularly presented to healthcare providers which could be used to personalise, improve and revise treatments further. However, there is presently a large gap between the data collected, the systems that store the data, and any ability to perform big data analytics to combinations of such data. This paper suggests a novel approach to integrate data from multiple sources and formats, by providing a uniform structure to the data in a healthcare data lake with multiple zones reflecting how refined the data is: from raw to curated when ready to be consumed or used for analysis. The integration further requires solutions that can be proven to be secure, such as patient-centric data sharing agreements (smart contracts) on a blockchain, and novel privacy-preserving methods for extracting metadata from data sources, originally derived from partially-structured or from completely unstructured data. Work presented here is being developed as part of an EU project with the ultimate aim to develop solutions for integrating healthcare data for enhanced citizen-centred care and analytics across Europe. Keywords. healthcare, data lake, integration, blockchain, data analytics 1. Introduction The EU project Serums2 addresses a recently exacerbated need - in the presence of a global pandemic - of improving the coordination of healthcare provision across Europe and beyond. As citizens move between countries, their newly produced medical data, including data from personal devices, must be continuously integrated to complement medical records across the countries where they have lived or where they need to be treated. This is essential to guarantee that all required information on a patient is available, and can thus be used to improve the quality of the treatment they receive. This vision requires novel mechanisms to exchange confidential medical records to personalise clinical advice and enhance treatment plans, whilst enabling trust in data security and privacy at all times. In order to be able to integrate personal medical data from multiple sources such as personal healthcare devices, primary, secondary and/or tertiary care, we need a GDPR-compliant solution, which entails a coherent and unified notion of a smart patient health record (SPHR). The integration further requires solutions that can be demonstrated to be secure, including in cases of cross-border processing. This includes 1 2 Corresponding Author, Juliana Bowles, University of St Andrews, UK; E-mail: jkfb@st-andrews.ac.uk. For more information please see www.serums-h2020.org. 18 J.K.F. Bowles et al. / Integrating Healthcare Data for Enhanced Citizen-Centred Care and Analytics patient-centric data sharing agreements (smart contracts) on a blockchain, and novel privacy-preserving methods for extracting metadata from data sources, originally derived from partially-structured or from completely unstructured data. Some aspects of our work within Serums are described next. 2. Methods A data lake is a universal data storage space which in our context is used for any healthcare data gathered from various healthcare providers and devices [6]. One advantage of a data lake is that it scales with ease, and its usage can range from simple storage, to a base from which to run analytics or big data processing, and machine learning (ML) at scale. A data lake consists of different zones (workspace, raw, structured, curated, consumer, analytic and trash) depending on the pre-processed state of the data it contains, and is responsible for carrying out data processing activities such as Retrieve, Assess, Process, Transform, Organise and Report (R·A·P·T·O·R). The R·A·P·T·O·R processing pipeline autocoder is scalable and a very efficient way of processing large amounts of data. It transforms the data according to a standard structure where data is classified into five groups: Time-Person-Object-Location-Event (T·P·O·L·E), forming what is known as a data vault model within the curated data lake zone. This model enables the standardisation of all data into an expandable hyperscalable structure that can load any kind of health or social care related data. This makes the process of combining varied data sources easier as well as the ability to gain new insights from considerably more data through data analytics and machine learning (in the respective analytic zone). To address security concerns, the Serums tool-chain [3] makes use of a blockchain to control data access through well defined rules. Rules can, for instance, limit what patient data can be seen by who and when, and encrypted logs are kept on every attempt to access patient records. Access within EU countries (Serums Use Cases [3]) is controlled under the General Data Protection Regulation (GDPR3) using smart contracts. For authenticated users, the blockchain controls the data that can be shown to the user, and the extraction is obtained from the T·P·O·L·E data lake. User-friendly interfaces are coded to enable the security and data visualisation features with language translation based on the users’ profile. The medical data is shown in its original language. Figure 1 shows a general overview of the Serums project components [1]. Patients and healthcare providers interact with the system through the front end (Serums Web Interface) which communicates with a backend (Serums API) responsible for managing the integration of all components including authentication (refer to [2]), blockchain and data lake modules. 3 Information on GDPR can be found at https://gdpr-info.eu/ J.K.F. Bowles et al. / Integrating Healthcare Data for Enhanced Citizen-Centred Care and Analytics 19 Figure 1. Serums Overview Figure 2 shows how the T·P·O·L·E data lake expands to hubs, links and satellites to enable the effective and efficient storage of the health and social care data into a globally universal data storage. The T·P·O·L·E model has the potential to resolve many challenges including the one identified in [4] on bringing together multiple sources of information on medications to provide a so-called My Medication Passport (MMP) for patients. Studies have shown that MMPs help patients understand their medications and promote adherence [5] contributing to an improved quality of life. The flexibility of the R·A·P·T·O·R processing on healthcare data lakes and the T·P·O·L·E data vault means that we can combine varied data for a single patient more easily, and we can extract knowledge through analytics which is currently not directly possible. A further benefit is that we are able to bring new data sources into the data lake at all times without conflicting with existing data, as the data within the lake is split into zones. Figure 2. The T·P·O·L·E structure within a data lake 20 J.K.F. Bowles et al. / Integrating Healthcare Data for Enhanced Citizen-Centred Care and Analytics 3. Results and Discussion The integration result is a data lake with advanced analytic capabilities that can handle the complexities of new global healthcare requirements. Data from various data sources enter the R·A·P·T·O·R processing ecosystem and is structured following the T·P·O·L·E model. Within Serums we explore three use cases provided by hospitals in the Netherlands (sensor information on patient mobility for patients that have received a hip replacement), Catalonia (device information to monitor elderly patients with diabetes and cardiovascular disease from home) and Scotland (cancer patients that report daily on their symptoms in between chemotherapy treatments) [3]. The data is stored in a Linux shared file system within the proof-of-concept. The data processing is done using custom Python code. The production grade solution will be secure to deal with personal healthcare data, since each hospital is set up with their own data lake acting as an intermediary between their source systems and the outside world, with only data that the healthcare provider allowed access to being shared with the data lake. Whether, in the future, hospitals adopt cloud based or a physical on-site system depends on both local legislation and hospitals own guidelines. Whichever solution was applied, the repositories would be similarly accessed by the Serums API relying on the authentication module, the blockchain rules, and the basic access controls. In addition, the data lake can grow and further adapt to new health and social care data that is added to it enhancing the information we may have on individual patients, on general cohorts of patients (e.g., cancer patients) and on novel treatments, further improving knowledge we can gain through ML and data analytics. Figure 3 shows the steps in which the data lake interacts with the Serums API to connect and process the data into a SPHR. Figure 3. T·P·O·L·E connections Health and social care providers, in our case three hospitals, share their data with the data lake via a Serums API gateway that was custom build for the proof-of-concept. The providers use a Web Interface (cf. Figure 1) to request the data in accordance with an underlying agreed smart contract data request from the health care blockchain. The T·P·O·L·E data factory then prepares the data and places an encrypted version in the consumer and analytics zone of the data lake ready for the SPHR API gateway to process. J.K.F. Bowles et al. / Integrating Healthcare Data for Enhanced Citizen-Centred Care and Analytics 21 The health and social care providers collect the data via the API gateway after it has been decrypted internally with the keys they receive from the blockchain. Most healthcare systems today consist of distributed heterogeneous systems that do not necessarily communicate with each other making it very challenging, if not impossible, to readily integrate data from medical practices, hospitals, medical devices, and so on, in real-time and in a straightforward manner. The approach followed in Serums with a healthcare data lake allows us to combine different data sources because they are pre-processed in the same way through the T·P·O·L·E model. The data lake concept thus removes the complexities of healthcare systems while opening novel and unprecedented capabilities to deploy any T·P·O·L·E compliant analytics and ML algorithms to process the data lake at scale. 4. Conclusion Serums comes with a methodology that can easily be expanded into a global health and social care data model to address current and future requirements to support near-realtime analytics on all citizens. Serums will supply a base model for a selected set of healthcare providers initially (cf. [3] for further details), however, it is not limited to this selection. The vision of Serums is to provide flexible structures which can be expanded to a European-wide solution for integrated medical records accessible anywhere in Europe. Acknowledgements This research is funded by the EU H2020 project SERUMS: Securing Medical Data in Smart Patient-Centric Healthcare Systems (grant code 826278). References [1] Bowles JKF, Mendoza-Santana J, Webber T. Interacting with next-generation smart patient-centric healthcare systems. In: UMAP’20 Adjunct: Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization; 2020 July: ACM; 192-193. [2] Constantinides A, Belk M, Fidas C, Pitsillides A. Design and Development of the Serums Patient-centric User Authentication System. In: UMAP’20 Adjunct: Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization; 2020 July: ACM; 201-203. [3] Janjic V, Bowles JKF, Vermeulen AF, Silvina A, Belk M, Fidas C, Pitsillides A, Kumar M, Rossbory M, Vinov M, Given-Wilson T, Legay A, Blackledge E, Arredouani R, Stylianou G, Huang W. The SERUMS tool-chain: Ensuring Security and Privacy of Medical Data in Smart Patient-Centric Healthcare Systems. In: Proceedings of the 2019 IEEE International Conference on Big Data, Big Data 2019; 2019 Dec: IEEE; 2726-2735. [4] Jubraj B, Blair M. Use of a medication passport in a disabled child seen across many care settings. BMJ Case Reports. 2015 Feb 25. [5] Leavey G, Abbott A, Watson M, Todd S, Coates V, McIlfactrick S, McCormack B, Waterhouse-Bradley B, Curran E. The evaluation of a healthcare passport to improve quality of care and communication for people living with dementia (EQuIP): A protocol paper for a qualitative, longitudinal study. BMC Health Serv Res. 2016 Aug 9;16(a):363. [6] Vermeulen AF. Practical Data Science: A Guide to Building the Technology Stack for Turning Data Lakes into Business Assets. 1st ed. Apress: 2018. 22 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200687 Implementing an Urban Public Health Observatory for (Near) Real-Time Surveillance for the COVID-19 Pandemic Whitney S. BRAKEFIELDa,f, Nariman AMMARa, Olufunto OLUSANYAa, Esra OZDENEROLb, Fridtjof THOMASc, Altha J. STEWARTd, Karen C. JOHNSONc, Robert L. DAVISa, David L. SCHWARTZc,e,g and Arash SHABAN-NEJADa,1 a University of Tennessee Health Science Center-Oak Ridge National Laboratory (UTHSC-ORNL) Center for Biomedical Informatics, Department of Pediatrics, College of Medicine, Memphis TN, USA b Department of Earth Sciences, University of Memphis, Memphis, TN, USA c Department of Preventive Medicine, UTHSC, Memphis, TN, USA d Department of Psychiatry UTHSC, Memphis TN, USA e Department of Radiation Oncology, UTHSC, Memphis, TN, USA f The Bredesen Center for Data Science, University of Tennessee, Knoxville. TN, USA g Department of Radiation Oncology, University of Texas M.D. Anderson Cancer Center, Houston, TX, USA Abstract. The COVID-19 pandemic is broadly undercutting global health and economies, while disproportionally impacting socially disadvantaged populations. An impactful pandemic surveillance solution must draw from multi-dimensional integration of social determinants of health (SDoH) to contextually inform traditional epidemiological factors. In this article, we describe an Urban Public Health Observatory (UPHO) model which we have put into action in a mid-sized U.S. metropolitan region to provide near real-time analysis and dashboarding of ongoing COVID-19 conditions. Our goal is to illuminate associations between SDoH factors and downstream pandemic health outcomes to inform specific policy decisions and public health planning. Keywords. Pandemic Surveillance, Data Integration, COVID-19, Urban Health Observatory, Precision Public Health 1. Introduction The COVID-19 pandemic is an international health crisis; it represents a leading direct and indirect cause of death in many countries. This burden falls disproportionally onto populations facing social disadvantage. Our prior work has explored associations between social determinants of health (SDoH) and several health outcomes (e.g., asthma, diabetes, and hospital readmission) [1-4]. Integrating SDoH indicators with the relevant health indicators is now an integral step for the implementation of intelligent public health surveillance solutions [5]. Health Intelligence [6] can assist researchers to explore the causal pathways between drivers (e.g., SDoH) and outcomes (e.g., COVID-19 1 Corresponding Author, Arash Shaban-Nejad, Centre for Biomedical Informatics, 492R-50 N. Dunlap Street, Memphis, TN 38103; E-mail: ashabann@uthsc.edu. W.S. Brakefield et al. / Implementing an Urban Public Health Observatory 23 positive cases, COVID-19 morbidity, and mortality) as well as correlations between the different outcomes. The process involves classifying the collected data into drivers and outcomes and studying to what extent we can identify or develop interventions to mitigate drivers that lead to the undesired outcomes. In this paper, we describe an Urban Public Health Observatory (UPHO) (Figure 1) for (near) real-time surveillance of the current pandemic. The UPHO assists public health authorities, epidemiologists, and researchers to collect data from several resources, foster the integration of surveillance data consistently across jurisdictions to estimate the incidence and prevalence of different health conditions, as well as related risk factors. Figure 1: An abstract representation of the Memphis Urban Public Health Observatory that integrates data from several sources, including individual-level COVID-19 indicators collected through a regional registry, population-level SDoH indicators, clinical data in the patients' EMRs, and patient-reported outcomes. As demonstrated in Figure 1, UPHO integrates data from several sources, including individual-level COVID-19 indicators collected through a regional registry, populationlevel SDoH indicators, clinical data in patients' EMRs, and patient-reported outcomes. The COVID-19 registry systematically collects Individual-level COVID-19 indicator variables. UPHO aligns those individual-level indicators with population-Level Social Determinants of COVID-19 and data collected by tracking observations of daily living. In this article, we provide a classification of Social Determinants of COVID-19, and explain how they have been collected, and integrated to be used for intelligent queryanswering to formally interrogate hypothesis-driven research questions. 24 W.S. Brakefield et al. / Implementing an Urban Public Health Observatory 2. Methods and Results 2.1. Study Area, Population and Study measures A Metropolitan Statistical Area (MSA) consists of Core Based Statistical Areas that have a core urbanized area with at least 50,000 people. It is comprised of the central county that contains the core urban area and all adjacent counties that are linked to that county through social or economic ties, typically measured by commuting patterns. The registry currently collects data from several regional health systems in the Memphis MSA, which includes counties in Tennessee, Mississippi, and Arkansas. Memphis MSA has a land area of about 4,985 square miles [7]. According to the U.S. Census Bureau, Memphis MSA has a population of 1,350,064 and the different counties are affected to quite different extend, see Fig. 2. 43% of the population is white, 47% is African American, 6% is Hispanic or Latino, 2% is Asian, and 2% is two or more races. 52% of the Memphis MSA population are females and 48% are males. For this study, SDoH and social distancing metrics represent the predictor variables while COVID-19 indicators serve as the outcome variables. As for the outcome variables, individual-level indicators include the date of testing, testing locations, residential address, medical insurance coverage, COVID-19 symptoms and exposure history, comorbidities, and occupation as a health worker or first responder. We discuss predictor variables in the following section. Figure 2: Counties in the Memphis Metropolitan Area and their COVID-19 confirmed cases per 100,000 population as the pandemic unfolds. 2.2. Social Determinants of COVID-19 The Center for Disease Control and Prevention (CDC) provides a taxonomy for SDoH that is comprised of 6 domains: economic stability (e.g., income), education (e.g., W.S. Brakefield et al. / Implementing an Urban Public Health Observatory 25 educational attainment and graduation rates), health and healthcare access ( access to health and how well an individual or group understands the health information to make the appropriate decisions), social and community context (variables reflect the social setting that an individual resides and their community involvement), demographics (e.g., race/ethnicity, sex, age), and neighborhood and built environment (variables that relate to the physical surrounding environment and have the potential to overlap other domains) [8, 9]. SDoH are associated with COVID-19 transmission and mortality [10-17]. SDoH overlap and interact in real-world setttings, requiring careful disentanglement of their individual downstream inpact on health outcomes such as COVID-19 spread. This project directly addresses this complexity, providing a crucial first step towards intelligent surveillance solutions to assist pandemic recovery [5]. We classify the SDoH associated with the COVID-19 pandemic into their 6 domains [8, 9]: i) SDoH that affect access to resources, ii) SDoH that increase disease exposure, susceptibility, and severity; iii) SDoH affecting adherence to policies; iv) SDoH that are community characteristics; v) SDoH that enable increasing awareness, knowledge dissemination, and health education; and vi) SDoH specific to neighborhood and built environment which can impact COVID-19 associated comorbidities 2.3. Data Collection, Analytics, and Visualization As shown in Fig. 1, UPHO collects 3 types of data, SDoH, social distancing metrics, and COVID-19 related outcomes, and aggregates them at Census Block Group (CBG) level. To obtain SDoH variables, we utilize the U.S. Census Bureau 2018 American Community Survey data [18]. We collect social distancing metrics since social distancing and shelter-in-place were among the most effective early interventions during the pandemic. For that purpose, we utilize the publicly available SafeGraph [19] movement behavior dataset, taking into account the phased interventions announced in the host MSA area starting from March 30, 2020, through a phased opening, including how often people visit specifically categorized public locations, the duration of their stay, where they come from, etc. This CBG-level data is collected anonymously from personal mobile phone use. We utilize the dataset to assess relationships among population movement behavior, transportation patterns, and COVID-19 transmission rates. UPHO enables conducting exploratory geospatial analyses of COVID-19 transmission patterns, including neighborhood-level clusters and hot spots. We then apply machine learning to test novel epidemiological models by linking COVID-19 outcomes with the SDoH and social distancing metrics at the CBG level for the identification of the geographic, sociodemographic, and disease-specific risk factors predictive of COVID-19 positivity, the spread of COVID-19 across the MSA region, and downstream clinical outcomes. UPHO integrates multidimensional SDoH and epidemiology data and makes them accessible via a public dashboard. Future directions will focus on further implementation of the dashboard that queries the observatory through its API and visualizes the results at different geographical resolutions. The dashboard can be used to answer several research questions such as i) to what degree is the disease spread associated with specific actionable or measurable determinants, ii) to what degree public adherence to pandemic mitigation policies is influenced by these determinants, and iii) how can data-driven insights directly inform policy changes to accommodate different populations and areas, especially as most cities prepare to enter re-opening phases. 26 W.S. Brakefield et al. / Implementing an Urban Public Health Observatory 3. Conclusion Our UPHO provides first-of-kind insights for immediate and long-term health policy response to COVID-19. The application of the dashboard is not limited for use to only scientific investigators, epidemiologists and healthcare professionals. Measures of SDoH from the dashboard could also be accessible to the general public in the form of neighborhood-level data as well as government officials to inform policymaking. In addition to the epidemiological surveillance of infectious diseases such as COVID-19, the UPHO may also have utility for monitoring and learning about chronic diseases e.g. cancers in the urban population. Overall, these outcomes reduce health disparities, achieve health equity, and improves urban population health. The platform provides a reproducible, durable, and scalable model for data-driven, socially-informed policymaking for recovery and future-readiness for large-scale pandemic events. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] Shin EK, Mahajan R, Akbilgic O, Shaban-Nejad A. Sociomarkers and biomarkers: Predictive modeling in identifying pediatric asthma patients at risk of hospital revisits. NPJ digital medicine. 2018;1(1):1-5. Shin EK, LeWinn K, Bush N, Tylavsky FA, Davis RL, Shaban-Nejad A. Association of maternal social relationships with cognitive development in early childhood. JAMA network open. 2019;2(1):e186963. Shin EK, Shaban-Nejad A. Urban decay and pediatric asthma prevalence in memphis, tennessee: Urban data integration for efficient population health surveillance. IEEE Access. 2018;6:46281-46289. Shin EK, Kwon Y, Shaban-Nejad A. Geo-clustered chronic affinity: Pathways from socio-economic disadvantages to health disparities. JAMIA open. 2019;2(3):317-322. Shaban-Nejad A, Lavigne M, Okhmatovskaia A, Buckeridge DL. PopHR: a knowledge-based platform to support integration, analysis, and visualization of population health data. Ann N Y Acad Sci. 2017 Jan;1387(1):44-53. Shaban-Nejad, A., Michalowski, M. & Buckeridge, D.L. Health intelligence: how artificial intelligence transforms population and personalized health. npj Digital Med . 2018;53(1):1-2. Census profile: Memphis, TN-MS-AR Metro Area [Internet]. Census Reporter. 2020 [cited 2020 Sep 1]. Available from: https://censusreporter.org/profiles/31000US32820-memphis-tn-ms-ar-metro-area Social Determinants of Health | Healthy People 2020 [Internet]. Healthypeople.gov. 2020 [cited 2020 Sep 6]. https://www.healthypeople.gov/2020/topics-objectives/topic/social-determinants-of-health About Social Determinants of Health (SDOH) [Internet]. Centers for Disease Control and Prevention. 2020 [cited 2020 Sep 6]. https://www.cdc.gov/socialdeterminants/about.html. Fielding-Miller RK, Sundaram ME, Brouwer K. Social determinants of COVID-19 mortality at the county level. medRxiv [Preprint]. 2020 Jul 1[cited 2020 July 1]. Available from: https://www.medrxiv.org/content/10.1101/2020.05.03.20089698v2 doi: 10.1101/2020.05.03.20089698. Cyrus E, Clarke R, Hadley D, Bursac Z, et al. The impact of COVID-19 on African American communities in the United States. medRxiv [Preprint]. 2020 May 19 [cited 2020 July 1]. Available from: https://www.medrxiv.org/content/10.1101/2020.05.15.20096552v1 doi: 10.1101/2020.05.15.20096552. Bai S, Jiao J, Chen Y. Mapping the Intercounty Transmission Risk of COVID-19 in New York State. SSRN [Prepint]. 2020 [cited 2020 July 1]. Available from: https://ssrn.com/abstract=3582774 Chen Y, Jiao J, Bai S, Lindquist J. Modeling the spatial factors of COVID-19 in new york city. SSRN [Prepint]. 2020 [cited 2020 July 1]. Available from: https://ssrn.com/abstract=3606719 Millett GA, Jones AT, Benkeser D et al. Assessing differential impacts of COVID-19 on black communities. Ann Epidemiol. 2020 Jul;47:37-44. Jiwani SS, Antiporta DA. Inequalities in access to water and soap matter for the COVID-19 response in sub-Saharan Africa. International Journal for Equity in Health. 2020 Dec;19:1-3. Qiu Y, Chen X, Shi W. Impacts of social and economic factors on the transmission of coronavirus disease 2019 (COVID-19) in China. Journal of Population Economics. 2020 Apr 6:1. Del Brutto OH, Mera RM, Recalde BY, Costa AF. Social Determinants of Health and Risk of SARSCoV-2 Infection in Community-Dwelling Older Adults Living in a Rural Latin American Setting. Journal of Community Health. 2020 Jul 15:1-6. About the Survey [Internet]. The United States Census Bureau. 2020 [cited 2020 Sep 10]. Available from: https://www.census.gov/programs-surveys/acs/about.html SafeGraph. [Internet]. 2020 [cited 2020 Sep 10]. Available from: https://docs.SafeGraph.com/docs/faqs Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200688 27 Dashboard Visualization of Information for Emergency Medical Services Oliver M. CHRISTENa,1, Yannic MÖSCHINGa,1, Patrik MÜLLER b, Kerstin DENECKEa, 2 and Stephan NÜSSLI a a Bern University of Applied Sciences, Biel, Switzerland b Emmental Hospital, Burgdorf, Switzerland Abstract. In emergency situations, every minute counts. Therefore, staff of emergency medical services (EMS) require easily accessible sources of information to organize and coordinate their work as quickly as possible. Digital dashboards can visualize various information at a glance and have thus potential to meet this need. We developed in cooperation with the Emmental Hospital a prototype of a dashboard, which aims to improve organizational aspects of the EMS. Method: A literature search was conducted in PubMed, IEEE and ACM. The goal was to identify design principles for dashboards. Additionally, several interviews and meetings were held with the EMS staff of the Emmental Hospital and with those of another hospital. The aim was to identify requirements of the EMS staff towards such an organizational dashboard and to transform them into use cases. Results: Considering the collected requirements and standards of dashboard design, a prototype of a dashboard was developed. It consists of several modules that show relevant information items such as news or traffic information. Due to this modular development, content is easily interchangeable. The most important information for the EMS is shown on the dashboard aiming at saving time for information gathering. Conclusion: A digital dashboard offers many advantages and optimization possibilities compared to an analog whiteboard. For example, such a dashboard can be connected to other systems and data can be automatically included. Although we developed our dashboard in cooperation with the EMS of a specific hospital, it can easily be applied and adjusted to other EMS. As a next step, we will perform usability tests with the prototype and start implementing the dashboard. Keywords. Emergency medical service, information system, visualization, dashboard 1. Introduction In medical emergency situations every minute counts; as soon as an emergency alert arrives paramedics need to move out immediately. Therefore, they are in need to easily access information in order to organize and coordinate their work as fast as possible. Dashboards show relevant information at a glance and have thus potential to meet this need. More specifically, dashboards generally display the most important data needed to achieve a certain objective [1]. In the case of a dashboard for emergency medical services 1 Contributed equally and share first authorship Corresponding Author, Kerstin Denecke, Bern University of Applied Sciences, Institute for Medical Informatics, Quellgasse 21, 2502 Biel, Switzerland; E-mail: kerstin.denecke@bfh.ch 2 28 O.M. Christen et al. / Dashboard Visualization of Information for Emergency Medical Services (EMS) the objective is to display all the relevant information that the team members need to efficiently organize themselves. Dashboards have already proven to be helpful in displaying information needed in critical situations [2] or for supporting decision making. Reis et al. [3] developed a dashboard prototype to keep track of patients arriving at the hospital emergency service. Patients were given a bracelet which monitored their vital signs and sent the acquired data to a database. The dashboard displayed the data from the database to help health professionals in keeping track of patients’ condition. Pestana et al. [4] developed dashboards to improve health care management in hospitals. They designed and evaluated dashboards to display key performance indicators of hospital departments. In this context, dashboards served as a decision support tool for the hospital management. In the EMS of our cooperation partner Emmental Hospital a dashboard is already in use implemented as a whiteboard that organizes part of their information sources. The whiteboard displays information about the team and the vehicle department, as well as absences and shift information regarding team members. Team leaders as well as team members are responsible for keeping the displayed data up to date. Aside from the mentioned whiteboard, the team members use a file share to access information about traffic messages and internal directives. This requires that team members regularly log into their computers or tablets to check for updates, which is time consuming. The goal of this work is to develop a technical tool that consolidates the available information sources, summarizes the relevant data and present it in one central display, which is the dashboard. In this way, we intend to facilitate the work of the EMS team members, so that they do not have to collect their needed information from multiple sources and thus can save time. 2. Methods A literature search was conducted between February and June 2020 in the online databases PubMed, IEEE and ACM. The purpose of the literature search was to extract principles in dashboard design and evaluate standards, which we wanted to consider in the prototype development. Further, we wanted to find out whether such or similar solutions already exist on the market and whether positive results can be achieved with them. The terms and keywords for the search were (dashboard OR tablet application) AND (design OR evaluation), dashboard AND (emergency OR ambulance OR paramedic), dashboard AND (ambulance OR emergency services OR emergency medical services OR paramedic), located either within the abstract or listed as keywords. The search resulted in the following results: IEEE (331 results), Pubmed (238 results), ACM (284 results). Based on the title, irrelevant papers were filtered out by checking the title and reading the abstract. This process returned 22 relevant paper where the full text was read. Finally, 4 papers were considered relevant for our work. Papers have been excluded when they were inconsistent with our core topics. In this way, we removed publications that did not focus on dashboards, the management of EMS or the design of dashboards. The full texts of all remaining publications were read. In order to identify the requirements, five interviews and meetings were conducted with the EMS staff of the Emmental Hospital and with those of another hospital. These interviews were conducted on a small scale with a maximum of four persons. Among O.M. Christen et al. / Dashboard Visualization of Information for Emergency Medical Services 29 them were an IT specialist, EMS employees, but also a team leader who is taking care of administrative issues within the EMS We thus gained insights into the organization of EMS and revealed the information systems and structures that are currently in place. Furthermore, ideas and suggestions for information visualization were collected. The collected requirements were evaluated, prioritized and divided into functional and non-functional requirements. All requirements were described as use cases and the use cases were formalized in a use case diagram. Based on this diagram as well as the results of the literature search on dashboard design, a prototype was developed using Axure RP 9. 3. Results In this section the collected requirements and standards for dashboard visualization are summarized and our prototype is described. 3.1. Requirements and use case descriptions Traffic messages: The EMS currently receives information about the exact traffic situation such as road closures or traffic restrictions. These messages are received as Word or PDF file. On the dashboard these messages should be visible at a glance. Short news: To receive directives and general information, currently the platform Smedex (http://smedex.com) is used by the staff of the EMS. Smedex is an e-learning platform on which mandatory courses and educational certificates for the continuous education can be managed. When new entries are made in Smedex, the users of the dashboard should be informed. This functionality requires an interface to Smedex. Allocation of staff and vehicles: The allocation of staff and vehicles is done in a different system. This information should be taken from the system and clearly displayed on the dashboard. Weekly vehicle control (functional requirement): The ambulance vehicles are thoroughly checked according to a flexible checklist. Materials are replaced and maintenance is carried out if necessary. The checklist should be displayed on the dashboard. EMS staff attendances and absences: Attendances and absences are currently recorded in a different system. The dashboard should provide an interface to this system so that staff attendances and absences are displayed directly on the dashboard. External material: Material that was stored during emergency deployments in other hospitals and is to be returned should be displayed on the dashboard. Medication expiration dates: Muscle relaxants are stored in the emergency vehicles and expire within a short time. The expiration dates should be visible and editable on the dashboard. Usability (non-functional requirement): The dashboard is intended to facilitate and simplify the coordination of the EMS. Therefore, the design of the dashboard has to ensure high usability. This comprises that all relevant information should be visible at first glance or after one click at most. Further, adapting information should be as easy as possible. Display of the dashboard (non-functional requirement): The dashboard should be displayed on mobile devices as well as on large screens. 30 O.M. Christen et al. / Dashboard Visualization of Information for Emergency Medical Services 3.2. Dashboard Design The following definition states the main purpose of dashboards: “A dashboard is a visual display of the most important information needed to achieve one or more objectives” [1]. We could not identify any fixed criteria that determine the process or appearance of dashboard designs. Nevertheless, we collected some points that should be considered in order to optimize the presentation. A dashboard should only display as much data as necessary to meet the users' objectives [5]. Users want to see the most important information at a glance. If a dashboard provides a large amount of information, important data might get lost in the crowd. If the same measurements are presented, these data should be presented in a consistent manner, e.g. using the same measurement unit. Graphs, numbers or other elements should always be displayed in the same way. Consistency is not only important in the choice of data visualization, but should also be applied to other points such as font or layout. Some key figures on a dashboard are closely interwoven. The information content is rather higher when all values can be viewed simultaneously than when the key figures have to be perceived one after the other. On a well-designed dashboard, related metrics are grouped close together [1, 6]. 3.3. Prototype The resulting prototype can be seen in figure 1. The dashboard visualizes information from the individual requirements. For example, the traffic messages are displayed in the upper left corner and the allocation of staff and vehicles are displayed to the right. Much of the layout was inherited from the existing whiteboard to keep some consistency for the users. However, since new information was added, the order of the information pieces was modified. Figure 1. Dashboard prototype The requirement “Traffic messages” was met by using a table to display where and for how long traffic is blocked due to construction. For every entry in the table a symbolic link can be opened to display the corresponding document, which contains further O.M. Christen et al. / Dashboard Visualization of Information for Emergency Medical Services 31 information. The requirement “Short news” was met by listing new entries from Smedex and internal notifications next to each other. Panels with borders visually separate the different parts of the dashboard from each other to allow for a clear layout. For every panel a small label indicates when the respective panel was last updated. These labels are supposed to help viewers, to recognize parts of the dashboard that have changed recently. 4. Discussion and Conclusions Several software solutions for EMS are available on the market. We identified two commercial solutions that met partially our collected requirements. Both products offer comprehensive solutions, but do not meet all requirements of our cooperation partner. Therefore, we decided to develop a prototype specifically tailored to the collected requirements to achieve an optimal solution. The requirement analysis has shown that a digital dashboard offers many advantages and optimization possibilities compared to an analog whiteboard. For example, the dashboard can be connected to other systems via interfaces. Thus, data can be automatically transferred to the dashboard and displayed. In this way, the actuality of the displayed information can be ensured. Machine learning technologies can be used to identify and prioritize favorites of information - for example, in case of lists, the most frequently mentioned objects would be displayed at the top. Additionally, access to an external file share is no longer required which can save time. All data can be accessed via the dashboard. Colors or other visual features can be used to draw the attention of viewers to important information. A digital dashboard can also be displayed on tablets so users always have access to it when they are on the way to an intervention. The described dashboard is based on the requirements of the Emmental Hospital EMS. By interviewing staff of another EMS, we tried to ensure that our solution is not only addressing the needs of one hospital. Additionally, we developed our dashboard in a modular way. In this way, we can ensure that it can be adopted and adjusted for the EMS of another hospital. We assume that some small individual adjustments would be necessary when deploying our dashboard to another EMS. As a next step, we will conduct a usability test with the prototype. Afterwards, the necessary improvements will be integrated, and the dashboard will be implemented. References [1] [2] [3] [4] [5] Few S. Information dashboard design: The effective visual communication of data. Beijing, London: OReilly; 2006. Nascimento BS, Vivacqua AS, Borges MRS. A flexible architecture for selection and visualization of information in emergency situations. 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 3317-3322. Reis A, Coutinho F, Ferreira J, et al. Monitoring System for Emergency Service in a Hospital Environment. 2019 IEEE 6th Portuguese Meeting on Bioengineering (ENBENG), pp. 1-4. Pestana M, Pereira R, Moro S. Improving Health Care Management in Hospitals Through a Productivity Dashboard. Journal of Medical Systems 44 (4), p. 87. Sarikaya A, Correll M, Bartram L, Tory M, Fisher D. What Do We Talk About When We Talk About Dashboards? IEEE Transactions on Visualization and Computer Graphics 2018; 25(1):682–92. [6] Effective dashboard design: a step-by-step guide | Geckoboard; 2020 [From: 10.05.2020]. Available on: https://www.geckoboard.com/best-practice/dashboard-design/. 32 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200689 Latent COVID-19 Clusters in Patients with Chronic Respiratory Conditions Wanting CUI1a, Manuel CABRERAb and Joseph FINKELSTEINa a Icahn School of Medicine at Mount Sinai, New York, NY, USA b Columbia University Irving Medical Center, NY, USA Abstract. The goal of this paper was to apply unsupervised machine learning techniques towards the discovery of latent COVID-19 clusters in patients with chronic lower respiratory diseases (CLRD). Patients who underwent testing for SARS-CoV-2 were identified from electronic medical records. The analytical dataset comprised 2,328 CLRD patients of whom 1,029 were tested COVID-19 positive. We used the factor analysis for mixed data method for preprocessing. It performed principle component analysis on numeric values and multiple correspondence analysis on categorical values which helped convert categorical data into numeric. Cluster analysis was an effective means to both distinguish subgroups of CLRD patients with COVID-19 as well as identify patient clusters which were adversely affected by the infection. Age, comorbidity index and race were important factors for cluster separations. Furthermore, diseases of the circulatory system, the nervous system and sense organs, digestive system, genitourinary system, metabolic diseases and immunity disorders were also important criteria in the resulting cluster analyses. Keywords. Chronic lower respiratory diseases, cluster analysis, COVID-19 1. Introduction Chronic lower respiratory diseases (CLRD) comprise heterogeneous chronic airway disorders that consist of multiple phenotypes with diverse clinical characteristics [1, 2]. Unsupervised machine learning has been successfully used in CLRD to identify latent clusters in such conditions as asthma [1] and chronic obstructive pulmonary disease [2]. Cluster analysis allowed to identify subgroups of COVID-19 patients with differing risk factors, comorbidities, and prognosis using electronic health records (EHR) [3]. No unsupervised learning approach has been undertaken to identify COVID-19 latent clusters in patients with CLRD. The goal of this study is to conduct cluster analysis of EHR data of CLRD patients who were tested for presence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). A broad approach to discover latent clusters is critical for a comprehensive understanding of the COVID-19 risk factors in CLRD. 1 Wanting Cui, Icahn School of Medicine at Mount Sinai, 1770 Madison Ave, 2nd Fl, New York, NY, USA, 10035, E-mail: wanting.cui@mssm.edu W. Cui et al. / Latent COVID-19 Clusters in Patients with Chronic Respiratory Conditions 33 2. Method The initial dataset was generated by querying electronic health records at Mount Sinai Health System in New York to identify all patients who underwent SARS-CoV-2 testing between January 2020 and April 2020. The initial dataset contained 19,588 patients with 8,559 tested positive and 11,029 tested negative. The analytical dataset for this study was de-identified and comprised 2,422 patients over 18 years old with CLRD based on presence of ICD-10 codes in the range of J40 – J47. We further eliminated patients with missing values, the final dataset included 2,328 CLRD patients of whom 1,029 were tested positive. Variables in the dataset included age, sex, race, ethnicity, ICU status, alive indicator and COVID-19 status. We added comorbidity index and 18 body systems based on patients’ medical history using ICD-10 codes [4]. A body system was positive if a patient has one or more diagnoses related to this system and was negative if a patient has no diagnosis of this system. In addition, the age-adjusted comorbidity index was calculated using patient’s age and ICD-10 code of diagnoses [5]. We divided our study into 2 subsets: all CLRD patients and CLRD patients who tested positive for SARS-CoV-2. For each subset of patients we performed data processing and cluster analysis. We used the factor analysis for mixed data (FAMD) method for preprocessing. It performed principle component analysis (PCA) on numeric values and multiple correspondence analysis (MCA) on categorical values which would help convert categorical data into numeric [6]. In PCA, we scaled all numeric variables between 0 and 1. In MCA, all categorical variables were converted into dummy variables. A dummy variable was a numeric variable that represents categorical data. If a variable had n levels, we expanded the one variable into (n-1) new variables and used a Boolean value to indicate this. FAMD was also good at reducing multi-collinearity issues between variables and achieving dimension reduction. It extracted features, emphasized variation and combined input variables in specific ways. It allowed us to drop the least important information, while still retaining trends and patterns. The K-means algorithm was used for clustering. Cluster analysis ranged from 2 clusters to 18 clusters, because the number of clusters needed to be determined prior to running the algorithm. We calculated the Within Cluster Sum of Squares (WCSS) which was the sum of squares of the distances of each data point in all clusters to their respective centroids. We plotted WCSS against the number of clusters and used the elbow method to determine optimal number of clusters. All analyses were performed in Anaconda Jupyter Notebook, using Python 3.7.3. 3. Results First of all, in the subset of all CLRD patients, 2,328 patients were included and 3 clusters were found (Figure 1). The number of patients distributed evenly among the 3 clusters. According to Table 1, there was a significantly greater amount of female CLRD patients than male CLRD patients in all groups. Over 99% of patients in cluster 0 tested positive for COVID-19, while almost all patients in cluster 2 tested negative for COVID-19. Cluster 1 was a mixture of COVID-19 positive and negative patients; however, patients in this group were generally younger with less comorbidities. In contrast, patients in cluster 0 were the oldest and had the highest comorbidity index. The average age of this group was 66 years old and the comorbidity index was 5.3. In addition, these patients 34 W. Cui et al. / Latent COVID-19 Clusters in Patients with Chronic Respiratory Conditions had the most severe symptoms as they had the highest death rate (23.3%), ICU admission rate (16.35%), percent use of ventilator (8.39%), hospitalization rate (60.35%) and the longest ICU length of stay (1.14 days). Figure 1. Clustering of all COPD patients. Figure 2. Clustering of COVID-19 positive patients To further analyze the effects of COVID-19 on CLRD patients, we performed clustering analysis on CLRD patients who tested positive for COVID-19. 1,029 patients were included and 2 clusters were found (Figure 2). Patients in cluster 0 had more serious conditions when infected compared to those in cluster 1. They were older (65.78 years) and had significantly more comorbidities (5.41). In addition, there were significantly more African American patients and significantly less White patients in cluster 0 than those in cluster 1. Table 1. Descriptive statistics of clusters (SD – standard deviation) Subsets Clusters Count Numeric Variables AGE Mean SD Comorbidity Index Mean SD ICU Length Mean SD Categorical Variables Status All COPD Patients 0 1 691 881 2 756 COVID19 Positive COPD Patients 0 1 583 446 65.99 15.62 52.47 19.15 60.25 16.87 65.78 15.72 58.85 18.62 5.30 3.03 2.74 2.08 5.14 3.53 5.41 3.10 3.39 2.27 1.14 3.63 0.24 1.80 0.02 0.32 1.13 3.51 0.74 3.25 Alive Deceased Sex 76.70% 23.30% 93.42% 6.58% 97.62% 2.38% 78.73% 21.27% 82.74% 17.26% Female Male Race 59.91% 40.09% 55.16% 44.84% 67.72% 32.28% 62.95% 37.05% 48.88% 51.12% 35 W. Cui et al. / Latent COVID-19 Clusters in Patients with Chronic Respiratory Conditions American Indian or Alaskan Asian Black Islander Other White Ethnicity 0.00% 4.05% 30.25% 1.45% 44.14% 20.12% 0.00% 3.29% 27.47% 2.16% 37.80% 29.28% 0.13% 3.57% 27.91% 1.19% 37.70% 29.50% 0.00% 3.60% 31.56% 1.37% 43.74% 19.73% 0.00% 4.71% 22.65% 2.02% 43.05% 27.58% Hispanic Not Hispanic On Ventilator COVID19 Positive ICU HOSPITAL 2.89% 97.11% 8.39% 99.42% 16.35% 60.35% 0.68% 99.32% 3.41% 38.37% 3.97% 34.17% 4.10% 95.90% 1.59% 0.53% 0.40% 30.03% 3.09% 96.91% 7.89% 100.00% 16.47% 61.06% 0.67% 99.33% 6.95% 100.00% 10.99% 38.57% In body systems (Table 2), over 95% of COVID-19 positive patients had endocrine, nutritional and metabolic diseases and immunity disorders. In addition, around 90% of CLRD patients with COVID-19 had diseases of the circulatory system. Furthermore, patients with diagnoses in sense organs, digestive system and genitourinary system were more likely to have serious complications when infected. Table 2. Percentage affected based on body systems All COPD Patients 0 1 92.33% 42.34% 41.39% 11.24% 2 65.87% 51.85% COVID19 Positive COPD Patients 0 1 92.97% 70.40% 46.31% 11.21% 95.22% 57.60% 55.86% 41.20% 12.94% 26.22% 91.67% 55.42% 63.89% 95.71% 62.09% 63.46% 57.85% 16.37% 17.49% 6. Diseases of the nervous system and sense organs 7. Diseases of the circulatory system 8. Diseases of the respiratory system 9. Diseases of the digestive system 10. Diseases of the genitourinary system 85.96% 89.00% 100% 75.98% 79.16% 28.83% 37.46% 100% 24.63% 29.63% 87.83% 83.73% 100% 82.94% 77.65% 90.22% 88.51% 100% 83.88% 81.99% 37.89% 56.28% 100% 23.09% 38.34% 11. Complications of pregnancy, childbirth, and the puerperium 2.60% 10.22% 8.60% 2.74% 4.04% 12. Diseases of the skin and subcutaneous tissue 13. Diseases of the musculoskeletal system 14. Congenital anomalies 56.15% 86.54% 9.99% 14.98% 32.92% 2.27% 67.20% 90.61% 8.20% 63.81% 91.77% 10.46% 13.23% 37.44% 2.91% 15. Certain conditions originating in the perinatal period 0.58% 0.34% 0.66% 0.51% 0.22% 16. Symptoms, signs, and ill-defined conditions 17. Injury and poisoning 99.57% 63.24% 83.09% 19.86% 99.21% 67.59% 99.83% 69.64% 90.36% 17.94% 18. Factors influencing health status and contact with health services Body System “None” 95.66% 22.00% 62.09% 5.33% 98.81% 23.28% 97.60% 24.53% 62.11% 6.05% Body System 1. Infectious and parasitic disease 2.Neoplasms 3. Endocrine, nutritional, and metabolic diseases and immunity disorders 4. Diseases of blood and blood-forming organs 5. Mental disorders 36 W. Cui et al. / Latent COVID-19 Clusters in Patients with Chronic Respiratory Conditions 4. Discussion Around 44% of CLRD patients tested positive for COVID-19, which was similar to the statistic of all patients (45%) at Mount Sinai Health System. In the first part of this study, we found 2 cluster of patients who were older and had higher comorbidity index. However, those patients who had COVID-19 had a 23% death rate, compared to the 2% death rate in the non COVID-19 cluster. In addition, patients with immunity disorders or diseases of the circulatory system were more likely to be subjected to the illness. The second part of the study confirmed that age and comorbidities were crucial factors. Race also emerged as an important part to differentiate seriously ill patients. Patients who developed severe symptoms had significant history of concurrent conditions of the nervous system, digestive system and genitourinary system. Cluster analysis provided initial insights of COVID-19 subgroups and risk factors in patients with CLRD. This methodology could be applied in the future towards similar studies. Our results are congruent with previous reports which used similar clustering techniques for CLRD phenotyping [7-8]. 5. Conclusion Cluster analysis was an effective means to both distinguish subgroups of CLRD patients with COVID-19 as well as identify patient clusters which were adversely affected by the infection. Age, comorbidity index and race were important factors for cluster separations. Furthermore, diseases of the circulatory system, the nervous system and sense organs, digestive system, genitourinary system, metabolic diseases and immunity disorders were also important criteria in the resulting cluster analyses. References [1] Horne E, Tibble H, Sheikh A, Tsanas A. Challenges of Clustering Multimodal Clinical Data: Review of Applications in Asthma Subtyping. JMIR Med Inform. 2020 May;8(5):e16452. [2] Pikoula M, Quint JK, Nissen F, Hemingway H, Smeeth L, Denaxas S. Identifying clinically important COPD sub-types using data-driven approaches in primary care population based electronic health records. BMC Med Inform Decis Mak. 2019 Apr;19(1):86. [3] Cui W, Robins D, Finkelstein J. Unsupervised Machine Learning for the Discovery of Latent Clusters in COVID-19 Patients Using Electronic Health Records. Stud Health Technol Inform. 2020 May;272:1-4. [4] Sundararajan V, Henderson T, Perry C, Muggivan A, Quan H, Ghali WA, New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality, J Clin Epidemiol. 2004 Dec;57(12):1288-94. [5] Ho CH, Chen YC, Chu CC, Wang JJ, Liao KM. Age-adjusted Charlson comorbidity score is associated with the risk of empyema in patients with COPD. Medicine (Baltimore). 2017 Sep;96(36),e8040. [6] Mori Y., Kuroda M., Makino N. Multiple Correspondence Analysis. In: Nonlinear Principal Component Analysis and Its Applications. Springer Briefs in Statistics. Springer, Singapore. 2016. [7] Weatherall M, Shirtcliffe P, Travers J, Beasley R. Use of cluster analysis to define COPD phenotypes. Eur Respir J. 2010 Sep;36(3):472-474. [8] Bourbeau J, Pinto LM, Benedetti A. Phenotyping of COPD: challenges and next steps. Lancet Respir Med. 2014 Mar;2(3):172-174. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200690 37 Security and Privacy when Applying FAIR Principles to Genomic Information Jaime DELGADO1 and Silvia LLORENTE Information Modeling and Processing (IMP) group – DMAG, Computer Architecture Dept. (DAC), Universitat Politècnica de Catalunya (UPC BarcelonaTECH) Abstract. Making data Findable, Accessible, Interoperable and Reusable (FAIR) is a good approach when data needs to be shared. However, security and privacy are still critical aspects. In the FAIRification process, there is a need both for deidentification of data and for license attribution. The paper analyses some of the issues related to this process when the objective is sharing genomic information. The main results are the identification of the already existing standards that could be used for this purpose and how to combine them. Nevertheless, the area is quickly evolving and more specific standards could be specified. Keywords. FAIR, FAIRification, de-identification, anonymization, license attribution, privacy, rules, genomics 1. Introduction The FAIR data principles consist on making data Findable, Accessible, Interoperable and Reusable. They were first formally introduced in [1]. When data (very often “scientific data”) is to be made publicly available, even subject to some conditions, a good approach is to achieve these principles. The process by which data is converted or adapted to be FAIR is very often called FAIRification. There are many aspects to be considered when FAIRifying data. This paper focuses in the security and privacy aspects. In addition, we also focus on a specific kind of data: health data, including genomic data. Part of this work has been done in the context of the FAIR4Health European Project [2], which provides real scenarios where to apply FAIR principles and privacy aspects. The Methods section analyses the FAIRification process and its impact in security and privacy, while section 3 on Results provides details on the available international standards dealing with the de-identification, anonymization and pseudonymization issues. On the other hand, the Discussion concentrates on the License attribution step and all the related problems that need to be solved. Finally, the Conclusions point to some more ideas on future work. 1 Corresponding Author, Jaime Delgado, Universitat Politècnica de Catalunya (UPC), Barcelona, Spain; e-mail: jaime.delgado@upc.edu 38 J. Delgado and S. Llorente / Security and Privacy when Applying FAIR Principles 2. Methods – FAIR concepts The FAIR principles need to be applied, through a process, to have health information Findable, Accessible, Interoperable and Reusable. This FAIRification process consists of a set of steps that need to be followed to prepare the data. There are several initiatives for the specification of the FAIR workflow or FAIRification process. Moreover, there are different definitions for those processes, although most of the approaches are very similar. For example, GO FAIR, an initiative that aims to implement the FAIR data principles, specifies its own FAIRification process [3]. They propose guidelines to help in making the data FAIR. On the other hand, the FAIR4Health project [2] has developed its own workflow based on the FAIRification process adopted by GO FAIR. The FAIR4Health specification is the starting point for our analysis. The different steps of the FAIR4Health’s FAIRification workflow could be summarized as: 1) Raw data analysis, 2) Data curation & validation, 3) Data deidentification / anonymization, 4) Semantic modeling, 5) Make data linkable, 6) License attribution, 7) Data versioning, 8) (Meta)data aggregation, and 9) Archiving. A first consideration of these steps from a Security and Privacy (S&P) point of view leads to the issues described in Section 3 on Results. The relevant steps are “Data deidentification / anonymization” (step 3) and “License attribution” (step 6). If we compare with the GO FAIR initiative, they also define a step on licenses (called “Assign license”), making clear that, “although license information is part of the metadata, they have incorporated the license assignment as a separate step in the FAIRification process to highlight its importance”. It is very important to take into account that in many situations having a license is the only way to access the data. The Research Data Alliance [4] is also very active in the area. Together with FORCE11 [5], they have jointly created the FAIRsharing.org registry of standards and other resources [6]. The registry collects metadata to ensure that the information is FAIR, claiming that one way to achieve accessibility (the “A” from “FAIR”) might be “by identifying their level of openness and/or license type”. Finally, in relation to the S&P aspects, GO FAIR refines the 4 principles. For example, with A1.2 (The protocol allows for an authentication and authorization where necessary) and R1.1 ((Meta)data are released with a clear and accessible data usage license). From this, the Research Data Alliance identifies the importance of the evaluation of the fulfillment of these principles, what they call the “FAIR Data Maturity Model”. In the S&P identified aspects, it means that data providers should evaluate if the access protocol supports authentication and authorization and if metadata refers to a standard license. 3. Results - Analysis of Security and Privacy aspects The first results of our work are an analysis of the S&P relevant FAIRification steps previously identified. Specifically, de-identification, pseudonymization, anonymization and license attribution. J. Delgado and S. Llorente / Security and Privacy when Applying FAIR Principles 39 3.1. De-identification, anonymization and pseudonymization Data de-identification/anonymization, step 3 of the FAIRification process, is the first step that explicitly refers to S&P. It recommends applying de-identification, anonymization or both operations to the dataset with the objective of enabling data sharing without compromising data subjects’ rights regarding privacy issues. For de-identification, the simplest approach is to drop data elements from the dataset. However, different understandings of the terminology for these concepts should be taken into account, as those from ISO/IEC 20889:2018 (Privacy enhancing data deidentification terminology and classification of techniques) [7]. In addition, ISO 25237:2017 on Pseudonymization [8] introduces several definitions to understand the relationship between the concepts of “de-identification”, “anonymization” and “pseudonymization”. In particular, anonymization is understood as the “process by which personal data is irreversibly altered in such a way that a data subject can no longer be identified directly or indirectly, either by the data controller alone or in collaboration with any other party”. However, there is a very relevant note to this definition clarifying that “the concept is absolute, and in practice, it may be difficult to obtain”. Therefore, anonymized data could be still considered as personal data if it is not possible to guarantee the absolute impossibility of re-identifying the data. On the contrary, it would no longer be personal data, so there would be no need to comply with the data protection requirements. Next, de-identification is defined as a “general term for any process of reducing the association between a set of identifying data and the data subject”, and pseudonymization as a “particular type of de-identification that both removes the association with a data subject and adds an association between a particular set of characteristics relating to the data subject and one or more pseudonyms”. A trusted third party may be able to obtain the normal personal identifier from the pseudonym. There is no specific ISO standard on anonymization. However, ISO/IEC 20889:2018 [7], introduced before, focuses on commonly used techniques for de-identification of structured datasets as well as on datasets containing information about data principals. The use of de-identification techniques is good practice to mitigate re-identification risk, but does not always guarantee the desired result. This de-identification standard [7] “establishes the notion of a formal privacy measurement model as an approach to the application of data de-identification techniques”. In any case, the application of these techniques should be considered as a privacy risk in the Privacy Impact Assessment. 3.2. License attribution License attribution, step number 6, is the second FAIRification step that refers to S&P. The objective of this step is to make clear the need for a regulatory framework for data owners to provide licensing attributions. The purpose of licenses is to support the proper reusability. Although use of Creative Commons [9] is a possible approach, other licensing options might be considered, since there might be very different needs for research datasets including health or genomic data. As the FAIR4Health project states, the license attribution for the dataset should be always clearly stated, together with the process by which an external requester could demand the permission for reusing the dataset. It should be also taken into account, as mentioned before, that the absence of an explicit license may prevent others to reuse data, even if the data is intended to be open access. 40 J. Delgado and S. Llorente / Security and Privacy when Applying FAIR Principles The previous issues raise the fact that there are additional problems to consider when licenses are in place, as developed in section 4. 4. Discussion Our discussion focuses on proposing solutions to help implementing the step 6 of the FAIRification process; i.e. “License attribution”. The related issues include: x x x x How to express the licenses. How to protect them and guarantee their provenance. How to evaluate their authorization. How to enforce what they are controlling. The proposed approach is based on the idea of access authorization using privacy rules, which describe the conditions for accessing the information, including allowed actions, analysis purposes or algorithms. It is also very important to support different levels of granularity in the allowed access to the information. A second focus on the consideration of these potential problems on license management, is the selection of a specific type of information with high privacy requirements: genomic information. There are different ways and standards to represent this kind of information. For our analysis, we start with MPEG-G [10], an ISO Standard for the representation of genomic information. We do not consider this as a limitation since MPEG-G already integrates different aspects of security and privacy, which could be used for our purposes. If we handle genomic information in different formats, we still would have very similar S&P issues. Regarding license expression (our first issue) and protection and provenance (the second one), MPEG-G, in its part 3 [10] provides an access control mechanism based on privacy rules, exactly as we are proposing. These rules are expressed in XACML [11], a general purpose language for access control rules definition. It allows a high level of granularity, which is very convenient for our case. The rules (that are in fact metadata) are included in the genomic information structure to be protected, and an authorization mechanism is also defined in the standard, based on the genomic file structure and the hierarchy of elements inside it. Privacy rules are located inside special protection elements associated to different kinds of genomic information (and also metadata) inside the file. MPEG-G defines mechanisms to ensure rules integrity, like digital signatures associated to them. Provenance can be checked from these signatures. Moreover, protection elements may contain encryption parameters for protecting both the genomic file and its metadata, also providing the required protection. Finally, authorization and enforcement mechanisms are also considered in MPEGG. [12] graphically explains how MPEG-G authorization works based on the hierarchical file structure, which can represent from several complete genomic studies to the more basic data units. Enforcement is guaranteed by the information described in the rule. Only the actions defined inside the rule over the corresponding data will be allowed by the authorization process. To sum up, MPEG-G is a suitable example of how license related issues can be solved when trying to apply FAIR principles to genomic information. J. Delgado and S. Llorente / Security and Privacy when Applying FAIR Principles 41 5. Conclusions This paper has presented the issues to consider when providing security and privacy in the process of applying FAIR principles to health and genomic information. To do so, we have firstly presented some FAIR initiatives related to health information, like GO FAIR or FAIR4Health. From FAIR4Health, we have taken the steps of the FAIRification workflow. From them, we have identified steps 3 (data de-identification / anonymization) and 6 (license attribution) to be the ones related to security and protection aspects. In section 3, we have presented the analysis of the different standards associated to de-identification, pseudonymization and anonymization. Moreover, some issues related to license attribution are also introduced. They are further developed in section 4, which describes how MPEG-G [10], an ISO standard to represent genomic information, may provide some of the mechanisms required to solve license attribution issues. Also related to genomic information, the GA4GH [13] has been working on several recommendations and tools related to security and privacy aspects. One of their produced resources is the Data Use Ontology (DUO) [14], which provides the matching between data use restrictions on genomic data and intended research use requested by researchers. We will study how DUO and other GA4GH specifications may provide some mechanisms to apply FAIR principles to genomic information. Acknowledgements This work is partly supported by the Generalitat de Catalunya (2017 SGR 1749). Part of this work is also supported by the FAIR4Health project (Grant agreement 824666, European Commission) through EFMI (European Federation for Medical Informatics). References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] Wilkinson, M. et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 3, 160018 (2016). https://doi.org/10.1038/sdata.2016.18 FAIR4Health project, https://www.fair4health.eu GO FAIR, FAIRification process, https://www.go-fair.org/fair-principles/fairification-process Research Data Alliance, https://www.rd-alliance.org FORCE11 (the Future of Research Communications and e-Scholarship, https://www.force11.org FAIRsharing.org, https://fairsharing.org ISO/IEC 20889:2018, Privacy enhancing data de-identification terminology and classification of techniques, https://www.iso.org/standard/69373.html, https://www.iso.org/obp/ui/#iso:std:isoiec:20889:ed-1:v1:en ISO 25237:2017, Health informatics — Pseudonymization, https://www.iso.org/standard/63553.html, https://www.iso.org/obp/ui/ #iso:std:iso:25237:ed-1:v1:en Creative Commons licenses, https://creativecommons.org/share-your-work/licensing-types-examples/ ISO/IEC 23092 MPEG-G, Genomic Information Representation, 2020. https://www.iso.org/standard/57795.html, https://mpeg-g.org OASIS, eXtensible Access Control Markup Language (XACML) v3.0, 2017. http://www.oasisopen.org/specs/index.php#xacmlv3.0 Naro, D., PhD Thesis (Advisors: Delgado, J. and Llorente, S.), Security strategies in genomic files, 2020. https://www.tdx.cat/handle/10803/669108 Global Alliance for Genomics and Health (GA4GH), 2018. https://www.ga4gh.org/ GA4GH, Data Use Ontology (DUO), 2019. https://www.ga4gh.org/news/data-use-ontology-approvedas-a-ga4gh-technical-standard/ 42 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200691 SLEEPexpert App – A Mobile Application to Support Insomnia Treatment for Patients with Severe Psychiatric Disorders Kerstin DENECKEa,1, Carlotta L. SCHNEIDERb, Elisabeth HERTENSTEINb and Christoph NISSENb a Bern University of Applied Sciences, Biel, Switzerland b University Hospital of Psychiatry and Psychotherapy, University of Bern, Switzerland Abstract. Cognitive behavior therapy for insomnia (CBT-I) is the first-line treatment for patients with insomnia disorder, including patients with severe mental disorders and comorbid insomnia. However, CBT-I is not sufficiently implemented in acute psychiatry settings. To make this treatment more accessible, we are currently adapting CBT-I to the needs of patients with severe psychiatric disorders in the form of a treatment program entitled SLEEPexpert. A core element of SLEEPexpert is keeping a sleep diary and restricting time in bed to increase sleep pressure. Here, we present a mobile application which supports the implementation of SLEEPexpert. The app is kept very simple, specifically designed for the target user group, and offers four main functionalities: entering information into the sleep diary, calculating the sleep efficiency and adapting the sleep window, delivering information on sleep and sleep disorders and accessing the recorded data in the sleep diary. Currently, we are preparing a usability test for the app aiming at fixing usability issues before running a clinical trial to assess the efficacy of this mHealth intervention. Keywords. mHealth, sleep disorders, insomnia, behavioral therapy 1. Introduction Mental disorders are highly prevalent with a lifetime-prevalence of about one fourth of the population and lead to significantly reduced quality of life worldwide. Insomnia, i.e. persistent difficulties falling and staying asleep, is very common in patients with mental disorders [1]. Insomnia is diagnosed based on disturbed sleep continuity and associated daytime impairment such as tiredness or reduced concentration reported by the patient. According to current guidelines, the first-line treatment is Cognitive Behavioral Therapy for Insomnia (CBT-I) [2]. Many patients with severe mental disorders suffer from cognitive impairment, reduced motivation and disorganized behavior, which often complicates the implementation of CBT-I. For this reason, we recently developed a pragmatic behavioral treatment program based on CBT-I (“Become your own SLEEPexpert”) [3]. This program focuses on bedtime restriction as the most effective component of CBT-I [4]. SLEEPexpert uses a simplified sleep diary that can 1 Corresponding Author, Kerstin Denecke, Bern University of Applied Sciences, Institute for Medical Informatics, Quellgasse 21, 2501 Biel / Switzerland; E-mail: kerstin.denecke@bfh.ch. K. Denecke et al. / SLEEPexpert App 43 offer valuable information about the individuals’ sleep-wake pattern [5]. Sleep diaries have been established as a gold standard for subjective sleep assessment [6]. Although formats of those sleep diaries vary, they collect bed and sleep times and information on sleep satisfaction. The sleep diary forms the basis for bedtime restriction therapy [7] by providing the therapist with the information needed to restrict the bedtime to the actual total sleep time (TST) [8]. The diary should be continued throughout therapy for adapting the bedtime restriction when applicable. It serves as a self-monitoring tool, and may help the patients with insomnia to provide a more accurate picture of night-to-night sleep variation compared to a retrospective questionnaire. It may even offer relief to the individual by showing variation in sleep satisfaction [9], opposing to the common (mis)perception of persistent poor sleep quality in patients with insomnia. In addition, the sleep diary can be used to verify or falsify the common belief that sleep and wake quality are absolutely dependent, i.e. "bad" nights are always followed by a "bad" day. The treatment program SLEEPexpert was developed to suit the needs of patients with severe psychiatric disorders. It combines an app-assisted behavioral intervention with face-to-face support and consists of three phases (therapist-guided treatment initiation, self-management with nurse support, and self-management). The mobile application aims at making this reduced CBT-I more accessible. In contrast to existing CBT-I apps, the app is tailored to the specific user group. As part of the treatment concept, it will enable patients with mental disorders to become their own sleep experts, i.e. be able to manage their sleep problems. In this paper, we describe the development of the app and its functionalities. 2. Methods The application was developed in two phases in close collaboration with psychologists, medical doctors, nurses and patients. They accompanied the entire development process and provided feedback on mockup and prototype. In a first phase, requirements were collected by interviewing 3 experts. Based on these requirements a mockup was generated with Axure RP. The mockup was tested with patients hospitalized in a psychiatric ward to collect their feedback. This usability test with the mockup aimed at determining usability issues before the actual programming started and at collecting feedback on the interaction with the app, design issues and functionalities. The usability test was task-based, i.e. the participants had to solve a given task with the app and were asked to answer a questionnaire afterwards. The four tasks included: try to get information on sleep in general, get an overview on individual sleep behavior of the last days, have a look at the exercises, create a sleep diary entry. The questionnaire comprised the following five statements for which one option from a 5-item Likert scale (1=totally disagree, 5 = totally agree) had to be selected after solving a task: x x x x x I could quickly solve the task. I was able to find the function quickly. Accessing the function is well designed. The functionality’s result is as expected. The functionality is convenient. Additionally, the participants were asked to suggest additional functionalities. The results from the usability test were considered for developing the final design and 44 K. Denecke et al. / SLEEPexpert App selecting the functionalities of the app. During the second phase, a prototype was implemented with the programming language Kotlin (https://kotlinlang.org). Data collected by the app is stored in a Google Firebase. 3. Results 3.1.Requirements Our aim is to provide a digital application reflecting the pragmatic behavioral treatment program (“Become your own SLEEPexpert”) customized to the needs of patients with acute psychiatric disorders and insomnia within a psychiatry setting. This means the application should be very simple with respect to data entry, but also regarding the suggestions for changing sleep behavior. The application should be attractive in using, e.g. by providing an intuitive visualization of the personal progress. It should motivate the patients to increase the number of entries in the sleep diary. The app has to support the collection of data on sleep behavior as it is relevant to support the treatment. The app has to be integrated in the three-step approach of the treatment program, i.e. it should be possible to use the app during the stay in the clinic, but also to continue using it afterwards in the personal environment to ensure a long-term improvement of the individual sleep behavior. Thus, it should encourage patients to maintain the newly acquired habits and should enable them to deal with their sleep problems in the long term. The app has to coach and educate the patient with respect the sleep disorders and has to provide means to improve the sleep behavior. 3.2.Usability test results Based on these requirements, a mockup was created that allowed to keep a sleep diary and to access educational content. The usability test with the mockup was conducted with four patients. All patients were hospitalized in a psychiatric clinic at the time of the test. They were able to solve all given tasks in a short amount of time. For all tasks and questions, the value 4 (rather agree) was selected by all four participants. As additional functionalities, the participants suggested to include an alarm clock, music for falling asleep, the possibility to add comments in the diary, integrate with a mobile sensor such as Fitbit, and enabling voice input for the diary. Out of these suggestions, we included the alarm clock and the commenting option for the diary in the final version of the app. Since Android mobile phones provide the possibility to dictate, we resisted on implementing a voice user interface. The other suggestions were not integrated to avoid an overload with functionalities and limit complexity of user interfaces. It is to note that the target user group are patients with acute psychiatric disorders with varying health literacy, mental capacity and educational background. 3.3.SLEEPexpert App The implemented prototype of the SLEEPexpert app provides the following functionalities: 1) Keeping a sleep diary, 2) Providing exercises and information on sleep, 3) Showing the progress on improving sleep behavior, 4) Alarm clock (see Figure 1). A surfer was used as a metaphor to illustrate sleep pressure and circadian variation. K. Denecke et al. / SLEEPexpert App 45 Like a surfer who has to wait for a big wave to build up, patients have to wait for sleep pressure to build up (e.g., should not go to bed too early). In addition, the correct time is critical for both surfing and sleeping (whether conditions for surfing, circadian type for sleep. Figure 1. SLEEPexpert app: The home screen provides access to the four functionalities: diary, information, progress bar, alarm clock. The diary entry page (Tagebuch) asks for a judgement of the sleep quality on a scale of 1 to 8, for entering the time a person went to bed, got up and an estimation of the time a person slept The sleep diary enables data entries when a person went to bed, when he or she got up in the morning, how long a person slept and asks for a judgement of the sleep quality. During the onboarding process, the user is asked to set an initial sleep window. There are two options: people who go to bed early and wake up early in the morning and those who go to sleep late at night and get up later in the morning. This initially selected sleep window is adapted automatically when a sufficient number of diary entries has been made. Sleep efficiency (percentage of time in bed that is actually spent asleep) is calculated by considering the last three entries in the sleep diary considering sleep time and time spent in bed. The value is only updated when data from 3 consecutive days is available. A progress bar shows the number of hours a person slept per night over time. The values are taken from the diary entries. When started for the first time, the user is informed on the functionalities of the SLEEPexpert app by a guided tutorial. The starting screen gives access to the main functionalities through four buttons. The number of functionalities is kept at a minimum to address the specific needs or backgrounds of the target user group. In this way, the interaction with the app can be kept as simple as possible. In its current implementation, the app is only running on Android. To store the data in the Google Firebase, an internet connection is required. 4. Discussion and Future Work While several apps that deliver CBT-I are already available in English (e.g. Sleepio, CBT-I Coach), there is still no app available that targets at supporting users with acute psychiatric disorders in a psychiatric setting. Lyla Sleep coach and CBT-I Coach allow to keep a sleep diary and provide access to relaxation exercises. Lyla offers a six week 46 K. Denecke et al. / SLEEPexpert App program for better sleep and is available in Dutch. Sleepio offers a virtual character that serves as a coach who delivers weekly therapy sessions. Additionally, a sleep diary can be kept. In contrast to those apps, the SLEEPexpert app is characterized by its simplicity; it limits the requested data entries and interactions with the app. Furthermore, it is integrated in an entire treatment program that starts in the clinic with therapist-guided treatment initiation, followed by self-management with nurse support, and continues with self-management at home. SLEEPexpert focuses on the patients becoming “their own SLEEPexpert”. This strategy is specifically important for patients with severe psychiatric disorders – the objective is to bring them into the position in taking over the responsibility for their own sleep behavior with support of the app. This might be of particular relevance in providing an alternative treatment approach to the frequent overprescription and over-use of hypnotics. A usability test with the app still has to be conducted for ensuring easy handling. Furthermore, data security and privacy has to be considered. Currently, the data is stored in a Google Firebase. Only a nick name and a password is used for patient identification. For a use in practice, the data could be stored in a health bank like MIDATA (https://www.midata.coop/) using FHIR. Such health bank ensures data privacy and security. This would also enable patients to give researchers their consent for using the data for studying the efficacy of the SLEEPexpert treatment concept in psychiatric wards. A web platform would be useful for therapists to access the data collected by their patients and as a basis to discuss follow-up treatment with the patient. We already started to design extensions of the application. A quiz will allow users to test and train their knowledge related to sleep, healthy behaviour and insomnia. 5. Acknowledgements We thank Andi Ademi, Luca Leuenberger, Andy Landolt, Janahan Sellathurai and Sugeelan Selvasingham for implementing the app as part of their semester project. References [1] Bei B, Asarnow LD, Krystal A, Edinger JD, Buysse DJ, Manber R. Treating insomnia in depression: Insomnia related factors predict long-term depression trajectories. J Consult Clin Psychol. 2018;86(3):282–293. doi:10.1037/ccp0000282 [2] Riemann D, Baglioni C, Bassetti C, et al. European guideline for the diagnosis and treatment of insomnia. J Sleep Res. 2017;26(6):675-700. doi:10.1111/jsr.12594 [3] Schneider C, Hertenstein E, Fehér K, et al. Become your own SLEEPexpert: design, implementation and first evaluation of a pragmatic behavioral treatment program for insomnia in patients with acute psychiatric disorders. Sleep Advances. 2020. Major Revision. [4] Miller CB, Espie CA, Epstein DR, et al. The evidence base of sleep restriction therapy for treating insomnia disorder. Sleep Med Rev. 2014;18(5):415-424. doi:10.1016/j.smrv.2014.01.006 [5] Riemann D, Baglioni C, Bassetti C, et al. European guideline for the diagnosis and treatment of insomnia. J Sleep Res. 2017;26(6):675-700. doi:10.1111/jsr.12594 [6] Carney CE, Buysse DJ, Ancoli-Israel S, Edinger JD, Krystal AD, Lichstein KL, Morin CM. The consensus sleep diary: standardizing prospective sleep self-monitoring. SLEEP 2012;35(2):287–302. [7] Troxel, WM, Germain A, Buysse DJ. Clinical management of insomnia with brief behavioral treatment (BBTI). Behav Sleep Med. 2012,;10: 266–279. doi: 10.1080/15402002.2011.607200 [8] Hertenstein E, Nissen C, Riemann D. Pharmacological and Non-Pharmacological Treatments of Insomnia. Vol 1. Oxford University Press; 2017. [9] Hertenstein E, Spiegelhalder K, Johann A, Riemann D. Prävention und Psychotherapie der Insomnie in Störungsspezifische Psychotherapie. Verlag W. Kohlhammer; 2015: 30-31. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200692 47 Measures of Decision Aid Quality Are Preference-Sensitive and InterestConflicted - 1: Normative Measures Jack DOWIEab1, Mette Kjer KALTOFTb and Vije Kumar RAJPUTc a London School of Hygiene and Tropical Medicine b University of Southern Denmark c Stonydelph Health Centre, Tamworth, UK Abstract. The belief that following rigorous inclusive methods will eliminate bias from ‘quality’ measures ignores the preferences necessarily embedded in any formative instrument. These preferences almost always reflect the interests of its developers when one uses the wide definition of ‘interest’ appropriate in healthcare research and provision. We focus on the International Patient Decision Aid Standards instrument, a popular normative measure of decision aid quality. Drawing on its application to a set of 23 breast cancer screening decision aids, we show the effects of modifications that reflect our own different interest-conflicted preferences. It is emphasised that the only objection is to the implication that any formative instrument should be promoted or treated as the ‘the gold standard’, without a conflict of interests disclaimer, and to the implication that other instruments cannot provide equally valid, high-quality measures. Keywords. Decision aid, preferences, normative, IPDASi, conflict of interest 1. Introduction There is growing agreement that the future will be dominated by the social and selfproduction of health by citizens, optionally supplemented by its co-creation with healthcare professionals. This means that the provision of apomediative decision support direct to the person in the community is of increasing relevance and importance, in addition to intermediative decision support direct from the clinician to the citizen-as-patient. For clarity we will refer to the former as apomediative Person Decision Support Tools (PnDSTs) and the latter as intermediative Patient Decision Aids (PtDAs). The terms ‘intermediative’ and ‘apomediative’ are those of Eysenbach [1] and characterise, respectively, the presence or absence of a dependent relationship between the supplier of the support and the professional involved in the decision. Despite this distinction the quality of both types must be assessed appropriately, and this includes addressing possible bias. Unfortunately, the belief that by following rigorous ‘scientific’ methods, all biases can be eliminated, involves confusing whether they are seen as acceptable, justifiable, or desirable, with their inevitability. This 1 Corresponding author, Jack Dowie, LSHTM, 15-17 Tavistock Place, London, UK WC1H 9SH; email: jack.dowie@lshtm.ac.uk 48 J. Dowie et al. / Measures of Decision Aid Quality – 1: Normative Measures inevitability is at its clearest in indexes developed to measure the quality of some state (e.g. of health), of some action (e.g. of surgery), of some process (e.g. shared decision making), or of some intervention (e.g. a decision aid or support tool). All quality indexes are formative measures and therefore dependent on the preferences - the value judgments - involved in (a) the selection of the component scales (instrument items) and (b) the weighting of those selected scales to form a quality index. Evaluations using such quality measures are therefore ‘preference-sensitive’ [2] and, given preferences are rarely in conflict with interests, they are usually biased in the sense of ‘interest-conflicted’. The definition of ‘interest’ used here is the wide one we regard as relevant in healthcare, including research and provision, not a narrow legal/financial one that ignores the serious conflicts arising for a variety of institutional and personal reasons [3]. Among these interests are the ones listed in the first section of Rodwin’s typology: Intellectual commitments (e.g. working within a theoretical framework, school of thought, or having proposed a hypothesis). Interest in a positive outcome to a study that will support your previous findings. Interest in maintaining professional reputation. Interest in career advancement. Interest in finding potential practical applications of research. Interest in maintaining good relations with entities that can provide future research funding [4]. With this wider definition of interest - particularly the inclusion of commitments to theoretical frameworks or schools of thought - the first issue is not whether a quality measure is ‘interest-conflicted’, but the nature and origins of the interest-conflicted preferences that are necessarily reflected in its development. Whether or not these interests are regarded by some (many) as acceptable or desirable, the second, but equally important issue, is the attitude and behaviour of those responsible towards alternative measures, which, by definition, also reflect interest-conflicted preferences. Quality being a formative construct, any alternative is a measure of a different construct (e.g. of decision aid quality), not an alternative measure of the same one. The wide range of interests underlying the preferences reflected in a formative measure should therefore be declared alongside any legal requirement, accompanied by a denial of any intention to seek to establish the measure as ‘the gold standard’, with an effective monopoly on professional endorsement and regulatory approval. An example of good behaviour is provided by the generic health-related quality of life measures, where alternatives co-exist in friendly, if robust, rivalry. In this first of two papers we focus on the normative measurement of the quality of decision aids, in other words on measures which consider only the content of the aid and/or its development process. In the companion paper [5], we see how the argument applies in the empirical measurement of decision aid quality, as implemented. In both we focus on the products of the International Patient Decision Aid Standards (IPDAS) consortium, but it is important to see that the argument is completely generic. We take the International Patient Decision Aid Standards instrument (IPDASi) as our specific focus [6]. It should already have been inferred that we will not be ‘criticising’ or even ‘critiquing’ IPDASi from some purportedly neutral, unbiased (interest-unconflicted) position. We will be pointing out the way the preferences and interests it reflects do not coincide with our own, and arguing why it should not be promoted and/or treated as the ‘gold standard’, as opposed to a standard based on a widely-agreed, but particular, set of preferences and interests. The setting of an IPDASi standard for decision aid ‘certification’ [7] is not in itself objectionable, only any implication or inference that aids that do not meet this certification standard should not J. Dowie et al. / Measures of Decision Aid Quality – 1: Normative Measures 49 be regarded as usable for this reason alone. To repeat, this is a widespread phenomenon and the same danger can be detected with other quality measures [8,9]. To give the argument empirical flesh we draw on the recent paper by Hild and colleagues [10]. They assessed the quality of 23 decision aids for women at average risk of breast cancer (and eligible for mammographic screening) using the original IPDAS 47 item instrument. We take advantage of the complete data set they provided to see the effects of modifying the instrument to reflect our different preferences. 2. Modifying IPDASi Our preferences are fully in agreement with IPDASi in distinguishing conceptually between binary checklist items and scalar index items. The former must be met. The latter may be only partially fulfilled and, most importantly, are compensable - lower ratings on some may be countered by higher ratings on others. IPDASi recognise this distinction in partitioning their later 44 item version into three parts [7] with a certification sub-set. Six binary qualifying (checklist) items are ones to be met for something to qualify as a decision aid: “1) the intervention should relate to a specific decision that has to be made; 2) patients should be helped to choose deliberately among options; 3) positive and negative features of the options should be presented; 4) outcomes given should be relevant to health status; 5) the intervention should not promote compliance with a recommended option; and 6) the intervention should help patients to clarify values.” Ten certification criteria, scored on a 1–4 scale (‘strongly disagree’ to ‘strongly agree’), are “deemed essential in order to avoid risk of harmful bias… Decision aids must meet all of these criteria to be certified. The 6 certification criteria selected relate to the quality of the evidence synthesis process, open disclosure of funding source, and a balanced presentation of options, with 4 additional items for screening/test aids.” Twenty-eight quality items are deemed “desirable because they would enhance a decision aid but are not essential for reducing risk of harmful bias… These items would improve the experience of using the decision aid, but absence of the item would not be expected to influence the individual’s decision in a negative way.” Tools should meet all qualifying criteria and score 3 or 4 on each of the 10 certification criteria in order to reach the certification standard. (Hild, et al. do not cite the 2013 paper in which this item classification and certification standard is introduced and hence make no reference to certification in their analysis.) Our preferences would reduce the IPDASi list from 44 to just 5 items. We retain the IPDASi wording here, as sufficient for the present purpose. 1. The aid makes it possible to compare the positive and negative features of the available options. 2. The aid provides information about outcome probabilities associated with the options (i.e. the likely consequences of decisions) 3. The aid asks patients to think about which positive and negative features of the options matter most to them (implicitly or explicitly). 4. The aid (or associated documentation) describes how research evidence was selected or synthesized 5. The aid was field tested with patients who were facing the decision All 5 of these fall only in the residual ‘quality’ category for IPDASi, in other words none are required for certification. It follows we must regard their 10 certification items 50 J. Dowie et al. / Measures of Decision Aid Quality – 1: Normative Measures as either merely ones necessary for acceptance as an aid (we indeed regard 5 of them as ‘qualifiers’), or as redundant (the other 5 certifying items, including all the screening test ones, are indeed covered in item 2 above if this is fulfilled properly. Spelling them out as separate items either implies that item 2 is not required to be fulfilled properly, or involves double counting. (There is a major omission in the IPDASi test set in our view. The prior odds of the target condition are essential in decision support, prevalence being the usual proxy. Providing only the True and False Positive and Negative rates, as IPDASi requires, is likely to be misleading if the user cannot also see the False Alarm and False Reassurance Rates. These appear in item 2 if done properly.) The correlation between the scores on our 5 item selection and the Hild 47 items scores is 0.89, confirming the feasibility of serious reduction. But the correlation with the certification items is only 0.66, confirming that preferences make a major difference. Applying the certification standard (rated 3 or 4), to our 5 items, only 3 of the 23 aids are certifiable. In contrast to IPDASi our preferences would certify the Schonberg aid, which fails IPDASi by not providing an update policy and not indicating the next steps if the target condition is not detected. (The latter is a redundant item for us.) Contrariwise, we would not certify the Keevil aid, because it is not describing well enough ‘how research evidence was selected or synthesized’. (The analysis is at http://bit.ly/hildanalysis, but the argument does not depend on its details.) 3. Interest-Conflicted Preferences Where and how do interests come in? First, in the inclusion of process criteria regarding the development process, criteria made redundant for us by rigorous testing using outcome criteria. If this testing is conducted properly, we see no justification for using any aspect of the development process (including the credentials of the people involved in it) in establishing the quality of an aid. Second, through the omission of items that preserve the interests of healthcare professionals in not having to deal with a preliminary opinion based on numerical analytic calculation, as opposed to one designed to fit a verbal deliberative reasoning process. Items 2 and 3, when fulfilled according to our preferences, provide all the ingredients necessary to calculate the expected value of each option, using the importance weights of the person. Calculating and displaying those scores is absent from IPDASi, whereas it is the central feature of aids based on other techniques. Third, through preference specification. Our preference is for decision support tools that are preference-sensitive at the point of care. These cannot be based on group average tariffs, let alone be those of a panel whose expertise and eminence relate to belief judgments about option performance rates, not value judgements about those criteria. In the online spreadsheet we enable the preference-sensitive weighting of the 47 items IPDASi set. To illustrate, we assign weight to only our 5 preferred items. The reader is free to explore alternative weights. 4. Conclusion That IPDASi was built by a large international consortium in a prolonged and rigorous Delphi process gives it unquestioned credibility and the right to have aids promote themselves as being ‘certified by the IPDASi standard’. But implying that it is a ‘gold J. Dowie et al. / Measures of Decision Aid Quality – 1: Normative Measures 51 standard’ and that an aid that fails to meet it cannot, by that fact, be a valid, possibly excellent, decision aid, conflicts with the scientific standards its developers - and the informatics community - would undoubtedly wish to uphold. The same applies to all alternative measures of decision support quality and indeed to all quality metrics. References [1] Eysenbach G. Medicine 2.0: Social Networking, Collaboration, Participation, Apomediation, and Openness. J Med Internet Res. 2008 10(3):e22. [2] Kaltoft MK, Nielsen JB, Dowie J. Formative preference-sensitive measures are needed in person-centred healthcare at both clinical and policy levels. Euro J Pers Cent Healthcare. 2017 5(4):495-500. [3] Saver RS. Is it really all about the money? Reconsidering non-financial interests in medical research, J Law Med Ethics. 2012 40(3):467–81. [4] Wiersma M, Kerridge I, Lipworth W, Rodwin M. Should we try to manage non-financial interests? BMJ. 2018:361:10 [5] Kaltoft MK, Dowie J, Rajput VK. Measures of decision aid quality are preference-sensitive and interestconflicted 1: normative measures. Stud Health Technol Inform. 2020: submitted [6] Elwyn G, O’Connor AM, Bennett C, Newcombe RG, Politi M, Durand M-A, et al. Assessing the quality of decision support technologies using the International Patient Decision Aid Standards Instrument (IPDASi). PLoS ONE. 2009 4(3):e4705. [7] Joseph-Williams N, Newcombe R, Politi M, Durand M-A, Sivell S, Stacey D, et al. Toward minimum standards for certifying Patient Decision Aids: A modified Delphi consensus process. Med Decis Making. 2013 34(6):699–710. [8] Dowie J, Kaltoft MK. Why a Global PROMIS can’t be kept, Stud Health Technol Inform. 2019 262: 114–17. [9] Kaltoft MK, Dowie J. The evaluation of decision support tools needs to be preference context-sensitive, Stud Health Technol Inform. 2019 265:163-68 [10] Hild S, Johanet M, Valenza A, Thabaud M, Laforest F, Ferrat E, Rat C. Quality of decision aids developed for women at average risk of breast cancer eligible for mammographic screening : Systematic Review and assessment according to IPDASi. Cancer. 2020 126(12):2765-74. 52 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200693 Measures of Decision Aid Quality Are Preference-Sensitive and InterestConflicted - 2: Empirical Measures Jack DOWIEab1, Mette Kjer KALTOFTb and Vije Kumar RAJPUTc a London School of Hygiene and Tropical Medicine b University of Southern Denmark c Stonydelph Health Centre, Tamworth, UK Abstract. Empirical measures of ‘decision aid quality’, like normative ones, are of a formative construct and therefore embody interest-conflicted preferences in their criteria selection and weighting. The preferences of the International Patient Decision Aid Standards consortium distinguish the quality of the decision-making process and the quality of the choice that is made ‘(i.e., decision quality)’. The Decision Conflict Scale features heavily in their profile measure of the former and Decision Quality Instruments (DQIs), have been developed by members of the consortium to measure the latter. We confirm that both of these, and other components, like the higher-level measures, are preference-sensitive and interestconflicted. Non-financial interest-conflicted preferences are endemic in healthcare research, policy-making, and practice. That they are inevitable means the main problem lies in the denial of this and attitude to and behaviour towards alternatives, equally interest-conflicted. Keywords: Decision aid, empirical evaluation, IPDAS, Decision Conflict Scale, Decision Quality Instrument, conflict of interest 1. Introduction In the companion paper [1] it was established that quality is a formative construct and measures of it are therefore preference-sensitive. They are sensitive to the preferences involved in the selection of the component scales (items) and to the weights used to aggregate those scales into an index measure. The measurement of the quality of a decision aid is no exception. The popular International Patient Decision Aid Standards measure, with its proposed ‘certification’ standard, reflects the preferences emerging from the consortium responsible for its development and maintenance. It was also established that, given the expanded definition of ‘interest’ relevant in healthcare research and provision, which include commitments to particular theoretical, methodological and ethical frameworks, institutional practices and schools of thought [2], the IPDASi measure is interest-conflicted. However, it was emphasised that this applies to all related measures, so the only possible objections can be to failure to 1 Corresponding author, Jack Dowie, LSHTM, 15-17 Tavistock Place, London, UK WC1H 9SH; email: jack.dowie@lshtm.ac.uk J. Dowie et al. / Measures of Decision Aid Quality – 2: Empirical Measures 53 acknowledge the formative ontology of the measure, to attempts to establish the measure as a ‘gold standard’, and to criticism of alternatives on the ground that they reflect interest-conflicted preferences. In this paper we focus on the empirical measurement of the quality of decision aids as implemented. We make clear from the outset here that our overriding preference is to avoid all normative measures, whether IPDASi or ones based on alternative interestbased preferences. Normative considerations should exert influence, but only via empirical outcome quality constructs, not independently or a priori. Notwithstanding, empirical measures, like normative measures, are of some formative construct of decision aid quality, so the task is again to demonstrate that they embody interestconflicted preferences through their criteria selection and weighting. Conceptually, these alternatives will be measures of different constructs of decision aid quality rather than different measures of decision aid quality, which does not exist until it is formed. While we again focus on the IPDAS position [3] it is appropriate to start with the conclusion from the latest Cochrane review, which was framed within it. “When people use decision aids, they improve their knowledge of the options (high-quality evidence) and feel better informed and more clear about what matters most to them (high-quality evidence). They probably have more accurate expectations of beneﬁts and harms of options (moderate-quality evidence) and probably participate more in decision making (moderate-quality evidence). People who use decision aids may achieve decisions that are consistent with their informed values (evidence is not as strong; more research could change results). People and their clinicians were more likely to talk about the decision when using an aid. Decision aids have a variable effect on the option chosen, depending on the choice being considered. Decision aids do not worsen health outcomes, and people using them are not less satisﬁed. More research is needed to assess if people continue with the option they chose and also to assess what impact decision aids have on healthcare systems.“ [4] (p3). While of some interest, both these overall conclusions and the underlying metaanalyses are of questionable benefit. It is hard to imagine any decision maker deciding whether or not to use ‘a decision aid’, any more than they would be deciding whether or not to prescribe ‘a drug’ or perform ‘an operation’. The meaningful decision is whether to use this or that or no decision aid - this or that type of drug or none, this or that type of surgery or none. So, it is the reports and assessments of the individual studies they review that are of real value. As implied in the overall conclusions, a wide range of ‘outcome’ criteria are reported in these and the way these are brought (or not brought) into the evaluation of a decision aid is preference-sensitive and interestconflicted. 2. IPDAS on Decision Aid ‘Effectiveness’ Apart from developing its normative instrument, the International Patient Decision Aid Standards consortium has also published its preferences in relation to the empirical measurement of individual decision aid quality [3]. An umbrella concept is introduced – effectiveness – under which two separate constructs and measures are advanced. As before the purpose of quoting at length is to establish that preferences are clearly embedded in making this split, as well as in the selection of items and metrics. This is not made explicit in the presentation, which includes several ‘shoulds’. 54 J. Dowie et al. / Measures of Decision Aid Quality – 2: Empirical Measures “To establish the effectiveness of a PtDA [Patient Decision Aid], it is critical to provide evidence that the PtDA improves two constructs: i) the quality of the decisionmaking process and ii) the quality of the choice that is made (i.e., ‘decision quality’)… For the quality of the decision-making process, the core attributes that should be measured include the extent to which PtDAs help patients to: • Recognize that a decision needs to be made (e.g., as measured by items in the Preparation for Decision Making Scale. • Feel informed about the options and about the risks, benefits, and consequences of the options (e.g. as measured by the “Feeling Uninformed” subscale of the Decisional Conflict Scale). • Be clear about what matters most to them for this decision (e.g. as measured by the “Unclear Values” subscale of the Decisional Conflict Scale. • Discuss goals, concerns, and preferences with their health care providers (e.g. as measured by items in the Perceived Involvement in Care Scale. • Be involved in decision making (e.g., as measured by the Control Preferences Scale and adaptations of it)… The quality of the choice that is made, or decision quality, is defined as the extent to which patients are informed and receive treatments that reflect their goals and treatment preferences. It follows from this construct definition that two core attributes should be measured: • Informed patient: This attribute is measured by assessing a patient’s knowledge of the options and outcomes. It is not assessed in terms of patient perceptions of their knowledge level; instead, factual items are used to assess objectively a patient’s understanding of the information. This may, when applicable, include an assessment of whether or not the patient holds realistic expectations of risks and benefits. • Concordance between what matters most to the patient and the chosen option: Most approaches to measuring this attribute require (1) the elicitation of a patient’s goals and/or treatment preferences; (2) the identification of the patient’s chosen or implemented option; and (3) a calculation of the extent to which the option best meets the patient’s stated goals or treatment preferences.” [4] (p2) (italics supplied). 3. Two Decision Quality Measures In the space available here we focus on the Decision Quality Instruments (DQIs), developed by members of the IPDAS consortium to measure ‘the quality of the choice that is made, or decision quality’, and on the Decision Conflict Scale which features heavily in their profile measure of the ‘quality of the decision making process’. As a profile measure, a DQI produces two scores. The DQI-Knowledge Score is the percentage of correct responses to a set of questions. A threshold for considering a patient to be ‘well-informed’ is set, using (if available) the mean knowledge score for a group of patients who have viewed a decision aid. The DQI-Concordance Score measures ‘the extent to which patients received treatments that reflected what is most important to them’. A binary Decision Quality Composite Score is created with a score of 1 for patients who were well-informed and received treatments matching their preferences, 0 for all others. The DQI composite score is only at the group level, available only after follow-up months after the decision, and being binary does not provide a scalar index measure. So DQIs are essentially research tools, not ones to be used in real time within clinical practice. They are not preference-sensitive index measures, assessed and available immediately after the point of decision, and before any deliberation occurs, any decision is taken, any actions engaged in, or outcomes J. Dowie et al. / Measures of Decision Aid Quality – 2: Empirical Measures 55 known. In all these respects they reflect interest-conflicted preferences orthogonal to ours. While the Decisional Conflict Scale may be a valid measure for the eponymous construct (i.e. decision conflict) - it lacks content validity for this task because of the 3 items which make up its Uncertainty subscale. (‘This decision is easy for me to make’, ‘I feel sure about what to choose’ and ‘I am clear about what choice is best for me’.) These penalize an aid that correctly reports the situation as one of decisional equipoise or near equipoise, a ‘false clarity’ bias being rewarded. In 20 of the Cochrane studies reporting all subscales, the Uncertainty score was 46% higher than the average of the other four in the decision aid arm, thereby reducing the effect of the decision aid relative to usual care. The 4-item SURE version of DCS is even more prone to this bias [5]. 4. Discussion It is time to summarise our interest-conflicted preferences, scattered throughout the above, or only hinted at. Our preferred decision support tool displays a Decision Opinion as the expected value of each option, produced by applying the criterion weights of the decision owner to the performance ratings of each option on each criterion. Our (interest-conflicted) preference is for the quality of the tool to be measured at each use, empirically (not normatively) and comparatively, with genuine usual care as the mandatory comparator in order to avoid interest-conflicted Partial Or Non-Comparative Evaluation (PONCE) [6]. This measurement is to occur in the decision making setting immediately after engagement with the tool, in order that the tool’s Decision Opinion Quality is minimally confounded by any subsequent discussion or decision. The Decision Opinion Quality of the ‘usual care’ comparator is to be measured separately and independently, in maximal ignorance of the contents or Opinion of the tool, to further avoid bias from PONCE. Our (interest-conflicted) preferences for this quality assessment exclude, in agreement with Elwyn and MironShatz [7], any objective assessment of the knowledge of any party to the decision. (Apart from its unknown relevance in the decision, such a ‘knowledge’- assessing instrument is of a formative construct and therefore preference-sensitive and interestconflicted.) They also exclude, again in agreement with Elwyn and Miron-Shatz, all ‘downstream’ outcomes of any sort, whether they relate to the health consequences of actions taken as a result of the eventual decision, or any later psychological/affective effects such as experienced regret. (Anticipated regret is assumed to be a key input into criterion weighting). Finally, they exclude any concern with the extent to which any elicited intention, or subsequent behaviour ‘matches’ the values expressed by the user during engagement with the tool. Historically, there was passing interest in PtDAs based on Decision Analysis Dolan 2002 [8], Montgomery 2003 [9], Bekker 2004 [10] - which came close to meeting the above preferences. These Cochrane-included trials had positive outcomes, but the reported ‘obstacles’ in delivery and clinician acceptance undoubtedly contributed to their demise, along with paternalistic projection on to patients: ‘… there are concerns that encouraging individuals to adopt this more systematic approach to making choices places an additional burden on the decision process that may lead to greater distress, decisional conﬂict and post-choice regret.’ [10] (p266). As seen above, our preferences rule out decision conflict and post-choice regret as relevant criteria. 56 J. Dowie et al. / Measures of Decision Aid Quality – 2: Empirical Measures Potential ‘distress’ becomes a criterion for the decision owner to weight in deciding whether or not to engage with ‘a systematic process’. 5. Conclusion Non-financial interest-conflicted preferences are endemic in healthcare research, policy-making and practice. The fact that they are essential, as well as inevitable, means that the problem lies in their denial or disguise. Paradoxically, much highquality research into decision support is undertaken by researchers hostile to positivist methodologies. But implying that a formative construct is reflective, or can be treated as such, because it embodies widely-supported preferences in line with current practices, is essentially positivistic. This needs to be explicitly acknowledged. Spelling out our preferred measure for evaluating decision aids, or introducing other non-IPDAS measures, has not been the aim. It is limited to establishing that a ‘level playing field’ must acknowledge that all quality measures are preference-sensitive and interestaligned, if not interest-conflicted. The preservation of existing structures and practices in healthcare research and provision may not be an interest embedded in some alternative constructs and measures of decision aid quality. References [1] Dowie J, Kaltoft MK. Measures of decision aid quality are preference-sensitive and interest-conflicted 1: normative measures. Stud Health Technol Inform. 2020. [2] Wiersma M, Kerridge I, Lipworth W, Rodwin M. Should we try to manage non-financial interests? BMJ. 2018 361:k1240. [3] Sepucha KR, Borkhoff CM, Lally J, Levin CA, Matlock DD, Ng CJ, et al. Establishing the effectiveness of patient decision aids: key constructs and measurement instruments. BMC Med Inform Decis Mak. 2013 13:S12. [4] Stacey D, Légaré F, Lewis K, Barry MJ, Bennett CL, Eden KB, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database of Syst Rev. 2017:4. [5] Kaltoft MK, Nielsen JB, Dowie J. The evaluation of decision support tools requires a measure of decision quality that has content and construct validity in person-centred care. Stud Health Technol Inform. 2018 247:331–335. [6] Dowie J. The danger of partial evaluation. Health Care Anal. 1995 3:232–4 [7] Elwyn G, Miron-Shatz T. Deliberation before determination: The definition and evaluation of good decision making. Health Expect. 2010 13(2):139–47. [8] Dolan JG, Frisina S. Randomized controlled trial of a patient decision aid for colorectal cancer screening, Med Decis Making. 2002 22(2):125–39. [9] Montgomery AA, Fahey T, Peters TJ. A factorial randomised controlled trial of decision analysis and an information video plus leaflet for newly diagnosed hypertensive patients. Br J Gen Pract. 2003 53 (491):446–53. [10] Bekker HL, Hewison J, Thornton JG. Applying decision analysis to facilitate informed decision making about prenatal diagnosis for Down syndrome: A randomised controlled trial. Prenat Diagn. 2004 24(4): 265–75. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200694 57 Acceptance Study on the Usage of HealthEnabling Technologies in Therapy and Diagnostics for People with Mental Disorders Bastian DROEGEMUELLERa,1, Corinna MIELKEa, Reinhold HAUXa and Alexander DIEHLb a Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, Braunschweig, Germany b Department of Psychiatry, Psychotherapy and Psychosomatics, Braunschweig Medical Center, Germany Abstract. Mental disorders are widespread among the world's population and place a high burden on both the people affected and the economy. In this area of health care and prevention major deficits can be found. Health-enabling technologies are being developed in order to provide support in the therapy and diagnostics of mental disorders. However, it is not clear whether patients are open to these technologies and what they expect from a suitable usage. The main goal of this study is to find out what opinions, hopes and fears mentally ill persons have towards a supporting treatment with health-enabling technologies. Personal interviews were conducted with psychiatric patients for that purpose. The evaluation of the interview data revealed a predominantly positive mindset of the participants. In addition to the general question according to the acceptance, requirements and expectations for the use of health-enabling technologies were acquired. In this context the concern of an invasion of privacy was exposed as a major barrier. Keywords. Mental disorders, mentally ill persons, health-enabling technologies, patient acceptance of health care, data collection, interview, expectations 1. Introduction Mental illness is becoming increasingly relevant within our society. Not only the proportion of disability to work and the number of ‘Disability-Adjusted Life Years’ (DALY) caused by these disorders is increasing, but also the associated economic loss through direct and indirect costs [1, 2]. On the other hand, there is an acute medical shortage for patients due to a lack of medical specialists and nursing staff [3]. In order to reduce the burden on the health care system, it is becoming increasingly common to use health-enabling technologies, for example ambient assisted living systems for the elderly. Comparable sensors and technologies are now supposed to be used to support 1 Corresponding Author, Bastian Droegemueller, Peter L. Reichertz Institute for Medical Informatics (PLRI), University of Braunschweig – Institute of Technology and Hannover Medical School, Mühlenpfordtstr. 23, D-38106 Braunschweig; E-mail: b.droegemueller@tu-braunschweig.de. 58 B. Droegemueller et al. / Acceptance Study on the Usage of Health-Enabling Technologies planned therapies or the general life of patients in psychiatric treatment [4, 5]. A literature search using PubMed showed that only a few studies could be found that deal with this exact topic. Therefore our assumption is that it is unclear whether psychiatric patients are openminded about the use of these technologies and what they expect from a supportive implementation. The overall objective of the study is to collect as many statements as possible from psychiatric patients in terms of their opinions, fears and hopes regarding therapy support by health-enabling technologies, so that first impressions about the general acceptance of these technologies can be derived. Studies with a similar starting point suggest that the acceptance is likely to be predominantly positive [6, 7, 8]. However, the concern about an intrusion into intimacy and privacy seems to be a major barrier to use [9]. In the state of research mentioned here, people with mental disorders are only a small part of the total study collective. Furthermore, many different technologies are usually considered there. In this study psychiatric patients are directly associated with healthenabling technologies in the form of both wearable and smart home sensors. 2. Method In close cooperation with the Department of Psychiatry, Psychotherapy and Psychosomatics of the Braunschweig Medical Center, an acceptance study was conducted among psychiatric patients. For this study we used individual oral interviews with suitable patients. In our estimation, the patients did not suffer any disadvantages from participating in the study. In addition, there were no points of contact with the medical care and therapy of the patients, which is why we have refrained from involving the ethics committee. 2.1. Design of the study A separate questionnaire was self-developed for the interviews, as no suitable questionnaire could be found in the literature for this specific topic. For the data collection the participants were asked a total of 13 questions. Most of these are of a quantitative nature. But there are also questions that required a qualitative answer. Previous experiences with health-enabling technologies were enquired in the interview as well as opinions on the possible use of wearable and smart home sensors. Therefore imaginable expectations and requirements were collected from the patients. Due to the different types of questions, the data evaluation consists of qualitative (e.g. clustering of similar answers) and quantitative methods, like the descriptive analysis. All in all, 15 to 20 minutes were set for conducting a single interview. 2.2. Details of the study population When selecting patients, care was taken to ensure that the study population was reasonably limited. For this task we defined the following main inclusion and exclusion criteria. B. Droegemueller et al. / Acceptance Study on the Usage of Health-Enabling Technologies 59 Inclusion criteria: • • The potential participant must be diagnosed with a mental disorder - according to ICD-10 F00-F99 ‘Mental and behavioural disorders’ The potential participant must be cognitively and mentally able to participate in the study Exclusion criteria: • The potential participant is not able to participate in the study due to the severity of the mental illness (e.g. psychoses or suicidal tendencies) After the recruitment process, the study collective resulted in a size of n = 27. Table 1 shows an overview of the characterization of the study participants. Due to the high number of individual diagnoses, the groups ‘F30-F39’ and ‘Other’ were considered in the further study. Table 1. Characterization of the study participants per interview 2.3. Implementation of the study The actual course of the study was largely similar to the previously developed study plan, which suggests that the planning was well thought out. The interviews were conducted in the period from mid-February to mid-March 2020 and the participants were distributed among a total of five different psychiatric wards of the Braunschweig Medical Center. 3. Results For the study collective it can be stated that there is a predominant acceptance of the use of health-enabling technologies in therapy and diagnostics. By differentiating between age and type of mental disorder, almost no differences in basic consent are apparent. 60 B. Droegemueller et al. / Acceptance Study on the Usage of Health-Enabling Technologies The gender-specific differences, on the other hand, can be seen in a slightly higher acceptance by male patients. In addition to that, it is noticeable that younger patients seem to be much more expectant and joyful when it comes to the application of those technologies. It can also be seen that for a large proportion of patients, the use of such technologies entails a fear of an excessive invasion of privacy. As expected, this concern can be understood as a major barrier to the use of these type of technologies. Furthermore, the evaluation of the interviews made it possible to identify basic requirements for the use of health-enabling technologies. As it can be seen in figure 1, besides several functional and non-functional requirements, boundary conditions could be derived from the given answers. Confidentiality Data integrity Neutral appearance Inconspicious p behaviour Look and behaviour Reliable functionality Reliability Enlightenment IT -Security Requirements for the use of health - enabling g technologies g in the therapy py and diagnostics g of people with mental disorders Reminder function Simple p and intuitive handling Correctness Low error rate App pp connection Right g to be heard Requirement Usability Cost absorption Non functional requirement Functional requirement Boundaryy condition Legend Figure 1. Requirements for the use of health-enabling technologies in the therapy and diagnostics of people with mental disorders 4. Discussion 4.1. Limitations Since no suitable questionnaire could be found in the literature, a separate one had to be self-developed, which means that this questionnaire must be considered as not validated. This study is not free of limitations due to the time constraints. Because of the small study collective, it is recommended that in future acceptance studies a much higher number of participants should be interviewed. That would also make it possible to distinguish between different types of mental disorders. In addition to that, a greater B. Droegemueller et al. / Acceptance Study on the Usage of Health-Enabling Technologies 61 inclusion of people at a higher age would be useful. During the recruitment process it was obvious that many of the elderly already refused to be informed about the study due to a lack of understanding of the technology. It is probable that the study's research collective largely comprises people who have a higher affinity for technology than a representative population. This assumption has to be counteracted in future studies. 4.2. Conclusion With a size of n = 27, the study collective is too small to draw thematically meaningful conclusions regarding a differentiated consideration of the individual disorders. In this case, an acceptance study should again be conducted on the basis of a modified study plan. But the results of the study allow a first impression about the general acceptance of patients regarding the use of appropriate technologies. Overall, the results fit very well into the latest state of the art. Not only the assumption after a predominant acceptance could be substantiated here [6, 7, 8], but also the fear about an intrusion into intimacy and privacy, which is also described in a previous study [9]. The investigation that took place here made it possible to set a focus whose content statements can supplement the previous findings of the literature. In this context, the concept of this acceptance study can also serve as a helpful support for further studies. For this purpose, the study implementation, in connection with the identified limitations, offers a practical example in research on the acceptance of psychiatric patients with regard to the use of healthenabling technologies in therapy and diagnostics. References [1] [2] [3] [4] [5] [6] [7] [8] [9] Plass D, Vos T, Hornberg C, Scheidt-Nave C, Zeeb H, Kramer A. Trends in disease burden in Germany: results, implications and limitations of the Global Burden of Disease Study. Dtsch Arztebl Int. 2014 Sep; 111(38): 629-38. Trautmann S, Rehm J, Wittchen HU. The economic costs of mental disorders: Do our societies react appropriately to the burden of mental disorders. EMBO Rep. 2016 Sep; 17(9):1245-1249. Butryn T, Bryant L, Marchionni C, Sholevar F. The shortage of psychiatrists and other mental health providers: Causes, current state, and potential solutions. Int J Acad Med. 2017 Jul; 3:5-9. Haux R, Koch S, Lovell NH, Marschollek M, Nakashima N, Wolf KH. Health-Enabling and Ambient Assistive Technologies: Past, Present, Future. Yearb Med Inform. 2016 Jun 30; 1:76-91. Doraiswamy PM, London E, Varnum P, Harvey B, Saxena S, Tottman S, Campbell P, Ibanez F, Manji H, Al Olama MAAS, Chou I, Herman H, Jeong SJ, Le T, Montojo C, Revel B, Rommelfanger KS, Stix C, Thakor N, Chow KHM, Welchman AE, Candeias V. Empowering 8 Billion Minds: Enabling Better Mental Health for All via the Ethical Adoption of Technologies. NAM Perspectives. 2019 Oct 28; 28 p. Wallbaum T, Frenken M, Meyer J, Hein A. Acceptance of sensor networks by dementia patients. Berlin: VDE Verlag GmbH. 2014 Jan 22; 7 p. Botros A, Schutz N, Camezind M, Urwyler P, Bolliger D, Vanbellingen T, Kistler R, Bohlhalter S, Muri RM, Mosimann UP, Nef T. Long-Term Home-Monitoring Sensor Technology in Patients with Parkinsons Disease – Acceptance and Adherence. MDPI. 2019 Nov 26. 1:1-13. Botella C, Mira A, Moragrega I, Garcia-Palacios A, Bretopn-Lopez J, Castilla D, Lopez del Amo AR, Soler C, Molinari G, Quero S, Gullien-Botella V, Miralles I, Nebot S, Serrano B, Majoe D, Alcaniz M, Banos RM. An Internet-based program for depression using activity and physiological sensors: efficacy, expectations, satisfaction, and ease of use. Dove Press Journal: Neuropsychiatric Disease and Treatment. 2016 Feb 23. 12:393-406. Patientenprojekte GmbH. Patientenperspektiven 2018 – Qualitative Studie zur Digitalisierung im Gesundheitswesen aus Sicht von Patientinnen und Patienten in Deutschland. Berlin: Kassenarztliche Bundesvereinigung (KBV); 2018 Aug. 39 p. German. 62 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200695 From Personalised Predictions to Targeted Advice: Improving Self-Management in Rheumatoid Arthritis Ali FAHMI a,1, Hamit SOYEL a, William MARSH a, Paul CURZON a, Amy MACBRAYNE b, and Frances HUMBY b a School of Electronic Engineering and Computer Science, Queen Mary University of London, United Kingdom b The William Harvey Institute, Queen Mary University of London, United Kingdom Abstract. Rheumatoid arthritis (RA) is a chronic inflammatory autoimmune disease, that can lead to joint damage but also affects quality of life (QoL) including aspects such as self-esteem, fatigue, and mood. Current medical management focuses on the fluctuating disease activity to prevent progressive disability, but practical constraints mean periodic clinic appointments give little attention to the patient’s experience of managing the wider consequences of chronic illness. The main aim of this study is to explore how to use patient-derived data both for clinical decisionmaking and for personalisation, with the first steps towards a platform for tailoring self-management advice to patients’ lifestyle changes. As a result, we proposed a Bayesian network model for personalisation and have obtained promising outcomes. Keywords. mHealth, personalised prediction, rheumatoid arthritis, Bayesian networks 1. Introduction Rheumatoid arthritis (RA) is a chronic inflammatory autoimmune disease, causing swollen, painful joints, and characterized by fluctuating inflammatory activity [1]. The long-term prognosis has changed significantly over recent years, largely due to aggressive early treatment with combination medications aiming to achieve remission and prevent disability [2]. However, outcomes remain varied and may affect not only physical functioning but also psychological aspects such as self-esteem, role, relationships, control perceptions, and mood [3]. Recently, mobile health (mHealth) applications have targeted this challenge and have an active role in patient-centered healthcare [4]. By enabling people to access and share their health information, mHealth applications can empower individuals to take a more active role in self-managing their health and well-being [5]. They can increase disease acceptance [6] and self-management [7] capabilities, with regular use related to behavioral change and health improvement [8]. Although clinicians are actively assessing the broader impact using quality of life (QoL) instruments, which measure the patient’s evaluation of life across different 1 Corresponding Author: Ali Fahmi, School of Electronic Engineering and Computer Science, Queen Mary University of London, London, UK; E-mail: a.fahmi@qmul.ac.uk. A. Fahmi et al. / From Personalised Predictions to Targeted Advice 63 domains such as having a positive outlook on life, having a good social network and living conditions, the definition and measurement of QoL is not standardized [9]. It is not clear whether disease-specific QoL tools (e.g. the RAQoL scale [10]) are applied effectively in mHealth applications to capture the processes behind patients’ changing priorities and adjustment to their long-term conditions impacting on QoL outcomes [11]. The National Institute for Clinical Excellence recommends that access to a multidisciplinary team should provide the “opportunity for assessments of the effect of RA on patients’ lives (such as pain, fatigue, physical activities, sleep quality, self-care, financial status, belonging and social activities, QoL, and mood)” [12]. However, there is little evidence that the psycho-social aspects of RA are formally assessed in clinical practice or that health services are equipped to support these issues in a personalised manner. Bayesian networks (BNs) are a promising technology that may be able to provide this support. They are directed acyclic graphs consisting of a set of variables and their dependencies [13]. They can combine expert knowledge with data, but also be used when no data is available. The structure of a BN represents the knowledge about a problem which is usually elicited from the experts or taken from literature. The underlying probabilities in the BN allow one to model the embedded uncertainty of a given problem. Here, we developed a BN model producing personalised predictions for selfmanagement in RA through a patient-centered process. The aim is to provide a holistic patient-centered support system, leading to greater patient participation and improved health outcomes and reduced economic costs. QoL is supported in three different ways: independence in terms of physical functioning and financial resources; empowerment in how to manage life; and participation in the experience of belonging in a social context [14]. The proposed model also reflects on disease acceptance, to be a process whereby patients begin to make choices that maximize their QoL, and estimates the probability of flares happening associated with functional disability, disease duration, functional deterioration, pain, morning stiffness and fatigue [15]. Although the personalisation aspect in self-management for RA is researched relatively broadly, to the best of our knowledge there is no study investigating the uncertainty involved in understanding the needs for a long-term interaction with an mHealth platform from the patients’ perspective. 2. Method For this study, we developed a knowledge-based BN model for personalised prediction, where the structure of the model shows the variables and their causal or associational dependencies derived from the literature. To build the BN structure, firstly, we determined the main variables for selfmanagement in RA. This was done by first engaging with members of a Patient and Public Involvement (PPI) group. Informal interviews and discussion led to knowledge elicitation based on both research and patient-centered publications on the issues raised. A series of patient personas were developed describing fictional patients and scenarios around their lives. These were used in a formal focus group with PPI members to elicit further information around the important issues, with changes validated by follow up discussion with PPI members. We used expert knowledge to specify the probabilities of the BN variables as no data was available. The probability elicitation was simplified using ‘ranked nodes’ as defined in [13]. 64 A. Fahmi et al. / From Personalised Predictions to Targeted Advice As the proposed BN model receives evidence about a patient and predicts the output variables, we used interviews from a formal semi-structured interview study (AtTRA) about life with RA as a basis to initially validate the model from a patients’ perspective. We developed 6 patient ‘scenarios’ directly from the interviews. We coded these scenarios and attained the evidences and expected state of output variables in a blind way. From these, inputs to the BN were extracted matching the patient scenario. Outputs were obtained from the BN and compared with the corresponding description in the scenarios. 3. Results Key variables that emerged from established literature and interviews included QoL, disease acceptance, flare-up, pain, morning stiffness, and fatigue. We grouped these variables into four groups: disease activity, QoL characteristics, lifestyle choices, and disease manifestations as well as two additional groups representing the risk factors namely personal factors and environmental factors. As shown in Figure 1, the evidence variables or input variables are displayed by orange ovals. The white dashed oval shows a synthetic variable which combines its parent variables and simplifies the model. The white ovals represent the output variables, namely: Flare-up, Current Disease Activity, Overall Disease Activity, Disease Acceptance, Independence, Participation, Empowerment, and QoL. Flare-up has three states: None, Mild, and Severe. Current Disease Activity and Overall Disease Activity have four states: Remission, Low, Moderate, and High. The rest of the output variables have three states of Low, Medium, and High. Figure 1. BN model for personalisation in self-management of RA. The outputs of the BN and the expected states for two of the scenarios - a mild case and a severe case - are shown in Table 1 for illustration purposes. The comparison A. Fahmi et al. / From Personalised Predictions to Targeted Advice 65 between the predicted and expected states indicates that the proposed BN model is highly consistent with the information from patient interviews. Table 1. BN outputs and expected states for mild and severe scenarios. Output Variables Flare-up Current Disease Activity Overall Disease Activity Disease Acceptance Independence Participation Empowerment QoL Mild Case BN Prediction Expectation None None Remission Low Low Low High High Low Low Medium Low Medium Medium Medium Medium Severe Case BN Prediction Expectation Severe Severe Moderate Moderate Moderate Moderate High High Medium Low Medium Medium Low Low Medium Low 4. Discussion Our results are only indicative given the small number of scenarios and with limited personal/environmental factors covered. However, they suggest the proposed BN model is on the right track on understanding the uncertainty in RA. It has the potential to form the basis of a prediction system by bringing external and patient-derived data into the clinical decision-making cycle. It would do this by generating personalised predictions for disease status and QoL aspects. This offers a promising direction to increase the efficiency of health service delivery by tailoring healthcare to patients’ individual needs. The proposed approach does not allow firm conclusions about the exact contribution of each factor to the BN model’s predictions. Future studies will shed further light on the usability of the proposed BN based approach. In the context of self-management, the ability to indicate and predict which advice will work best for a certain person at a certain time and in a certain context may be possible. From a methodological point of view, alternative approaches and techniques for data collection from the patient may have the potential to further increase the precision of the prediction model. Current medical management focuses on the fluctuating disease activity to prevent progressive disability, but practical constraints mean periodic clinic appointments give little attention to the patient’s experience of managing the wider consequences of chronic conditions. Instead, patients must rely on generic resources, such as those provided by patient associations, to gradually learn how to adapt their lives to RA. Our initial validation suggests that our BN has potential to help in this regard, pointing patients at appropriate advice in a timely way. 5. Conclusion In this paper, we presented a BN model that predicts QoL factors based on a patientcentered knowledge acquisition process. It forms the basis of a system to give advice to patients based on its predictions of the QOL issues most needing attention. It thereby has the potential to increase the response rate to a smartphone-based targeted advice platform in terms of disease acceptance and adherence to lifestyle changes. We are in the process of designing a prototype of such a system. This approach is in line with the precision 66 A. Fahmi et al. / From Personalised Predictions to Targeted Advice medicine initiative. The proposed model could also be used to identify relationships between multiple behavioral factors to enable the assessment of opportunities and risks associated with RA. This could, for example, be used to flag patients ready for tapering or to conduct individual targeted preventive actions towards high-risk patients. Acknowledgements: This research is supported by the Engineering and Physical Sciences Research Council (EPSRC) under project EP/P009964/1: PAMBAYESIAN: patient managed decision-support using Bayesian networks. References [1] Arnett FC, Edworthy SM, Bloch DA, McShane DJF, Cooper NS and Healey LA. The American Rheumatism Association 1987 revised criteria for the classification of rheumatoid arthritis. Arthritis Rheum. 1988 March;31(3):315-24. [2] Goekoop-Ruiterman YD, de Vries-Bouwstra JK, Allaart CF, Van Zeben D, Kerstens PJ, Hazes JM, Zwinderman AH, Ronday HK, Han KH, Westedt ML, Gerards AH. Clinical and radiographic outcomes of four different treatment strategies in patients with early rheumatoid arthritis (the BeSt study): a randomized, controlled trial. Arthritis Rheumatism. 2005 Nov;52(11):3381-90. [3] Ryan S. Living with rheumatoid arthritis: a phenomenological exploration. Nursing Standard (Royal College of Nursing (Great Britain): 1987). 1996 Jul 1;10(41):45-8. [4] Dixon WG, Michaud K. Using technology to support clinical care and research in rheumatoid arthritis. Current opinion in rheumatology. 2018 May;30(3):276. [5] Gossec L, Cantagrel A, Soubrier M, Berthelot JM, Joubert JM, Combe B, Czarlewski W, Wendling D, Dernis E, Grange L, Beauvais C. An e-health interactive self-assessment website (SanoiaR©) in rheumatoid arthritis. A 12-month randomized controlled trial in 320 patients. Joint Bone Spine. 2018 Dec1;85(6):709-14. [6] Grainger R, Townsley H, White B, Langlotz T, Taylor WJ. Apps for people with rheumatoid arthritis to monitor their disease activity: a review of apps for best practice and quality. JMIR mHealth and uHealth.2017;5(2):e7. [7] Luo D, Wang P, Lu F, Elias J, Sparks JA, Lee YC. Mobile apps for individuals with rheumatoid arthritis: a systematic review. JCR: Journal of Clinical Rheumatology. 2019 Apr 1;25(3):133-41. [8] Revenäs Å, Opava CH, Martin C, Demmelmaier I, Keller C, Åsenlöf P. Development of a web-based and mobile app to support physical activity in individuals with rheumatoid arthritis: results from the second step of a co-design process. JMIR research protocols. 2015;4(1):e22. [9] Moons P. Why call it health-related quality of life when you mean perceived health status?. European Journal of Cardiovascular Nursing. 2004 Dec;3(4):275-7. [10] De Jong Z, Van der Heijde DM, McKenna SP, Whalley D. The reliability and construct validity of the RAQoL: a rheumatoid arthritis-specific quality of life instrument. British journal of rheumatology. 1997Aug 1;36(8):878-83. [11] Druce KL, Cordingley L, Short V, Moore S, Hellman B, James B, Lunt M, Kyle SD, Dixon WG, McBethJ. Quality of life, sleep and rheumatoid arthritis (QUASAR): a protocol for a prospective UK mHealth study to investigate the relationship between sleep and quality of life in adults with rheumatoid arthritis. BMJ open. 2018 Jan 1;8(1). [12] NICE. Rheumatoid arthritis: The management of rheumatoid arthritis in adults. 2009;CG79. [13] Fenton N, Neil M. Risk assessment and decision analysis with Bayesian networks. CRC Press; 2012. [14] Steger MF, Kashdan TB. Depression and everyday social activity, belonging, and well-being. Journal of counseling psychology. 2009 Apr;56(2):289. [15] Connolly D, Fitzpatrick C, O’Toole L, Doran M, O’Shea F. Impact of fatigue in rheumatic diseases in the work environment: a qualitative study. International journal of environmental research and public health. 2015 Nov;12(11):13807-22. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200696 67 What is Digital Health? Review of Definitions Farhad FATEHI a,b, Mahnaz SAMADBEIK c,1 and Azar KAZEMI d a Monash University, Melbourne, Australia b Tehran University of Medical Sciences, Tehran, Iran c Lorestan University of Medical Sciences, Khorramabad, Iran d Mashhad University of Medical Sciences, Mashhad, Iran Abstract. Digital technologies are transforming the health sector all over the world, however various aspects of this emerging field of science is yet to be properly understood. Ambiguity in the definition of digital health is a hurdle for research, policy, and practice in this field. With the aim of achieving a consensus in the definition of digital health, we undertook a quantitative analysis and term mapping of the published definitions of digital health. After inspecting 1527 records, we analyzed 95 unique definitions of digital health, from both scholar and general sources. The findings showed that digital health, as has been used in the literature, is more concerned about the provision of healthcare rather than the use of technology. Wellbeing of people, both at population and individual levels, have been more emphasized than the care of patients suffering from diseases. Also, the use of data and information for the care of patients was highlighted. A dominant concept in digital health appeared to be mobile health (mHealth), which is related to other concepts such as telehealth, eHealth, and artificial intelligence in healthcare. Keywords. digital health, definition, term mapping, content analysis 1. Introduction Digital Health is an emerging field of study at the intersection of healthcare and digital technologies, which has attracted lots of attention in the past decade in many countries around the world. In 2019, the American Medical Association reported that companies have invested billions of dollars on new digital health entrepreneurship [1]. The US Food and Drug Administration considers a broad scope of technologies as digital health; mobile health, wearable devices, telehealth and telemedicine, health information technologies, and personalized medicine [2]. WHO emphasizes that digital health can be beneficial to achieving the Sustainable Development Goals by making health and wellbeing services accessible with high standards for all people globally [3]. The term “digital health” is broadly used in the various disciplines such as health informatics, but there is no agreed upon definition for this term. Due to different perspectives of academia, scientific institutions, industry, and individuals, there is a lack of comprehensive and precise definition of digital health. A systematic review of the literature identified the following components of digital health innovation ecosystem: e1 Corresponding Author, Mahnaz Samadbeik, School of Allied Medicine, Lorestan University of Medical Sciences, East Goldasht, Khorramabad, Iran. Postcode: 6819789741; Email: mahbeik@yahoo.com. 68 F. Fatehi et al. / What is Digital Health? Review of Definitions health, m-health, health 2.0, telehealth and telemedicine, public health surveillance, personalized medicine, health promotion strategies, self-tracking, wearable devices and sensors, genomics, medical imaging, and information systems [4]. According to the Health Informatics Society of Australia and Digital Health Workforce Academy paper, digital health uses not only electronic data but also traditional data to serve healthcare and research. Today, artificial intelligence and machine learning are used as essential methods in digital health scope, to combine with ICT and other technologies to solve consumer and patients’ problems [5]. The first reference to digital health in PubMed database dates back to 90s, when this concept was mainly used for digitization of health information and libraries [6]. In the 2000s, with the spread of the Internet worldwide, the concept of digital health changed. Later on, with the advancement of computer science and informatics, and their applications in health care, a number of new concepts such as artificial intelligence and genomics were also considered as part of digital health. However, ambiguity in the definition of digital health and its taxonomy remains yet to be addressed. There is, therefore, a need to consolidated digital health concepts for use in research, policy, and practice. We envisaged to reach a consensus definition for digital health that can satisfy most, if not all, of the stakeholders. In this study we collected, examined, and quantitatively analyzed the published English definitions of digital health both in scholarly literature and online sources. 2. Methods We conducted a term map analysis of the published definition of ‘digital health’ using VOSviewer software version 1.6.15 (Centre for Science and Technology Studies, Leiden, The Netherlands) [7]. The definitions were obtained from two sources: 1) peerreviewed publications, and 2) websites of relevant authorities and scientific bodies. We first searched Web of Science, PubMed, Scopus, and Google Scholar using this search string: “digital health” AND (definition OR defined). The publication period was between January 2000 and April 2020 (last search, 15 May 2020), limited to publications in English. Furthermore, we searched Google using two search queries in two separate runs: A) “digital health” AND (definition OR defined); B) “What is digital health”. As Google ranks the retrieved websites based on their importance and relevance, we reviewed the first 200 results for each of these two searches. This study included all resources (articles, reports, letters, guidelines, discussion papers, and websites) that have defined or attempted to define digital health in explicit terms. Documents were excluded if they did not provide an original definition, or they focused on the other aspects of digital health (e.g. digital health technology, digital health frameworks, and digital health interventions) rather than its definition. The search results were exported to EndNote, and duplicates were removed. Two authors independently assessed the titles, abstracts, full texts, and websites for eligibility. Disagreements about study eligibility were resolved through consensus discussion or by consulting the third author. The final set of selected references was then reviewed, and the definitions were extracted for term mapping. From the selected references, we extracted the following data: author name, publication year, title, source, URL, and the definition. We carried out a quantitative analysis to find the most common terms used in the included definitions. VOSviewer was used to visualize the main terms and concepts in the digital health definitions. A thesaurus file was used to perform data cleaning and F. Fatehi et al. / What is Digital Health? Review of Definitions 69 merging terms in VOSviewer (e.g. “mhealth”, “m health”, and “Mobile health” were merged as “Mobile health”). We excluded a number of general terms that were commonly used, but did not add value to the definition of digital health, such as “definition”, “term”, and “field”. We used the full counting method of occurrence of terms in the VOSviewer software. For the term occurrence map, terms that appeared in at least three of the definitions (threshold level of terms equal to 2) were selected, so terms with fewer than 3 occurrences are excluded. 3. Results We screened 1,527 sources (855 peer-reviewed articles and 672 web pages), which were returned by our scholar and general electronic search. After inspecting the contents of these references, we extracted 95 unique definitions of “digital health” in this study (30 definitions from journal articles and 39 from websites). The list of included articles and websites is available at https://osf.io/yfusw/.Total main terms included in the definitions were 410, and 60 terms were repeated at least 3 times. Table 1 lists the top ten terms with the highest frequency in the included definitions. Terms with a high relevance score tend to represent specific topics covered by the text data, including “disruptive technology” (6.25), “human health” (3.46), and “healthcare service” (2.49). However, terms with a low relevance score tend to be of a general nature, including “health” (0.09), “use” (0.12) and “technology” (0.18). Table 1. Ten top terms with the highest frequency in the definitions Term Health Technology Use Information Mobile health Healthcare Medicine Wellness Patient eHealth Occurrence 49 35 31 25 24 19 14 14 14 13 Relevance score 0.09 0.18 0.12 0.35 0.53 0.31 0.71 0.48 0.46 0.42 To visualize the most frequent terms used in the definitions, a network map and a density map of the terms were created. The terms included in the map are selected based on the calculation of occurrences and relevance scores. According to the map of words (Figure 1), among the terms that met the threshold, the terms “health,” “technology”, and “use” were occurred the most in the definitions of digital health. Six major clusters emerged and were classified according to the 52 most common terms with an occurrence of at least three times. Figure 2 shows the terms density in 95 definitions visualized by VOSviewer. The bubble size indicates the number of definitions containing each term. If the terms frequently co-appeared in the same definitions, their bubbles would be closer to each other. The color of each point in this visualization indicates the density of items at that point. By default, colors range from blue to green to yellow. The yellow color of the point shows the larger number and the higher weights of items in its neighborhood, which in this case were the terms “health”, “technology”, and “use”. The blue color of the point shows the smaller number and the lower weights of items in its neighborhood, such as the terms “health service”, “health risk”, and “illness”. 70 F. Fatehi et al. / What is Digital Health? Review of Definitions Figure 1. Terms occurrence network in the 95 definitions of digital health visualized by VOSviewer Figure 2. Terms density visualization of the 95 definitions of digital health. F. Fatehi et al. / What is Digital Health? Review of Definitions 71 4. Conclusion We reviewed, quantitively analyzed, and mapped the main terms of 95 unique definitions of 'digital health’. The results of this study show that in the field of digital health, the main focus is on the health, rather than technology. Under the concept of health, emphasis was on the health and wellbeing of individuals and population, rather the diseases and patients. It is evident that in the concept of technology, emphasis is on the (proper) use of technology, rather than its technical aspects. Mobile Health (mHealth) appeared to be a dominant concept in the field of digital health, and closely related to artificial intelligence and through it connected to genomics. Moreover, the concepts of data and information, were closely related to patients. Based on the results of this study, we can infer that digital health is about the proper use of technology for improving the health and wellbeing of people at individual and population levels, as well as enhancing the care of patients through intelligent processing of clinical and genetic data. References [1] [2] [3] [4] [5] [6] [7] O'Reilly KB. Digital health funding hits $8.1 billion. How to spend it wisely [Internet]. 2019 [cited 12 Sep 2020]. Available from: https://www.ama-assn.org/practice-management/digital/digital-healthfunding-hits-81-billion-how-spend-it-wisely. US Food and Drug Administration (FDA). Digital health innovation action plan [Internet]. 2019 [cited 12 Sep 2020]. Available from: https://www.fda.gov/medical-devices/digital-health. World Health Organization. Digital Health [Internet]. 2020 [cited 12 Sep 2020]. Available from: https://www.who.int/health-topics/digital-health Iyawa GE, Herselman M, Botha A. Digital health innovation ecosystems: From systematic literature review to conceptual framework. 2016. Rowlands D. What is digital health? And why does it matter? [White paper]. Health Informatics Society of Australia 2020. Available from: https://www.hisa.org.au/wpcontent/uploads/2019/12/What_is_Digital_Health.pdf Galvin JR, D'Alessandro MP, Erkonen WE, Smith WL, El-Khoury GY, Weinstein JN. The virtual hospital. Providing multimedia decision support tools via the Internet. Spine. 1995;20(15):1735-8. Van Eck NJ, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics. 2010 Aug;84(2):523-38. 72 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200697 Usability of Remote Assessment of Exercise Capacity for Pulmonary Telerehabilitation Program Joseph FINKELSTEIN1, In cheol JEONG, Mackenzie DOERSTLING, Yichao SHEN, Chenhao WEI and Herbert KARPATKIN Icahn School of Medicine at Mount Sinai, New York, NY, USA Abstract. Pulmonary rehabilitation [PR] has been successfully carried out via telemedicine however initial patient assessment has been traditionally conducted in PR centers. The first step in PR is assessment of patient's exercise capacity which allows individualized prescription of safe and effective exercise program. With COVID-19 pandemics assessment of patients in PR centers has been limited resulting in significant reduction of patients undergoing life-saving PR. The goal of this pilot study was to introduce approaches for remote assessment of exercise capacity using videoconferencing platforms and provide initial usability assessment of this approach by conducing cognitive walkthrough testing. We developed a remote assessment system that supports comprehensive physical therapy assessment necessary for prescription of a personalized exercise program tailored to individual fitness level and limitations in gait and balance of the patient under evaluation. Usability was assessed by conducting cognitive walkthrough and system usability surveys. The usability inspection of the remote exercise assessment demonstrated overall high acceptance by all study participants. Our next steps in developing user-centered interface should include usability evaluation in different subgroups of patients with varying socio-economic background, different age groups, computer skills, literacy and numeracy. Keywords. pulmonary rehabilitation, telemedicine, exercise capacity 1. Introduction Pulmonary rehabilitation [PR] is one of few treatments of chronic lung conditions which has been shown to slow down the disease progression and improve clinical outcomes [1-3]. PR has been successfully carried out via telemedicine [2] however initial patient assessment has been traditionally conducted in PR centers. Home based patient assessments decreases the difficulties associated with travel to a PR assessment and has been shown to be effective [3-4]. The first step in PR is assessment of patient's exercise capacity which allows individualized prescription of safe and effective exercise program. With COVID-19 pandemics assessment of patients in PR centers has been limited resulting in significant reduction of patients undergoing life-saving PR. Remote assessment via telemedicine may limit the risks associated with face to face visits. Telemedicine remote assessments have been found to be effective for other 1 Corresponding Author, Joseph Finkelstein, Icahn School of Medicine at Mount Sinai, 1425 Madison Ave, Rm. L2-36, New York, NY, USA, E-mail: Joseph.Finkelstein@mssm.edu J. Finkelstein et al. / Usability of Remote Assessment of Exercise Capacity 73 conditions but have not been examined for exercise capacity evaluation [5] The goal of this pilot study is to introduce approaches for remote assessment of exercise capacity using videoconferencing platforms and provide initial usability assessment of this approach by conducing cognitive walkthrough testing. 2. Method 2.1. System Design A system for remote exercise capacity assessment has been designed to support the connection between patients at home and rehabilitation providers including physical therapists (PT) at their office. This system utilizes secure videoconferencing platforms such as zoom or webex. The remote assessment system allows carry out comprehensive physical therapy assessment necessary for prescription of a personalized exercise program tailored to individual fitness level and limitations in gait and balance of the patient under evaluation. Figure 1. System design. A system design of the remote assessment system is depicted in Figure 1. The system comprises: 1) personal computer with Zoom for PT; 2) personal computer with Zoom for a patient; 3) wrist oximeter and portable arm bike for a patient. The system supports three roles: 1) PTs who will assess patient fitness and prescribe exercises; 2) patients who will follow PT’s instruction to be assessed; 3) operation assistant (OA) who will support the patient on both instructions and techniques during the assessment. The procedure is designed with minimal requirements for the participating parties. The OA will help with setting up the vital monitoring system in patient home and help the PT and patient to communicate via Zoom meeting. The PT can give the instructions and read the heart rate and SpO2 data through the system in real time. Thus, PT can also monitor patient’s condition during the whole remote assessment via the system. 2.2. Study Design Participants were given a packet of instructions and surveys to carry out a cognitive walkthrough of the system. Surveys consisted of standardized questions with answers 74 J. Finkelstein et al. / Usability of Remote Assessment of Exercise Capacity Table 1. Tasks performed by study participants during cognitive walkthrough. Task 1: login, enter the zoom meeting, and meet the meeting participants Steps 1. Schedule and set up the Zoom meeting with both patient and physical therapist 2. Join the Zoom meeting and check the setting with operation assistant 3. Help patient to wear the wrist Oximeter and make sure the heart rate and SpO2 data can be read by physical therapist Task 2: five times sit to stand test Steps 1. Introduce the purpose of the exercise test 2. Explain the procedures of the exercise test 3. Confirm the preparation of the exercise 4. Perform sit to stand test. 5. Monitor patient’s reactions, heart rate, and SpO2 data during the exercise test. Task 3: prolong phonation test Steps 1. Introduce the purpose of the exercise test. 2. Explain the procedures of the exercise test. 3. Confirm the preparation of the exercise 4. Perform vital capacity test. 5. Monitor patient’s reactions, heart rate, and SpO2 data during the exercise test. Role OA PT OA, Patient Role PT PT OA, Patient OA, Patient PT Role PT PT OA, Patient OA, Patient PT Figure 2. System Interface. Table 2. Participant profile. Age (years) Gender Female Male Race White Asian Born in United States No Yes Job Permanent Internet use Once a day Mean (SD) 35.6 (13.3) % 40 60 40 60 80 20 100 100 % ATM use Once a month or less Once a day Never Computer use at home Once a day Computer use at work/school Once a day English proficiency (self-reported) Good Excellent Proficiency of using the internet Excellent 40 40 20 100 100 60 40 100 75 J. Finkelstein et al. / Usability of Remote Assessment of Exercise Capacity Table 3. Post-task survey Questions asked after Each Task 1. How difficult or easy was it to complete this task? 2. How satisfied are you with using this application/system to complete this task? 3. How would you rate the amount of time it took to complete this task? Score Range Sub Session 1, “Very Difficult,” to 5, “Very Easy.” Task X.1 1, “Very Unsatisfied,” to 5, “Very Satisfied.” Task X.2 1, “Too Much Time,” to 5, “Very Little Time.” Task X.3 Table 4. Results of patient testing of the remote tele-assessment system Accomplished time (sec) Exercise time (sec) Group Session Mean SD Mean SD Score Sub Session Mean SD Task1.1 4.6 0.5 Task 1 151.0 38.6 Task1.2 5.0 0.0 Task1.3 4.0 1.4 Task2.1 5.0 0.0 PT Task 2 111.0 19.0 Task2.2 4.6 0.5 Task2.3 4.6 0.9 Task3.1 5.0 0.0 Task 3 95.4 33.3 Task3.2 4.8 0.4 Task3.3 4.2 1.3 Task1.1 4.0 1.0 Task 1 253.8 44.1 Task1.2 4.0 1.0 Task1.3 3.8 0.8 Task2.1 5.0 0.0 OA Task 2 104.0 28.8 Task2.2 4.2 1.1 Task2.3 5.0 0.0 Task3.1 5.0 0.0 Task 3 100.0 27.4 Task3.2 4.2 1.1 Task3.3 5.0 0.0 Task1.1 5.0 0.0 Task 1 271.0 33.2 Task1.2 4.8 0.4 Task1.3 4.4 1.3 Task2.1 5.0 5.0 Patient Task 2 107.8 22.2 15.4 2.3 Task2.2 5.0 5.0 Task2.3 4.8 0.4 Task3.1 4.4 0.4 Task 3 95.2 32.8 23.1 13.5 Task3.2 5.0 0.0 Task3.3 4.8 0.4 PT: physical therapist, OA: operation assistant, Task Accomplished- PT: 100%, OR: 100%, Patient: 100%, Help needed- PT: 0%, OA: 0% Table 5. Exit survey and System Usability scale Items The zoom is visually appealing† The zoom is easy to navigate† System usability scale (0-100) † 1: strongly disagree – 5: strongly agree Group PT OA Patient PT OA PT OA Patient Mean SD 4.8 4.4 4.6 4.4 3.8 0.4 1.3 0.9 0.9 1.1 86.0 88.0 91.0 16.5 6.9 11.3 76 J. Finkelstein et al. / Usability of Remote Assessment of Exercise Capacity arranged as Likert-type scales and additional written responses. Participants were instructed to perform three representative tasks while being timed. If participants needed additional help to complete a task, these requests were also noted. Each cognitive walkthrough experiment consisted of three participants representing PT, patient and OA. After completing each task, each participant was asked to grade that task on a scale of 1 (very difficult) to 5 (very easy) using a 3-item survey that included the following questions: 1) How difficult or easy was it to complete this task? 2) How satisfied are you with using this application/system to complete this task? 3) How would you rate the amount of time it took to complete this task? Once all tasks were completed, the participants were given an exit survey including the System Usability Scale (SUS). Data analysis has been carried out using IBM SPSS Statistics. 3. Results The resulting user interface is depicted in Figure 2. Five cognitive walkthrough experiments have been completed by different teams including 3 participant each. Overall, 15 reports were generated and analyzed. The profiles of the 15 study participants are presented in Table 2. The usability analysis is presented in Tables 2-5. SUS scores ranged between 86 and 91 representing high usability of the system. 4. Conclusion The usability inspection of the remote exercise assessment demonstrated overall high acceptance by all study participants. Our results are congruent with previous reports demonstrating significant potential of patient-centered digital health [7]. Our next steps in developing user-centered interface should include usability evaluation in different subgroups of patients with varying socio-economic background, different age groups, computer skills, literacy and numeracy. References [1] [2] [3] [4] [5] [6] [7] Zeng X, Chen H, Ruan H, Ye X, Li J, Hong C. Effectiveness and safety of exercise training and rehabilitation in pulmonary hypertension: a systematic review and meta-analysis. J Thorac Dis. 2020;12(5):2691-705. Bedra M, McNabney M, Stiassny D, Nicholas J, Finkelstein J. Defining patient-centered characteristics of a telerehabilitation system for patients with COPD. Stud Health Technol Inform. 2013;190:24-26.. Holland AE, Mahal A, Hill CJ, Lee AL, Burge AT, Cox NS, Moore R, Nicolson C, O'Halloran P, Lahham A, Gillies R. Home-based rehabilitation for COPD using minimal resources: a randomised, controlled equivalence trial. Thorax. 2017;72(1):57-65. Lahham A, McDonald CF, Mahal A, Lee AL, Hill CJ, Burge AT, Cox NS, Moore R, Nicolson C, O’Halloran P, and Gillies R. Home-based pulmonary rehabilitation for people with COPD: A qualitative study reporting the patient perspective. Chron Respir Dis. 2018;15(2):123-30. Wood J, Wallin M, Finkelstein J. Can a low-cost webcam be used for a remote neurological exam? Stud Health Technol Inform. 2013;190:30-32. Bangor A, Kortum PT, Miller JT. An Empirical Evaluation of the System Usability Scale. International Journal of Human-Computer Interaction 2008;24(6):574-594. Finkelstein J, Jeong IC. Feasibility of interactive biking exercise system for telemanagement in elderly. Stud Health Technol Inform. 2013;192:642-646. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200698 77 Personalization Dimensions for MHealth to Improve Behavior Change: A Scoping Review Laëtitia GOSETTOa,1, Frédéric EHRLERb and Gilles FALQUET a a University of Geneva, Switzerland b University Hospitals of Geneva, Switzerland Abstract. Due to the large number of smartphone users, mHealth has become a popular support to foster users' health behavior change Personalization is an important factor to increase the effectiveness of mHealth interventions. Based on a literature review, we have listed and categorized personalization concepts associated with behavior change in mHealth into 4 dimensions, users, system functionalities, information, and app properties. The users dimension refers to user-related characteristics such as personality, player profile, need for cognition and perception of social norms. The system functionalities contain the functionalities that can be found in applications such as reminders as well as gamification functionalities such as collectibles. The information dimension concerns the way information is transmitted, such as the source of the message must be expert or the type of feedback to be provided. Finally, there are app properties such as the aesthetics of the application. For the next part, it would be interesting to discover the links we can make between the dimensions. Keywords. MHealth, Mobile, Application, Health, Behavior Change Theory, Personalization, Gamification 1. Introduction MHealth can be defined as the use of mobile computing and communication technologies in health care and public health [1]. About 79% of the European population used their smartphone in 2016 to go online [2]. Smartphones possess some features, including apps text messaging, Bluetooth, and others, that can be useful to change user behavior towards healthier ones [3]. Integrating behavior change theories (BCT) is one of the popular techniques employed in mHealth. BCT, is defined by Michie et al. as "like an observable, replicable, and irreducible component of an intervention designed to alter or redirect causal processes that regulate behavior" [4]. Researchers are therefore integrating interventions such as goal setting or self-monitoring of behavior [5]. Personalization is another mechanism that can be incorporated into mHealth interventions to promote behavioral change [6]. Personalization can be defined as the incorporation of recognizable aspects of a person into tailored content, such as a person's name [7]. The importance of personalization is already widely recognized since it is 1 Corresponding author, Laëtitia Gosetto, University of Geneva, Route de Drize 7, 1227 Carouge; Email: laetitia.gosetto@unige.ch 78 L. Gosetto et al. / Personalization Dimensions for MHealth to Improve Behavior Change found as criteria in many rating scales for mobile health applications such as the Mobile Application Rating Scale (MARS) [8], or the App Behavior Change Scale (ABACUS) [9]. Since personalization in mHealth can be applied in many ways, from simply adding the user's name to adapting the content to the user's personality, one may ask on which dimensions can we customize? The purpose of this article is to present the different dimensions of personalization of mHealth intervention to promote behavioral change based on a review of the literature. 2. Method Searches were conducted on the ScienceDirect, ResearchGate online databases for articles from 2008 to 2020. 2008 being the release of the first smartphone and thus the current mHealth. The selected articles had to treat personalization and applications for behavioral change as well as the evaluation of applications for behavior change. They also had to be written in English. The terms used for the search were: personalization, mHealth, gamification, personality, tailoring, app features, app functionality, scale app mHealth, guideline app. A first selection was made base on the reading of the titles and abstracts. Then, a second reading of the full article allowed to determine if the article met the eligibility criteria. We have then listed the personalization techniques found in these articles. We have also listed the features that appear in different scales used to assess application for behavior change. We have selected four scales, the MARS [8], the ABACUS [9], the persuasive system design [10] and the ergonomic criteria grid for the assessment of ergonomic persuasion [11]. All concepts were organized into a conceptual map helping us to regroup them into several dimensions. 3. Results 1825 articles were extracted, and 27 articles met the eligibility criteria. From the 27 articles, we were able to extract a list of concepts that are used to personalize intervention. From this list, we organized them into a conceptual map and group them into four dimensions. 3.1. Definition of Dimensions The literature presents 39 personalization concepts, ranging from the personality of the user to the characteristics of the messages. We have organized these personalization concepts into a concept map summarized in the form of a table (see Table 1). This helped us to identify several dimensions to characterize the personalization concepts. We defined 4 dimensions: user, system functionalities, information, and app properties. These dimensions are detailed below. We identified 39 concepts, 5 for the dimension user, 17 for the dimension system functionalities, 13 for the dimension information and 4 for the dimension app properties. We also indicated which concepts were present in the four scales used to assess application for behavior change. L. Gosetto et al. / Personalization Dimensions for MHealth to Improve Behavior Change 79 3.1.1. User This dimension contains all user-specific characteristics that can be used for personalization. The literature shows links between user personality and gaming characteristics [12]. Personality is measured using the Big Five [13], a model with five factors neuroticism, openness, conscientiousness, altruism, and extroversion, defining personality. The profile of the players is another user characteristic for personalization of mHealth intervention derived from gamification theory. Several scales exist to define the user’s type of player, as well as his preferences for the games. We have chosen two scales for this representation, Tondello's Hexad Scale [14] and the taxonomy of player motivation by Yee [15]. Each one defines a type of player and the type of games or interaction he prefers. We chose these scales because according to the literature, there would be a link between the type of player and the gamification features [14][15]. For example, according to the Hexad Scale, philanthropists are motivated by a goal, are altruistic and willing to give without expecting a reward. It is therefore necessary to incorporate elements of collection and exchange into the game to appeal to this type of user. [14]. Another interesting feature is the need for cognition [16]. This characteristic defines people according to their individual differences in intrinsic motivation to engage in effortful cognitive endeavors [16],[17]. It may be interesting to consider this characteristic, as for example, individuals with high need-for-cognition are more influenced by quality messages while low need-for-cognition are more influenced by peripheral cues [18]. Finally, the last characteristic we have integrated is the perception of the subjective norm. This characteristic is common to many theories of behavior change, such as the Theory of planned Behavior [19] or the Integrated Behavior Mode [20]. This characteristic refers to the perceived social pressure to perform or not to perform the behavior. As a general rule, the more the subjective norm is in agreement with the behavior, the more the individual will intend to change behavior in accordance with this subjective norm [19]. 3.1.2. System Functionalities In this dimension we included the functionalities of applications that can be personalized according to the literature. Functionalities refer to the services the application provides to the user, such as reminders or self-monitoring. Self-monitoring as “occurring when an individual first self-assesses whether or not a target behavior has occurred, and then self-records the occurrence, frequency, duration, or so on of the target behavior”[21]. We have also included gaming features that can be personalized. Such as goal setting, rewards or levels and progression. 3.1.3. Information This dimension groups together characteristics that are related to the transmission of information in an application. One part concerns the knowledge and information to be transmitted, such as the importance of relying on an expert source to provide the content, 80 L. Gosetto et al. / Personalization Dimensions for MHealth to Improve Behavior Change or to provide basic information about the desired behavior. These characteristics are extracted from different scales such as MARS [8] or ABACUS [9]. Another part concerns feedback. Feedback consists in presenting individuals with information about themselves, obtained through the application. There are 3 types of feedback, descriptive (provides only a description of the user's behavior in relation to his data), evaluative (provides an interpretation based on the user's behavior) and comparative (provides feedback comparing the user with other people). Each may be more or less effective depending on the user. For example, comparative feedback will work best for a person who needs to have a high level of social norms [22]. 3.1.4. App Properties The App properties dimension regroups features that are specific to mobile applications. In particular, it includes the aesthetic features, extracted from the MARS scale [8]. As well as one feature, customization. Customization means that “the user explicitly states interests and preferences through direct configuration of human-computer interfaces (HCI), system’s options or screens” [23]. Table 1. Representation of the dimension to personalize mHealth Users Personality (e.g BigFive) [12][13] Gamer Profil (e.g Hexad scale) [14][15] System Functionalities Information App Properties App Functionalities (e.g reminder, selfmonitoring)** [21] Knowledge and information (e.g. expert source, quantity of information) **** [8][9] Aesthectics (Layout, visual appeal, graphics)* [8] Feedbacks (evaluative, descriptive, comparative)* [22] App Features (customizable ****) Gamification Features (e.g Rewards, cooperation)**** [12][14][15] [23] Need-for-cognition [16][17][18] Perception of the social norm [19][20] *present in one scale; **present in two scales; ***present in three scales; ****present in four scales 4. Discussion From the literature, we have identified and classified personalization concepts into 4 dimensions, users, system functionalities, information, and app properties. From this classification, we can identify on which characteristics it is possible to personalize. For example, what kind of feedback to provide for each user etc... It would be interesting as a future research to study the notion of design, such as design with empathy or emotional design. In particular, emotional design has the potential to bring personality to the application in order to make it more attractive to the user. It would be also interesting to explore the relationship between the characteristics belonging to different dimensions. For example, what kind of gamification features are favored by people who fit the big-five's extroversion profile. In this way, one could personalize features of the application according to the user's personality. It would also L. Gosetto et al. / Personalization Dimensions for MHealth to Improve Behavior Change 81 be interesting to define how to obtain information about the user in order to personalize according to the other dimensions. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] Free C, Phillips G, Felix L, and others. The effectiveness of M-health technologies for improving health and health services: a systematic review protocol. BMC Res Notes. 2010;3(250):1–7. Eurostat. 2016 [cited 2020 Sept 17]. Available from: https://ec.europa.eu/eurostat/documents/2995521/7771149/9-20122016-BP-FR.pdf/7e0f0705-cabd4a1c-b57d-f893c5093947 WHO. World Health Organization - mHealth New horizons for health through mobile technologies. 2011. [cited 2020 Sept 17]. Available from: https://www.who.int/goe/publications/goe_mhealth_web.pdf Michie S, Richardson M, Johnston M, and others. The Behavior Change Technique Taxonomy (v1) of 93 Hierarchically Clustered Techniques: Building an International Consensus for the Reporting of Behavior Change Interventions. Ann Behav Med. 2013;46:81–95. Direito A, Pfaeffli Dale L, Shields E, and others. Do physical activity and dietary smartphone applications incorporate evidence-based behaviour change techniques? BMC Public Health. 2014 Jun;14(1):1–7. Wanyonyi KL, Themessl-Huber M, Humphris G, Freeman R. A systematic review and meta-analysis of face-to-face communication of tailored health messages: Implications for practice. Patient Educ Couns. 2011;85(3):348–55. Dijkstra A. The Psychology of Tailoring-Ingredients in Computer-Tailored Persuasion. Soc Personal Psychol Compass. 2008;2(2):765–84. Stoyanov SR, Hides L, Kavanagh DJ, and others. Mobile App Rating Scale: A New Tool for Assessing the Quality of Health Mobile Apps. JMIR mHealth uHealth. 2015 Mar;3(1):1-10. McKay FH, Slykerman S, Dunn M. The App Behavior Change Scale: Creation of a Scale to Assess the Potential of Apps to Promote Behavior Change. JMIR mHealth uHealth. 2019;7(1):1–18. Wang Y, Eng M, Fadhil A, Lange J-P, Reiterer H. Integrating Taxonomies Into Theory-Based Digital Health Interventions for Behavior Change: A Holistic Framework. JMIR Res Protoc. 2019;8(1):1-11. Nemery A, Brangier E, Kopp S. Proposition d’une grille de critères d’analyse ergono-miques des formes de persuasion interactive. Proceedings of the IHM’2010 ; 2010 ; Luxembourg. 153–6 p. Orji R, Nacke LE, Marco C Di. Towards Personality-driven Persuasive Health Games and Gamified Systems. Proceedings of the CHI. Denver; 2017. 1015–27. Mccrae RR, Costa PT. Validation of the Five-Factor Model of Personality Across Instruments and Observers. Journal of Personality and Social Psychology. 1987;52(1):81–90. Tondello GF, Wehbe RR, Diamond L, and others. The Gamification User Types Hexad Scale. Proceeding in CHI Play’16; Austin; 2016. 229-243p. Yee N. Motivations for Play in Online Games. Cyberpsychology & Behavior. 2006;9:772-775. Cacioppo JT, Petty RE. The Need for Cognition. J Pers Soc Psychol. 1982;42(1):116–31. Cacioppo JT, Petty RE, Feng Kao C. The Efficient Assessment of Need for Cognition. J Pers Assess. 1984; 48(3):306–7. Axsom D, Yates S, Chaiken S. Audience response as a heuristic cue in persuasion. Journal of Personality and Social Psychology. US: American Psychological Association. 1987; 53:30–40. Ajzen I. The Theory of Planned Behavior. Organizational Behavior And Human Decision Processes. 1991;50:179-211. Fishbein M, Yzer MC. Using Theory to Design Effective Health Behavior Interventions. 2003;164–83. Nelson RO, Hayes SC. Theoretical Explanations for Reactivity in Self-Monitoring. Behav Modif. 1981 Jan;5(1):3–14. Hawkins RP, Kreuter M, Resnicow K, Fishbein M, Dijkstra A. Understanding tailoring in communicating about health. Health Educ Res. 2008;23(3):454–66. Madeira RN, Germano H, Macedo P, Correia N. Personalising the user experience of a mobile health application towards patient engagement. Procedia Computer Science. 2018;141: 428–33. 82 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200699 Mobile Access and Adoption of the Swedish National Patient Portal a Maria HÄGGLUNDa,1, Charlotte BLEASEb and Isabella SCANDURRAc Department of Women’s and Children’s Health, Uppsala university, Uppsala, Sweden b Harvard Medical School and Beth Israel Deaconess Medical Center, Boston, USA c Informatics, School of Business, Örebro University, Örebro, Sweden Abstract. Patient portals are used as a means to facilitate communication, performing administrative tasks, or accessing one’s health record. In a retrospective analysis of real-world data from the Swedish National Patient Portal 1177.se, we describe the rate of adoption over time, as well as how patterns of device usage have changed over time. In Jan 2013, 53% of all visits were made from a computer, and 38% from a mobile phone. By June 2020, 77% of all visits were made from a mobile phone and only 20% from a computer. These results underline the importance of designing responsive patient portals that allow patients to use any device without losing functionality or usability. Keywords. Patient portals, adoption, mobile access, 1. Introduction Patient portals are used to facilitate communication between patients and healthcare professionals, as well as for performing administrative tasks, such as appointment bookings and prescription renewals [1]. Patients are also increasingly provided with access to their electronic health records (EHRs) [2], sometimes referred to as patient accessible EHRs (PAEHRs) [3][4] or open notes [5], through portals. Patient portals have been widely implemented, yet adoption often remains low [1]. In Sweden, a national patient portal is used that connects to all EHR systems used in the 21 regions (who are responsible for providing healthcare) [4], through a national health information exchange platform [6][7]. Authentication with a national e-ID gives access to a number of administrative services as well as the PAEHR Journalen. Although the regions are autonomous and can prioritize which eHealth services to focus on, the national eHealth strategy stipulates that there should be only one online healthcare access point for patients [8]. Thus, a national patient portal ‘1177.se’ is available for everyone seeking healthcare or health-related information in Sweden, consisting of three parts; 1. 1177 on the phone - a telephone advice service reached through the national phone number 1177, 2. 1177.se on the web - without authentication the public can access and search among information about illnesses, symptoms and treatments, as well as information about healthcare in the region. The portal is national, but each 1 Corresponding Author, Maria Hägglund, Department of Women’s and Children’s Health, Uppsala University, Dag Hammarskjölds väg 14B, 752 37 Uppsala, Sweden. Email: maria.hagglund@kbh.uu.se. M. Hägglund et al. / Mobile Access and Adoption of the Swedish National Patient Portal 83 region in Sweden can adapt the information to its inhabitants, and users can switch between regions, 3. 1177.se personal eHealth services – after authentication (using a nationally approved e-ID) individuals have access to personalized e-services where they can e.g. send secure messages, request, reschedule or cancel appointments, renew prescriptions and access documents such as sick-leave. Functionality can vary based on region/healthcare provider. Similar national patient portals are implemented throughout the Nordic countries, and healthcare provider specific portals are common beyond the Nordic context. Yet, implementation does not equal adoption, and in order to reap the expected benefits, we have to increase our knowledge about how patients access and use patient portals in order to better adapt them to users’ needs. A key issue in this is what devices patients prefer to use. The aim of this paper is therefore to analyze and describe how patients access the Swedish national patient portal and how adoption has evolved over time. 2. Methods A retrospective analysis of real world data on the use and adoption of the National Patient Portal has been performed. De-identified and aggregated usage data from the National Patient Portal and associated eHealth services are provided online by Inera AB [9], and is used for this study. Adoption is assessed by analyzing the number of visits to the National Patient Portal from Jan 2013 until June 2020, as this is the data that is provided as open access. The open pages of the National Patient Portal (1177 on the web) does not require log-in, and therefore it is not possible to keep track of demographic data on the users. Therefore we also include data on the number of users of the personal eHealth services on 1177.se, although it may not be representative to all users of 1177.se. Data on what device (computer, mobile phone, or tablet) a patient use to access the open pages of 1177.se is also presented, to highlight how access to the National Patient Portal has evolved over time from 2013 to 2020. This is closely related to Internet access and usage among the Swedish population overall. 2.1. Internet use among the Swedish population Internet usage is high in Sweden, and according to the most recent survey “the Swedes and the Internet” from the Swedish Internet Foundation, 95% of the Swedish population use the internet and 91% uses the internet on a daily basis [10]. Both computers and mobile phones are frequently used to access the internet (91% and 90% respectively). 53% of the respondents also use digital services to manage their healthcare, e.g. book appointments, check lab results etc. 3. Results Late 2019, Sweden had approximately 10.3 million inhabitants, compared to 9.6 million in 2013, an increase with 7%. The open pages of the national patient portal 1177 had 3 210 189 visits in January 2013, whereas the monthly number of visits in January 2020 84 M. Hägglund et al. / Mobile Access and Adoption of the Swedish National Patient Portal was 13 325 793 (an increase of over 400%). Figure 1 shows the development of monthly visits to 1177 from Jan 2013 to June 2020. Dips in the curve occurs during the summer months (June-Aug) every year, whereas visits peak in Jan-March, coinciding with the Swedish flu season. In 2020, the COVID-19 pandemic created an outstanding peak of visits in March with 18 268 469 visits. Figure 1. Number of monthly visits to 1177.se January 2013 – June 2020. The personal eHealth services provided on the National Patient Portal require for an individual to sign in to their account. Therefore, it is also possible to keep track of how many individuals access these services. In June 2020, 7 230 063 persons had accessed their personal eHealth services at least once. These services have also seen a dramatic increase in use; in Jan 2013 a total of 29 228 logins were made to the portal, made by 17 572 unique users, and the corresponding numbers in June 2020 was 7 189 878 log ins made by 1 724 735 unique users. Number of log-ins follows a similar pattern of dropping in the summer months and increasing in winter, however 2020 stands out with an increase in log-ins during June. This is likely due to an increase in COVID-19 testing (both for the virus and for antibodies), where patients use the national patient portal to both book appointments and to access their test results. In addition to the increase in visits to the National Patient Portal, a possibly more striking change has occurred in how people access the website. In January 2013, 1 716 175 visits were on a computer (53% of all visits), 1 210 561 from a mobile phone (38%) and 278 635 from a tablet (9%) (Figure 2). In June 2020, mobile phones dominates as the most common device for visiting the National Patient Portal (Figure 2); computers 2 805 176 (20%), mobile phones 11 001 189 (77%), and tablets 468 227 (3%). This is however not due to a decrease in visits from computers and tablets, since the number of visits from these devices has increased over time; rather, the proportion of mobile phone visits has increased greatly during this period. M. Hägglund et al. / Mobile Access and Adoption of the Swedish National Patient Portal 85 Figure 2. Devices used to access 1177.se January 2013 and June 2020 respectively. The increase in visits from mobile phones is perhaps even clearer if we look at a graph showing development over time (Figure 3). Where access from a computer or tablet remains on a slow increase, the visits from mobile phones starts to increase more rapidly in 2014 and even further in 2018. Figure 3. Monthly visits to 1177.se from different devices Jan 2013 - June 2020. 4. Discussion and Conclusions Adoption takes time. In this analysis, we have looked at real-world usage data captured over 7 years of use of the Swedish National Patient Portal. From a modest start, the numbers have steadily increased every year. A recurring pattern of high usage during the winter months and lower during summer can also be seen. Adoption does not seem to have reached a plateau yet in Sweden, rather the 2020 COVID-19 pandemic seem to have further increased use of both the open and the personal eHealth services on the platform. This increase may be temporary, but it could also indicate that new user groups are finding their way to the National Patient Platform, and it will be of interest to follow-up over time. 86 M. Hägglund et al. / Mobile Access and Adoption of the Swedish National Patient Portal As indicated by the data presented in this analysis, mobile phones have become the far most common way to access the National Patient Platform. Still, computers make up for 20% of the visits, and it remains unknown whether users choose their device depending on which type of information or task they are looking to perform. Responsive design is key to ensure that different devices can be used based on the users’ needs or preferences in different contexts and for different purposes. Further research is needed to deepen our understanding of when users choose a specific device over another; what are their preferences for usage when they go online e.g. accessing electronic health records, and how does mobile access affect adoption and use of eHealth services? Another area that is important to explore further relates to socio-economic differences in accessing the National Patient Portal. Does the increase in mobile phone access correspond to new populations gaining access to the portal? Similarly, do patterns of mobile access correlate with a lack of broadband access among some patient populations? Might limitations associated with mobile phone data restrict frequency of portal usage? The analysis presented in this paper provides a starting point to continue exploring further questions relating to patient portal adoption and use. Acknowledgement FORTE – the Swedish Research Council for Health, Working Life and Welfare funds the research project “PACESS” (2016-00623). This research was also supported by Uppsala MedTech Science & Innovation (https://medtech.uu.se/), a joint strategic initiative between Uppsala University and Uppsala University Hospital. References [1] Fraccaro P, Vigo M, Balatsoukas P, Buchan IE, Peek N, van der Veer SN. Patient Portal Adoption Rates: A Systematic Literature Review and Meta-Analysis. Stud. Health Technol. Inform. 2017;245:79–83. [2] Hägglund M, DesRoches C, Petersen C, Scandurra I. Patients’ access to health records. BMJ. 2019 Oct; 367:l5725. [3] Moll J, Rexhepi H, Cajander Å, Grünloh C, Huvila I, Hägglund M, Myreteg G, Scandurra I, Åhlfeldt RM. Patients’ Experiences of Accessing Their Electronic Health Records: National Patient Survey in Sweden. J Med Internet Res. 2018; 20(11): e278. [4] Hägglund M, Scandurra I. Patients’ Online Access to Electronic Health Records: Current Status and Experiences from the Implementation in Sweden. Stud Health Technol Inf. 2017;245: 723–727. [5] Walker J, Leveille S, Bell S, Chimowitz H, Dong Z, Elmore JG, Fernandez L, Fossa A, Gerard M, Fitzgerald P, Harcourt K, Jackson S, Payne TH, Perez J, Shucard H, Stametz R, DesRoches C, Delbanco T. OpenNotes After 7 Years: Patient Experiences With Ongoing Access to Their Clinicians’ Outpatient Visit Notes. J Med Internet Res. 2019; 21(5):e13876. [6] Davoody N, Koch S, Krakau I, Hägglund M. Accessing and sharing health information for postdischarge stroke care through a national health information exchange platform - a case study. BMC Med. Inform. Decis. Mak. 2019; 19(1):95. [7] Sellberg N, Eltes J. The Swedish Patient Portal and Its Relation to the National Reference Architecture and the Overall eHealth Infrastructure, in Information Infrastructures within European Health Care. Springer, 2017, pp. 225–244. [8] Ministry of Health & Social Affairs., National eHealth – the Strategy for Accessible and Secure Information in Health and Social Care, 2010. [9] Inera, “Inera Statistics.” [Online]. Available: https://www.inera.se/aktuellt/statistik/ . [10] Internetstiftelsen, “Svenskarna och internet 2019,” 2019. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200700 87 Chronic Disease Self Management Using a Social Networking PHR/UHR Jeremy S. KAGAN1a Corporate, CardioLync, Israel and USA a Abstract. This viewpoint paper presents a potential solution to the “information islands” that are holding back PHR/UHR from becoming truly effective diagnostic information care management tools for patients especially those who suffer from chronic diseases. The solution involves integrating patient portal with a diagnostic data interface layer to create a single access point for caregivers and patients. Keywords. Digital Health, PHR, UHR, PACS, EHR, Diagnostic Images, Diagnostic Data, Interpretive Reports, Patient Portal 1. Introduction Aging populations, increased prevalence of chronic conditions, and commensurate rising costs significantly challenge health care systems worldwide. One proposed solution to these challenges has been health information technologies (ITs) that empower patients to be partners in their care, and that support evidence-based individualized care. But there is a tendency to downplay the complexity of implementation and user adoption [1]. Patients who suffer from chronic disease often have multiple concurrent chronic conditions and complications that require regular visits with a number of different specialists in addition to their primary care physician (PCP). They also may have intermittent interactions with emergency rooms and other care settings. This puts them at increased risk for severe adverse events if information does not flow between care settings timely and accurately. Personal Health Records (PHR) together with Universal Health Records (UHR) can mitigate this problem. PHR Allows patient to maintain all their care information as well as uploaded data from personal monitoring devices, health smartphone apps, and smart home devices like medical toilets, and daily diaries of their wellbeing. UHR is Patient-centered information available to and often controlled by patients, that contains all health care information and history. A PHR combined with a UHR would drastically reduce the risk that care-providers may overlook pertinent information. Despite massive effort and investment in health information systems and technology, and many years of widespread availability, the reality is that most physicians still have to fax and mail patient records the way they did a decade ago [2]. A fragmented system of storing and retrieving essential patient data impedes optimal care. If they cannot exchange data with other health care systems, PHRs will become “information islands” that contain subsets of patients' data, isolated from other information about patients, with limited access and transient value [3]. When PHRs are 1 Corresponding author, Jeremy S. Kagan, email: jeremy.kagan@cardiolync.com 88 J.S. Kagan / Chronic Disease Self Management Using a Social Networking PHR/UHR integrated with EHR systems, they provide greater benefits than would stand-alone systems for consumers. This viewpoint paper presents an innovative solution to the “information islands” for combined PHR/UHR to become a truly effective care management tool for patients. The solution involves integrating patient portal with a cross-modality diagnostic data interface layer to create a single integrated access point for caregivers and patients. 2. Methods - Integrated Access Point for Care-Givers and Patients Effective communication and coordination among doctors, specialists and other caregivers could mean the difference between life and death for patients. A crosssystem data interface layer can integrate images and data across diagnostic modalities, and thus simplify the process of accessing information by doctors for medical interpretations, reporting and treatment recommendations, and aggregate this information for patients. Diagnostic interface layer technology can help care providers improve patient outcomes by facilitating initial risk stratification and remote consults with experts, thereby reducing admissions and readmissions [4]. Since the adoption of PACS in the late nineties and early 2000s, imaging exams have been stored digitally. There is a misconception that digital imaging helped medical image exchange eliminate any potential image loss. Unfortunately, this is not the case: despite the most recent advances in digital imaging, most hospitals still often lose their imaging data, with these losses going completely unnoticed. As a result, not only does image loss affect the faith in digital imaging but it also affects patient diagnosis and daily quality of clinical work [5]. Furthermore, when patients travel from provider to provider, image availability at the point of care can be a problem. Patients may receive CDs with copies of their imaging studies, but there is risk that this fragile media could be damaged and become unreadable. CDs also can be easily misplaced or lost by the patient, and some patients simply forget to bring their CDs to the appointment. At the receiving end, images are sometimes viewed using the viewer on the CD, but more commonly copied from the CD, updated with local patient and order information, and loaded into the local PACS. This requires significant manual local effort. Electronic exchange provides opportunities for improved operational workflows that can positively impact patient care, reduce cost, improve patient and clinician satisfaction, and can even increase revenue opportunities in key service lines. CDbased image exchange has laid an important foundation to support the emerging next generation of interoperable, standards-based image exchange [6]. A data interface layer can support standardized integration of clinical technologies using existing industry standards to facilitate automatic upload of images and data. Using DICOM file exports from all modalities such as ECG/EKG, ECHO, ultrasound, CT, PET, MRI, nuclear imaging, it is possible to interface directly to the diagnostic machine or to the PACS for the diagnostic data, and to the EHR system with interpretation results in HL7. A data interface layer can also accept data from Holter monitors, mobile telemetry apps, home patient follow-up visit reports for monitoring and evaluation. It allows for data from patient monitoring devices, smartphone apps, and future tools such as Artificial Intelligence algorithms. By converting files of images, clips, and data from DICOM or other modality specific formats into web friendly jpeg, mp4, and PDF formats, a data interface layer can provide single point J.S. Kagan / Chronic Disease Self Management Using a Social Networking PHR/UHR 89 access from any computer without installation of special client software like a DICOM reader or modality specific application, and can sit atop PACS systems for messaging and reporting via the web. 3. Results - Social Networking Paradigm Social networking may be the answer. It is a proven communication and coordination model that can be applied to facilitate interdisciplinary and cross-institution imaging interpretation and expert consults on a UHR. Social networking can be implemented via a web-based interface portal to link doctors, diagnostic imaging techs, experts and patients for remote diagnosis; and to speed access to images, data, and history; which ultimately cuts the time to the correct treatment plan. The social networking communication and information sharing medium has the power to revolutionize the way physicians interact with their patients and fellow health care workers. When managed correctly, it can provide a great way for doctors to communicate and educate others. For example, a surgeon used Twitter during a robotically assisted partial nephrectomy to let other surgeons know that a total nephrectomy was not necessary, despite the large tumor size [7]. A patient portal that creates a secure social network of physicians with patients is a clinical tool that allows a primary care physician to retain interaction with the patient and seek specialty consult when they jointly determine the need together. This empowers patients to take a more active role in their treatment and saves money. Online shared access to critical data is secured by pre-authorized user restrictions and permissions. Single access to all diagnostic and history event data facilitates remote interpretation and reporting by specialists. Using an interface layer, diagnostic tests can be administered locally by a tech, and then be interpreted by a specialist who may be sitting remotely. Pre-screening evaluation can determine if the patient needs to travel for a physical exam or procedure, thus reducing referrals and hospital admissions. 4. Discussion In addition to access, the integrated patient portal and data interface layer can provide a private messaging platform for secure communication among doctors, experts, technicians and patients. Table 1 shows the main advantages of the integrated interface portal compared to the current state of the art, as observed by the author in clinical use of an implementation of the integrated interface portal. Table 1. Observed Advantages of the Integrated Interface Portal model Integrated Interface Portal Pitfall that it Addresses What it Provides Social Networking Model with History Communication Access Population Coverage Improves care coordination among doctors and other experts to cut decision time and drives engagement with patients via access to history and recommendations at point of care Workflow Speeds the workflow for the Interpreting Physician Structured Reporting 90 J.S. Kagan / Chronic Disease Self Management Using a Social Networking PHR/UHR Single Access for data and images Interoperability Physicians have access to review studies Patient Portal Patient Engagement Encourage consults and access in emergencies Private caregiver Messaging Coordination Secure communication by and between doctors, specialists, and technicians for clarifications 4.1. A Model that Keeps the Patient in Center Focus The integrated interface portal consolidates information- data, measurement, history, and images; and facilitates sharing of that information among caregivers and patients at the point of care and remotely. The social networking model places the patient squarely in control of their condition with their individual timeline of medical events. The patient portal functionality gives patients access to their history and reports so that they can seek other expert advice as well as never be caught short in an emergency. The level of information that patients can upload themselves or access can be limited by their primary care provider or other entity that manages their diagnostic data. As far as the patient’s caregivers are concerned, questions and answers between the doctors and specialists can remain private and not available to the patient. In cases where a primary care physician is advised to have a psychologist present when presenting an expert’s findings and opinions, they can limit patient access to these reports until after the primary care physician discusses them directly with the patient. In addition, with the right permission settings, an integrated interface portal can best fulfill the role of a UHR, as it is patient-centered, and the information available can be controlled. Similarly, as the portal interfaces to diagnostic modalities like ultrasound, nuclear imaging, and PACS systems; technically the portal can just as effectively interface with other portals of the patient’s various health care providers. If there is an operational or regulatory mandate available that allows the sharing of PHI (private health information) by and between healthcare organizations, as per a patient’s permission, then all of the patient’s data and history can be accessed via the portal. This would eliminate the major shortcoming of most patient portals today, namely, data access across multiple organizations and institutions. With an integrated interface portal, patients and caregivers will have a non-fragmented view of patient data. Similarly, the interface portal model can facilitate a patient’s PHR, by allowing patients to maintain all of their care information as well as upload data from personal monitoring devices, smartphone apps, smart home devices, and daily wellness diaries. PHRs can be set so that if any of the patient/home device generated data exceeds a certain pre-defined threshold, the patient’s primary care physicians can be automatically notified. This would make it easier for caregivers to maintain continuous communication with patients, not just episodic, and to track chronic conditions and illnesses and post-discharge follow-up so that they could enact early interventions. Technology takes time to be deployed in hospitals and healthcare networks, and introducing a new technology will require clearly defined and demonstrable added value. An interface portal offers such added value. PACS system vendors continually expand the features and use cases of their systems, but the applications were built to meet the needs of specific specialties and disciplines which restrict them. An integrated interface portal on the other hand introduces a layer that sits atop of and connects all PACS, EMR, and native diagnostic modalities for requisite access and/or sharing. J.S. Kagan / Chronic Disease Self Management Using a Social Networking PHR/UHR 91 Organizing patient information around a single access point enables specialists to remain focused on analyzing the data and translating it into an appropriate plan of care. This is a vast improvement over current inefficiencies whereby doctors log in to multiple systems to interpret various test results, which could lead to frustration and possible distraction from their primary focus on the patient’s plan of care. The social networking aspect improves patient care by connecting the principal people involved with the necessary and relevant information. It consolidates patient medical diagnostic tests, ongoing monitoring, and event data from multiple sources and enables a consolidated visualization in a timeline shared with patient and caregivers. Doctors can interpret native diagnostic images, and encourage patients to securely access and update their medical histories when they experience health related events. The author has led a team that implemented this model in a technology which is in clinical use. Each type of user: doctor, technician, patient has specific permissions for what they are able to access, view, upload and download. 5. Conclusions An integrated interface portal presents patient health events, diagnostic data and images; and creates social networking interaction between patients and physicians for communication, questions and answers about a patient’s specific chronic condition and history. Primary care physicians and patients alike can use the interface portal alike to trigger and facilitate teleconsults with specialists and subspecialists. The integrated interface portal allows specialists who may be located remotely to access all relevant medical material including patient history, data and images and have these data before their eyes in an integrative and systemic manner; while engaging patients with controlled but direct access. This improves joint decision-making and encourages patients to more effectively self manage their chronic conditions. References [1] [2] [3] [4] [5] [6] [7] Avdagovska M, Stafinski T, Ballermann M, Menon D, Olson K, Paul P. Tracing the Decisions That Shaped the Development of MyChart, an Electronic Patient Portal in Alberta, Canada: Historical Research Study. Journal of medical Internet research. 2020; 22(5): e17505. Reisman M. EHRs: The Challenge of Making Electronic Data Usable and Interoperable. P & T : a peerreviewed journal for formulary management. 2017; 42(9): 572–575. Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. Journal of the American Medical Informatics Association JAMIA. 2006; 13(2): 121–126. https://doi.org/10.1197/jamia.M2025 Kagan JS. Doctor-Patient Social Networking to Improve Specialist Care Coordination. In McHaney R W, Reychev I, Azuri J, McHaney ME, & Moshonov R. (Eds.) Impacts of Information Technology on Patient Care and Empowerment (pp. 267-289). 2020 IGI Global. http://doi:10.4018/978-1-7998-00477.ch014 Oglevee C, Pianykh O. Losing Images in Digital Radiology: More than You Think. J Digit Imaging 2015; 28: 264–271. https://doi.org/10.1007/s10278-014-9748-2 Vreeland A, Persons KR, Primo HR, Bishop M, Garriott KM, Doyle MK, et al. Considerations for exchanging and sharing medical images for improved collaboration and patient care: HIMSS-SIIM collaborative white paper. Journal of Digital Imaging. 2016; 29(5): 547-558. Prasanna PM, Seagull FJ, Nagy P. Online social networking: a primer for radiology. Journal of Digital Imaging. 2011; 24(5): 908-912. 92 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200701 A Secure Protocol for Managing and Sharing Personal Healthcare Data Athanasios KIOURTISa,1, Argyro MAVROGIORGOUa, Sofia-Anna MENESIDOUb, Panagiotis GOUVASb and Dimosthenis KYRIAZISa a Department of Digital Systems, University of Piraeus, Piraeus, Greece b Ubitech Ltd, Athens, Greece Abstract. Current technologies provide the ability to healthcare practitioners and citizens, to share and analyse healthcare information, thus improving the patient care quality. Nevertheless, European Union (EU) citizens have very limited control over their own health data, despite that several countries are using national or regional Electronic Health Records (EHRs) for realizing virtual or centralized national repositories of citizens’ health records. Health Information Exchange (HIE) can greatly improve the completeness of patients’ records. However, most of the current researches deal with exchanging health information among healthcare organizations, without giving the ability to the citizens on accessing, managing or exchanging healthcare data with healthcare organizations and thus being able to handle their own data, mainly due to lack of standardization and security protocols. Towards this challenge, in this paper a secure Device-to-Device (D2D) protocol is specified that can be used by software applications, aiming on facilitating the exchange of health data among citizens and healthcare professionals, on top of Bluetooth technologies. Keywords. Device-to-Device protocol, Health Information Exchange, HL7 FHIR 1. Introduction The current medical world is surrounded by healthcare information stored either locally (on each device) or remotely (on computer clouds) – among others, with the overall purpose of being exchanged among authorized people who can gain value from it [1]. The exchange of this data can be performed through multiple ways (wired, wireless, physical documents), at various distances, achieving different goals in terms of transmission rate, security, or platform applicability. In the electronic healthcare domain, the exchange of information between citizens - patients and healthcare practitioners (HCPs), is characterized of great importance, since a medical condition and solution can be found much faster, while the overall life quality can be improved. Currently, European Union (EU) citizens have very limited control over their own health data, despite the fact that several countries are using national or regional Electronic Health Records (EHRs) for realizing virtual or centralized national repositories of citizens’ health records. Among others, what is missing is to complement and integrate the current interoperability infrastructures with new technologies for health data exchange that is 1 Corresponding Author, Athanasios Kiourtis, 80, M. Karaoli & A. Dimitriou St., 18534 Piraeus, Greece; E-mail: kiourtis@unipi.gr. A. Kiourtis et al. / A Secure Protocol for Managing and Sharing Personal Healthcare Data 93 centered on the citizen, which does not demand the coordination by a superior authority, thus leaving more control of the health data to its owner. While there are specific cases (e.g. authorization to buy a medication abroad) that demand coordination among government institutions, there are many cases (e.g. a medical visit abroad) that do not, and for which the exchange of personal health data could be better handled directly by the citizen. At present, citizens often carry their own paper medical documents for such medical visits. However, it would be more effective if citizens could carry or access their data in a digital form. This paper addresses the current lack of standardization and security, by presenting a set of integrated protocols, supporting secure data exchange and portable local storage, released as open specifications in order to perform short-range distance Health Information Exchange (HIE) [2] among the different stakeholders. Hence, a secure Device-to-Device (D2D) protocol is specified, based on small-scale wireless technologies and in particular Bluetooth technologies [3], with the overall goal to be adopted at a pan-European level for the safe exchange of medical records between a smart mobile device and a health information system. The remainder of this paper is organized as follows. In Section 2, the methodology followed to specify the D2D protocol is being provided, while Section 3 depicts the overall evaluation results of the proposed D2D protocol. Section 4 includes a short discussion of the derived results, presenting our concluding remarks. 2. Methods In order to conclude to the usage of the Bluetooth short range wireless communication technology in the D2D protocol, an exhaustive research took place among the most widely used short range distance communication protocols. The top-four candidates [4] for the D2D protocol were the Wi-Fi direct, Bluetooth v4.0, Bluetooth Low Energy (BLE), and Near Field Communication (NFC). In this context, since it was within our plans to exchange large files of healthcare data (e.g. medical images), for BLE and NFC we concluded that they should not be included due to their low data rates. As a result, Wi-Fi direct and Bluetooth v4.0 were the top-two choices. In that case, Wi-Fi direct had 10 times better data rate than Bluetooth, but since Wi-Fi direct supports unidirectional communication instead of the Bluetooth that supports bidirectional communication, we concluded that Bluetooth would be the best option for our needs (Table 1). Table 1. Short-range wireless communication technologies comparison. Criterion Range Data Rate Security Power Consum Communication Wi-Fi Direct Up to 180m Up to 250 Mbps High High Unidirectional Bluetooth v4.0 Up to 100m Up to 25 Mbps High Medium Bidirectional BLE Up to 10m Up to 200 kbps High Low Bidirectional NFC Up to 4cm Up to 424 kbps Medium Low Unidirectional Regarding the D2D protocol, it can be best described as a series of different Bluetooth messages that contain the information that is being exchanged, in terms of healthcare related data, between an HCP and a citizen, without using internet connection. Before continuing the description of the D2D protocol, the following terms should be identified: “medical application of a citizen (smart Electronic Health Record application (S-EHR-app))” and “application of medical staff (Healthcare Practitioner application (HCP-app))”. A S-EHR-app is any application installed on a personal mobile device that 94 A. Kiourtis et al. / A Secure Protocol for Managing and Sharing Personal Healthcare Data can store a user’s personal health data securely (encrypted). Such an application may contain user health information generated and signed by the healthcare provider, but may also contain data stored and produced directly by citizens or sensors (e.g. smartwatches). An HCP-app is a software application designed to provide medical staff with access to and use of patient data from a S-EHR-app, with the goal of securely exchanging health data with any S-EHR-app, using different protocols. The overall specification of the D2D protocol is based on exchanging HL7 FHIR healthcare data [5], following the steps of Fig. 1, that occur between the citizen and the HCP, using their S-EHR-app and HCP-app. Figure 1. Data exchange phases of the D2D protocol. In deeper detail, the steps of the D2D protocol have been categorized into five main phases, based on the different functionalities that they offer: (i) in the Connection phase, the HCP-app gets the advertised connection request from the side of the S-EHR-app so as to initiate the Bluetooth connection, (ii) in the Demographic Data Exchange phase, as soon as the connection has been done, the S-EHR-app gets the Healthcare Organization identity in order to identify the HCP and the Healthcare Organization. After that, in the case that the citizen approves the Healthcare Organization identity, the HCP-app gets the Personal Identity data from the side of the citizen, in order for the HCP to identify the citizen. Again, in the case that the HCP approves the Personal Identity data of the citizen, the S-EHR-app receives a consent request from the side of the HCP-app, in order for the HCP-app to have access to the citizens’ healthcare data, (iii) in the Consent exchange phase, it is included the approval of the consent from the side of the S-EHR-app, and as a result the exchange of Healthcare Data. Hence, if the S-EHR-app approves the consent, then the HCP-app receives as a reply the requested healthcare data. On the contrary, if the consent request is not approved, the connection terminates, (iv) in the Data Exchange phase, as described before, the reply to the consent request is the healthcare related data from the side of the S-EHR-app. Hence, the HCP examines this data, and sends back to the citizen her consultation results, (v) in the Connection Closure phase, the S-EHR-app, as soon as the consultation results’ data has been received, sends to the HCP-app a specific connection closure message, in order for the Bluetooth connection to terminate. A. Kiourtis et al. / A Secure Protocol for Managing and Sharing Personal Healthcare Data 95 On top of the D2D protocol phases, a security protocol has been specified for performing encryption in transit, consisting of five phases, towards establishing an encrypted communication channel (Fig. 2). More particularly, the existing phases are as follows: (i) in the Bootstrap phase, the prerequisites regarding certificate acquisition on both entities are performed, (ii) in the Identity Management (IDM) phase, each entity verifies the identity of the other entity by certificate exchange and signature verification, (iii) in the Consent Management phase, the citizen gives her consent for process upon her data, (iv) in the Key Establishment phase, a symmetric key establishment happens for secure communication, while (v) in the Encrypted Communication phase, both parties use the established symmetric key to transfer data in an encrypted form. Figure 2. Security phases of the D2D protocol. 3. Results In order to evaluate the proposed overall protocol, two applications were created in Java for Android and Windows, using Android Studio and NetBeans accordingly, for sending and receiving healthcare data based on the described flow. The scenario on top of which we have based in order to specify the D2D protocol and implement its functionality, was the one of a medical visit abroad. Shortly, in this scenario it is assumed that an Italian citizen who already has her data stored on her S-EHR-app, is visiting an HCP in Greece. Hence, these two parties have to connect to each other, identify themselves, and finally exchange healthcare related data. The overall flow of Fig. 1 was followed, providing us with the expected results. Fig. 3 displays a few screenshots of the developed applications that confirm the functionality of the protocol on top of the medical visit scenario, 96 A. Kiourtis et al. / A Secure Protocol for Managing and Sharing Personal Healthcare Data showcasing the scanning of a Quick-Read (QR) code for performing the Bluetooth connection, the personal details of the HCP, and the closure of the Bluetooth connection. Figure 3. Medical visit abroad scenario. 4. Discussion & Conclusions In this paper, a secure protocol was specified, based on small-scale wireless technologies (Bluetooth) that aims to be adopted at a pan-European level. The current specification is based on a globally used short-range distance data exchange protocol, being compatible by the main market operating systems (e.g. Android, Apple, Windows). Among the most innovative novelties is the fact that through the D2D protocol, it happens a secure data exchange process with minimum user interactions and fast response times, while the citizens are given the ability to manage their own healthcare data, with consultation data being provided back to them, without the interaction of any other third party. For our next goals, we are planning to redesign some operations on exchanging health data, to provide the option for the citizen to send partial information (upon request) of a HL7 FHIR resource, to add operations for transmitting additional types of health data, and finally to perform evaluations with different communication technologies (e.g. Wi-Fi Direct), respecting privacy issues, based on the mechanism developed in [6]. Acknowledgment The research has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 826106 (InteropEHRate project). References [1] [2] [3] [4] [5] [6] Ezekiel E, Wachter R. Artificial intelligence in health care: will the value match the hype?. Journal of the American Medical Association. 2019;321(23):2281-2282. Mello M, et al. Legal barriers to the growth of health information exchange̅boulders or pebbles?. The Milbank Quarterly. 2018;96(1):110-143. Zeadally S, et al. 25 years of Bluetooth technology. Future Internet. 2019;11(9):194. Abdulnabi M, et al. A distributed framework for health information exchange using smartphone technologies. Journal of biomedical informatics. 2017;69:230-250. Saripalle R, et al. Using HL7 FHIR to achieve interoperability in patient health record. Journal of biomedical informatics, 2019;94:103188. Kiourtis A, et al. Towards a Secure Semantic Knowledge of Healthcare Data Through Structural Ontological Transformations. Joint Conf. on Knowledge-Based Software Engineering. 2018;178-188. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200702 97 Technology Supporting Nursing at Homecare ̶ Seems to Be Lacking Eija KIVEKÄSa,1 , Santtu MIKKONENb, Samuli KOPONENa and Kaija SARANTO a a Department of Health and Social Management, University of Eastern Finland, Kuopio, Finland b Department of Applied Physics, University of Eastern Finland, Kuopio, Finland Abstract. The use of welfare technologies in the home setting has drawn increased attention in healthcare. From a historical perspective, medical technologies were designed for hospital settings. Digitalization and internet of things have changed the structure of our society. The aim of this paper is to describe the factors that determine a user’s intent to adopt new welfare technologies in the context of homecare. The phenomenon was being examined by the unified theory of acceptance and use of technology. This study was to show that performance expectancy, effort expectancy, and facilitating conditions are significant factors in determining a user’s intention to use new welfare technologies. While, the use of welfare technologies was rare in homecare. Keywords: welfare, technology, homecare, UTAUT 1. Introduction Digitalization and Internet of things (IoT) have changed the structure of our society. This structural change has a continuous effect on job descriptions in the healthcare sector. The main challenges in launching and using technology are a lack of usability, inadequate communication between participants, and poorly resourced implementation processes. The need for competence is affected by internal changes in professional operating environments that arise from the knowledge base in those professions. Technology use in healthcare always create challenges in nurse-patient relationship. This creates external expectations for professional competencies [1]. Theoretical models have been developed to understand the acceptance and use of information systems (IS). The acceptance and use of information systems and information technology (IT) have received extensive attention from researchers in the last few decades [2]. Different technological and contextual factors that influence the adoption of technologies in individual and organizational contexts has been focused by various theories. Venkatesh and his colleagues (2003) developed a unified model that brings together alternative views on user and innovation acceptance [3]. The unified theory of acceptance and use of technology (UTAUT) is a behavioral model that aims to explain the behavior of people or organizations in their use of IT/IS. The UTAUT has four key constructs: 1 Corresponding Author, Eija Kivekäs, Department of Health and Social Management, University of Eastern Finland, Kuopio Campus, 70211 Kuopio, Finland; E-mail: eija.kivekas@uef.fi. 98 E. Kivekäs et al. / Technology Supporting Nursing at Homecare performance expectancy, effort expectancy, social influence, and facilitating conditions. These are direct determinants of behavioral intention and ultimate behavior and are, in turn, moderated by gender, age, experience, and voluntariness of use [3]. Performance expectancy defines as the level to which an individual believes that using an IT system will help them improve their job performance, whereas effort expectancy is the level of ease associated with the use of such a system. Social influence defines as the degree to which an individual’s important relations believe that the individual should use the system, and facilitating conditions are the measure of infrastructural support available for use of the system. Performance expectancy, effort expectancy, social influence, and facilitating conditions have both a direct and indirect influence on behavioral intention to use IT systems. The UTAUT model also posits that the attitudes construct has both direct and indirect (via behavioral intention) effects on use behavior. [3.] The focus of this study is for understanding individual adoption of IT/IS in homecare using the UTAUT. The purpose of this paper is to describe the factors that determine nurses and students intent to adopt new welfare technologies in homecare settings after educational sessions. 2. Welfare Technology in Homecare Welfare technologies are being increasingly used in elderly care. Assistive technologies have been positively evaluated by elderly clients, healthcare professionals, and family members [1]. In Finland, municipalities have a legislative responsibility to organize homecare services in collaboration with private sectors, various associations, and older clients to plan and realize homecare services for older clients at home by offering care based on clients’ personal needs [4]. Therefore, the goal of welfare services for older people is to provide homecare services that support independent living and maximize clients’ resources. This requires homecare services to make meaningful activities and social relationships possible in relation to clients’ quality of life and psychological wellbeing despite their decline in functional, cognitive, psychological, and social abilities and their need for the highest level of care [1,2,5]. In general, elderly care can and needs to develop using welfare technology and robotics. The elderly population is living at home longer, requires more nursing and care resources. An increasing number of elderly people have a pressing need for solutions to how independent living and high-quality care can be achieved in the circumstances where the number of nurses is decreasing and resources are becoming limited [4,5,6]. Coco and colleagues showed that according to patients, interacting with robots has been useful and pleasant [4]. Patients do not consider them replacements for human interactions [4]. The attitudes of care personnel have to also been considered, as we do in this article. The model of UTAUT is explored through five hypotheses, which described relationships of four key constructs by the model (Table 1). 3. Methodology The questionnaire used in this study was modified from the question items of Venkatesh et al. [3,7]. The questionnaire was pretested on a technology pilot in homecare and was then modified according to their feedback. All items, excluding the use behavior, were measured using a five-point Likert scale, with the anchors being strongly disagree and E. Kivekäs et al. / Technology Supporting Nursing at Homecare 99 strongly agree. Examples, an item of performance expectancy “Using welfare technology increases my productivity” and an item of effort expectancy “Learning how to use welfare technology is easy for me” were used in the measurements. The use behavior was measured using tripartite scale (daily – weekly – rarely). Variables’ internal consistency were assessed using Cronbach’s alpha and a sum variable were constructed for performance expectancy (α = .942), effort expectancy (α = .888), behavioral intention (α = .665) and facilitating conditions (α = .805). Data collection was carried out in connection with the training of the WelTech project [8]. This project was launched to develop welfare technology training courses for social and healthcare professionals and students. The questionnaire was used at the end of the course in the WelTech project. To analyze the data, we use SEM in Amos 25 (IBM SPSS). SEM is a combination of confirmatory factor analysis (CFA) and path analysis. Confirmatory factor analysis allows the specification of construct–item relationships so that they can be tested against the UTAUT theory. CFA and SEM are therefore used for testing the UTAUT theory. We use a root-mean-square error (RMSE) less than or equal to 0.08 and a comparative fit index (CFI) greater than or equal to 0.95. We also use a Bentler-Bonett Normed Fit Index (NFI) and an incremental fit index (IFI) greater than or equal to 0.90 to indicate that the model fits the data adequately [9]. 4. Results A total of 124 participants answered the questionnaire in 2019. The subjects were comprised of 102 women (84%) and 20 men (16%). They included 61 social and healthcare professionals (50%), 24 other professionals (19%), 22 students (18%) and 17 missing information (13%). One third of the participants (n = 44) were less than 27 years old, nearly one third of the participants (n = 36) were between 28 and 37 years, and one third of the participants (n = 44) were more than 38 years old. We examined our proposed research model with the key constructs of performance expectancy, effort expectancy, social influence, facilitating conditions in relation to behavioral intention, and use behavior. The results of SEM are shown in Fig. 1, and the results of the hypotheses are presented in Table 1. Performance expectancy proved a strong construct, whereas social influence did not prove to be effective in this study. Figure 1. Assessment of the research model (standardized solution, p < 0.5, RMSEA = .023, CFI = .996, NFI = .942, IFI = .996). 100 E. Kivekäs et al. / Technology Supporting Nursing at Homecare The various indices confirmed that the UTAUT model was supported (Table 1). Performance expectancy, effort expectancy and facilitating conditions were associated with behavioral intention. Performance expectancy and facilitating conditions were associated with use behavior. Behavioral intention weekly associated with use behavior, and social influence did not have any statistical correlation in this data. Table 1. Results of examining the hypotheses. Hypotheses Results Performance expectancy positively affects users’ intention to use welfare technology in homecare. Supported Effort expectancy positively affects users’ intention to use welfare technology in homecare Supported Social influence positively affects users’ intention to use welfare technology in homecare. Not supported H4 Facilitating conditions of welfare technology positively affects users’ use behaviors of actually using welfare technology in homecare Supported H5 Users’ behavioral intentions to use welfare technology in homecare positively affect the users’ use behavior of actually using welfare technology in homecare. Not supported H1 H2 H3 5. Discussion and Conclusion The use of technology has been perceived to be useful, particularly when it diminished the workload of care personnel [5] and when the solutions were user-friendly [1]. This study shows that performance expectancy, effort expectancy, and facilitating conditions are significant factors in determining a user’s intention to use new welfare technologies. The results are in line with previous studies [5,6]. Social influence did not prove to be as strong a factor in the model as we expected. Previous research results for UTAUT relationships have shown inconsistencies [2,7]. Weakly association between behavior intention and use behavior could revealed that technology is lacking in homecare. The weak social influence factor could reflect the role on management support, which seemed to be weak in implementing welfare technologies. It is important to help social and healthcare personnel accept technology and to reduce fears that technology could take their jobs [4,6]. Education plays a crucial role in technology acceptance, and it is important that care personnel notice that welfare technology is credible. Education is crucial in changing attitudes and helping social and healthcare personnel understand that welfare technologies may perform routine tasks, allowing personnel to focus on providing improved care. The WelTech project was launched to develop welfare technology training courses for social and healthcare professionals and students. This study proved that performance expectancy was the most important factor in the early stages of development. The UTAUT model has been extensively tested in various fields and promises to be a great tool for analyzing users’ acceptance of health technology [6]. However, the UTAUT does have some limitations; an analysis of acknowledged limitations across studies indicates that focusing on a single subject, community, organization, department, or age group has been the most widespread constraint [7]. The limitations in this study included a small amount of data, consisting only of first students. The training course continues, and this study could be seen as a pilot study. Another limitation is that welfare technologies are still rare in homecare, and therefore, the answers from this study could largely be a view of the future. E. Kivekäs et al. / Technology Supporting Nursing at Homecare 101 The UTAUT also demonstrates the role of facilitating conditions and intentions for directly predicting use behavior, citing the theory to support the proposed relationships across a range of contexts, including social and healthcare professionals’ behavioral intentions toward the use of welfare technology in homecare in general. To ensure the content validity of the scales, the selected items must represent the concept about which generalizations are to be made. Therefore, items selected for the constructs were adapted from previous studies and modified to fit welfare-technology adoption in the context of homecare. Our study shows that the UTAUT is a useful framework. In the future, it should be extended with relevant constructs so that it can contribute to the understanding of important phenomena. References [1] [2] [3] [4] [5] [6] [7] [8] [9] Melkas H, Hennala L, Pekkarinen S, Kyrki V. Impacts of robot implementation on care personnel and clients in elderly care institutions. Int J of Med Inf, 2020; 134(104041). Dwivedu YK, Rana NP, Tamilmani K, Raman R. A meta-analysis based modified unified theory of acceptance and use of technology (meta-UTAUT): A review of emerging literature. Current Opinion in Psychology, 2020; 36:13–8. Venkatesh V, Morris MG, Davis GB, Davis FD. Use acceptance of information technology: Towards a unified view. MIS Quarterly, 2003; 27(3):425–78. Coco K, Kangasniemi M, Rantanen T. Care personnel’s attitudes and fears toward care robots in elderly care: A comparison of data from the Care Personnel in Finland and Japan Journal of Nursing Scholarship, 2018; 50(6):634–44. doi:10.1111/jnu.12435 Peek STM, Wouters EJM, van Hoof J, Lujikx KG, Boeije HR, Vrijhoef HJM. Factors influencing acceptance of technology for aging in place: A systematic review. Int J Med Inf, 2014; 83:235–48. Magsamen-Conrad K, Upadhyaya S, Youngnyo Joa C, Dowd J. Bridging the divide: Using UTAUT to predict multigenerational tablet adoption practices. Computers in Human Behavior, 2015; 50:186–96. Venkatesh V, Thong JY, Xu X. Consumer acceptance and use of information technology: Extending the unified theory of acceptance and use of technology. Forthcoming in MIS Quarterly, 2012; 36(1):157–78. WelTech project. Wellbeing technology education product: Introduction and training 2018–2020. Savonia University of Applied Sciences. Retrieved from: https://portal.savonia.fi/amk/fi/tutkimus-jakehittaminen/projekti-ja-hanketoiminta/projektit-ja-hankkeet?id=973. Hair J, Black W, Babin B, Anderson R. Multivariate data analysis. A global perspective. 7th ed. Upper Saddle River, NJ:Pearson Education, 2010. 102 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200703 Health Professionals' Perceptions and Reactions to ICT-Related Patient Safety Incidents Jouni KOUVOa,1, Samuli KOPONEN a, Hanna KUUSISTO a, b and Kaija SARANTO a a University of Eastern Finland b Tampere University Hospital, Department of Neurology Abstract. Patient incident reporting is an important way to promote safer health care. The barriers for reporting can be organizational (leadership, culture, lack of feedback, etc.) or individual (time pressure, perceived competence, attitude, etc.). In this study, we examined what kinds of ICT-related incidents health professionals observe in Finland, how they react to them and the reasons for non-reporting. Our data was collected using a nationwide survey during the Spring of 2020. The theory of planned behaviour by Ajzen served as our framework for explaining nonreporting behaviour. While we found that attitudes, subjective norms and perceived behavioural control all explain non-reporting, our factor model based on our confirmatory factor analysis did not directly match Ajzen’s theory. Keywords. Health care informatics, patient incident reporting, the theory of planned behaviour. 1. Introduction The importance of patient safety and the need for safer health care practices became apparent after a leading-edge report by the Institute of Medicine in 2000 [1]. Patient safety incidents are a major concern also in Finland, with varying consequences ranging from damage to an institution’s reputation to loss of lives. Finland introduced a national incident reporting system in 2007 and the current trend is a yearly increase in incident reporting by health professionals [2], indicating an increasing awareness of patient safety culture. However, there is still room for improvement, especially since new patient safety risks are emerging, many of which are related to information systems and communication [2]. There are organizational barriers to reporting patient safety incidents. Reporting very rarely leads to recommendations, let alone their implementation [2]. Lack of transparency and feedback decreases willingness to report incidents [3, 4, 5], as well as insufficient managerial support [3, 4] and uncertainty about what types of events and what level of severity should be reported [3, 4, 5]. A punitive culture and fear of consequences can also be barriers to incident reporting [4, 5]. 1 Corresponding Author, Jouni Kouvo, University of Eastern Finland, P.O.BOX 1627, 70211 Kuopio, Finland. E-mail: jouni.kouvo@gmail.com J. Kouvo et al. / Health Professionals’ Perceptions and Reactions 103 One of the most popular socio-cognitive theories to predict and explain behaviour is the theory of planned behaviour [6, 7]. According to the model, behavioural intention predicts actual behaviour and three factors affect behavioural intention: attitude (beliefs about the consequences and experiences of behaviour), subjective norms (beliefs about the expectations and behaviours of others) and perceived behavioural control (beliefs about resources and opportunities) [8]. While there is evidence that the model predicts patient incident reporting intentions quite well [9, 10], there is no clear understanding as to which factors have the greatest impact [11, 12]. The model has also been expanded with other psychological concepts, such as altruism [12], psychological safety [10] and self-efficacy [10]. Our research questions are: 1) What kind of ICT-related patient safety incidents do health professionals experience in their work? 2) What kinds of actions do they take when they are noticed? 3) What are the reasons for not reporting incidents? 4) Do attitude, subjective norms and perceived behavioural control explain behaviour, as predicted by the theory of planned behaviour? 2. Methods In Spring 2020, the Finnish Institute for Health and Welfare conducted a nationwide survey (STePS 3.0) on information system services in health care. An anonymous web questionnaire was sent to 58 276 health care professionals, of whom 10 094 opened the link and 3912 completed the survey. The 3610 replies were sufficient for the analysis, representing 35.8% of those who opened the link. The survey was designed to assess how users experience information systems’ functionality, usability, and support for daily practice, as well as to describe the current status and needs for improvements of the electronic health care system [13]. As some of the questions were directly related to patient safety incidents and their reporting, the following variables were included in this study:    “If during the last 12 months you have noticed patient safety incidents caused by use of information systems, what kinds of errors occurred?” (See options in Figure 1). “What did you do when you noticed incidents?” (Figure 2). “If you didn’t report incidents, what caused you to make that decision?” (Figure 3). Data analysis was carried out with SPSS (version 25.0) and Amos, and it included descriptive and inferential statistics. For inferential statistics, we used principal component analysis and confirmatory factor analysis. 3. Results Altogether 92.5% of the participants were women, and most of them worked as a nurse or similar (78.1%) in a public-owned organization run by a municipality (85.2%). Age was more evenly spread, with a majority of the respondents being born in the 1960s (30.6%) or 1970s (28.2%). 104 J. Kouvo et al. / Health Professionals’ Perceptions and Reactions Almost all the participants had experienced at least one ICT-related incident during the last 12 months (94.6%). A majority of incidents were caused by human errors, not system malfunctions. The most typical adverse findings were related to medication lists or patient registrations (Figure 1). 0% 25% 50% 75% 100% Medication list wasn't verified Registration was missing Registration lacked information Registration was done in the wrong place Patient personal data was incomplete System crashed or froze (T) System didn't open (T) System integrations didn't work (T) Registration had errors in it The wrong patient was registered Software update caused issues (T) Quality register didn't open (T) Monthly Weekly Daily Never/No answer Figure 1. ICT-related patient safety incidents observed by health professionals during the last 12 months (%). (T) = Technological issue, not human mistake. (N = 3610). When health professionals witnessed an incident, the most common action they took was to discuss it with their colleague or manager (Figure 2). In almost half of the cases (45.7%) they created a patient safety incident report. Principal component analysis showed that there were three different patterns of reactions (in order of popularity): 1) Discussing (with colleague, manager or patient), 2) Reporting (in the patient incident reporting system) and 3) Contacting (help desk or super user). A KMO measure of .612 and Bartlett’s test p < .001 suggest that the model is appropriate. I spoke with my colleague I spoke with my manager I created a patient incident report I informed the patient or his/her relatives I created a help desk ticket I informed the system super user I didn't do anything 0% 25% 50% 75% 100% Figure 2. Actions taken with regards to ICT-related patient safety incidents (%). (N = 3158). The last questions covered health professionals’ explanations for not reporting (Figure 3). About half of the respondents said that they did not have time to report or they did not report because no actual harm was done to the patient. Approximately 10% of the people felt that their organization did not expect them to report patient safety incidents or that they did not even have access to a reporting tool. J. Kouvo et al. / Health Professionals’ Perceptions and Reactions 105 Figure 3. The reasons for not reporting ICT-related patient safety incidents (%). (N = 3022). Confirmatory factor analysis was conducted to see how the indicators loaded on predicted factors and how the factors were correlated. The following three dimensions were identified:  Subjective norm (perceived expectations of the organization)  Attitude (beliefs about the need to report incidents)  Costs vs. Benefits (the amount of time and effort needed for reporting vs. the expected benefits) Figure 4. The factors for not reporting ICT-related patient safety incidents (%). (N = 3022. RMSEA .067, NFI .967, IFI .969, CFI .969). 4. Discussion and Conclusion Practically all participants (N=3610) had observed ICT patient safety incidents during the last 12 months. Incidents were mainly caused by human mistakes, so it is understandable that the most typical reaction was to discuss the matter with a colleague or manager. However, in almost half of the cases respondents created an incident report, which confirms the findings that reporting in Finland is at a quite good level [2]. 106 J. Kouvo et al. / Health Professionals’ Perceptions and Reactions For non-reporting, we could not find all three factors predicted by Ajzen’s theory. Instead of perceived behavioural control, we found a factor which could be called costs vs. benefits. Literature has shown that a lack of feedback and recommendations decrease reporting willingness [2-5]. It is interesting that in our study, respondents seemed to have considered available resources (time, competence) and expected outcomes together. We did find a component representing a subjective norm, i.e. insufficient managerial support, which is known to negatively affect reporting [3,4]. Our data also confirmed that people sometimes skip reporting because they consider events as not severe enough [3-5]. That may indicate an attitude issue or lack of proper instructions. The three components of the theory of planned behaviour are known to covary and their exact impact is still unclear [7,11,12], so further research is needed. It would also be interesting to compare how the model works for different groups of people in health care: public vs. private sector staff, nurses vs. physicians, etc. Our questionnaire was crafted and reviewed by scholars from various Finnish research organizations, and our sample size was quite large (3610), so we can assume adequate reliability of this study. However, external validity of our results is limited: they cannot be directly generalized outside the (mainly public) nursing community in Finland. Lastly, we did not use pre-existing scales from the literature to measure some of our key concepts, such as attitudes or subjective norms, which might compromise our study’s internal validity. References [1] Kohn LT, Corrigan JM, Donaldson MS. To Err is Human: Building a safer health system. Institute of Medicine Report. Washington, DC: National Academy Press. 2000. [2] Liukka M, Hupli M, Turunen H. Problems with incident reporting: Reports lead rarely to recommendations. Journal of Clinical Nursing. 2019;28(9-10):1607-1613. [3] Pfeiffer Y, Briner M, Wehner T, Manser T. Motivational antecedents of incident reporting: evidence from a survey of nurses and physicians. Swiss Medical Weekly. 2013;143; w13881. [4] Winsvold Prang I, Jelsness-Jørgensen L. Should I report? A qualitative study of barriers to incident reporting among nurses working in nursing homes. Geriatric Nursing. 2014;35(6):441–447. [5] Rashed A, Hamdan M. Physicians’ and Nurses’ Perceptions of and Attitudes Toward Incident Reporting in Palestinian Hospitals. Journal of Patient Safety. 2019;15(3):212–217. [6] Godin G, Belanger-Gravel A, Eccles M, Grimshaw J. Healthcare professionals’ intentions and behaviours: A systematic review of studies based on social cognitive theories. Implementation Science. 2008; 3:36. [7] Ajzen I. The theory of planned behaviour: reactions and reflections. Psychology & Health. 2011;26(9):1113–1127. [8] Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes. 1991;50(2):179–211. [9] Ben Natan M, Sharon I, Mahajna M, Mahajna S. Factors affecting nursing students’ intention to report medication errors: An application of the theory of planned behavior. Nurse Education Today. 2017; 58:38–42. [10] Lee Y, Yang C, Chen T. Barriers to incident-reporting behavior among nursing staff: A study based on the theory of planned behavior. Journal of Management and Organization. 2015;22(1):1–18. [11] Ekayani N, Wardhani V, Rachmi A. Nurses’ intention and behavior in reporting adverse event: Application of theory of planned behavior. Kesmas: National Public Health Journal. 2017;11(3):138– 144. [12] Hung C, Lee B, Liang H, Chu T. Factors influencing nurses’ attitudes and intentions toward medication administration error reporting. Japan Journal of Nursing Science. 2016;13(3):345–354. [13] Saranto K, Kinnunen U-M, Koponen S, Kyytsönen M, Hyppönen H, Vehko T. Nurses’ competences in informatics/information management as well as experiences/opinions in health and social care information system support for daily practice. FinJeHeW. 2020;12(3): 212-228. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200704 107 From Atomic Guideline-Based Recommendations to Complete Therapeutic Care Plans: A KnowledgeBased Approach Applied to Breast Cancer Management Hicham KOUZa , Jacques BOUAUD b,a, Gilles GUÉZENNECa and Brigitte SÉROUSSIa,c,1 a Sorbonne Université, Université Sorbonne Paris Nord, INSERM, LIMICS, UMR_S 1142, Paris, France b AP-HP, DRCI, Paris, France c AP-HP, Hôpital Tenon, Paris, France Abstract. How textual clinical practice guidelines are written may have an impact on how they are formalized and on the kind of recommendations issued by the clinical decision support systems (CDSSs) that implement them. Breast cancer guidelines are mostly centered on the description of the different recommended therapeutic modalities, represented as atomic recommendations, but seldom provide comprehensive plans that drive care delivery. The objective of this work is to implement a knowledge-based approach to develop a care plan builder (CPB) that works on atomic recommendations to build patient-centered care plans as sequences of chronologically ordered therapeutic steps. The CPB uses the atomic recommendations issued by the guideline-based decision support system (GL-DSS) of the DESIREE project. The domain knowledge is represented as the list of all care plans that apply to breast cancer patients. Scenarios are introduced to locate the patient on these theoretical care plans. The CPB has been evaluated on a sample of 99 solved clinical cases leading to an overall performance of 89,8%. Keywords. Decision support systems, Clinical Practice Guidelines, Patient Care Planning, Breast Cancer. 1. Introduction Breast cancer is the most common cancer among women worldwide. In France, the mortality rate of breast cancer is decreasing, which is partly due to the early stage of the disease at diagnosis, and the progress of therapeutic drug protocols. However, although studies have reported that following clinical practice guidelines (CPGs) does improve survival rates of patients [1], the compliance rate of multidisciplinary tumor board (MTB) decisions remains variable. Guideline-based decision support systems (GLDSSs) have been developed to promote MTB implementation of CPGs [2]. 1 Corresponding Author, Brigitte Séroussi, DSP, Hôpital Tenon, 4 rue de la Chine, Paris, France; E-mail: brigitte.seroussi@aphp.fr 108H. Kouz et al. / From Atomic Guideline-Based Recommendations to Complete Therapeutic Care Plans DESIREE is a European-funded project 2 that aims at developing a web-based platform for the management of primary breast cancer. It offers different decision support modalities to support the decision at the various stages of patient care, from diagnosis to treatment and follow-up [3]. However, the GL-DSS of DESIREE mostly produces “atomic” recommendations, i.e., recommendations that are focused on one therapeutic modality like surgery or chemotherapy. Such recommendations are regularly redundant, sometimes conflicting, and very rarely organized as comprehensive care plans. To answer MTB physicians’ needs for operational decision support, we have developed a “Care Plan Builder” (CPB) that relies on a knowledge-based approach to build the recommended care plans as a sequence of chronologically ordered therapeutic steps from the atomic recommendations generated by the GL-DSS of DESIREE. 2. Material and Methods 2.1. DESIREE atomic recommendations The Breast Cancer Knowledge Model (BCKM) represents the central element for the DSS components of DESIREE. It describes in a common ontology following the EntityAttribute-Value model both the data model and the termino-ontological knowledge used for representing breast cancer concepts and clinical cases. Relying on BCKM concepts, and decision rules that model CPG contents, the GL-DSS produces patient-specific recommendations as atomic recommendations (see Figure 1) at different levels of abstraction (surgery, but also lumpectomy; radiotherapy, but also radiotherapy of the lymph nodes; chemotherapy, but also 3-4 cycles of Epirubicin, etc.). Each recommendation has a level of conformance that may be either positive (SHALL, SHOULD, MAY, and MAYNOT) or negative (SHALLNOT, SHOULDNOT). Figure 1. Example of three atomic recommendations issued by the GL-DSS of DESIREE, one of chemotherapy, one of radiotherapy, and one of endocrine therapy. 2.2. Care plans models for breast cancer management Cancer care plans are organized around a number of treatment methods such as surgery (SUR), chemotherapy (CHEM), targeted therapies, endocrine therapy (HO), and radiotherapy (RAD). The diagram displayed in Figure 2 illustrates all the possible 2 The DESIREE project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 690238. H. Kouz et al. / From Atomic Guideline-Based Recommendations to Complete Therapeutic Care Plans109 trajectories for the management of non-metastatic breast cancer patients, from the moment the diagnosis is made, and passing through the different scenarios (A, B, C, and D). Each path can have a single step method or an ordered combination of methods defined by a branch. Figure 2. Diagram of all medically relevant care plans for breast cancer patients. 2.3. Care plan representation To represent the structure of the care plan model, we borrowed from the conceptual frameworks of existing data models, e.g., FHIR [4] and Open EHR [5]. From FHIR, we used the “CarePlan” resource to represent the general information of a patient (identifier, date, etc.), and the “PlanDefinition” resource to represent a group of actions defined as a step in our care plan. We also used the “Task plan” model of “Open EHR task Planning” for the definition of each action named “activity”. 2.4. Care plan building processes We proceeded in six stages to build the care plans from atomic recommendations: 1. Analysis of recommendations generated by the GL-DSS, identification of R+, defined as the set of recommendations with a positive conformance level and R-, defined as the set of recommendations with a negative conformance level; 2. Elimination of conflicting recommendations that exist when there is a positive conformance level (e.g., Tumorectomy SHOULD) and a negative conformance level (BreastSurgery SHOULDNOT) taking into account the subsumption relationship. The process consists in browsing all the recommendations of R+ in order to check if there is a comparable recommendation in R- and to proceed with the elimination of both of them; 3. Elimination of recommendations that have a negative conformance level which are not useful for the construction of care plans since only those with a positive conformance level have to be actually performed; 4. For each recommendation, identification of the corresponding therapeutic category (chemotherapy, endocrine therapy, radiotherapy, surgery) to reach the level of abstraction of the care plan models (see 2.2); 110H. Kouz et al. / From Atomic Guideline-Based Recommendations to Complete Therapeutic Care Plans 5. 6. Taking into account the therapeutic categories previously identified and the scenario of the patient management, identification of the model of care plan from the set of all possible care plans (see 2.2); Generation of instantiated care plans based on the remaining recommendations and the model identified in the previous stage. The CPB has been assessed on a sample of solved clinical cases for which we had (i) the set of atomic recommendations issued by the GL-DSS, (ii) the MTB decision expressed as a care plan, and (iii) the compliance status of the MTB decision with CPGs as previously established by clinicians (gold standard). The CPB performance was defined by the frequency with which MTB decisions acknowledged as compliant with CPGs were retrieved in the care plans generated from the atomic recommendations of the GL-DSS. 3. Results Figure 3 illustrates how care plans are built from atomic recommendations. We use the case of a patient in “scenario C” that generated seven recommendations among three therapeutic categories (chemotherapy with three instances, radiotherapy with one instance, and endocrine therapy with three instances), and the care plans generated by the CPB (an excerpt with four out of the nine care plans generated is displayed). We used a sample of 99 clinical cases with CPG-compliant MTB decisions that were solved using DESIREE to produce atomic recommendations processed by the CPB. For 89 clinical cases, the MTB decision was found in the list of CPB-generated care plans, which corresponds to an overall performance of 89.8%. Figure 3. Care plans generated from DESIREE atomic recommendations in scenario C. 4. Discussion and Conclusion Not all GBPs recommend comprehensive care plans that can be directly encoded for decision support [6]. We have developed a care plan builder allowing the consistent processing of atomic recommendations issued by the GL-DSS of DESIREE to generate the corresponding recommended complete care plans. The CPB gives satisfactory results with a performance of 89.8%. For 10 clinical cases, the MTB decision was not found H. Kouz et al. / From Atomic Guideline-Based Recommendations to Complete Therapeutic Care Plans111 among the care plans generated. In seven cases, DESIREE outputs were at the origin of the issue (a MTB decision was badly entered; some chemotherapies were missing in the recommendations (n=3); target therapies wrongly included in the BCKM as sort of chemotherapies (n=3)). Only three badly processed cases were imputable to the CPB due to a mismanagement of the subsumption relationship. Despite being focused on one pathology (breast cancer) and on one guideline, building care plans from atomic recommendations is part of the general scientific research topic on guideline reconciliation, e.g., for the management of multimorbidity. In these situations, CDSSs generate several recommendations that might conflict or may be combined, and for which varied approaches have been proposed (see [7]). In our case, it is as if we had several guidelines, one per therapeutic modality, and the additional knowledge used to build care plans (e.g., no chemotherapy after radiotherapy) can be considered as constraints to be satisfied in building the care plan. This work has some limitations. We based the CPB development on the assumption of atomic recommendations, allowing to use the FirstStepCategory. However, in some cases, so-called atomic recommendations were in fact semi-care plans. The resolution of conflicts (removing pairs of similar recommendations that had a positive conformance level for one of them and a negative conformance level for the other one) is a pragmatic and empirical approach but it means that the negative conformance level is favored which should be fine-tuned by considering additional domain knowledge (to select the recommendations to be removed instead of removing them both). Finally, the definition of the performance measure has imperfections. Indeed, we considered the frequency with which MTB decisions acknowledged as compliant with CPGs were retrieved in the care plans generated from the atomic recommendations of the GL-DSS. Thus, we do not have any evaluation of the CPB when MTB decisions were not compliant with CPGs and we didn’t evaluate the generated care plans that were different from the compliant MTB decision. Further work is needed to improve the CPB (to take into account non-atomic recommendations) and the performance indicator. References [1] [2] [3] [4] [5] [6] [7] Kreienberg R, Wöckel A, Wischnewsky M. Highly significant improvement in guideline adherence, relapse-free and overall survival in breast cancer patients when treated at certified breast cancer centres: An evaluation of 8323 patients. Breast. 2018;40:54-59. Eccher C, Seyfang A, Ferro A. Implementation and evaluation of an Asbru-based decision support system for adjuvant treatment in breast cancer. Comput Methods Programs Biomed. 2014;117(2):308-321. Seroussi B, Lamy JB, Muro N, et al. Implementing Guideline-Based, Experience-Based, and Case-Based Approaches to Enrich Decision Support for the Management of Breast Cancer Patients in the DESIREE Project. Stud Health Technol Inform. 2018;255:190-194. HL7. CarePlan - FHIR v4.0.1. [available at https://www.hl7.org/fhir/careplan.html] Anani N, Chen R, Prazeres Moreira T, Koch S. OpenEHR-based representation of guideline compliance data through the example of stroke clinical practice guidelines. Stud Health Technol Inform. 2012;180:487-491. Laleci Erturkmen GB, Yuksel M, Sarigul B, et al. A Collaborative Platform for Management of Chronic Diseases via Guideline-Driven Individualized Care Plans. Comput Struct Biotechnol J. 2019;17:869–885. Bilici E, Despotou G, Arvanitis TN. The use of computer-interpretable clinical guidelines to manage care complexities of patients with multimorbid conditions: A review. Digit Health. 2018;4:2055207618804927. 112 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200705 Automatic Exploitation of YouTube Data: A Study of Videos Published by a French YouTuber During COVID-19 Quarantine in France Gery LAURENTa,b,1 Benjamin GUINHOUYAa,b, Marielle WHATELETc and Antoine LAMERa,b,c a Univ. Lille, CHU Lille, ULR 2694 - METRICS: Évaluation des Technologies de santé et des Pratiques médicales, 59000, Lille, France b Univ. Lille, Faculté Ingénierie et Management de la Santé, 59000, Lille, France c CHU Lille, Public Health Department, F-59000 Lille, France Abstract. The objective of this study was to test the feasibility of automatically extracting and exploiting data from the YouTube platform, with a focus on the videos produced by the French YouTuber HugoDécrypte during COVID-19 quarantine in France. For this, we used the YouTube API, which allows the automatic collection of data and meta-data of videos. We have identified the main topics addressed in the comments of the videos and assessed their polarity. Our results provide insights on topics trends over the course of the quarantine and highlight users sentiment towards on-going events. The method can be expanded to large video sets to automatically analyse high amount of user-produced data. Keywords. COVID-19, YouTube, citizen, Natural Language Processing 1. Introduction YouTube is an online video-sharing platform, used by individuals, professionals or institutions. It provides a huge range of Health-Related Content and was used during COVID-19 outbreak as a source of information [1-3]. In most of the studies using YouTube, videos are watched and then analysed manually [4]. However, there exist automatic methods to extract useful information from online content, and in particular YouTube [5]. The objective of this study is to test the feasibility of automatically extracting and exploiting data from YouTube. For this, we will seek to identify the main topics addressed in the videos of a French well-known Youtuber, and the audience's reactions to these topics. 1 Corresponding Author, Géry Laurent, ULR2694, Pôle Recherche 1, place de Verdun 59045 - Lille, France; E-mail: gery.laurent.etu@univ-lille.fr G. Laurent et al. / Automatic Exploitation of YouTube Data 113 2. Material and Methods A YouTube video is characterized by a title, an author, the video itself, subtitles, a number of positive/negative rates (like/dislike), comments. In this work, we analysed the YouTube channel HugoDécrypte followed by more than 868 000 subscribers [6]. We collected and exploited a maximum of data from videos of the “daily news” playlist released between March 17th 2020 and May 04th 2020, as it provides a chronological overview of daily news regarding the COVID-19 outbreak in France. YouTube Data API provides a way to collect metadata of a video: title, author, duration, publication date, number of like/dislike, number of views, number of comments, text description or its comments: author, publication date, text, number of like/dislike. The API can be implemented in Python. Some of the metrics retrieved by the API, namely number of views or like/dislike count, are representing a snapshot of current data at the time of the API call and thus cannot be used retroactively. Retrieval of number of views of a video overtime necessitates data collection stream. Thus, for the present study, data used were the video title, publication date, comments publication date and text. In order to identify the main topics addressed in the videos, we have implemented the following steps. (i) We extracted comments published in the 24 hours following the release of the videos. (ii) For each video, we detected the 10 most frequent words from which topics were drawn. Data management followed Natural Language Processing (NLP) steps: accent and stopwords removal, tokenization, stemming. (iii) Topic cooccurrence was assessed by computing the frequency of comments sharing both topics. The two most co-occurring topics were linked in the network graph as well as the topics presenting at least half the co-occurring frequency. (iv) Data from all the videos were pooled to analyse the most discussed topics globally and compute the relative daily frequency of comments for each topic. To evaluate the audience's reactions to these topics, we used the polarity score obtained using SAS Viya VisualTextAnalytics software [7]. This method is using a sentiment lexicon attributing polarity scores to individual words. The overall comment score is then computed based on each word polarity score as well as sentence structure, punctuation and emoticons, on a scale of -1 (extremely negative) to +1 (extremely positive). Average polarity, polarity distribution and number of positive, negative and neutral comments were calculated for each topic on a video per video basis. The following python libraries were used for this study: pandas and numpy (data management), matplotlib.pyplot and seaborn (data visualization), nltk, sacremoses and sklearn (NLP). 3. Results Between March 17th 2020 and May 04th 2020, 49 videos related to the COVID-19 outbreak were published on the playlist “daily news” of “HugoDécrypte” channel. These videos received 38 725 comments in the 24 hours following the release. For each video, the median [1st quartile; 3rd quartile] number of comments was 771 [682 ; 865]. After removal of stopwords, 135 unique topics were identified across the 49 videos. The top 10 discussed topics are presented in Table 1 and Figure 1. Table 1 is presenting the polarity distribution of the comments across the 10 most discussed topics based on SAS polarity score. The median [1st quartile; 3rd quartile] polarity of all the comments is 0.00 [-0.20 ; 0.00]. The two topics with the most percentage of positive comments were “thank you” and “Hugo” with respectively 30% 114 G. Laurent et al. / Automatic Exploitation of YouTube Data and 27%. The two topics with the most percentage of negative comments were “death” and “people” with respectively 87% and 62%. Overall, all the topics displayed a lower ratio of neutral comments and a higher ratio of negative comments compared to the global ratio for all the comments except for the two most represented topics “thank you” and “hugo”. Figure 1 represents a heatmap of the relative occurrence for each topic, across the 49 videos. Amongst the topics we have: “thank you”, “Hugo”,“France”, “quarantine”, “masks”, “individuals”, “virus”, “deaths”, “people” and “country”. Main topics are evolving and fluctuating based on video content although most topics have a recurring pattern throughout the time of study. Table 1. Number of comments and polarity distribution of the 10 most frequently mentioned topics. Polarity is expressed in median [1st quartile ; 3rd quartile]. Topic Percentage of comments (Number) Polarity Total thank you Hugo France quarantine masks individuals virus deaths people country Total Positive Negative Neutral 38 725 4 898 3 837 2 652 2 534 1 992 1 993 1 780 1 774 1 850 1 432 15% (5 763) 30% (1 473) 27% (1 049) 13% (344) 13% (323) 14% (273) 11% (217) 8% (147) 4% (63) 10% (193) 11% (164) 33% (12 720) 16% (800) 19% (743) 53% (1 411) 49% (1 233) 43% (866) 60% (1 204) 60% (1 061) 87% (1 540) 62% (1 153) 59% (840) 52% (20 242) 54% (2 625) 53% (2 045) 34% (897) 39% (978) 43% (853) 29% (572) 32% (572) 10% (171) 27% (504) 30% (428) 0.00 [-0.20;0.00] 0.00 [0.00;0.00] 0.00 [0.00;0.20] -0.20 [-0.38;0.00] 0.00 [-0.38;0.00] 0.00 [-0.20;0.00] -0.20 [-0.38;0.00] -0.20 [-0.38;0.00] -0.38 [-0.54;-0.20] -0.20 [-0.54;0.00] -0.20 [-0.38;0.00] Figure 1. Frequency of the 10 most frequently mentioned topics in the comments across the 49 videos Figure 2 represents the 10 topics discussed in the comments of the 9 th video published on March 22th 2019, their average polarity and their co-occurrence. The three topics with a positive polarity are related to the “work” of “information” realized by the YouTuber Hugo, and the subscribers “thank” him for that. The other topics present a negative polarity, from -0.17 for “quarantine” to -0.33 for “deaths”, compared to the global video polarity of -0.07. G. Laurent et al. / Automatic Exploitation of YouTube Data 115 Figure 2. Topics discussed in the comments of the 9th video published on March 22th 2019, their polarity and their co-occurrence. Central circle represents the number of comments of the video. Outer circles represent the 10 most discussed topics in the videos, with a radius proportional to their frequency, and a color related to their average polarity. Edges are drawn between the most co-occurring topics. 4. Discussion In this study, we have automatically extracted and exploited the comments of 49 videos of a French youtuber, HugoDécrypte, who produced videos on current events during the French lockdown of COVID-19 outbreak. From this automatic analysis came out the main topics discussed in relation to the videos, and their polarity. The work of the YouTuber was received positively by subscribers, while the topics discussed have a negative polarity. From a methodological point of view, the main difference with previous works is that we were able to perform an automatic exploitation of data from the YouTube platform [1]. It presents some advantages compared to the manual method: (i) the study can cover a larger number of videos, (ii) it can be replicated several times over time (iii) the methodology can be applied to any video to retrieve main topics and polarity. Even if the YouTube API provides an easy way to automatically extract data from the YouTube platform, it also presents some limitations. First, all data are not available: subtitles can only be extracted by the owner of the channel. Secondly, when submitting retrospective queries, the API returns a limited amount of content. In order to have completeness, the query may be submitted in real time in streaming. Furthermore, the YouTube API is returning a non-exhaustive sample of the videos and comments that may vary from a query to another and based on the time between query and publication date. Last, the NLP methods for the treatment of comments used in this study delivered decent results with the selected videos, but it depends very much on the community, the vocabulary and language used as well as the topics discussed. This has yet to be tested in other contexts. Some parts of the extraction and cleaning process of the video content may depend of the context and need to be updated for each study. Authors have to be cautious when interpreting results from polarity score. Indeed, we experimented with another French lexicon besides SAS sentiment analysis, from the library TextBlob_fr [8], which returned different raw polarity score. The first method is 116 G. Laurent et al. / Automatic Exploitation of YouTube Data biased towards negative score while the second method is biased towards positive score. Sentiment analysis studies on French content is lacking compared to English corpus. The development of a more complete and up to date lexicon for French content, especially focused on social network corpus, is required to improve the results and reliability. To go further, emotion analysis (happy, sad, angry, fearful, excited, bored) can also enhance results by providing a more precise picture of the community feeling towards the different topics [9]. While the study is aimed at studying the community interaction with the main discussed topics, there is no current way to retrieve topics discussed in the video without watching it and manually analysing audio and video content. Subtitles retrieval by other means than using the YouTube API could be considered. Besides, YouTube recently released a new feature allowing content creators to timestamp their video and split it in several chapters based on the topic discussed at that point. This could provide an easy way to automatically extract the different topics mentioned in the video [10]. 5. Conclusions Social Media and YouTube represent a novel and fast-growing way to share information and discuss about trending topics worldwide. With the explosion of video content, we proposed an automatic method to collect and exploit citizens produced data by highlighting main discussed topics in the comment section of a video and user sentiment towards it. References [1] Khatri P, Singh SR, Belani NK, Yeong YL, Lohan R, Lim YW, et al. YouTube as source of information on 2019 novel coronavirus outbreak: a cross sectional study of English and Mandarin content. Travel Med Infect Dis. March 20th 2020;101636. [2] Kocyigit BF, Akaltun MS, Sahin AR. YouTube as a source of information on COVID-19 and rheumatic disease link [published online ahead of print, May 23th 2020 ]. Clin Rheumatol. [3] Cinelli M, Quattrociocchi W, Galeazzi A, Valensise CM, Brugnoli E, Schmidt AL, et al. The COVID19 Social Media Infodemic. ArXiv200305004 Nlin Physicsphysics [Internet]. March 10 th 2020 [4] MacLeod MG, Hoppe DJ, Simunovic N, Bhandari M, Philippon MJ, Ayeni OR. YouTube as an information source for femoroacetabular impingement: a systematic review of video content. Arthroscopy. 2015;31(1):136-142. [5] Liu J, Siegel L, Gibson LA, et al. Toward an Aggregate, Implicit, and Dynamic Model of Norm Formation: Capturing Large-Scale Media Representations of Dynamic Descriptive Norms Through Automated and Crowdsourced Content Analysis. J Commun. 2019;69(6):563-588. [6] HugoDécrypte. Actus du jour [Internet]. [cited July, 10th 2020]. Available on : https://www.youtube.com/playlist?list=PLKDC6DUkHXj28ptnJ44sveHIKFgJ50U_v [7] SAS Visual Text Analytics [Internet]. [Cited on July, 15 th 2020]. Available on: https://www.sas.com/en_us/software/visual-text-analytics.html th [8] Loria S. textblob-fr: French language support for TextBlob. [Internet]. [Cited on July, 14 2020]. Available on: https://github.com/sloria/textblob-fr [9] Abdaoui A, Azé J, Bringay S, Poncelet P. FEEL: a French Expanded Emotion Lexicon. Lang Resour Eval. Sept 2017;51(3):833-55. [10] Add chapters to a progress bar - YouTube Help [Internet]. [cited on July, 14 th 2020]. Available on: https://support.google.com/youtube/answer/9884579?hl=en Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200706 117 Reuse of Clinical COVID-19 Patient Data: Pre-Processing for Future Classification Elena LAZAROVAa,1, Sara MORAa, Antonio DI BIAGIOb, Antonio VENAb and Mauro GIACOMINIa a Department of Informatics, Bioengineering, Robotics and System Engineering (DIBRIS), University of Genoa, Italy b Division of Infectious Diseases, IRCCS Ospedale Policlinico San Martino, Genoa, Italy Abstract. One of the most important challenges in the scenario of COVID-19 is to design and develop decision support systems that can help medical staff to identify a cohort of patients that is more likely to have worse clinical evolution. To achieve this objective it is necessary to work on collected data, pre-process them in order to obtain a consistent dataset and then extract the most relevant features with advanced statistical methods like principal component analysis. As preliminary results of this research, very influential features that emerged are the presence of cardiac and liver illnesses and the levels of some inflammatory parameters at the moment of diagnosis. Keywords. COVID-19, feature extraction, principal component analysis, imputation of data, pseudo-anonymous data 1. Introduction Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) is the novel coronavirus that provoked the pandemic of COVID-19. It was first reported in Hubei (China) in December 2019, but it quickly spread across the globe. In Italy, at July 15,2020, the total number of reported positive cases were 243.506 including 34.997 deaths. One of the most important challenges at the moment is to analyse what happened to investigate guidelines and appropriate instruments to face the disease in a more prepared way in the event that a new local pandemic episode occurs. In particular, it is important to work on previously collected data from COVID-19 patients in order to design and develop Decision Support Systems (DSS) that can help medical staff to identify a cohort of patients that is more likely to have worse clinical evolution. To achieve this objective, it is useful to define which clinical and laboratory parameters influence the outcome most, for example the death of a patient or admission to the Intensive Care Unit (ICU). In this way, advanced statistical procedures and Machine Learning (ML) techniques can be applied to identify, extract and analyse significant features for COVID-19 management. This paper first describes the applied preprocessing operations done on a sample of data collected from COVID-19 patients. The dataset was obtained through an already existing platform for the automatic collection of 1 Corresponding Author, Elena Lazarova, Department of Informatics, Bioengineering, Robotics and System Engineering (DIBRIS), University of Genoa, Italy; elena.lazarova@dibris.unige.it 118 E. Lazarova et al. / Reuse of Clinical COVID-19 Patient Data data from HIV infected patients, used for multicentric clinical studies since 2011 [1,2]. The second part reports the comparison between the considered advanced statistical algorithms to determine relevant features. Some preliminary results are reported and discussed. 2. Methods 2.1. Data collection The complete list of parameters included in the protocol of data collection was approved by the Liguria ethics committee and for data storage and usage for research purposes, after a process of pseudo-anonymization. Patients also signed the informed consent. The system used to collect pseudo-anonymous clinical and laboratory data of COVID-19 patients was derived directly from the Liguria HIV Network. It is based on a Service Oriented Architecture (SOA) to ensure the interoperability between the hospital information system and the Liguria HIV Network [3] with the appropriate privacy and security level [4]. As the system was designed and developed to collect useful data in the HIV (but also HCV and TB) field, it was necessary to update the structure and to insert new parameters specific for COVID-19, for example the ones related to arterial blood gas test. In order to also collect the anamnestic information included in the approved protocol, specific sections were inserted into the platform so that medical staff could insert types of data that are not originally digital. Data included present and previous illnesses of the patient; home therapies; reported information about the symptoms and parameters measured at the moment of the hospitalization. Data on study participants, indicated as COVID-19 hospitalized patients, were collected from 22th February to 15th June, 2020. For each patient, data were collected starting from COVID-19 diagnosis until in-hospital death or discharge. The attributes used in the analysis reported in this manuscript are 71 in total, the input parameters included: gender, age, previous underlying diseases (for example those included in the Charlson Comorbidity Index) and treatments, laboratory findings at baseline (tests done 48hours prior to and following the first nasopharyngeal swab positive for SARS-CoV-2). The targets were death of the patient, ICU admission and discharge of the patient without any of the previous events. In the study a total of 912 patients were selected from a more numerous group (about 1200 patients), including criteria were: conclusion of the hospitalization so that information about patient death or discharge was present and definitive, complete insertion of almost all anamnestic information, including the treatments. 2.2. Advanced Statistical Procedure The aim of this section is to briefly present the advanced statistical procedure that the authors considered appropriate to discriminate between features to single out the most influencing ones. The software used to pre-process and analyse data was MatLab (version 2018a). The authors proposed two different approaches with the common objective of reducing the high dimensionality of data before using classification methods. Principal Component Analysis (PCA) is one of the most popular dimensionality reduction procedures. It computes the identification of a smaller number of uncorrelated variables from a larger dataset and its outputs are a transformed dataset with weights of E. Lazarova et al. / Reuse of Clinical COVID-19 Patient Data 119 individual instances and the weights of principal components. It is used in predictive models and exploratory data analysis [5,6]. 2.3. Missing data management Missing data in medical research is a common problem because, in general, real data contains several missing values. There are different types of “missingness” that can occur and this may influence how the researchers should analyse the data that they have collected. Missing completely at random (MCAR): Patients with complete data cannot be distinguished from others with complete data. When data are MCAR, the missing values can be thought of as a random sub-sample of the actual values. Missing at random (MAR): Patients with incomplete data differ from patients with complete data, but the pattern of “missingness” is traceable or predictable from other variables in the dataset, rather than being due to the specific variable on which the data are missing. Not missing at random (NMAR): Missing values do depend on unobserved ones. There are several methods for handling missing data [7,8]. Listwise deletion (or complete case analysis): If a row of the dataset has missing data for any of the parameters, then simply exclude that record from the analysis. It is the easiest way to deal with missing data and it requires minimal computing, but it probably excludes a great fraction of the entire dataset. Imputation methods: Attempt to estimate the values of the missing data and ‘fillin’ or impute new values. Once this has been achieved the analysis can proceed as if the dataset were ‘complete’. During this research, the second option, imputation methods, was chosen to deal with missing data. Dataset parameters were divided into two groups: binary variables and continuous ones; while population was stratified by sex (M and F) and age (under 50 years, between 50 and 59, between 60 and 69, between 70 and 79, over 80 years) into 10 groups. Then a specific function was created to fill missing values for each patient for each specific parameter with the mode (binary variable) / mean (continuous variable) of the not null values of the group it belonged to. 3. Results 3.1. Study population This section briefly presents the characteristics of the study’s sample, it consisted of 912 patients, 546 males (60%) and 366 females (40%) with a combined mean age of 69 (SD = 16) years. The mean value of the Charlson Comorbidity Index adjusted by age is 4 (± 3) [9,10]. After a preliminary analysis we decided to only consider features that had a percentage of missing data less than 25%, so we excluded: albumin, absolute number of CD4 and CD8 T cells at baseline; systolic and diastolic blood pressure, FiO2 and PO2 at hospitalization. 120 E. Lazarova et al. / Reuse of Clinical COVID-19 Patient Data 3.2. PCA The PCA was used to identify the most robust and representative features within the considered and previously mentioned group. In order to better underline how each variable contributes to the principal components, we decided to analyse the features normalized modules in the space identified by the first two principal components. Table 1 shows the first quartile of the features list calculated on the module of the first two components in the PCA space ordered by PCA weights. Table 1. 1st features quartile division according to PCA weights. Feature Width Feature Width Lactate Dehydrogenase 0,366 Alanine Transaminase 0,232 Azotemia 0,297 Prothrombin time PT% 0,230 Charlson Comorbidity Index (CCI) 0,296 Peripheral vascular disease 0,223 Age 0,294 Cerebrovascular disease 0,213 Aspartate Aminotransferase 0,293 Chronic antiplatelet home therapy 0,208 Ferritin 0,274 Cerebrovascular pathology 0,207 Heart failure 0,242 Coronary pathology 0,204 Congestive heart failure 0,242 Total bilirubin 0,200 Troponin I 0,241 SO2 ART / peripheral SO2 0,198 4. Discussion The preliminary results of this research basically show that influential features in COVID-19 patients include: Charlson Comorbidity Index; increased troponin levels and the levels of some inflammatory parameters at the moment of diagnosis. Regarding CCI, our study supports recent findings showing that previous history of cardiovascular disease, cerebrovascular disease, liver disease or acute kidney injury (e.g. all features included in the CCI) are the most important determinants for developing severe COVID-19 [11]. Because of these factors are even more important for outcome than the virulence of Sars-CoV-2 strains [12], we believe that they should always be considered as determinant features for decision support systems. As for the prognostic value of troponin, it is important to mention that COVID-19 remains associated with high risk for developing cardiovascular complication [13], so that it is essential to identify high-risk patients who may benefit from early aggressive treatment strategies. Previous studies have been focused on the prognostic impact of troponin levels in patients with COVID-19 [14]. Troponin elevation has been found to be associated with an increased risk of myocardial injury and death for COVID-19 patients. Lastly, confirming the association between some inflammatory parameters (e.g. ferritin) and disease severity [15], our study supports the involvement of a cytokine storm in the clinical outcome of the patients [16]. The implication of the host immune response in the disease process among COVID-19 patients suggests a potential role of antiinflammatory drugs as adjunctive therapy. Therefore, our preliminary analysis can be also used to define a subset of parameters to be rapidly considered to enhance the safety in terms of treatment for COVID-19 patients. However, follow-up studies evaluating the role of antiinflammatory drugs in well-defined sub-groups are warranted. E. Lazarova et al. / Reuse of Clinical COVID-19 Patient Data 121 5. Conclusion This manuscript’s aim is to present preliminary results of the analysis conducted on a dataset related to COVID-19 patients, that are quite aligned with current medical knowledge. We believe that a pre-processing of this type is adequate for the correct preparation of further and more accurate classification models based on machine learning to help medical staff in the therapeutic decisions related to the infection. Moreover, we can assess that a moderate level of missing data, if correctly addressed in the preprocessing phase, cannot prevent a correct classification in situations like the presented one. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] Fraccaro P, Pupella V, Gazzarata R, et al. The ligurian human immunodeficiency virus clinical network: a web tool to manage patients with human immunodeficiency virus in primary care and multicenter clinical trials. Med 2 0. 2013 Aug;2(2):e5. Giannini B, Riccardi N, Cenderello G, Di Biagio A, Dentone C, Giacomini M. From Liguria HIV Web to Liguria Infectious Diseases Network: How a Digital Platform Improved Doctors’ Work and Patients’ Care. AIDS Res Hum Retroviruses. 2018 Mar;34(3):239-240. Gazzarata R, Giannini B, Giacomini M. A SOA-Based Platform to Support Clinical Data Sharing. J Healthc Eng. 2017 May;Article ID:2190679. Gazzarata G, Gazzarata R, Giacomini M. A standardized SOA based solution to guarantee the secure access to EHR. Procedia Computer Science. 2015;64:1124-1129. Lee SA. Coronavirus Anxiety Scale: A brief mental health screener for COVID-19 related anxiety. Death Stud. 2020 Apr;44(7):393-401. Adiwijaya WU, Lisnawati E, Aditsania A, Kusumo DS. Dimensionality reduction using principal component analysis for cancer detection based on microarray data classification. Journal of Computer Science 2018;14(11):1521-1530. Bennett DA. How can I deal with missing data in my study?. Australian and New Zealand journal of public health. 2001;25(5):464-469. Soley-Bori M. Dealing with missing data: Key assumptions and methods for applied analysis. Boston University. 2013;23. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373-383. Glasheen WP, Cordier T, Gumpina R, Haugh G, Davis J, Renda A. Charlson Comorbidity Index: ICD9 Update and ICD-10 Translation. Am Health Drug Benefits. 2019;12(4):188-197. Chen N, Zhou M, Dong X, Qu J, Gong F, et al. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. The Lancet. 2020;395(10223):507-513. Xiaobo Y, Yuan Y, Jiqian X, Huaqing S, Jia’an X, et al. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. The Lancet Respiratory Medicine. 2020;8(5):475-481. Bansal M. Cardiovascular disease and COVID-19. Diabetes & Metabolic Syndrome: Clinical Research & Reviews. 2020;14(3):247-250. Imazio M, Klingel K, Kindermann I, et al. COVID-19 pandemic and troponin: indirect myocardial injury, myocardial inflammation or myocarditis?. Heart. 2020;106:1127-1131. Wu C, Chen X, Cai Y, et al. Risk Factors Associated With Acute Respiratory Distress Syndrome and Death in Patients With Coronavirus Disease 2019 Pneumonia in Wuhan, China. JAMA Intern Med. 2020;180(7):934–943. Ruan Q, Yang K, Wang W, Jiang L, Song J. Clinical predictors of mortality due to COVID-19 based on an analysis of data of 150 patients from Wuhan, China. Intensive care medicine. 2020;46(5):846848. 122 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200707 Development and Validation of Standardized Pain Management Documentation a Pia LILJAMOa,1 and Ulla-Mari KINNUNEN b Oulu University Hospital, Northern Ostrobothnia District, Oulu, Finland b University of Eastern Finland, Kuopio, Finland Abstract. Pain management, assessment and documentation is a crucial part of patient care. However, several studies show flaws in pain management processes. Documentation is not unified or even sufficient. The aim of this study was to describe how patient pain management has been recorded using the nursing diagnoses and nursing interventions of a standardized terminology, the Finnish Care Classification, (FinCC), and how that terminology should be further developed. The research data consisted of the daily nursing documentation notes of patient care episodes (n=806) during inpatient days (n=2564) at several specialty units (n=9). The documentation of pain management was found inadequate and insufficient. The results support the development of a new component, Pain management, and its attendant categories in the new version, FinCC 4.0, to help nurses document pain management in their daily work. Keywords. Documentation, Nursing Informatics, Pain Management, Standardized Nursing Terminology 1. Introduction Effective pain management is of great importance regardless of a patient’s illness, severity of the illness, the patient’s age, gender or any other circumstance [e.g. 1-3]. Pain management documentation, including information quality and availability, plays a significant role in patient medication safety, patients’ and health care professionals’ legal protection and the quality control, assessment and development of care [4]. Several studies show that there is room to improve in both the assessment and documentation of pain [5-8]. According to a review, the documentation of pain assessment and management is all too often unsystematic, insufficient or totally lacking. The same applies to the nursing decision-making process, which is left unclear. In addition, the patient’s insight into the pain symptoms is not documented. Common agreements or instructions for nurses regarding pain documentation vary greatly [5]. In addition to poor pain documentation, there is a lack of effort to evaluate the effectiveness of pain management interventions. Thus, educational interventions and standardization of pain management and documentation are urgently needed [5,9]. There is evidence that 1 Corresponding Author, Pia Liljamo, Oulu University Hospital, P.O. Box 10 90029 OYS, Oulu, Finland, E-mail: pia.liljamo@ppshp.fi. P. Liljamo and U.-M. Kinnunen / Standardized Pain Management Documentation 123 if patient care can be documented structurally using standards and common terminology, that documentation can yield more complete and reliable data that better meet the requirements placed on patient records, including in secondary use [10,11]. In Finland, the national nursing documentation model is based on the nursing process model in decision-making, the essential structured data components (nursing diagnoses, nursing interventions, nursing outcomes, nursing intensity and nursing discharge summary) and the standardized terminology Finnish Care Classiﬁcation (FinCC). The structure of the FinCC involves a three-level hierarchy featuring three separate classiﬁcations: the Classiﬁcation of Nursing Diagnoses (FiCND), Nursing Interventions (FiCNI) and Nursing Outcomes (FiCNO) [12]. FiCND and FiCNI have the same hierarchical structure, with component, main category and subcategory levels. The component level represents the most abstract level of documentation, while the main category and subcategory levels are more concrete levels of documentation. Nursing outcomes can be evaluated by means of the three qualiﬁers of FiCNO: ‘improved’, ‘stabilized’ and ‘deteriorated’. Version 3.0 of the terminology was implemented in 2012. For pain management documentation, a component named Sensory and neurological functions is used in FinCC 3.0 [12,13]. The FinCC expert group begun the terminology update process in 2018. First, evidence was gathered, e.g. national clinical practice guidelines, other scientific evidence and national guidelines, and legislation. Second, the update conducted a survey of end users, i.e. nurses, to receive feedback and development suggestions for the first version of the FinCC 4.0, finally published at the end of 2019 [14]. The aim of this study is to describe how patient pain management has been recorded using the standardized nursing diagnoses and nursing interventions of the FinCC 3.0. The information obtained from this study was utilized when updating the new FinCC 4.0 [14] with a new component, named Pain management, which has received positive feedback from nurses. 2. Methodology The retrospective EHR data, i.e. coded nursing data with free-text in all phases of the nursing process, were collected from one Finnish university hospital representing 36 specialized care inpatient units and 671 beds over a 15-day period in November 2014. The FinCC has been employed in that research hospital since 2007. Certain criteria for inpatient units were set before data were pooled out of the databases: the unit had received a good or excellent quality level of nursing documentation measured by an audit instrument [15]. Research units that passed the selection criteria (n=9) represent a variety of medical specialties: maternity, sensory system and respiratory disease, neurology, traumatology, gastric and plastic surgery, internal medicine and cardiac monitoring. Research data consisted of the daily nursing documentation notes of 806 patient care episodes over 2564 inpatient days, including the morning, evening and night shifts. For this study, all coded nursing data with free-text related to the patient’s pain management were selected from the total research material [12]. Records of pain medication were excluded from the data. To describe the research data, descriptive statistics were used. Qualitative methods were used to analyze the free-text nursing notes. Permission for this study was obtained from the research organization pursuant to the Guidelines of the Finnish Advisory Board on Research Integrity [12,16]. 124 P. Liljamo and U.-M. Kinnunen / Standardized Pain Management Documentation 3. Results The research data consisting of the different phases of the nursing process included 36 179 coded / structured nursing notes in total. Of all these coded nursing notes 2 139 (5.9%) were related to pain management (Table 1). In the first phase of the nursing process, Care planning/determining need for care, 278 (7.4%) FiCND nursing diagnoses related to pain management were employed. The most frequently encountered nursing diagnoses were ‘Chest pain’ (27.3%), ‘Acute pain’ (23.0%) and ‘Pain related to an intervention (e.g. surgical operation)’ (20.5%). Table 1. Main and subcategories of FiCND 3.0 and FiCNI 3.0 related to pain management used in the different phases of nursing process Phase of nursing process Care planning / Determining need for care Care planning / Setting goals of care Implementation of interventions Evaluation of nursing outcomes All phases of the nursing process FiCND 3.0 and FiCNI 3.0 categories FiCND 3.0 nursing diagnoses Chest pain (sc) Acute pain (mc) Pain related to an intervention (sc) Persistent pain (mc) Traumatic pain (sc) Inflammatory pain (sc) Headache (sc) Pain related to tissue damage (sc) Neuropathic pain (sc) Idiopathic pain (sc) Cancer pain (sc) Need for information related to pain (mc) Sensory and neurological functions (c) Number of categories related to pain management n (%) 278 (7.4) 76 (27.3) 64 (23.0) 57 (20.5) 37 (13.3) 24 (8.6) 12 (4.3) 5 (1.8) 2 (0.7) 1 (0.4) 0 (0.0) 0 (0.0) 0 (0.0) Number of all FiCND and FICNI 3.0 categories in the whole data n (%) 3 754 (10.4) 187 (6.5) 2 867 (7.9) FiCNI 3.0 nursing interventions Monitoring of the pain (mc) Assessment of the pain (sc) Pain management (mc) Assessment of the intensity of pain (sc) Guidance of the pain management (mc) Sensory and neurological functions (c) 1545 (5.6) 1470 (95.1) 38 (2.5) 35 (2.3) 2 (0.1) 0 (0.0) 129 (6.5) 27 566 (76.2) Total 2139 (5.9) 36 179 (100) 1 992 (5.5) *component = c; main category = mc; subcategory = sc Overall nurses made the most of coded nursing notes in the phase Implementation of nursing interventions (76.2%). Pain-related nursing interventions comprised 5.6% of the notes. The most-used nursing intervention related to the patient’s pain management P. Liljamo and U.-M. Kinnunen / Standardized Pain Management Documentation 125 was ‘Monitoring the pain’ (95.1%). Free-text annotations associated with ‘Monitoring the pain’ were related to the intensity and location of the pain. Pain intensity expressions included pain is under control (n=309), patient is pain-free or has no pain (n=288) or patient has headache or headache is relieved (n=29). Pain intensity, as documented by the numerical rating scale (NRS), was used 17 times. 4. Discussion Pain management, assessment and documentation is unsatisfactory [5-8], which hinders good quality care, patient care coordination and patient safety [4], and, as importantly, may result in unnecessary suffering and an unpleasant patient experience. The results show that patient pain management and assessment have been documented in a variable and generalized manner. At the Care planning / Determining need for care phase, only three categories of the FiCND 3.0 were used in a majority of cases. At the Implementation of interventions phase, one FiNCI 3.0 nursing intervention, ‘Monitoring of the pain’, was used in 95% of the cases. There was no indication in the nursing records that patient had been given guidance with pain management. These results are consistent with previous research, showing that patient education is inadequately documented, and nurses may not see the importance of documenting it [5,12]. In addition, the frequent use of free-text for documentation in lieu of the FinCC components and main and sub categories gives rise to terminology which is incompatible with good quality documentation [10,11]. Managing pain is one of the most important aspects of patient care [2,5]. In the FinCC 3.0, five categories permit the recording of pain management related nursing interventions [12]. Based on the results of this study and the feedback received from the nurses, the FinCC expert group is substantially vindicated in their decision to improve the terminology to better support the recording of pain management, and to include a new component ‘Pain management’ in FinCC 4.0 [14]. In the Pain management component of FiCND 4.0, there are 15 nursing diagnoses, with eight main categories with 23 concrete subcategories of nursing interventions in FiCNI. One new main category is ‘Non-pharmacological management of pain’ with 11 interventions like ‘Postural therapy’ and ‘Mental imagery’. There is also a category for ‘Assessment of the effects of non-pharmacological management of pain’, as well as ‘Assessment of the intensity of pain at rest’, and ‘Assessment of the intensity of pain when mobile’ [14]. The goal for these new terms in the revised terminology is to remedy nurses’ skills deficit to record more than just ‘painkillers given’, and more completely document the content of the care provided to patients [2,9], as well as facilitate better quality data within nursing records [10,11]. The documentation of medication, prescribed by the physician and administered by the nurse, is an essential part of pain management and its documentation. In this study, medication management was excluded. In the FinCC, the component ‘Medication’ is used for medication management and it bears consideration that pain management could also have been recorded using that component. 126 P. Liljamo and U.-M. Kinnunen / Standardized Pain Management Documentation 5. Conclusion Patient care must be evidence-based. In addition, the standardized terminology must derive from scientific evidence. Thus, daily patient care will be documented in a unified manner, and it will become more distinctly visible and transparent. This supports patient care quality and continuity, patient safety, and protects health care professionals from legal liability. ‘Pain management’-component will support the documentation of e.g. acute or chronic pain, or a newborn or elderly patient’s. Further, the FinCC 4.0 requires the validation of all its components to support the documentation needs. There is also interest of cross mapping the FinCC with the SNOMED CT in order to benefit from the use of different terminologies and to allow international health care data comparisons and benchmarking. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] Fillingim RB, Loeser JD, Baron R, Edward RR. Assessment of Chronic Pain: Domains, Methods, and Mechanisms. J Pain. 2016 September; 17(9 Suppl): T10–T20. doi:10.1016/j.jpain.2015.08.010. Alotaibi K, Higgins I, Day J, Chan S. Paediatric pain management: knowledge, attitudes, barriers and facilitators among nurses – integrative review. Int Nurs Rev. 2018; 65: 524–533. Minaya-Freire A, Ramon-Aribau A, Pou-Pujol G, Fajula-Bonet M, Subirana-Casacuberta M. Facilitators, Barriers, and Solutions in Pain Management for Older Adults with Dementia. Pain Manag Nurs. Available online 29 May 2020 (In Press) Available: https://doi.org/10.1016/j.pmn.2020.03.003 WHO, Medication Safety in Transitions of Care, Geneva, World Health Organization, 2019 (WHO/UHC/SDS/2019.9). Licence: CC BY-NC-SA 3.0 IGO. Available: https://apps.who.int/iris/bitstream/handle/10665/325453/WHO-UHC-SDS-2019.9-eng.pdf?ua=1 Heikkilä K, Peltonen L-M, Salanterä S. Postoperative pain documentation in a hospital setting: a topical review. Scand J Pain. 2016; 11;77–89. Heikkilä K, Axelin A, Peltonen L‐M, et al. Pain process of patients with cardiac surgery—Semantic annotation of electronic patient record data. J Clin Nurs. 2019; 28:1555–1567. Ardon A, Warrick M, Wickas T. A Multi-faceted Educational Approach for Pain Metric Recording Prior to Knee and Hip Arthroplasty: Effects on Documentation by an Acute Pain Service. Cureus. 2019; 11(6): e5030. DOI 10.7759/cureus.5030 Chineke I, Adams Curry M, Bell W, Flood D, Mishra P, Power S, Bernal-Mizrachi L. Improving Documentation of Pain and Constipation Management Within the Cancer Center of a Large Urban Academic Hospital. J Oncol Pract. 2020; 16 (3): e251– e256. Shoqirat N, Mahasneh D, Dardas L, Singh C, Khresheh R. Nursing Documentation of Postoperative Pain Management, A Documentary Analysis. J Nurs Care Qual. 2019; 34: 279–284. Saranto K, Kinnunen U-M, Kivekäs E, Lappalainen AM, Liljamo P, Rajalahti E, Hyppönen H. Impacts of structuring nursing records: a systematic review. Scand J Caring Sci. 2014; Dec; 28(4):629-47. doi: 10.1111/scs.12094. Epub 2013 Nov 18. Review. Macieira TGR, Chianca TCM, Smith MB, Yao Y, Bian J, Wilkie DJ, Lopez KD, Keenan GM. Secondary use of standardized nursing care data for advancing nursing science and practice: a systematic review. J Am Med Inform Assoc. 2019; 26(11): 1401-1411. Liljamo P, Kinnunen U-M, Saranto K. Assessing the relation of the coded nursing care and nursing intensity data: Towards the exploitation of clinical data for administrative use and the design of nursing workload. Health Informatics J. 2020; 26(1): 114-128. https://doi.org/10.1177/1460458218813613 Kinnunen U-M, Junttila K, Liljamo P et al. FinCC and the national documentation model in EHR – user feedback and development suggestions, Stud Health Technol Inform 2014; 201: 196–202. Kinnunen U-M, Liljamo P, Härkönen M, Ukkola T, Kuusisto A, Hassinen T, Moilanen K. User Guide, The Finnish Care Classification System, 4.0, Finnish Institute for Health and Welfare, Helsinki, 2019. Mykkänen M, Saranto K, Miettinen M. Nursing audit as a method for developing nursing care and ensuring patient safety. Nurs Inform 2012; 2012: 301. Finnish National Board on Research Integrity TENK. The ethical principles of research with human participants and ethical review in the human sciences in Finland. Finnish National Board on Research Integrity TENK guidelines 2019. Available: https://www.tenk.fi/sites/tenk.fi/files/Ihmistieteiden_eettisen_ennakkoarvioinnin_ohje_2019.pdf Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200708 127 Patient-Centered Development of a Digital Care Pathway for Arrhythmia Patients Pia LILJAMOa,1 Hanna SÄILYNOJAa, Kirsi TUOMIKOSKI a, Anja HENNER b and Kirsi KOIVUNENb a Oulu University Hospital, Northern Ostrobothnia District, Oulu, Finland b Oulu University of Applied Sciences, Oulu, Finland Abstract. Citizens are ready and willing to use various kinds of e-health services and Web-based portals. The purpose of this study was to describe the experiences of patients who underwent an arrhythmia procedure of the guidance they received as well as their needs and expectations for a future digital care path. The goal for the future is to utilize the results in other patient-centered digital service development activities. The research material was collected in a two-part thematic interview with patients who underwent an electrophysiology examination and supraventricular tachycardia catheter ablation procedure (n=7) or ablation treatment for atrial fibrillation (n=4). The preliminary digital care path was modified based on the results. The arrhythmia patient’s digital care path was tested in a workshop using a test group consisting of patients (n=3) and nursing staff (n=6). As a result, a digital care pathway for arrhythmia patients was completed. Keywords. Arrhythmia, digitalization, eHealth, patient guidance 1. Introduction In many countries, digital technologies are expected to bridge the rapidly growing gap between healthcare service demand and capacity. There are increasing demands for healthcare systems to shift to supporting consumers and patients in managing their own health and wellbeing. Digital services are becoming a recognized and integral part of all healthcare services [1,2]. Citizens are ready and willing to use different e-health services and Web-based portals. [3-5]. At this point, patients and citizens have decades of experience in using the Internet to search for health-related information. Even though patients trust healthcare professionals, they want look up symptoms on the Internet because information is easily accessible [6]. Patient portals are promising instruments for improving patient-centered care, as they provide patients with information and tools to better manage their health. The implementation of portals in both inpatient and outpatient settings gives health care providers more opportunities to support patients during hospitalization and after discharge [5]. Healthcare professionals are also optimistic about patient portals, provided that they are adequately informed in advance and that their organization is able to implement them well [7]. 1 Corresponding Author, Pia Liljamo, Oulu University Hospital, P.O. Box 10 90029 OYS, Oulu, Finland, E-mail: pia.liljamo@ppshp.fi. 128 P. Liljamo et al. / Patient-Centered Development of a Digital Care Pathway There are many different types of e-health services in use globally, many designed to support self-medication of chronic diseases [8]. Chronic disease self-care e-services have brought benefits to patients [9]. In Denmark, the Active Heart portal was developed for self-treatment of cardiovascular disease, and patient experiences have been positive [10]. Mobile applications developed for atrial fibrillation patients facilitate communication between patients and professionals, increase patient participation in treatment decisions, and encourage self-care [11]. Mobile applications significantly improved patient informedness, drug adherence, satisfaction with anticoagulation therapy, and quality of life [12]. The use of mobile health applications has a positive effect on interaction between patients and healthcare providers, which correlates with better health outcomes and supports patient self-care [13]. A digital care pathway (DCP) is a secure digital service channel for patients in a care relationship with a specialized health care hospital in Finland. It is part of the Health Village portal built as a cooperative effort of five Finnish university hospitals, led by Helsinki University Hospital, within the Virtual Hospital project [3,14]. The DCP application enables patients to interact online with professionals and receive information about specific illnesses or symptoms and help for self-care. They can also access frequently asked questions, various exercises, questionnaires and a tool for monitoring personal health data. Accessing the DCP requires strong identification using e-banking identifiers or mobile ID. A doctor’s referral or an existing care relationship is required [14]. The Virtual Hospital project established an e-health Development Model, which includes training and material to develop new CDPs. By the end of June 2020, eleven DCPs for different patient group had been implemented in Oulu University Hospital, with many more on the way. The goal of all DCPs is patient-oriented development, but with the arrhythmia path, patients were systematically involved from the outset. The aim of this study was to describe the experiences of patients who underwent an arrhythmia procedure of the guidance they received, as well as their needs and expectations regarding the content of the future digital care path. Looking forward, the aim is to standardize patient-centered guidance material for patients entering an arrhythmia care procedure and utilize the results to make the e-health Development Model more patient-oriented. 2. Methodology The topic of the second author’s developmental research thesis was chosen so that it is adjacent to both practical nursing and development work, thus harnessing existing professional competence to the best effect in the development work [15]. Research material was collected through thematic interviews. The study utilized service design methods and a qualitative research approach. The research material was analyzed using content analysis. The research was conducted in the following main phases. The definition phase described the current method of treating arrhythmia patients, and outlined the content of the DCP and its proposed integration to the care protocol. The research phase incorporated the experiences of patients, who underwent an arrhythmia procedure of the guidance they received, as well as their needs and expectations regarding the content of the future DCP. The material for this phase was collected in a two-part thematic interview with patients who underwent an electrophysiology examination and supraventricular tachycardia (SVT) catheter ablation procedure (n=7) and patients who underwent ablation treatment for atrial fibrillation (n=4). The age of P. Liljamo et al. / Patient-Centered Development of a Digital Care Pathway 129 the patients interviewed ranged from 24 to 70 years and the sample included both males (n=6) and females (n=5). The thematic interview was done in two phases. The first interview was conducted after the procedure while the patients were still in hospital, and the second over the phone about a week after the procedure. Telephone interviews were appropriate because of long distances between home and hospital. The aim of the interview was to obtain information about the guidance received at discharge and how it helped patients to cope at home. The thematic interview gathered patient experiences of guidance at different stages of the treatment path, including guidance from the referring physician, written guidance in the appointment letter, and a pre-call two weeks before the operation, hospitalization, follow-up after the operation, discharge, and aftercare at home. The researcher emphasized that she did not work in that unit and the content of the interview had no effect on patients’ care. The preliminary DCP was modified based on the study results during the development phase. The arrhythmia patient DCP was further tested using a test group composed of patients who underwent an arrhythmia procedure (n=3) and nursing staff from the Cardiology and Medical Day Wards (n=6). After testing the DCP, feedback and development ideas, including a written review by one patient, were collected in a workshop. The workshop was recorded and the material studied using content analysis, after which the material of the DCP was modified into its final form. 3. Results Patients’ experiences of the traditional guidance they received varied. Patients felt that the guidance provided by the referring physician was inadequate and poorly applicable to their own situation, arrhythmias, and prognosis. Information about treatment options, the operation and associated risks was perceived as insufficient. The content of the appointment letter was generally considered clear and informative enough, but some felt that all necessary instructions should have been included in the appointment letter alone. Some patients were confused when the pre-call came before other information, and the guidance received over the phone was considered difficult to absorb. More guidance, partly the same as during the pre-call, was given at admission to the hospital. The guidance given during the procedure itself was good, but some would have liked more information in advance. Guidance on coping at home was generally considered adequate, but there were patients who felt uncertain what to do if symptoms occurred. Two of the interviewed patients said they preferred a traditional control model. The reason was that they did not own a computer or a smartphone or use the Internet. According to one patient, the traditional way is easy: instructions arrive by mail and the caregiver calls you, so you do not have to search for information. Although the call was well regarded, nine out of 11 patients would have been willing to try the DCP. The digital service sought to preserve what patients preferred in the traditional guidance. Many were pleased with the content of the appointment letter and hoped the path would contain the same information. The path should have preparatory instructions and an electronic pre-information form, as well as content related to aftercare and recovery, and track the patient’s post-operation sensations. A more detailed description of the operation and the associated risks was requested, as was a FAQ section. Based on the analysis of the interview material, the preliminary DCP for arrhythmic patients was modified. The patients interviewed felt that guidance was generally good, but a lack of information was felt at each stage of the care pathway. Especially at the 130 P. Liljamo et al. / Patient-Centered Development of a Digital Care Pathway referral phase, patients experienced deficiencies in guidance. Patients craved information about arrhythmia, its prognosis, treatment options, and the planned operation. Feelings of anxiety, as reported by the patients, were most prevalent at the time of arrhythmia diagnosis, but subsided as they learned more about their condition. As requested by the patients, the content of the DCP sought a clear and simple writing style, avoiding medical terminology. The structure of the path was divided into preparatory instructions, description of the procedure, and aftercare. The associated risks were placed under their own heading so that the patient knew them before entering the hospital. Patients wanted photographs and videos of the operation as well as anatomical drawings of the structure of the heart. Based on the analysis of the workshop work, the development worker compiled a summary of the test patients’ requested improvements to content, visuality and usability using photos, text sequencing, and highlight boxes. The cardiology unit staff completed the final DCP and it will go into pilot operation in August 2020. 4. Discussion In this process, the results of involving patients closely in the development of the DCP for arrhythmia patients are encouraging and productive. Patients were willing to participate in the process and, according to the answers, interested in using a digital service, equally in previous research [3-5]. Many patients felt that the DCP was a good addition to current services. Good knowledge of one’s own arrhythmia and its treatment promotes patients’ ability to influence their own care [13]. Health Village emphasizes the active role and equality of citizens in promoting their own wellbeing by implementing online and digital self-care services as part of the care process [3]. In the case studied here, patients wanted to have increased, complete, and timely information about their illness. Some, lacking technological competencies, still preferred the traditional information letter by mail. Experience with web portals, e.g. using a cardiac telerehabilitation web portal, can be beneficial for patient education and may increase patients’ eHealth literacy skills [10]. Even though patients trust their physicians and their expertise, many prefer the Internet because it provides easy access to information [6]. Interviews and workshops revealed that in addition to text, the material of a DCP should include images and videos. The absorption and recall of patient guidance can be improved by using a variety of guidance materials [16]. An important addition was a video where a patient who had undergone the same procedure shared their experience; there is evidence that peer messaging in guidance reduces patient anxiety [17]. Digital services are expected to improve patient access to care and facilitate the workflow of healthcare professionals. Expectations for the cost-effectiveness and impact of digital transactions are high [3]. New e-health solutions must provide evidence-based benefits and be safe to use, and their impact on patients and organizations needs to be clarified and evaluated [1,2]. Recent studies [e.g. 18] show that an organization’s view of the health services and care can differ in many ways from patients’ experiences. Patients are experts in their own well-being and therefore an important resource in the development of care. Because the study was qualitative and participants were selected non-randomly, based on their willingness to cooperate, the results cannot be generalized. However, the goal of this study was to provide a rich, contextualized understanding of arrhythmia patients’ experience through the intensive study of particular cases. Through this study, we gained evidence and experience on how to involve patients more systematically in the development of DCP. P. Liljamo et al. / Patient-Centered Development of a Digital Care Pathway 131 5. Conclusion The content of the digital care pathway for arrhythmia patients was produced in collaboration with patients and caregivers. Patient experiences and suggestions for the guidance material were central to the result. These patient-centered methods can be utilized in the development of digital pathways for other patient groups. By involving patients in the development, the quality of service and commitment can be promoted. References [1] Ministry of Social Affairs and Health, Finland. Information to support well-being and service renewal. eHealth and eSocial Strategy 2020. Available in Finnish: http://urn.fi/URN:ISBN:978-952-00-3548-8. [2] Schofield P, Shaw T, Pascoe M. Toward Comprehensive Patient-Centric Care by Integrating Digital Health Technology With Direct Clinical Contact in Australia. J Med Internet Res 2019;21[6]:e12382. [3] Saranto K, Kivekäs E, Kuosmanen P, Kinnunen U-M. Electronic Health Services in the Patients’ Daily Activities - Willingness to Use Health Village Services. Stud Health Technol Inform. 2018; 247:586-90. Doi: 10.3233/978-1-61499-852-5-586 [4] Hyppönen H, Aalto AM. Citizens experiences of e-health and e-welfare services. In: Vehko T, Ruotsalainen S, Hyppönen H (Eds). E-health and e-welfare of Finland. Checkpoint 2018. National Institute for Health and Welfare [THL], Helsinki, Finland. p. 148-167. Available: http://urn.fi/URN:ISBN:978-952-343-326-7 [5] McAlearney AS, Sieck CJ, Gaughan A, et al. Patients’ Perceptions of Portal Use Across Care Settings: Qualitative Study. J Med Internet Res 2019;21[6]:e13126. [6] Champlin S, Mackert M, Glowacki EM, Donovan EE. Toward a Better Understanding of Patient Health Literacy: A Focus on the Skills Patients Need to Find Health Information. Qual Health Res. 2017;27(8):1160-1176. doi:10.1177/1049732316646355 [7] Kujala S, Hörhammer I, Kaipio J, Heponiemi T. Health professionals' expectations of a national patient portal for self-management. Int J Med Inform. 2018;117:82-87. doi:10.1016/j.ijmedinf.2018.06.005 [8] Rush KL, Hatt L, Janke R, Burton L, et al. The efficacy of telehealth delivered educational approaches for patients with chronic diseases: A systematic review. Patient Educ Couns. 2018;101(8):1310-1321. [9] Neubeck L, Coorey G, Peiris D, et al. Development of an integrated e-health tool for people with, or at high risk of, cardiovascular disease: The Consumer Navigation of Electronic Cardiovascular Tools [CONNECT] web application. Int J Med Inform. 2016; 96:24-37. doi:10.1016/j.ijmedinf.2016.01.009 [10] Melholt C, Joensson K, Spindler H, et al. Cardiac patients' experiences with a telerehabilitation web portal: Implications for eHealth literacy. Patient Educ Couns. 2018;101(5):854-861. [11] Kotecha D, Chua WWL, Fabritz L, et al., European Society of Cardiology (ESC) Atrial Fibrillation Guidelines Taskforce, the CATCH ME consortium and the European Heart Rhythm Association (EHRA). European Society of Cardiology smartphone and tablet applications for patients with atrial fibrillation and their health care providers. EP Europace, 2018; Feb. 20(2):225-33. [12] Guo Y, Lane DA, Wang L, Chen Y, Lip GYH; mAF-App II Trial investigators. Mobile Health (mHealth) technology for improved screening, patient involvement and optimising integrated care in atrial fibrillation: The mAFA (mAF-App) II randomised trial. Int J Clin Pract. 2019;73(7):e13352. [13] Qudah B, Luetsch K. The influence of mobile health applications on patient - healthcare provider relationships: A systematic, narrative review. Patient Educ Couns. 2019;102(6):1080-1089. [14] Digital Health Village. https://www.digitalhealthvillage.com/en/home [Accessed on 15th of July, 2020]. [15] Säilynoja H. Development of a digital treatment path for an arrhythmia patient at the Department of Cardiology, Oulu University Hospital. Oulu University of Applied Sciences Master´s Degree Program in Development and Management of Health and Social Care. 2020. Available: http://urn.fi/URN:NBN:fi:amk-202005098116. [16] Villanueva C, Talwar A, Doyle M. Improving informed consent in cardiac surgery by enhancing preoperative education. Patient Educ Couns. 2018; 101(x):2047-53. [17] Salmasi S, Kwan L, MacGillivray J, et al. Assessment of atrial fibrillation patients’ educations need from patient and clinician perspectives: A qualitative descriptive study. Thromb. Res. 2019;173:109-16. [18] Bombard Y, Baker G.R, Orlando E. et al. Engaging patients to improve quality of care: a systematic review. Implementation Sci 2018; 13, 98. https://doi.org/10.1186/s13012-018-0784-z 132 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200709 Word-Final Phoneme Segmentation Using Cross-Correlation Emilian-Erman MAHMUTa,1, Stelian NICOLAa and Vasile STOICU-TIVADARa a Department of Automation and Applied Informatics, Politehnica University Timisoara, Romania Abstract. The goal of this paper is to present a word-final target phoneme automated segmentation method based on cross-correlation coefficients computed between a reference sound wave and a sample sound wave. Most existing Speech Sound Disorder (SSD) Screening solutions require human intervention to a greater or lesser extent and use segmentation methods based on hard-coded time frames. Moreover, existing solutions extract features from the frequency domain, which entails large amounts of computational power to the detriment of real-time feedback. The preprocessing algorithm proposed in this paper, implemented in a Python version 3.7 script, automatically generates 2 new .wav files corresponding to the phonemes found in word-final position in the initial sound waves. The newly-generated .wav files are meant to be used as valid and homogeneous input in a subsequent classification stage aimed at rigorously discriminating mispronunciations of the target phoneme and assist Speech-Language Pathologists (SLPs) with the SSD screening. Keywords. Cross-correlation, audio segmentation, SSD 1. Introduction Using over 100 distinct muscles in order to control minute movements, triggered by nerve impulses traveling through the cortical and subcortical structures of the brain at speeds over 100 m/s, the articulatory apparatus displays the most complex behavior in the human body [1-2]. If undetected and untreated in due time, language disorders may have severe consequences on the development of children’s personality and behavior, including scarcity at school and poor social skills. The ever-increasing prevalence of persistent SSDs (Speech Sound Disorders) among preschoolers and elementary schoolers [3] in conjunction with the key role played by early diagnosis and subsequent treatment in the therapeutic outcome reinforce the need for an automated mispronunciation screening solution. The automated screening output stored in an anonymized, online database would provide access to analyses and statistics based on various demographic parameters of interest. The screening application should rigorously assess the similarity between a reference segment (the Speech Language Pathologist’s pronunciation of a word or logatome containing the target phoneme) and a sample segment (the subject’s pronunciation of the same word or logatome) of a target phoneme 1 Corresponding Author, Emilian-Erman Mahmut, Politehnica University Timisoara, P-ta Victoriei no. 2, Timisoara, Romania; E-mail: emilian.mahmut@aut.upt.ro E.-E. Mahmut et al. / Word-Final Phoneme Segmentation Using Cross-Correlation 133 within the same phonetic context. In any given utterance, the neighboring phonemes affect the target phoneme both progressively (sound n affects sound n+1) and regressively (sound n+1 affects sound n). Most existing audio segmentation algorithms serve as a preliminary (pre-processing) step whereby new segments are created to be used as input for subsequent feature extraction, analysis and/or classification. Such preprocessing algorithms are mainly devised and used in automatic speech recognition (ASR) and multimedia applications, such as, for instance, music information retrieval (MIR). Speech processing algorithms extract features from the time and frequency domains and aim mainly to provide solutions for a robust classification of several categories of sounds: noise, silence, voiced and unvoiced phonemes and/or parts thereof. Real-time output is a ubiquitous requirement and it is achieved at the cost of a large amount of computational power, usually involving a large amount of training data in the subsequent classification stage. The fixed frame size and rate (FFSR) technique is widely-used in the ASR systems with solid results, except for recognition of speech in noisy environments. Paper [4] gives a comprehensive presentation of the challenges of this research field and proposes a speech envelope-based segmentation solution (inspired and supported by the neuroscientific perspective) to the shortcomings of the FFSR technique. An extensive classification of speech segmentation algorithms and feature extraction techniques is given in paper [5]. In reference [6] we presented the cross-correlation based audio segmentation method for phonemes in word-initial position and briefly discussed the corresponding results. The method presented in this paper focuses on segmenting target phonemes in the final position within an utterance. The pre-processing algorithm is meant to provide adequately-extracted reference and sample segments that are homogeneous in terms of duration and context, to serve as valid input for a subsequent processing stage, i.e. an automated SSD screening solution, which is the main objective of our research project. Several criteria were adopted in the development of the SSD Screening application: noninvasiveness (reduced emotional stress), cost-efficiency (using open-source frameworks), time-efficiency (real-time feedback), mobility (access to remote/rural areas), and modularity (connectivity with computer-aided speech therapy applications). 2. Method The homogeneously-trimmed segments are obtained using a Python version 3.7 script. The flowchart below (Figure 1) describes the segmentation of the phoneme found in final position within an utterance. The algorithm consists of 5 main steps: • • • In step 1 the algorithm reads both audio files (SLP and SUB) in reverse order and generates 2 corresponding .csv (comma-separated value) files based on the .wav (waveform audio file format) file amplitude data; The two .csv files consisting of the amplitude data in reverse order are read in step 2. A data range encompassing the first 5000 values was considered sufficient to cover the target phoneme found in final position. Cross-correlation equates the lag value with the number of indexes by which the sample signal (SUB) is shifted to the left or to the right of the reference signal (SLP); The following step (step 3) declares and initializes two variables, max_corr_l and max_corr_r, in order to compute the maximum cross-correlation corresponding to each displacement, respectively to the left (lag_l) and to the right (lag_r); 134 • E.-E. Mahmut et al. / Word-Final Phoneme Segmentation Using Cross-Correlation The cross-correlation coefficients are computed for every single lag to the left and to the right. If the correlation coefficient computed for the current lag is larger than the correlation coefficient computed for the previous lag, then max_corr_l respectively max_corr_r is assigned the new maximum value. The algorithm stores the index of the maximum correlation (lag_max_l or lag_max_r). Step 4 is completed once all the 10,000 correlation coefficients have been computed for the displacement to the left (lag range: 0; - 4999) and, respectively, to the right (lag range: 0; 4999). Two maximum correlation coefficients are identified, one for each direction. Figure 1. Pre-processing algorithm flowchart. • In step 5, the 2 maximum correlation coefficients (left and right) are compared and 2 new audio segments (new WAV SLP and new WAV SUB) are generated. If max_corr_l > max_corr_r, the newly-generated audio files will consist of the amplitude data within the (leng_slp - k; leng_slp) range for the SLP, and within the (leng_sub – lag_max_l-k; leng_sub-lag_max-l) range for the SUB. If max_corr_l < max_corr_r, the newly-generated audio files will consist of the amplitude data within the (leng_slp - lag_max_r-k; leng_slp – lag_max_r) range for the SLP, and within the (leng_sub-k; leng_sub) range for the SUB. The value of the k constant appearing in the aforementioned ranges determines the number of amplitude data contained in the newly-generated audio files. The value of k (7,000) was determined empirically so as to cover the target phoneme in the final position within the analyzed utterance. The Python script allows for a fairly easy modification of the value of k. However, higher values of k determine the inclusion of larger portions of the preceding phoneme into the newly generated .wav files (reference and sample segment). The 2 newly-generated audio files have the following parameters: sample rate = 44100.0 Hz, maximum duration = 1.0s, frequency = 440.0 Hz. E.-E. Mahmut et al. / Word-Final Phoneme Segmentation Using Cross-Correlation 135 3. Results The pronunciations of a population of 30 primary school pupils (subjects aged 5-7 from the CNB College in Timisoara) were fed to the pre-processing segmentation algorithm. For 63.33% of the subjects the maximum cross-correlation values were obtained by shifting the sample signal (SUB) to the left of the reference signal (SLP), while for the remaining 36.77% the maximum cross-correlation values were identified by moving the sample signal to the right. Figure 2 shows the polynomial trendline of the initial .wav files (whole word, /f-a-r/, Romanian word for headlight): reference (SLP, left side) and sample (SUBJECT, right side). Figure 3 displays the polynomial trendline for the newlygenerated segments: reference versus sample (final phoneme, /r/). As it may be observed, the R-squared value of the automatically-generated segments is higher (i.e. major goodness-of-fit) as opposed to the corresponding R-squared value of the manually segmented initial audio file (Figure 2). The maximum and minimum amplitude values (crests and troughs) are marked by orange squares in the diagram. Figure 2. Initial .wav files, reference (left, R2 = 0.444) versus sample (right, R2 = 0.3704) (whole word /f-a-r/). Figure 3. Newly-generated segments: reference (left, R2 = 0.5206) versus sample (right, R2 = 0.4282) (final phoneme /r/). Table 1 contains the output data: the maximum lag values to the left (lag_max_l) are contained in the [-3858; -113] range and the maximum lag values to the right (lag_max_r) are included in the [0; 2036] range. The output data confirms the effectiveness (in terms of computational workload) of the empirically-determined data range of 5000 amplitude data values used in the Python script. Increasing such data range does not produce better cross-correlation values. The R2 value is constant (0.5206) for all the segments where the maximum cross-correlation value is to the left while in the 136 E.-E. Mahmut et al. / Word-Final Phoneme Segmentation Using Cross-Correlation cases where the maximum cross-correlation value is found to the right, the R 2 value is variable. Table 1. Maximum cross-correlation values and corresponding lags Subject (SUB) 1 2 3 4 … 30 max_l (lag_max_l) 0.010026 (-2918) 0.034544 (-1791) 0.021644 (-266) 0.024533 (-113) 0.018012 (-270) max_r (lag_max_r) 0.027897 (1493) 0.018945 (863) 0.016214 (388) 0.019913 (152) 0.019100 (161) R2_SLP 0.4939 0.5206 0.5206 0.5206 R2_SUB 0.4441 0.4706 0.4812 0.4146 0.5305 0.2765 4. Discussion and Conclusions The cross-correlation based pre-processing algorithm is an efficient solution that generates homogeneous segments to be used as valid input for the classification stage. It does not have a temporal limitation (such as the FFSR fixed-size frames and shifts [4]) and it is language-independent. The R2 values obtained for the newly-generated segments are better than the R2 values corresponding to the initial, manually-segmented audio files. The value assigned to the k constant was validated by the newly-generated audio files. Comparing an utterance issued by an adult voice (SLP) with that of a child (primary schoolers) is a limitation of the current state of our algorithm. Therefore, our new approach to this research thread entails the calculation of the autocorrelation coefficient for the 2 newly-generated segments so as to determine the energy level of each segment. Subsequently, the ratio between the 2 aforementioned autocorrelation coefficients (autocorrel_slp/autocorrel_sub) will be used to increase the energy level of the sample files (subject signals). The current classification stage performed in our C# (.NET) application [7] is based on the representation of the polynomial trendline of the audio files. To increase the precision of the screening solution, a logarithmic function will also be added, in an attempt to obtain higher R2 values (better goodness-of-fit) for the new segments. References [1] [2] [3] [4] [5] [6] [7] Nuwer MR, Pouratian N. Monitoring of neural function: electromyography, nerve conduction, and evoked potentials, Youmans and Winn Neurological Surgery (7th ed.), Winn HR, Philadelphia, PA: Elsevier, 2017. Conant D, Bouchard KE, Chang EF. Speech map in the human ventral sensory-motor cortex, Current Opinion in Neurobiology, 2014; 24(1): 63–67. Wrenn Y, Miller LL, Peters TJ, Emond A and Roulstone S. Prevalence and Predictors of Persistent Speech Sound Disorder at Eight Years Old: Findings From a Population Cohort Study, J Speech Lang Hear Res. 2016 Aug; 59(4): 647–673. Lee B and Cho KH, Brain-inspired speech segmentation for automatic speech recognition using the speech envelope as a temporal reference, Scientific Reports 6, Article number: 37647; 2016 Nov. Alaa ES, Sherif MA, Salah EH, Mohsen R. A Review: Automatic Speech Segmentation, International Journal of Computer Science and Mobile Computing; 2017 Apr; Vol. 6 Issue.4, p. 308-315. Mahmut EE, Nicola S and Stoicu-Tivadar V. Cross-correlation based automated segmentation of audio samples, Studies in Health Technology and Informatics, 2020 Jun 26; (Ebook) Volume 272: The Importance of Health Informatics in Public Health during a Pandemic, IOS Press, p. 241-244 Mahmut EE, Berian D, Della Ventura M and Stoicu-Tivadar V, Optimization of Entropy-based automated Dyslalia Screening Algorithm, Studies in Health Technology and Informatics, 2020 Jun 16; (Ebook) Volume 270: Digital Personalized Health and Medicine, IOS Press, p. 357 – 361. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200710 137 A Decentralized Framework for Biostatistics and Privacy Concerns Paul MANGOLDa,b,c,1, Alexandre FILIOTa, Mouhamed MOUSSAd, Vincent SOBANSKIa, Gregoire FICHEURa,e, Paul ANDREYa and Antoine LAMERe a CHU Lille, INCLUDE: Integration Center of the Lille University Hospital for Data Exploration, 59000, Lille, France. b INRIA Lille Nord Europe, Magnet Team, 59650, Villeneuve d’Ascq, France. c ENS de Lyon, 69007, Lyon, France. d CHU Lille, Pole d’Anesthesie-Reanimation, 59000, Lille, France. e Univ. Lille, CHU Lille, ULR 2694 - METRICS: Evaluation des Technologies de sante et des Pratiques medicales, 59000, Lille, France. Abstract. Biostatistics and machine learning have been the cornerstone of a variety of recent developments in medicine. In order to gather large enough datasets, it is often necessary to set up multi-centric studies; yet, centralization of measurements can be difficult, either for practical, legal or ethical reasons. As an alternative, federated learning enables leveraging multiple centers’ data without actually collating them. While existing works generally require a center to act as a leader and coordinate computations, we propose a fully decentralized framework where each center plays the same role. In this paper, we apply this framework to logistic regression, including confidence intervals computation. We test our algorithm on two distinct clinical datasets split among different centers, and show that it matches results from the centralized framework. In addition, we discuss possible privacy leaks and potential protection mechanisms, paving the way towards further research. Keywords. federated learning, data privacy, biostatistics 1. Introduction The advent of machine learning methods and the ongoing movement towards wide and high-quality data collection have made biostatistics a crucial component in medical research. Constituting large and representative datasets, which are mandatory either to have enough statistical power or to improve models’ generalization, is not always feasible within a single medical center. A popular approach is thus to centralize data from multiple centers in one leading site and conduct the study there. With medical data, this centralization is often a practical challenge, as data is sensitive and must be handled within a controlled environment abiding by strong legal and ethical constraints. An alternative approach, known as federated learning, consists in training statistical models in a decentralized way, leaving the data on each site, running computations 1 Corresponding Author, Paul MANGOLD, INRIA Lille Nord Europe, 40 avenue Halley, 59650 Villeneuve d’Ascq, France; E-mail: paul.mangold@inria.fr 138 P. Mangold et al. / A Decentralized Framework for Biostatistics and Privacy Concerns locally and communicating aggregated information between centers during the training phase. Such an approach has already been applied to medicine in a few studies, with the goal of preserving the privacy of sensitive data [1,2], as well as data owners’ sovereignty. This work takes a first step towards defining and implementing a decentralized learning framework for medicine, which differs from previous works in that it allows full decentralization, meaning that it does not require any center to play a central role in the computation (although the latter case remains an option). We aim at proving that this framework can produce results virtually identical to the ones obtained in a centralized setting on actual clinical data. To do so, we use two distinct datasets, fit logistic regressions and compute confidence intervals of the estimates. Finally, we put our work into perspective by highlighting some privacy concerns, together with privacy-preserving mechanisms that could address them, depending on desired privacy levels. 2. Methods 2.1. Decentralized Protocol for Logistic Regression Logistic regression is fit by estimating the parameters that maximize the likelihood over the observed dataset. Iterative algorithms, such as gradient descent, are commonly used to do so. A decentralized version of gradient descent, as described in [3], can thus be used. In this setting, each center runs the following protocol: • Initialization. Initialize local variables, and divide features by agreed-upon maximum values. This ensures faster convergence, without sharing private data. • Training. Iterate until convergence: ‫ כ‬Local Update. Compute a local gradient and update local parameters. ‫ כ‬Communication and Aggregation. Send local parameters to other centers and await theirs. Average the local and received parameters. Assign results as the new local parameters. • Confidence intervals computation. Compute Fisher information on local dataset, and send it to others. Use these values to compute global confidence intervals. Note that categorical variables are encoded as dummy variables, whose proper encoding requires either a set of agreed-upon values, or extra communications to determine those. 2.2. Datasets and Learning Scenarios The first clinical dataset used in our experiments consists in measurements collected during caesarean sections performed at the Lille University Hospital. We aim at predicting fetal acidosis at birth based on six explanatory variables, including blood pressure drops during the operation. To simulate a multi-centric environment, the 775 records were randomly assigned to four equally-sized (up to one sample) chunks. So as to provide reproducible results, the UCI heart disease dataset [4], available at https://archive.ics.uci.edu/ml/datasets/heart+Disease, was also used. We aim at predicting the presence of a heart disease based on twelve explanatory variables, which mainly encompass clinical measurements at rest and during a controlled P. Mangold et al. / A Decentralized Framework for Biostatistics and Privacy Concerns 139 physical effort. This data was collected in four distinct medical centers, with variable sample sizes (respectively 303, 261, 130 and 46 records, for a total of 740). Three different learning scenarios are studied. The “centralized” scenario, in which the entire collated dataset can be used by a single center. The “all alone” scenario, in which each center tries to perform the study using only its local dataset. The “decentralized” scenario, in which centers communicate together without directly exchanging data records, following the protocol detailed in section 2.1. To compare those three scenarios, we observe the estimated coefficients and their confidence intervals, checking whether they match, and if so, how precisely. 3. Results 3.1. Implementation Our decentralized framework is implemented as a R package, available under the MIT license at https://gitlab.com/include-project/federate. The developed package handles network communications, and provides a way to simulate decentralized algorithms locally for testing purposes. It currently implements logistic regression with basic gradient descent, but may easily be expanded to comprise new algorithms, as only logical parts need to be re-implemented. Algorithms are implemented in R with C++ integration using Rcpp. C++ libraries Armadillo and Asio are used for linear algebra and networking, with their respective R bindings RcppArmadillo and AsioHeaders, available at CRAN. 3.2. Experimental Results The three learning scenarios were run on both the caesarean section and heart disease datasets. In the decentralized scenario, the algorithm is run for a few thousands iterations, inducing as many communication rounds. For both datasets, resulting odds ratios and confidence intervals are the same as in the centralized scenario up to 10−5 precision on each coefficient. Better precision may be achieved at the cost of more communication rounds and tuning. As for the all alone scenario, it results in unsatisfactory estimations in each center, due to insufficient population size. Odds ratios obtained in these various settings are shown in Figure 1. Clinical results on the caesarean dataset match those reported in [5]. Scripts for the heart disease dataset are provided in the git repository. 3.3. Privacy Concerns Although decentralized machine learning naturally favors privacy preservation, keeping the data on site does not fully prevent sensitive information leaks. For federated deep learning, [6] show that shared information may reveal parts of the training dataset. Since logistic regression shares the same underlying optimization procedure as deep learning, it may be vulnerable to similar attacks. This raises major concerns as local datasets are often small (e.g. for studies on rare diseases), the whole purpose being to gather enough data records to achieve statistical significance. Furthermore, individual 140 P. Mangold et al. / A Decentralized Framework for Biostatistics and Privacy Concerns Figure 1. Odds ratios and 95% confidence intervals learned in the three distinct scenarios for the caesarean dataset (left, randomly uniformly split in 4), and heart dataset (right, split across the 4 actual sources, showing a subset of variables for readability). Circles, triangles and squares respectively represent the “centralized”, “all alone” and “decentralized” scenarios. Each color represents odds ratio learned by a center. Horizontal grey line is 1, and confidence intervals not crossing it suggest a correlation exists between the variable and the outcome. records are not the only sensitive information that may be revealed: local aggregated values, e.g. mortality rate, may be retrieved, which can expose centers’ internal practices. 4. Discussions 4.1. Privacy Improvement Mechanisms Local aggregates can be protected through secure aggregation [7], a protocol that consists in adding random masks on sent information that cancel out when computing the result. This yields an exact global average while preventing sent information from being revealed. It could be used during aggregation and confidence interval computations steps. However, this does not protect individual records from leaking. Differentially Private (DP) mechanisms [8] address this problem, by adding noise that blurs individual contribution on shared values, making it almost impossible to guess the presence of an individual in the dataset. This, however, widely impacts results’ precision, and obtaining good accuracy while guaranteeing privacy generally requires very fine tuning of algorithms. Such mechanisms could be used at every communication step of our protocol, either before sending values (Local DP) or after their aggregation (Global DP), depending on trusted parties. Table 1 summarizes the impact of these mechanisms and describes who can infer information, thus requiring others’ trust. Table 1. Privacy mechanisms and who can infer what about records and aggregated values from local datasets. Mechanism Who can infer Data Records Local Aggregates Precision None All Not Protected Not Protected Exact Sec. Agg. All Only Origin Protected Protected Exact Global DP Aggregator(s) Protected Not Protected Inexact Local DP No one Protected Not Protected Inexact Sec. Agg. + DP No one Protected Protected Inexact P. Mangold et al. / A Decentralized Framework for Biostatistics and Privacy Concerns 141 4.2. Perspectives Our framework gives accurate results, echoing the conclusions of [2], within an acceptable number of communications rounds. Its modularity allows further experiments with more advanced optimization algorithms. It can also be extended to different learning tasks, including training deep neural networks, e.g. to classify medical images or learn word embeddings from hospital records. Besides, keeping data on site does not guarantee privacy. The latter should thus receive more attention in the future, notably by implementing secure aggregation and differentially private mechanisms. Precisely quantifying required privacy levels is mandatory to make informed choices of protection mechanisms. Therefore, a comprehensive study of effective data leakage appears to be the next step towards this direction. Full decentralization could further improve privacy, by enabling network topologies in which pairs of centers are distanced based on their mutual trust level. It may also lead to developing broader studies, directly leveraging measures from connected devices at patients’ homes, or allow learning personalized parameters adjusted to local specificities. 5. Conclusion Our decentralized framework gives very promising results, near-exactly matching those of the centralized scenario when fitting logistic regressions on two distinct clinical datasets. Its design and open-source implementation allow for its re-use, improvement and extension to other learning tasks. We have also identified a set of privacy-preserving mechanisms whose informed use can ease collaborations between clinical data holders. References [1] Jochems A, Deist TM, El Naqa I, Kessler M, Mayo C, Reeves J, et al. Developing and Validating a Survival Prediction Model for NSCLC Patients Through Distributed Learning Across 3 Countries. Int J Radiat Oncol Biol Phys. 2017 Oct 1;99(2):344-352. [2] Bogowicz M, Jochems A, Deist TM, et al. Privacy-preserving distributed learning of radiomics to predict overall survival and HPV status in head and neck cancer. Sci Rep. 2020 Mar 11;10(1):4542. [3] Lian X, Zhang C, Zhang H, Hsieh C, Zhang W, Liu J. Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17); 2017 Dec 04-09; Long Beach, CA, USA.; 2017. p. 5336–46. [4] UCI Machine Learning Repository [Internet]. Irvine (CA): Dua D, Graff C. 1988 - [cited 2020 Jul 22]. Available from: http://archive.ics.uci.edu/ml. [5] Ngan Kee WD, Lee A. Multivariate analysis of factors associated with umbilical arterial pH and standard base excess after Caesarean section under spinal anaesthesia. Anaesthesia. 2003 Feb;58(2):125-30. doi: 10.1046/j.1365-2044.2003.02888.x. PMID: 12562407. [6] Geiping J, et al. Near optimal neural network estimator for spectral¨ x-ray photon counting data with pileup. arXiv:2003.14053v1 [Preprint]. 2020 [cited 2020 Jul 22]: [26 p.]. Available from: https://arxiv.org/abs/2003.14053. [7] Bonawitz K, Ivanov V, et al. Practical Secure Aggregation for Privacy-Preserving Machine Learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS ’17); 2017 Oct 30-Nov 3; Dallas, TX, USA. 2017. p. 1175–91. [8] Dwork C, Roth A. The Algorithmic Foundations of Differential Privacy. Found. Trends Theor. Comput. Sci. 2014 Aug;9(3–4):211-407. Available from: http://dx.doi.org/10.1561/0400000042 142 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200711 Do You Know Who Is Talking to Your Wearable Smartband? Andrei KAZLOUSKIa,b,1, Thomas MARCHIOROa,b, Harry MANIFAVASa,b and Evangelos MARKATOSa,b a Computer Science Department, University of Crete, Greece b Institute of Computer Science, Foundation for Research and Technology, Greece Abstract. We study seven fitness trackers and their associated smartphone apps from a wide variety of manufacturers, and record who they are talking to. Our results suggest that some of them communicate with unexpected third parties, including social networks, advertisement websites, weather services, and various external APIs. This implies that such unanticipated third-parties may glean personal information of users. Keywords. fitness trackers, wearable devices, security, privacy 1. Introduction Having shipped more than 17 million smartbands during the first quarter of 2020, the smart wearable devices market is expected to reach more than 60 million devices per year2. The increasing trend towards an active lifestyle, and growing health concerns are likely to boost the sales of wearables, and to reach a much higher penetration in the worldwide population. Although the increasing use of wearables in general, and smartbands in particular, promotes healthier habits, it may have raised public concerns with respect to the privacy they provide. Such concerns are mainly related to the possible leakage of fitness data and other private information. Health data. Wearable smartbands collect personal and fitness-related data that might include user’s heartbeat, sleep patterns, habits, and the exercising routine. Additionally, sensitive data like age, height, gender, weight, and body fat can be inserted manually. Other sensitive data. At present vendors store personal data of users on proprietary servers. However, since the capability for remote communication is there, apps may use it to contact not only the manufacturer cloud, but other third-party servers as well. During these communications various other sensitive information can be leaked, including location, IP and MAC addresses, an email address, and possibly the phone name/model. 1 Corresponding Author, Andrei Kazlouski, E-mail: andrei@ics.forth.gr. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skodowska Curie grant agreement No 813162. The content of this paper reflects the views only of their author (s). The European Commission/ Research Executive Agency are not responsible for any use that may be made of the information it contains. 2 https://www.tizenhelp.com/huawei-xiaomi-dominated-in-chinese-wearable-market-for-q1-2020/ A. Kazlouski et al. / Do You Know Who Is Talking to Your Wearable Smartband? 143 Given the above concerns, we gathered a set of smartbands from various manufacturers, and investigated the following questions: Who is talking to these smartbands as part of their operation? Or similarly, who are these devices talking to? Are they connecting only to the cloud of their manufacturer in order to permanently and securely store their data, or are they communicating with third parties as well? In the latter case, who are these third parties? Related work. Previous works focused on privacy of fitness trackers, and on the data that are shared with third parties. Contacting third parties. Sharing users’ data with third parties is regulated by privacy policies. However, the associated terms and conditions tend not to be always clearly expressed [1]. Also, making it optional to read the agreement often induces users put less effort in understanding it [2,3]. Vague policies authorize vendors to legally sell personal data of users to third parties without their explicit consent. Privacy of smartbands. A number of prior works have studied how the advance of wearables and ubiquitous data collection impacts privacy [4,5,6,7,8,9]. Mass surveillance of users has been studied in [4,6]. Some works [5,9] investigated how concerned people are about disclosure of their data. Lack of control over data by users have been reviewed in [7,8]. It is worth noting that privacy updates for modern wearables often emerge from non-academic research. Unlike previous academic works discussing potential privacy risks, in this paper we aimed to analyze third-party services that are contacted in practice. We provide the following contributions: we analyze the entities that are contacted by seven variously priced wearable devices; we identify unexpected/undesired (from the standpoint of privacy) third parties that the bands communicate with; we provide guidelines for preserving privacy while retaining essential functionality of the fitness trackers. 2. Methodology In order to determine what kinds of IP addresses, domains and ISPs communicate with the studied bands, we followed a three-steps pipeline. Traffic Capture. Smartbands send data to mobile applications using Bluetooth, and apps send/receive data over the Internet. We utilized WireShark3 to capture the traffic, and learn contacted domains. To analyze what data are sent, we set up a MITM Proxy 4. Retrieval of domains and IP addresses. After capturing the traffic, we aimed to find the domain names of the servers the smartphone app talks to. We obtained URLs, and IP addresses from our MITM setup. In some cases, we utilized the SNI field of TLS. Identification of the domains’ nature. Once we learned both domain names, and transmitted data, we set out to find what kind of business are these domains in. This final step turned out to be the most challenging. While for some domain names (e.g., graph.facebook) it is clear who the owner is, for others (e.g., plbslog.umeng) it is less obvious. To determine physical location of servers we employed Geoip 5 . To study origins of the domain names we utilized the Whois 6 service. 3 https://www.wireshark.org/ https://portswigger.net/burp 5 https://geoip.com 6 https://www.whois.com/whois/ 4 144 A. Kazlouski et al. / Do You Know Who Is Talking to Your Wearable Smartband? 3. Results Table 1 illustrates third parties contacted by each smartband/app pair. Arbily Smartwatch (China). Arbily Smartwatch connects to VeryFitPro, a popular fitness app that counts more than 5 million of downloads (July 2020). VeryFitPro connects mainly to its API at the domain veryfitproapi.veryfitplus, which for Europe has servers in Germany. Third Parties. The VeryFitPro app connects to the aliyuncs domain to upload profile pictures of the users, in case they decide to use one. Information about the user’s phone is also sent to ido-ble-lib.cn-hongkong.log.aliyuncs - a server located in Hong Kong. In particular, when the app synchronizes with the band, a Zlib encoded file that contains information about the OS of the phone, the time zone, the phone name, and a timestamp is transmitted. This info might enable third parties to profile app’s activity. To enable GPS tracking of user’s path during exercise (walking, running, cycling) VeryFitPro contacts the amap domain. Amap API is a mapping service provided by Alibaba Group (China) which owns servers located both in China and the United States. Table 1. Third parties that are contacted by the bands. Origin refers to the country of origin for ISPs. The Site column implies physical location of the server. Role describes why the domain is contacted (Social = Social Networks). For domain name * replaces .com; IdoBleLogs is the alias for the ido-ble-lib.cn-hongkong. log.aliyuncs.com domain. Ger = Germany; HK = Hong Kong; C = China (i.e. China Unicom). App Domain name IP address ISP Origin Site Role IdoBleLogs 47.244.67.196 Alibaba China HK Logs abroad.apilocate.amap* cgicol.amap* 205.204.101.28 198.11.136.99 Alibaba Alibaba USA China USA USA control.aps.amap* 140.205.230.4 Alibaba China China restapi.amap* 47.246.74.109 Alibaba China USA api.weibo* cgi.connect.qq* graph.facebook* 114.134.80.166 203.205.254.62 31.13.84.8 HGC Tencent Facebook HK China USA HK HK Austria logs.amap* abroad.apilocate.amap* 203.119.211.252 47.88.68.79 Alibaba Alibaba China China China USA apilocate.amap* 205.204.101.31 Alibaba China USA restapi.amap* 47.246.74.104 Alibaba China USA login.sina.com.cn 58.63.236.212 ChinaNet China China xtrapath2.izatcloud.net 52.85.156.111 Amazon USA Greece Samsung app-measurement* 172.217.21.78 Google USA Ger Analytics Huawei api.geetest* 54.77.192.2 Amazon USA Ireland API TBand iwhop* 47.56.106.31 Alibaba China China Weather hmma.baidu* openrcv.baidu* dxp.baidu* 111.202.114.42 39.156.66.235 39.156.66.180 C Unicom C Mobile C Mobile China China China China China China Ads plbslog.umeng* 203.119.214.123 Alibaba China China iwhop* 47.56.106.31 Alibaba China China VeryFit MiFit Wearfit Location Social Location Ads Weather 145 A. Kazlouski et al. / Do You Know Who Is Talking to Your Wearable Smartband? Yoho plbslog.umeng* ulogs.umeng* 203.119.214.124 203.119.214.124 Alibaba Alibaba China China China China log.umsns* 203.119.215.106 Alibaba China China Ads Xiaomi Mi band 4 (China). MiBand 4 connects to the MiFit app (50 million downloads), developed by Xiaomi. The app mainly connects to api-mifit.huami, an Amazon hosted API domain that collects health data about users. The connected servers are located in Germany, if the app is used from Europe. However, if a user registers from the USA, the app will mostly share health information with American servers. Third Parties. Similarly to VeryFitPro, MiFit also relies on Amap to track user’s position during fitness activities. The correspondent IP addresses can be from Europe, China or Hong Kong. A number of requests are automatically sent to three popular social networks (Tencent QQ, Weibo and Facebook) regardless of whether the user is registered there. Moreover, a user consent for sharing data with these networks is never asked. QQ, for instance, is contacted with a plain text GET request that contains the phone name and the OS version in the query. Although this can be considered minor information, it still enables the social network to gather data about people beyond its userbase. Overall, the app talks to servers from a number of Chinese ISPs: ChinaNet Guangdong, Alibaba, Shenzhen Tencent. Gear Fit 2 Pro (South Korea). Gear Fit 2 Pro is a smartwatch produced by Samsung which must be linked to Samsung Health. The app has been installed more than 1 billion times through Google Play Market, and it mostly connects to servers owned by Google and Amazon. Most of the domains that are contacted by the app belong to Samsung and can be considered “safe”. Nevertheless, the amount of traffic that is generated for advertisement purposes, mainly towards dls.di.atlas.samsung, is quite consistent. Creating a large quantity of undesired traffic causes bandwidth and power consumption. Samsung Health utilizes an analytics service by Google. Huawei Band 3 Pro (China). We used Huawei band 3 Pro with the Huawei Health application. The app has been downloaded more than 100 million times as of July 2020. The domains hicloud and dbankcdn (and others with similar names) are owned by Huawei. To execute its functions Huawei Health contacts servers in China, Germany, United States, and Ireland. In Germany it uses servers of T-Systems, in Ireland it communicates with Amazon servers, in China and USA it talks to Huawei and Alibaba IPs. Since Huawei Band 3 pro is endowed with an inbuilt GPS, there is no need for the app to contact third-party APIs for tracking user’s location during training. To our surprise it also appears that Huawei Health does not contact any third-party ads services. Third Parties. Huawei Health employs a CAPTCHA service Geetest to prevent botting. Low-cost Bands. These smartbands (price <e15) include RoHs, M4, and Naxius. Due to the absence of dedicated vendor servers, the corresponding apps (Wearfit, Tband, and Yoho Sport) do not send away any health data of the users. Third Parties. Since the mentioned manufacturers do not produce their own applications, they rent them from other companies. Thus, every entity contacted by these apps can be considered a “third party”. Tband and Wearfit obtain current temperature in Celsius from iwhop. Wearfit and Yoho sport communicate with various servers of Alibaba for advertisement. 146 A. Kazlouski et al. / Do You Know Who Is Talking to Your Wearable Smartband? 4. Discussion and Conclusion It appears that the saying “if you are not paying for the product, you are the product” applies to fitness trackers: although the apps can be used free of charge, users are giving their data in return. Manufacturers aim to maximize their profit by collecting as much information as possible and eventually sharing it with third parties. Although no illicit activity emerged from our studies, once users accept the privacy agreement (which is mandatory in order to use the fitness tracker) they are likely to lose control over their own data. Moreover, it is often the case that the agreement does not even specify who are these third parties. However, privacy-conscious consumers are still able to protect their data from being uncontrollably shared. It is possible to restrict access of applications to particular domains by using mobile firewalls. Such services allow customers to block any connection to any domain, including advertisement and tracking services. Although this might cause the app to stop working properly. Alternatively, it is possible to utilize an open-source “jail break” application Gadgetbridge (https://github.com/Freeyourgadget/Gadgetbridge). This app allows users to use their smartbands without transmitting any data to vendors’ servers. Currently it supports more than 30 popular models of wearables. With an immense number of various smartbands readily available, we expect the majority of them to contact “unexpected” services. We analyze traffic of seven commercial wearable devices. We show that their official mobile applications contact many unexpected or even “unwanted” third-party servers such as location services, advertisement and analytics providers, and various APIs. Every person who wears a fitness tracker on her wrist is likely “donating” private information to the device manufacturer. We recommend every privacy-conscious individual to study the privacy policy before purchasing a desired wearable to learn which sensitive data can be shared. In case of unacceptable policies, we suggest consumers to consider more transparent vendors. It is still feasible, however, to use the majority of smartbands without leaking sensitive data. Mobile firewalls, and/or dedicated “no-traffic” apps are able to restrict third parties from gathering private information. Note that in these cases some of the device functionality might fail. References [1] Balebako R, Shay R, Cranor LF. Is your inseam a biometric? evaluating the understandability of mobile privacy notice categories. CMU, Tech. Rep. CMU-CyLab-13-011. [2] Acquisti A, Grossklags J. Privacy and rationality in individual decision making. IEEE security & privacy 2005; 3(1): 26–33. [3] Steinfeld N. I agree to the terms and conditions:(how) do users read privacy policies online? an eyetracking experiment. Computers in Human Behavior 2016; 55: 992–1000. [4] Hilts A, et al. Every step you fake: A comparative analysis of fitness tracker privacy and security, Open Effect Report. Available at: https://openeffect.ca/reports/Every_Step_You_Fake.pdf. [5] Vitak J, Liao Y, Kumar P, Zimmer M, Kritikos K. Privacy attitudes and data valuation among fitness tracker users, in: International Conference on Information, Springer. 2018; 229–239. [6] Ball K, et al. Big data surveillance and the body-subject. Body & Society 2016; 22 (2): 58–81. [7] Crawford K, Lingel J, Karppi T. Our metrics, ourselves: A hundred years of self-tracking from the weight scale to the wrist wearable device. European Journal of Cultural Studies 2015; 18 (4-5): 479–496. [8] Peppet SR. Regulating the internet of things: first steps toward managing discrimination, privacy, security and consent. Tex. L. Rev. 2014; 93:85. [9] Raij A, et al. Privacy risks emerging from the adoption of innocuous wearable sensors in the mobile environment. SIGCHI Conference on Human Factors in Computing Systems, 2011: 11–20. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200712 147 The Effect of Chronic Diseases on the Use of Health Technology and Digital Services in the Elderly Population in Finland Jukka MIELONENa,1, Ulla-Mari KINNUNEN b, Kaija SARANTOb, Anssi KEMPPIc and Hanna KUUSISTOb a,b Department of Health and Social Management, University of Eastern Finland c Finnish Pensioners’ Federation, Finland Abstract. Digital services are growing in the health-care field. The population in Europe is aging, and digital services are on the rise. There are also plenty of new health-care devices on the market. The aim of this study was to survey how elderly people cope with digital services or devices, especially if they are chronically ill. This quantitative study focuses on the impact of chronic diseases on the use of health technology and digital services. The target group of this study is Finnish people aged 65 or over. Based on the results, a chronic disease or disability is not an obstacle to the use of digital services or health-care technology in the Finnish elderly population. The main obstacles to the use of health technology or digital services are complexity, obscure text, or small font size. According to this study, elderly people seem to trust the device or application. Devices, applications, and online services should be designed so that elderly people’s diseases or ability to function are considered. Keywords. Health care, Digital services, Health technology, Disease 1. Introduction The cost of health care is threatening to rise in Europe due to the aging population. The cost-effectiveness of health care can be improved by implementing digital technology. Increasing the role of self-care enables health professionals to monitor the progress of symptoms of certain diseases. Home-based health-care devices should be designed for various age-groups and diseases. Even though care processes and interventions are intended to support the use of digital technologies at home, the level of health technologies to be used at home has remained low [1,2]. The potential effects of various diseases must be considered when designing digital social and health-care services and devices [3]. A certain disease may have a negative impact on the used health-care device. In this study, health-care devices and meters mean, for example, blood glucose meters or spirometry. Digital services are least used by those who would benefit most from them, and this can lead to a digital divide. Sociodemographic factors, and especially age, have an influence on the use of digital health-care services in people with chronic diseases. Elderly people do not have devices, or they do not know how to use the technology or ¹Jukka Mielonen, PhDs, University of Eastern Finland, Department of Health and Social Management, Yliopistonranta 1, 70210 Kuopio, Finland; E-mail: jukmi@student.uef.fi 148 J. Mielonen et al. / Use of Health Technology and Digital Services digital services. They might be afraid of losing personal contacts when using digital services. Income and education seem to influence the use of digital services. Higher income or higher education may increase usage [4,5]. Aging also affects vision, hearing, motor functions, and coordination. The text of an application may be too small, and difficulties in eye-hand coordination and motor disabilities may slow down keyboard or mouse usage [6,7]. Many diseases, such as arthritis, may cause fine motor control and coordination changes. Cognitive capabilities may decrease, and elderly people may perceive technology differently from younger adults. These constraints force designers to create better products for elderly people [8,9]. However, no relationship between self-management of internet-based health information technologies and technology acceptance of patients with heart disease has been found [10]. This article describes how various chronic diseases or disabilities affect the use of digital health-care services, devices, or applications among elderly people. The following research question was set: How does a possible disease affect the use of digital health-care services or devices? 2. Methods The target group of this quantitative study is Finnish people older than 64 years of age. There were approximately 1.2 million people, or about 22% of the population, aged 65 and higher in Finland at the end of 2018, according to Statistics Finland [11]. The questionnaire was formed by operationalizing the variables of UTAUT (Unified Theory of Technology Acceptance) theory [12]. There was a total of 39 questions, of which two were open questions. The questionnaire was carried out using an Eduix E-form. The responses consisted of 'yes – no' answers (for which some of the answers requested more information in the text field), multiple choice, and rankings on a 5-point Likert scale (very often, often, sometimes, rarely or I never and completely agree, partially agree, partially disagree, completely disagree or I cannot say). The form also provided readymade response options for diseases. Some of the questions were not addressed to the respondent if the preceding value on the form was not met. These values included a disease or a device used to treat illness [13]. Information on the study and a link to the questionnaire were sent to members of the Finnish Pension Association by e-mail. Finnish Pensioners' Federation advertised the research on its own website and on the SeniorSurf website. The research was also advertised on social media, websites, and various publishing sites (e.g. LinkedIn and Facebook). It was possible to respond to the survey with assistance if a respondent was unable to open the survey themselves. The data was collected during the three-month period of March to May 2019. The research data was analyzed using IBM SPSS Statistics 25 and 26. Both sociodemographic and disease-related ratios were calculated from the data. During the analysis phase, Likert-scale responses were reclassified by combining categories such as very often and often as one answer. A chi square test was used to examine the statistical dependence of background variables on the variables to be studied, and Pearson's correlation coefficient was used to examine the links between the variables. Open responses were analyzed using inductive content analysis. In content analysis, open 149 J. Mielonen et al. / Use of Health Technology and Digital Services responses were simplified and grouped into parent categories using descriptive expressions. The statistics to be reported were calculated from the main categories. 3. Results Of the respondents to the survey (N = 978), almost half had some underlying disease. Most of the respondents had some form of heart disease (22.2%). The diseases and their incidence in subjects are presented in Table 1. Almost half of the respondents (44.9%) had high incomes (> €3000 per month) and a high level of education (a college or university degree). Table 1. Respondents’ diseases (n = 978). Disease Heart disease Musculoskeletal disorders Diabetes Rheumatic disease Psychiatric disorder Parkinson's disease Memory illness n % 217 192 125 46 10 9 6 22,2 19,6 12,8 4,7 1 0,9 0,6 In addition to the pre-completed answers, the questionnaire included an open followup question: I have another disease, what? This question was answered by 52.2% of the respondents, and 18 of the responses were rejected due to incomplete answers. The replies were classified in the upper classes of the ICD-10 classification [14]. The results are shown in Figure 1. 28,3 14,8 Diseases of the circulatory system Endocrine, nutritional and metabolic diseases 166 87 14,0 Diseases of the respiratory system Diseases of the musculoskeletal system and… Diseases of the digestive system Diseases of the nervous system Tumors 6,3 4,3 4, Diseases of the eye and adnexa 68 77,,7 7,3 2,0 Diseases of the skin and subcutaneous tissue 82 1 ,6 11 45 43 37 25 112 1,7 10 Diseases of the ear and mastoid process 0 20 % 40 60 80 100 120 140 160 n Figure 1. Responses classified according to the ICD-10 disease classification (n = 586). 180 150 J. Mielonen et al. / Use of Health Technology and Digital Services The respondents’ functional ability or disease does not appear to prevent the use of digital services. Only a few of the respondents (8.1%) reported that their functional ability or disease prevents the use of digital services. The chi square test showed the relation between morbidity and the use of digital services to be statistically very significant. Only 13.0% of the respondents were using a device or an application for a disease or health monitoring. The majority (68.1%) of the respondents who were using healthcare applications or devices were using a device or application (e.g. blood pressure or blood glucose meter) for diagnosing, controlling, treating, or alleviating a disease. Devices for the diagnosis, monitoring, treatment, alleviation, or compensation of an injury or defect (e.g. pacemaker) were used by 12.5% of the respondents, and 4.2% of the respondents were using the device for the study, replacement, or modification of an anatomical or physiological function (e.g. spirometers). The majority (81.9%) trusted the device or application. The obstacles to the use of the IT device or application are shown in Figure 2. Figure 2. Barriers to use of the IT device or application. 4. Reflection and Conclusions According to previous studies elderly people may not have necessary digital health devices or they are afraid to use them [4]. Furthermore, it has been shown that motor disabilities may slow down or cause errors in the use of a keyboard or mouse and eye– hand coordination problems may affect the use of digital health-care services [6,7] However this study shows that in Finland elderly people have the necessary devices at home, and they can use them. Furthermore they trust the devices or applications, regardless of their disease or disability. As a person’s chronic disease or disability does not appear to inhibit the use of digital health services or devices at home, a part of face to face visits among elderly could be replaced with telecare combined with remote monitoring increasing patient involvement. According to this study, obstacles to the use of a health-care device or application include obscure text and the complexity of the application or device. However, there was a small proportion of respondents with these claims. The main obstacle for elderly people in the use of a health-care device or application is its complexity. Software developers and device engineers should focus more on elderly people’s demands and should consider their possible disease or functional ability, as shown previously [8]. The devices should have bigger buttons or a larger font size. Since Finnish elderly people are willing and competent to use electronic health-care services and devices. We recommend that health- J. Mielonen et al. / Use of Health Technology and Digital Services 151 care providers increase the amount and variety of digital health solutions for elderly. A previous study shows that there is no relationship between technology acceptance and internet-based health information technologies [10]. The UTAUT model seeks to predict the use of technology based on intended use. Use behavior is influenced by performance expectancy, effort expectancy, social influence, and facilitation conditions [12]. Based on our research, elements such as effort expectancy and facilitation conditions that enable the use of technology are good. Limitations to the study was that it was an electronic survey, but there was also an option to recruit a person to assist when answering the questionnaire. It seems that most of the respondents were well educated and their incomes were high. As shown in previous studies, higher income or a higher degree may increase usage [3]. It is recommended that future studies should focus on lower income or lower education groups and those who are not able to answer electronic questionnaires. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] Hollmark M, Lefevre Skjöldebrand A, Andersson C, Lindblad R. Technology Ready to be Launched, but is there a Payer? Challenges for implementing eHealth in Sweden. Studies in Health Technology and Informatics. 2015;211:57-68. Craven J. Access to electronic resources by visually impaired people. In Information Research. 2003 Jul; 8(4):156. Reiners F, Sturm J, Bouw LJW, Wouters EJM. Sociodemographic Factors Influencing the Use of eHealth in People with Chronic Diseases. International Journal of Environmental Research and Public Health. 2019;16(4), 645. Liu L, Stroulia E, Nikolaidis I, Miguel-Cruz A, Rincon AR. Smart homes and home health monitoring technologies for older adults: A systematic review. International Journal of Medical Informatics. 2016 Apr; 91:44-59. Cresci M, Yarandi H, Morrel R. The Digital Divide and Urban Older Adults. Computers, Informatics, Nursing. 2010;28(2):88-94. Wildenbos GA, Jaspers MWM, Schijven MP, Dusseljee-Peute LW. Mobile Health for Older Adult Patients: Using an Aging Barriers Framework to Classify Usability Problems. International Journal of Medical Informatics. 2019;124:68-77. Trewin S, Pain H. Keyboard and mouse errors due to motor disabilities. International Journal of HumanComputer Studies. 1999b;50(2):109-144. Charness N, Boot WR. Aging and Information Technology Use: Potential and Barriers. In Current Directions in Psychological Science. 2009;18(5):253-258. Czaja S, Charness N, Fisk A, Hertzog C, Nair S, Rogers W, et al. Factors predicting the use of technology: Findings from the center for research and education on aging and technology enhancement (create). Psychology and Aging. 2006:21(2):333-352. Calvin KL, Karsh BT. The Patient Technology Acceptance Model (PTAM) for Homecare Patients with Chronic Illness. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. October 2006 Oct; 50(10):989-993. Official Statistics of Finland (OSF): Population structure [e-publication]. ISSN=1797-5379. Helsinki: Statistics Finland. Referred: 2.12.2019. Access method: http://www.stat.fi/til/vaerak/index.html Venkatesh V, Morris MG, Davis GB, Davis FD. User Acceptance of Information Technology: Toward A Unified View. MIS Quarterly 2003;27(3):425-478. National Supervisory Authority for Welfare and Health (Valvira). Health care Equipment and Supplies. Referred: 25.1.2020 Access method: https://www.valvira.fi/terveydenhuolto/terveysteknologia/tuotteen_markkinoille_saattaminen/terveyden huollon_laitteet_ja_tarvikkeet Finnish Institute for Health and Welfare (THL). Classification of Diseases. ICD-10. Referred: 14.5.2020. Access method: https://www.julkari.fi/bitstream/handle/10024/80324/15c30d65-2b96-41d7aca81a05aa8a0a19.pdf?sequence=1&isAllowed=y, 152 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200713 Personalized Predictive Models for Identifying Clinical Deterioration Using LSTM in Emergency Departments Amin NAEMI1,a ,Thomas SCHMIDTa, Marjan MANSOURVARa and Uffe Kock WIILa a SDU Health Informatics and Technology, The Maersk Mc-Kinney Moeller Institute, University of Southern Denmark, Denmark Abstract. Early detection of deterioration at hospitals could be beneficial in terms of reducing mortality and morbidity rates and costs. In this paper, we present a model based on Long Short-Term Memory (LSTM) neural network used in deep learning to predict the illness severity of patients in advance. Hence, by predicting health severity, this model can be used to identify deteriorating patients. Our proposed model utilizes continuous monitored vital signs, including heart rate, respiratory rate, oxygen saturation, and blood pressure automatically collected from patients during hospitalization. In this study, a short-time prediction using a sliding window approach is applied. The performance of the proposed model was compared with the Multi-Layer Perceptron (MLP) neural network, a feedforward class of neural network, based on R2 score and Root Mean Square Error (RMSE) metrics. The results showed that the LSTM has a better performance and could predict the illness severity of patients more accurately. Keywords. clinical deterioration, machine learning algorithms, time series, health informatics, LSTM, recurrent neural network, emergency department. 1. Introduction Research has demonstrated that about 31 percent of acutely admitted patients, who seem normal upon arrival deteriorate during their stay, which could lead to an increase in mortality and morbidity rate [1]. The probability of deterioration and adverse events such as unexpected transfer to the intensive care unit can be reduced by using consistent and rigorous methods of vital signs monitoring and applying deterioration prediction models [2,3]. Therefore, many departments utilize expensive patient monitoring equipment. However, most of the automatic measurements recorded by these devices are never used in clinical decision making, due to the large amount of information that is timeconsuming and difficult for clinicians to process. Consequently, potentially useful information for identifying impending deterioration is neglected. Through monitoring of patients and conversations with clinicians, it is evident that the health condition of a patient is difficult to quantify [4]. However, dynamic changes in patients’ vital signs can be used to detect those who are at risk of adverse events and deterioration. Several Machine Learning (ML) algorithms such as Neural Network (NN), Support Vector 1 Corresponding Author, Amin Naemi, E-mail: amin@mmmi.sdu.dk. A. Naemi et al. / Personalized Predictive Models for Identifying Clinical Deterioration 153 Machine (SVM), K-Nearest Neighbors (K-NN), Gaussian Process (GP) have been utilized to predict patient severity, instability, and clinical deterioration [5]. Most of these studies showed that ML techniques could recognize complex patterns and predict the health condition of patients and clinical deterioration more precisely than traditional approaches. However, a majority of these studies neglected previous monitored information, long dependencies, and dynamic changes in illness severity of patients; however, the current health status of patients depends on their previous health conditions. One of the appropriate algorithms that can address this limitation and detect temporal dependencies and dynamic changes in a time series is Long Short-Term Memory (LSTM) neural network. The main idea behind using LSTM is to investigate whether there is a temporal and long-term dependency in a time series. LSTM has been used in Emergency Departments (ED) such as prediction of admission [6], and pain classification [7]. Moreover, recent studies in medicine indicate that the population of patients is heterogeneous, i.e., every patient has his/her unique characteristics, and it is necessary to have targeted, patient-specific predictions and treatments [8]. Nevertheless, most of the studies have proposed a single model for the whole population. Hence, it is important to examine the impact of building personalized models on the quality of model performance. Therefore, in this paper, a LTSM neural network model is proposed to predict clinical deterioration and patient illness severity on the individual level at ED. This model utilizes continuous monitored vital signs and considers the previous health conditions and characteristics of each patient to predict the future illness severity of them. This enables the clinicians to identify patients who are at risk of deterioration and helps them to manage their medical resources and attention more effectively. 2. Materials 2.1. Data Acquisition Vital signs of patients, including Heart Rate (HR), Respiratory Rate (RR), Arterial Blood Oxygen Saturation (SpO2), and systolic Blood Pressure (BP) of all patients admitted to the EDs of Odense University Hospital (OUH), and Hospital of South Western Jutland (HSWJ) between 2018 and 2019 were stored in the database. This data was gathered from the HL7 interface of Philips IntelliVue patient monitors and registered in 60-second intervals. Moreover, other information such as age, gender, admission date, length of stay, and clinical notes was stored in the dataset. Vital signs registration was authorized by the Danish Data Protection Agency under journal nr. 17/14630 and registered at ClinicalTrials.gov with number: NCT03375658. The data was stored in a database with restricted access according to Danish legislation on privacy concerns and analyzed by Python 3.7.7. Description of the data is shown in Table 1. Table 1. Data description Patients, n Age, Median (years) HR, Median (minute-1) RR, Median (minute-1) SpO2, Median (%) BP, Median (mmHg) Length of Monitoring, Median (minute) Total Vital signs, n OUH 31,234 (male: 16,271) 65 (IQR: 51 - 80) 83 (IQR: 74 - 111) 16 (IQR: 14 - 22) 97 (IQR: 92 - 99) 126 (IQR: 119 - 131) 181 (IQR: 98 - 721) 2,488,292 HSWJ 21,421 (male: 12,054) 66 (IQR: 54 - 81) 81 (IQR: 70 - 106) 18 (IQR: 14 - 23) 97 (IQR: 93 - 99) 129 (IQR: 118 - 136) 523 (271 - 1176) 5,619,021 154 A. Naemi et al. / Personalized Predictive Models for Identifying Clinical Deterioration 2.2. Data Preprocessing In the preprocessing phase, patients under 18 years old and patients not monitored during their admission, were removed from the dataset to prepare the data for the model development phase. To handle missing values in vital signs, Moving Average (MA) technique with a window size of 10 minutes was used. 3. Methods 3.1. Scoring System The primary goal of scoring systems is to stratify patients into different categories based on their health conditions and help clinicians to identify seriously ill patients [9]. In this study, Adaptive Process Triage (ADAPT) was used to assign patients to different categories based on their vital signs. ADAPT has been utilized in several Danish hospitals and could be used for scoring the severity of vital signs according to the ABCDE-principle [9]. Table 2 shows how the ADAPT triage categories are computed based on vital signs. For each patient, based on his/her vital signs over time, a sequence of severity scores is produced. ADAPT has four categories including Red, Orange, Yellow, and Green, where Red indicates more severe circumstances that require a higher priority and Green corresponds to less urgent situations that can be handled with a lower priority. Table 2. ADAPT triage model Airways Breathing Circulation Disability Exposure Red (1) Orange (2) Yellow (3) Green (4) Resuscitation 0 min Obstructed airway stridor SpO2 < 80% 8 > RR > 35 HR > 130 BPsys<80 Urgent 15 min threatened airway 80 < SpO2 <89 31 < RR < 35 121 < HR <130 HR < 40 80< BPsys < 89 9 < GCS < 13 Less urgent 60 min Not urgent 180 min 90 <SpO2 <94 26 < RR < 30 111 < HR <120 40 < HR <49 SpO2 ≥ 95 8 < RR < 25 50 < HR <110 GCS = 14 GCS = 15 38.1 < Tp < 40 32 < Tp < 34 34.1 < Tp < 38 GCS ≤ 8 Tp > 40 Tp < 32 3.2. Model Development In this paper, two different types of neural networks, including Multi-Layer Perceptron (MLP) and LSTM were utilized to predict patients’ severity. MLP is a well-known class of feedforward neural network while LSTM is a special kind of Recurrent Neural Networks (RNN) which has internal memory to process any arbitrary flow of inputs. In LSTM, usual hidden layers are substituted by LSTM cells. These cells consist of different gates, including input gate, output gate, and forget gate. 4-fold cross-validation was used to have a robust model, where the three folds were used to train the model each time, and the fourth fold was used as the validation set. Grid search was used to find the optimal hyperparameters and best models on the validation set were used as the final models to apply on the test data. Moreover, the sliding window approach was used for short-term prediction. This technique enables us to follow the dynamic and sudden changes in patients’ conditions time series. The window size was set to 90 minutes, and the step size which indicates the number of predicted samples was set to 30 minutes. In other words, A. Naemi et al. / Personalized Predictive Models for Identifying Clinical Deterioration 155 the severity of a patient for the next 30 minutes was predicted based on his/her severity in the last 90 minutes. The training process was interrupted for both models after 1000 epochs to estimate the generalization error on the validation data. Moreover, the training process was stopped whenever the generalization error was higher than the error of previous epoch. The loss function was Mean Square Error (MSE), and the optimization algorithm to update networks weights was Adaptive Moment Estimation (Adam) [10] which was designed specifically for training deep neural networks. This algorithm has some advantages, including easy implementation, accelerate training, and suitable for non-stationary or noisy objectives. 4. Results and Discussion 2000 patients with at least 10-hour monitoring time were selected randomly to evaluate the performance of LSTM. 75% of this data was used as a training set and the rest was used for testing. The training set was used for training and validation process based on cross-validation. The hyperparameters of LSTM and MLP are shown in Table 3. Table 3. Hyperparameters of models Hidden Layers LSTM MLP 2 2 Hidden Layers Neurons 4, 4 (LSTM cells) 8,4 Number of Epochs 1000 1000 Optimization Algorithm Adam Adam Models were tested on the 25% of the data, selected as test data. ܴଶ score and Root Mean Square Error (RMSE) were used to evaluate the performance of models. The ܴଶ scores used in time series analysis is a value between 0 and 1, indicates how much of variance in a given time series can be explained by a model. A closer ܴଶ score to 1 means that the model works better. The average ܴଶ scores and RMSE for 500 patients (test data) are shown in Table 4. A patient was randomly selected to investigate the prediction accuracy of models, and the patient’s severity for a sample period of 200 minutes was predicted, as shown in Figure 1. Table 4. Performance of models LSTM MLP R2 Score Training Data Test Data 0.92 0.87 0.89 0.81 RMSE Training Data Test Data 0.042 0.083 0.066 0.105 Figure 1. Prediction of a patient’s illness severity using LSTM and MLP over a sample period. Based on Table 4, the prediction performance of LSTM was better for both evaluation metrics. It means that there are meaningful information and long dependencies in patients’ severity trajectories that must be taken into account for the prediction of future severity of patients. Moreover, based on Figure. 1, LSTM followed 156 A. Naemi et al. / Personalized Predictive Models for Identifying Clinical Deterioration the real trajectory (blue time series) better than MLP, especially in sudden changes and high fluctuation situations. It shows that deep learning models such as LSTM have great capability in detecting dynamic changes of chaotic time series. 5. Conclusion In this paper, we proposed a model based on the LSTM neural network to predict patients’ illness severity, which is an index to quantify clinical deterioration, during hospitalization at EDs. The ADAPT triage system was used to calculate the severity of patients based on their vital signs. Our proposed method predicts patients’ severity of illness individually using continuous monitored vital signs, automatically collected during treatment. This means that a separate model is developed for each patient trained based on patient characteristics and his/her previous recorded vital signs. Moreover, sliding window technique was applied to detect and follow the dynamic and sudden changes in patients’ health conditions. The performance of the proposed model for a selected representative population of patients was investigated, and the results showed that the performance of LSTM, a recurrent deep neural network, is approximately 6% and 21% higher compared to the feedforward class of neural networks (MLP) in terms of RMSE and R2 score, respectively. This highlights the importance of considering long dependencies in patients’ vital signs and previous health conditions for prediction of their severity. It also demonstrates the capability of deep neural networks such as LSTM to detect hidden and complex dynamics of time series. Future studies will focus on the prediction of each vital sign trajectories independently and developing ML models to predict clinical deterioration based on vital signs trajectories. References [1] Henriksen DP, Brabrand M, Lassen AT. Prognosis and risk factors for deterioration in patients admitted to a medical emergency department. PLoS One. 2014;9(4):e94649. [2] Mansourvar M, Andersen-Ranberg K, Nøhr C, Wiil UK. A Predictive Model for Acute Admission in Aged Population. In: MIE. 2018. p. 96–100. [3] Churpek MM, Adhikari R, Edelson DP. The value of vital sign trends for detecting clinical deterioration on the wards. Resuscitation. 2016;102:1–5. [4] Schmidt T, Wiil UK. Identifying patients at risk of deterioration in the Joint Emergency Department. Cognition, Technology & Work. 2015;17(4):529–45. [5] Churpek MM, Yuen TC, Winslow C, Meltzer DO, Kattan MW, Edelson DP. Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards. Critical Care Medicine. 2016;44(2):368. [6] Kadri F, Baraoui M, Nouaouri I. An LSTM-based Deep Learning Approach with Application to Predicting Hospital Emergency Department Admissions. In: 2019 International Conference on Industrial Engineering and Systems Management (IESM). IEEE; 2019. p. 1–6. [7] Tsai F-S, Weng Y-M, Ng C-J, Lee C-C. Embedding stacked bottleneck vocal features in a LSTM architecture for automatic pain level classification during emergency triage. In: 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE; 2017. p. 313–8. [8] Ng K, Sun J, Hu J, Wang F. Personalized predictive modeling and risk factor identification using patient similarity. AMIA Summits Translational Science Proceedings. 2015;2015:132. [9] Nordberg M, Lethvall S, Castrén M. The validity of the triage system ADAPT. Scandinavian Journal of Trauma Resuscitation Emergency Medicine. 2010;18(S1):P36. [10] Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv Preprint arXiv14126980. 2014. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200714 157 Electronic Health Record System-Related Patient Safety Incidents – How to Classify Them? Sari PALOJOKI a,1, Riikka VUOKKOa, Anne VAKKURIb and Kaija SARANTOc a The Ministry of Social Affairs and Health, Finland b Helsinki University Hospital (HUS), Peijas Hospital, Finland c University of Eastern Finland, Finland Abstract. The implementation of electronic health record systems (EHRs) may cause multidimensional patient safety issues that deserve research attention. Our research aims to identify the current body of evidence on EHRs-related incident types and how incidents are classified in these studies. A literature search resulted in 44 peer-reviewed papers and six papers were included in the final analysis. The error types do not concern solely the technological features of the EHRs but may involve also non-technical aspects. Our review indicates that standard classification systems would facilitate comparisons across countries. To achieve the goal, more research evidence, testing and development of classifications are required. Keywords. Patient safety, incident reporting, electronic health record system, error, classification 1. Introduction Health and biomedical informatics communities have long been interested in unintended consequences arising from the implementation of electronic health record systems (EHRs). While EHRs may enhance the safety of patient care, it is also assumed that an increase in the implementation of information technology within healthcare systems will lead to patient safety incidents by introducing novel vulnerabilities and unique risks. Even if the body of research identifying technology-induced errors related to EHRs is growing, there is a lack of risk reporting, and data describing those risks is still limited. [1-4] There is a growing body of evidence pointing to several methods that can be used to address technology-induced errors. Patient safety incident reporting by end-users is the primary mechanism by which it is possible to learn about these concerns [3-5]. The European Council [6] recommends that Member States support blame-free reporting systems, which provide information about the extent, types, and causes of incidents. Incident reporting systems (IRS) have now been in place for more than a decade in many countries but it is not well established how to define and classify events in these systems. Moreover, comprehending the limitations of patient safety incident data is indispensable in avoiding the misinterpretation of the data [5, 7]. 1 Corresponding author, Dr. Sari Palojoki, sari.palojoki@stm.fi 158 S. Palojoki et al. / Electronic Health Record System-Related Patient Safety Incidents EHR concerns as a multidimensional patient safety issue deserve research attention. Analysis of patient safety incidents produces useful data [3-4, 14]. Based on a literature review our research aims to identify the current body of evidence on EHRsrelated incident types based on classification systems. Our research questions are: (1) Which are the most common types of electronic health record system-related patient safety incidents? 2) How EHRs-related patient safety incidents are classified in these studies? 2. Methods In this paper, we apply a method for our literature review, which is consistent with guidelines by Templier and Paré [8]. Steps consist of formulating the research questions, searching the literature, screening for inclusion, extracting data, and analyzing data. Step “assessing the quality of primary studies” was excluded due to the research focus. Terms for literature searches in the PubMed database were composed according to appropriate MeSH terms “patient safety”, ”incident reporting”, “voluntary incident reporting”, “patient safety reporting”, “electronic health record/system”, “EHR/s”, “information system” and “computer”. The terms were grouped into sets and terms were combined with the “OR” operator, and all sets were then combined with the “AND” operator. In this context, we conceptualize EHRs-related patient safety incidents as errors being realized in a complex healthcare environment during the use of EHRs [e.g. 5]. Research that focused on technology-induced errors associated with EHRs in connection with IRS were eligible for inclusion. Our search strategy covered the use of all types of EHRs in any type of clinical setting. Studies published in peer-reviewed journals or conference proceedings were included but editorials were excluded. Papers published in English without any restrictions of timeframe were included. A literature search in the middle of July 2020 resulted in 44 peer-reviewed papers. After removing duplicates and the first exclusion round based on the researcher reading the abstracts, 12 papers were selected for further reading. Criteria for exclusion were the following: the language was not English and the research was out of scope, e.g. the focus was on incident reporting system development. After full paper reading, an additional seven of the research papers were excluded as they were out of scope. Additionally, one peer-reviewed article was retrieved based on full paper review [3]. A total of six papers was included in the final analysis based on our research questions. 3. Results 3.1. Common Types of Electronic Health Record System-Related Patient Safety Incidents The results indicate that there is yet little evidence for research on types of EHRs-related patient safety incidents. Some studies reported underlying causes of incidents and mentioned e.g. failures related to communication with other care providers [9]. However, a more detailed EHRs-related analysis was not the scope of the study. Magrabi et al. [10] performed a study, which examined a broader scope of computerrelated patient safety incidents. Computer-related patient safety incidents in a national AIMS database were analyzed. Only 0.2% of all reports in the database were computer- S. Palojoki et al. / Electronic Health Record System-Related Patient Safety Incidents 159 related. Machine-related problems were more common than human-computer interaction issues. However, they also found human-computer interaction errors related to the selection of patient and clinical information, as well as display errors. A study on radiology systems revealed that communication breakdown was a contributing factor in 49% of incidents (n=209) reported. An association with data collection, storage, or retrieval of electronic information was found in 147 of the 209 incidents. One-tenth of the incidents indicated that EHRs contributed to errors. [11] Research on radiation oncology incident reports focused on potentially significant clinical consequences. Totally, 53% of events (n=1507) with a potential high severity rating were related to human error. The most common human error reported concerned about the design of a suboptimal treatment plan. Almost one-third of events were related to an error at the level of the human-software interface, 2% were hardware failures, 1% were software failures, and 1% concerned an error in the software-hardware interface. Additionally, events rated with a maximum potential severity were related to a mismatch of information between the treatment planning software and the treatment management system and with manual data entry errors. [12] Patient safety incidents, which included all aspects of IT within the healthcare context were studied in the UK. The majority of the reports (77%) were machine-related technical problems, such as software errors, access, and display problems, and system downtime. A further 10% of the reports were related to human-computer interaction issues, and 13% of the incidents could not be classified using the framework. Only rare human error events were identified. [13] EHRs-related safety issues reported within a voluntary reporting system by applying a sociotechnical conceptual model that included both technical and non-technical dimensions of safety. Non-technical dimensions, such as workflow, policies, and personnel, interacted frequently with technical dimensions, which included software/hardware, content, and user interface, to produce safety concerns. A total of 94% of incidents related to unmet data display needs in the EHR, data transmission problems and ‘hidden dependencies’ related to the EHR. [3] EHRs-related patient safety incidents were analysed in an incident reporting database in hospitals with 100% EHR implementation rate. Data from 23 hospitals during a 2-year period indicated that the proportion of electronic health record-related incidents was higher than in previous studies with similar data. Human-computer interaction problems were the most frequently reported. [14] 3.2. Classification Systems of EHRs -Related Patient Safety Incidents Our research illustrates that classification development is documented in heterogeneous ways. For example, in Australia [10] work was carried out to identify natural categories for classification based on the incident data available. Based on previous research and analysis of incident reports, the Advanced Incident Management System (AIMS) classifies incidents to 13 incident types. It distinguishes human-computer interactionbased errors (e.g. wrong patient selected) from machine-related problems. After that, incidents were subdivided based upon problems at the point of data entry (input), data transfer (transfer), or data retrieval (output). A category of ‘contributing factors’ was also included to account for other socio-technical contextual variables that contributed to computer-related incidents (e.g. multi-tasking while using a computer). Similarly, information technology–related incidents were analysed based on a Welsh voluntary IRS [13] to understand the implications of these incidents for 160 S. Palojoki et al. / Electronic Health Record System-Related Patient Safety Incidents healthcare. In the analysis, the AIMS classification was used. The results point out that the AIMS classification is dependent on the original data used to develop it, and thus, not all clinical relevance of the Welsh incidents could be captured with the classification. The research suggests that a different approach is needed to explore the clinical implications of incidents more appropriately. In a Dutch cohort study [9], the underlying causes of incidents were classified with three main classes (organizational, human, and patient-related), where the two first classes have several sub-classes. The research indicates that incomplete patient records increase the risk of incidents. In the English National Health Service (NHS) context [3], an EHR implementation research with a patient safety focus was carried out by applying a sociotechnical model and a three-phase patient safety model (safe technology, safe use of technology, and use of technology to improve safety) to data from 12 NHS hospitals. Patient safety concerns were classified into eight main classes where each class had defined characteristics, continued with a review of risks and incidents related to each class with the professionals to both review the incident and to develop the model. Although the classification relates to risks of EHRs during the implementation phase, it may have the potential to inform safety risks. The Finnish patient safety IRS (HaiPro) based classification was used to define safety incidents in hospitals with 100% EHRs implementation rate [14]. Here, incidents are classified with 13 main classes and their sub-classes, of which the most frequently used main classes are ‘Medication and Transfusions’, ‘Information Flow’ and ‘Information Management’ categories as well as ‘Laboratory’, ‘Imaging’ and ‘Other Patient Treatment Procedures’ categories. The classification was built into the HaiPro system, and as such could not be modified by the incident reporter. Research in an oncology setting [12] applied a French Nuclear Safety Authority (ASN) 5-point scale to classify events. ASN has seven classes: human, software and hardware errors, errors in communication between two humans, at the human-software interface, at the software-hardware interface, and at the human-hardware interface. The results indicate that the NRS could inform also other classification development. 4. Discussion Researchers have developed ways of identifying and addressing types of errors in EHRs. Patient safety incident reporting systems (IRS) are an important part of safety programs, but the difficulty in analyzing error reports has limited their utility [4,5]. Research data on IRS is still scarce. In our review, the error types are not related solely to the technological features of the EHRs but may involve users of EHRs, their workflows, and aspects of the organizations in which they function. In summary, presumably, patient safety risks associated with EHRs vary along the adoption and implementation timeline of EHRs [see also, 3]. Our review indicates that the use of standard classifications would facilitate data use across countries. However, research notes that there is limited evidence of the development of investigative frameworks or classifications to categorize and comprehend the nature and e.g. clinical implications of the incidents [13]. There is a need to continually standardize the incident categories as well as train health professionals about how to report on types of EHRs-related errors [5]. For example, in EHRs, a medication administration error may have been due to missing data, but it is reported as a medication incident rather than a data capture event [13, 14]. Moreover, narrative S. Palojoki et al. / Electronic Health Record System-Related Patient Safety Incidents 161 information and evidence-based classification development presented in research may serve as a basis for improving classifications and in turn, incident reporting. Although IRS serves a purpose to corrective actions, attention has to be paid to the potential bias in reporting patterns that comes from uneven participation [12]. One of the limitations of these studies is that number of events that are reported is likely low. Reports do not provide exact frequencies of incidents but rather a descriptive analysis of EHRs-related safety problem types [14]. As a conclusion, there are only a little research results on EHRs-related error types. Classifications are potential tools and key enablers for the identification of incidents and for better use of data across countries. To achieve the goal, more research evidence and testing and development of existing classifications are needed. References [1] Kuziemsky CE, Randell R, Borycki EM. Understanding Unintended Consequences and Health Information Technology:. Contribution from the IMIA Organizational and Social Issues Working Group. Yearb Med Inform. 2016 Nov 10;(1):53-60. doi: 10.15265/IY-2016-027. PMID: 27830231; PMCID: PMC5171563. [2] Denham CR, Classen DC, Swenson SJ, Henderson MJ, Zeltner T, Bates DW. Safe use of electronic health records and health information technology systems: trust but verify. J Patient Saf. 2013 Dec;9(4):177-89. doi: 10.1097/PTS.0b013e3182a8c2b2. PMID: 24257062. [3] Meeks DW, Takian A, Sittig DF, Singh H, Barber N. Exploring the sociotechnical intersection of patient safety and electronic health record implementation. J Am Med Inform Assoc. 2014 Feb;21(e1):e28-34. doi: 10.1136/amiajnl-2013-001762. Epub 2013 Sep 19. PMID: 24052536; PMCID: PMC3957388. [4] Palojoki S, Pajunen T, Saranto K, Lehtonen L. Electronic Health Record-Related Safety Concerns: A Cross-Sectional Survey of Electronic Health Record Users. JMIR Med Inform. 2016 May 6;4(2):e13. doi: 10.2196/medinform.5238. PMID: 27154599; PMCID: PMC4890731. [5] Borycki E, Dexheimer JW, Hullin Lucay Cossio C, Gong Y, Jensen S, Kaipio J, Kennebeck S, Kirkendall E, Kushniruk AW, Kuziemsky C, Marcilly R, Röhrig R, Saranto K, Senathirajah Y, Weber J, Takeda H. Methods for Addressing Technology-induced Errors: The Current State. Yearb Med Inform. 2016 Nov 10;(1):30-40. doi: 10.15265/IY-2016-029. PMID: 27830228; PMCID: PMC5171580. [6] The Council of the European Union. COUNCIL RECOMMENDATION of 9 June 2009 on patient safety, including the prevention and control of healthcare associated infections (2009/C 151/01). https://ec.europa.eu/jrc/sites/jrcsh/files/2_June_2009%20patient%20safety.pdf. [7] Shojania KG, Marang-van de Mheen PJ. Temporal trends in patient safety in the Netherlands: reductions in preventable adverse events or the end of adverse events as a useful metric? BMJ Qual Saf. 2015 Sep;24(9):541-4. doi: 10.1136/bmjqs-2015-004461. Epub 2015 Jul 6. PMID: 26150549. [8] Templier M and Pare G.A Framework for Guiding and Evaluation Literature Reviews. Communications of the Association for Information Systems 37(6):112-137. August 2015. DOI: 10.17705/1CAIS.03706. [9] van Dulmen SA, Tacken MA, Staal JB, Gaal S, Wensing M, Nijhuis-van der Sanden MW. Patient safety in primary allied health care: what can we learn from incidents in a Dutch exploratory cohort study? Med Care. 2011 Dec;49(12):1089-96. doi: 10.1097/MLR.0b013e3182367105. PMID: 22002643. [10] Magrabi F, Ong MS, Runciman W, Coiera E. An analysis of computer-related patient safety incidents to inform the development of a classification. J Am Med Inform Assoc. 2010 Nov-Dec;17(6):663-70. doi: 10.1136/jamia.2009.002444. PMID: 20962128; PMCID: PMC3000751. [11] Stewart MJ, Georgiou A, Hordern A, Dimigen M, Westbrook JI. What do radiology incident reports reveal about in-hospital communication processes and the use of health information technology?. Stud Health Technol Inform. 2012;178:213-218. [12] Terezakis SA, Harris KM, Ford E, Michalski J, DeWeese T, Santanam L, Mutic S, Gay H. An evaluation of departmental radiation oncology incident reports: anticipating a national reporting system. Int J Radiat Oncol Biol Phys. 2013 Mar 15;85(4):919-23. doi: 10.1016/j.ijrobp.2012.09.013. Epub 2012 Dec 19. PMID: 23265568. [13] Warm D, Edwards P. Classifying health information technology patient safety related incidents - an approach used in Wales. Appl Clin Inform. 2012 Jun 27;3(2):248-57. doi: 10.4338/ACI-2012-03-RA0010. PMID: 23646074; PMCID: PMC3613022. [14] Palojoki S, Mäkelä M, Lehtonen L, Saranto K. An analysis of electronic health record-related patient safety incidents. Health Informatics J. 2017;23(2):134-145. doi:10.1177/1460458216631072. 162 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200715 Semantic Clustering to Augment Qualitative Content Analysis in Exploring Reasons for Emergency Department Transfer Delays Laura-Maria PELTONENa,1, Sanna SALANTERÄa,b and Hans MOENc a Department of Nursing Science, University of Turku, Turku, Finland b Turku University Hospital, Turku, Finland c Department of Future Technologies, University of Turku, Turku, Finland Abstract. The aim of the study was to explore emergency department transfer delays and to assess the potential of using a semantic clustering approach to augment the content analysis of transfer delay data. Data were collected over a period of 5 months from two hospitals. A set of (unique) phrases describing reasons for transfer delays (n=333) were clustered using the k-means with 1) cluster centroids initiated in an unsupervised fashion and 2) a semi-supervised version where the cluster centroids were initiated with keywords. The unsupervised algorithm clustered 77 % and the semi-supervised 86 % of the phrases to suitable clusters. We chose the better performing approach to augment our content analysis. Three main categories for transfer delays were found as a result. These included 1) insufficient staffing resources, 2) transportation and bed issues, and 3) patient and care related reasons. The findings inform the audit of organisational processes, accuracy of staffing and workflow to reduce transfer delays. Future research should explore implications of semantic clustering approaches to other narrative data sets in health service research. Keywords. Emergency department, health service research, k-means clustering, qualitative content analysis, transfer delay 1. Introduction A transfer delay can be defined as a situation when a patient is medically ready to be transferred from a unit such as an emergency department (ED) into further care or home, but still occupies a hospital bed. Patient transfer delays are associated with treatment delay and negative patient outcomes, such as increased length of stay, higher hospital mortality [1–2] and higher costs of care [3]. Studies on transfer delays in critical care have reported that organisational issues account for some of the delays [1], with a common reason being insufficient availability of inpatient ward beds [3]. Some transfer delays, such as a deteriorating health condition, cannot be influenced by organisational arrangements, but others could be reduced to improve patient outcomes and reduce costs of care. To date, there is a lack of knowledge about the reasons for patient transfer delays 1 Corresponding Author, Laura-Maria Peltonen, Department of Nursing Science, 20014 University of Turku, Finland; E-mail: laura-maria.peltonen@utu.fi. L.-M. Peltonen et al. / Semantic Clustering to Augment Qualitative Content Analysis 163 from EDs into further care. Further, there is a need to explore these reasons to reduce them and to find ways to estimate their impact on care provision. Our ultimate goal is to describe the reasons for ED patient transfer delays. We want to test semantic clustering approaches to augment the qualitative content analysis of our narrative data with an inductive and a deductive approach. Although, clustering methods are widely used in other fields (see e.g. [4–5]) there is a lack of literature on the use of these methods on narrative data in health service research. Such methods have the potential to support the analysis of big e.g., hospital wide narrative data sets, which previously have been impossible to analyse as manually processing of large amounts of information is typically difficult and time consuming. 2. Methods 2.1 Data collection The study had an interrupted time series design. Data were collected in two EDs in Finland for four weeks at the time during five intervals in 2015–2016. Departments were purposefully chosen; one was from the north and the other from the south of the country. Nursing shift leaders manually documented transfer delay reasons. Ethical review was done by the University of Turku Ethics Committee (ID: 13/2015). 2.2 Clustering We used the word2vec toolkit [6] to train semantic word vectors (embeddings) for each unique word in a large corpus of clinical text consisting of nursing and physician notes from patients admitted to a Finnish university hospital. It consists of 136 million tokens (1.5 million unique tokens). We used NLTK (Natural Language Toolkit for Python) [7] for the pre-processing of our text. Initial testing showed that better scores were achieved when normalizing the words with stemming (Snowball stemmer for Finnish). In addition we performed tokenization, lowercasing and stopword removal. We generated a sentence vector for each documented transfer delay phrase by summing their constituent word vectors, with the addition of first normalizing and multiplying the word vectors by their inverse sentence frequency (c.f. inverse document frequency (IDF)) derived from the training corpus. These vectors were finally used as input into two clustering approaches using the k-means algorithm [8].   The unsupervised k-means clustering algorithm with cluster centers initiated in an unsupervised fashion (k-means++). Here we used the implementation in scikit-learn [9]. We set the expected number of clusters to be generated (n=8) based on a consultation with domain experts. K-means clustering with cluster centers initialized with keywords. As a second clustering approach we tested a semi-supervised clustering approach where we first manually defined keywords and key phrases for each of the eight clusters (one or more per cluster) provided by a domain expert. Next we generated a vector for each cluster centroid by averaging the vector of each keyword (where each keyword vector were generated in the same way as the care delay statements). Finally, we applied k-means with these vectors as cluster centroids. 164 L.-M. Peltonen et al. / Semantic Clustering to Augment Qualitative Content Analysis 2.3 Evaluation of automatically generated clusters We developed a gold standard for the data set by having domain experts manually cluster the transfer delays for our automated cluster evaluation. We used the adjusted Rand index [10] for the automatic evaluation of the generated clusters against the manually made gold standard. This index describes the agreement between two clusters (partitions) as a value or score between -1 and 1 (where 1 equals identical). Although this score can be difficult to interpret directly, it is useful for comparing two or more generated clusters when such a gold standard is available. Finally, we manually assessed the automatically generated clusters. Each item in each cluster was rated on a four-class scale: 1) suits this cluster, 2) suits this or another cluster, 3) suits another cluster but not this, and 4) cannot be analysed due to unclear phrase. 2.4 Synthesis of findings We used the automatically generated clusters of the better performing clustering approach (i.e. keyword initialized k-means) as a basis for our qualitative content analysis [11-12]. We continued with grouping the developed clusters into higher abstraction levels by merging similar categories by discerning them from those that were dissimilar until we discovered a set of main categories that no longer could be merged. We only focused on the manifest content as the data set consist of free text written by shift leaders. 3. Results A total of 333 unique reasons were used for the automatic clustering from a set of 600 documented phrases for patient transfer delays. Seven phrases were excluded as they lacked vector representation. 3.1 Performance of the automatic clustering approaches The automatic evaluation against the gold standard showed an adjusted Rand index score of 0.4508 for the k-means with cluster centers initiated in an unsupervised fashion and 0.5337 when cluster centers were initiated with keywords. A total of 277 out of 333 phrases were assessed to suit the suggested cluster (including classes 1 and 2) when using the keyword initialized clustering based on the manual evaluation, while the respective number for the unsupervised approach was 248 out of 333 (Table 1). Table 1. Contingency table of the manual evaluation with number of the evaluated phrases per rating class of the unsupervised k-means clustering and keyword initialized k-means clustering results. Rating class 1) Suits this cluster Unsupervised k-means clustering (n) 154 Keyword initialized k-means clustering (n) 178 2) Suits this or another cluster 94 99 3) Suits another cluster but not this 73 44 4) Cannot be analysed due to unclear phrase 12 12 248 277 1 and 2 L.-M. Peltonen et al. / Semantic Clustering to Augment Qualitative Content Analysis 165 3.2 Reasons for patient transfer delays Our augmented qualitative content analysis resulted in three main categories of reasons behind transfer delays from EDs into further care. These included 1) insufficient staffing resources, 2) transportation and receiving unit bed issues, and 3) patient and care related issues. The insufficient staffing resources main category included two sub categories that commonly covered examples of waiting for paperwork, such as the electronic health record notes to be completed or a physician’s order, as well as waiting for a particular transfer related task to be completed by what seemed to be busy professionals. Individual examples of the busyness of professionals included inability to find time to care for a patient or transfer the patient due to haste. The transportation and bed issues main category also contained two sub categories. A need to wait for permission from the receiving unit to transfer the patient and a lack of space on the receiving unit both within and beyond the hospital were commonly reported issues. The other sub category with frequently reported reasons for waiting regarded transportation means that included examples like waiting for an ambulance or a taxi. The final main category, namely, patient and care related reasons included four sub categories. The first of these focused on the wait of different examination and laboratory values, such as blood samples, xrays and CT-scans. The second regarded the time for obtaining consultations from specialist, such as neurologists and physiotherapists. The third sub category covered waiting for procedures, such as chest tube drainage or central catheter placement, or follow-up time after a cardioversion. The fourth sub-category included patient related issues, such as a change in a patient’s health condition that required attention and reassessment or a request by the patient for a rest before leaving. 4. Discussion Content analysis belongs to the most commonly used analysis methods in descriptive qualitative studies in health sciences as it is a feasible method for analysis in many different contexts and settings. But up until now, qualitative content analysis has been limited to the amount of workload possible to be completed by researchers manually. In our study, the clustering algorithms were able to cluster 77–86% of the phrases to suitable clusters. Despite the advantage of the approach where cluster centers are initialized with keywords provided by domain experts, the fully unsupervised approach performed comparable (only 0.08 below in adjusted Rand index). According to our results, it is feasible that both unsupervised and semi-supervised semantic clustering can be used in inductive and deductive qualitive content analyses [12]. An unsupervised approach is particularly useful when prior knowledge and a theoretical framework is lacking and no keywords can be provided. In the future, other clustering approaches could be tested if it is difficult to state a sensible number of clusters and for a more advanced approach to the qualitative content analysis. This includes hierarchical clustering and methods that try to estimate the most sensible number of clusters to be used (see e.g. [13]). It is important to acknowledge that the content analysis process is not free from interpretation [11]. It requires the researcher to be completely familiar with the data, which usually necessitates several iterations of reading the data [12]. A critique against qualitative content analysis states that the method is often used in a simple manner and a deeper data interpretation should be visible [11]. The purpose of using a machine-driven semantic clustering approach is not to shift towards simple or superficial 166 L.-M. Peltonen et al. / Semantic Clustering to Augment Qualitative Content Analysis analyses, but rather to augment the content analysis process when dealing with a large data set that would be difficult to analyse manually. Here, clustering shows potential in developing descriptive categories in the re-contextualisation phase of the analysis process [11]. The findings showed that patient transfer delay reasons in EDs can be classified into insufficient staffing resources, transportation and bed issues, as well as patient and care related reasons. These findings may be used to inform audits and developmental work of organisational processes, staffing adequacy and use, and changes in workflow to reduce transfer delays. This has the potential to improve quality of care and patient outcomes as well as reduce costs of care. It is important to find appropriate ways to analyze, classify and summarize data as ample information is collected in health services every instant. Clustering can be seen as one such approach. More research is warranted to explore the implications of these methods in other narrative health service data sets. Study limitations include the weakness of the manually collected data with missing entries and a possibly skewed representation between data collection months. In conclusion, semantic clustering has the potential to support researches in both inductive and deductive content analysis of big narrative data sets. The results showed organisational process, staffing and workflow -related issues that potentially could be addressed to reduce patient transfer delays from EDs. Acknowledgements This research was supported by the Academy of Finland (315376). References [1] Grech C, et al. The delay in transfer between the emergency department and the critical care unit for patients with an acute cardiac event--in hospital factors. Aust Crit Care. 2001 Nov;14(4):139-45. [2] Chalfin DB, et al. Impact of delayed transfer of critically ill patients from the emergency department to the intensive care unit. Crit Care Med. 2007 Jun;35(6):1477-83. [3] Johnson DW, et al. Delay of transfer from the intensive care unit: a prospective observational study of incidence, causes, and financial impact. Crit Care. 2013 Jul;17(4):R128. [4] Maimon O, Rokach L. Clustering methods. In: Rokach L, Maimon O. editors. Data mining and knowledge discovery handbook. Boston: Springer; 2005. p. 321-52. [5] Grira N, Crucianu M, Boujemaa N. Unsupervised and semi-supervised clustering: a brief survey. A review of machine learning techniques for processing multimedia content. 2004 Jul;1: 9-16. [6] Mikolov T, et al. Distributed representations of words and phrases and their compositionality. Proceedings of the 27th Annual Conference on Neural Information Processing Systems. Advances in Neural Information Processing Systems 26; 2013 Dec 5-10; Nevada. CA. 2014 p. 3111-9. [7] Bird S, Loper E, Klein E. Natural Language Processing with Python. O’Reilly Media Inc., Sebastopol, California, USA. 2009. [8] Lloyd SP. Least squares quantization in PCM. Information Theory IEEE Transactions 1982 28(2);129-37. [9] Pedregosa F. et al., Scikit-learn: Machine Learning in Python. JMLR. 2011;12: 2825-30. [10] Hubert L, Arabie, P. Comparing partitions. J. Classif. 1985 Dec;2(1);193–218. [11] Lindgren BM, Lundman B, Graneheim UH. Abstraction and interpretation during the qualitative content analysis process. Int J Nurs Stud. 2020 Aug;108:103632. [12] Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs 2008 Apr;62(1):107-15. [13] Kriegel HP, Kröger P, Sander J, Zimek A. Density‐based clustering. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. 2011 Apr;1(3):231-40. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200716 167 Health Data Privacy: Research Fronts, Hot Topics and Future Directions Javad POOLa,1, Farhad FATEHI b,c, Farkhondeh HASSANDOUST d and Saeed AKHLAGHPOUR a a The University of Queensland, Brisbane, Australia b Monash University, Melbourne, Australia c Tehran University of Medical Sciences, Tehran, Iran d Auckland University of Technology, Auckland, New Zealand Abstract. Health data privacy is an important research stream due to the high impacts on the success of digital health transformation and implementation. Neglecting to safeguard data confidentially and integrity and mitigate risks associated with unauthorized access will lead to failures in materializing benefit from digital health. This study aims to present a bibliometric analysis of health data privacy and provide a platform for future directions. We conducted a literature search between 2010 and 2020 in the Web of Science (WoS) database, resulted in 1,752 records. As part of the bibliometric analysis, concept mapping of health data privacy researches was depicted by network visualization and overlay visualization. These two visualizations represent five research fronts and emerging topics (e.g., digital health, blockchain, the internet of things (IoT)). Finally, we chart directions for future research on health data privacy, highlighting emerging topics, and boundary-breaking alternatives (e.g., GDPR, contact tracing apps in the context of pandemics). Keywords. Privacy, cybersecurity, digital health, health data, data protection 1. Introduction Implementation and effective use of digital health can revolutionize healthcare delivery and improve the quality of care. However, unlocking the net benefits of digital health cannot be achieved without protecting the confidentiality and privacy of health data. Investments in health data protection should be included in the healthcare strategy for digital transformations to actualize business value. According to the Cisco data privacy benchmark study in 2020, protecting clients’ data drives business value such as innovation, enabling agility and operational efficiency [1]. To gain competitive advantages in the age of Artificial Intelligence (AI), Internet of Medical Things (IoMT), and big data, healthcare industries need to be aligned with updated and new privacy regulations such as the European Union General Data Protection Regulation (GDPR). During the past decade, the health sector has experienced high profile data breaches. Research highlighted that failures in protecting health data privacy and security are associated with the reputational and financial cost to healthcare organizations and more 1 Corresponding Author, Javad Pool, The University of Queensland, Brisbane, Australia; E-mail: j.pool@uq.net.au. 168 J. Pool et al. / Health Data Privacy: Research Fronts, Hot Topics and Future Directions importantly, rated to care quality. For example, the Ponemon study reported the average of a health data breach total cost as $6.45 million, which was higher than other industries [2]. Also, a study in the US hospital context revealed that health data breaches were associated with deterioration of care delivery [3]. Health data privacy is closely linked to ‘privacy protection practices’ and ‘security measures’. These two protective safeguards facilitate ensuring the authorized use, confidentiality, and integrity of personal health data. The need for a profound understanding of security and privacy phenomena in the healthcare context has brought together researchers from different domains such as information systems [4] and medical informatics [5]. Research topics in this multidisciplinary field range from social to technical, and psychological perspectives. Therefore, it is important to provide an overview of the published research so that interested academics and practitioners can clearly understand the research profile so far. In this study, therefore, we conducted a bibliometric analysis to examine the academic research fronts in the field of ‘Health Data Privacy’ to inform scholars and provide impactful directions for future research. 2. Methods To conduct this bibliometric analysis, first, we developed a search strategy to capture peer-reviewed publications related to ‘Health Data Privacy’. Table 1 shows our search query in the Web of Science (WoS) database. Table 1. Search strategy Search queries Privacya AND (Health* OR Medic* OR clinic* OR hospital)b AND (electronic OR online OR digital OR Internet OR Virtual OR “Information system*” OR “information technolog*” OR “computer*” OR “information and communication technologies” OR ICT)a Limitation Years: 2010-2020 Index: SCIE, SSCI, A&HCI, ESCI Type: Articles (excluding reviews) a In Title/Abstract/Keywords b In Title The WoS search result was exported in Tab-delimited format as an input for the bibliometric analysis. The analysis has been conducted via VOSViewer version 1.6.15 [6]. Using this software, this study reports a concept mapping via co-occurrences analysis based on authors' keywords (unit of analysis). To demonstrate the meaningful concept mapping, we also created a thesaurus to perform data cleaning. This thesaurus, then, was loaded into VOSViewer to replace or merge synonym terms such as electronic medical records, electronic medical record, electronic medical record (EMR), EMR, and EHR. 3. Results Our search in the WoS database returned 1,752 records. To provide a concept mapping of health data privacy literature, two types of visualizations, namely ‘network visualization’ and ‘overlay visualization’ were represented in our study to illustrate research fronts (privacy-related clusters) and emerging topics. 3.1. Research fronts Our analysis of co-occurrence of authors’ keywords revealed five privacy-related clusters, which are depicted in Figure 1 with different colors. J. Pool et al. / Health Data Privacy: Research Fronts, Hot Topics and Future Directions 169 Figure 1. Network visualization of health data privacy. High-res image available at https://bit.ly/2P2rSUk We labeled these clusters as five research fronts: ‘privacy context’, ‘digital health and care delivery context’, ‘data right and use aspects’, ‘security context’, and ‘technical safeguards and impacts’. Table 2 summarizes these research fronts and hot topics in health data privacy. Table 2. Research fronts and hot topics in health data privacy research Cluster Research fronts Hot topics #1 Privacy context Privacy, anonymization, confidentiality, de-identification, EHR/EMR, health data, data protection #2 Digital health delivery context #3 Data rights and use aspects Ethics, HIPAA, trust, education, informed consent, policy, professionalism, information systems, internet, social media, Facebook #4 Security context Security, big data, data mining, blockchain, cloud, data security, data sharing, deep learning, encryption, health services, healthcare, interoperability, medical images, Internet of Things (IoT), machine learning, privacy preservation, cloud computing #5 Technical impacts and safeguards care and Digital health, e-health, mobile health, mobile devices, health information exchange, health information systems, mobile apps, medical informatics, mental health, personal health records, consent, telehealth Anonymity, authentication, biometrics, key agreement, mutual authentication, RFID, smart card, telecare medical information systems, privacy protection, technology adoption As evident in the network visualization, hot topics such as EHR/EMR, and security have received more attention among health data privacy clusters. 170 J. Pool et al. / Health Data Privacy: Research Fronts, Hot Topics and Future Directions 3.2. Emerging topics and future directions Figure 2 illustrates the emerging topics in health data privacy research. The network structure is similar to Figure 1, but the hot topics are colored based on years. The yellow color in the figure indicates hot topics, emerged from 2018 onwards. These ‘trending’ topics include IoT, blockchain, and digital health. However, topics such as HIPAA, and data mining as a general method are gradually ‘cooling off’. Figure 2. Overlay visualization of health data privacy. High-resolution image is available at https://bit.ly/3jHVg02 Based on the emerging topics (Figure 2) and boundary-breaking alternatives (which can arguably replace the cooling off topics), our suggestions for future research directions are summarized in Table 3. Table 3. Future research opportunities in health data privacy Directions IoT Emerging Topics Blockchain Machine learning and deep learning Digital health Data sharing Key Questions How and why the introduction and use of IoT in healthcare can increase the risk of health data breaches? How care providers can address and mitigate the cybersecurity and privacy risks associated with the use of IoT in healthcare? How do blockchain implementations in healthcare can influence data protection practices? What are the privacy protection opportunities and risks associated with the use of blockchain in healthcare? How do privacy rights (e.g. the right to restrict processing) can be considered in the use of machine learning and deep learning methods? How do privacy rights (e.g. the right to erasure) affect medical decision making based on these methods? How can data protection be designed and included in digital health transformations and contribute to improved healthcare performance? How do care providers effectively use and implement privacy policies for data sharing in the telehealth context (i.e., GP-to-Specialist, GPto-Nurse, Patient-to-GP)? J. Pool et al. / Health Data Privacy: Research Fronts, Hot Topics and Future Directions Directions Health services Boundary-breaking alternatives GDPR Artificial Intelligence (AI) 5G Internet Privacy protection value Contact tracing apps 171 Key Questions What are the impacts of data breaches on data sharing practices among providers and patients? How can general privacy frameworks (e.g., NIST) be contextually implemented in different health services (e.g., elderly home care)? How do medical device manufacturers consider GDPR in their processes of designing of digital artefacts? What challenges do Data Protection Officers (DPO) face in protecting health data in practices and how can these data protection challenges be addressed? How can AI play a role in detecting unauthorized access to health data and facilitate response to health data breaches? How do privacy concerns related to health data will influence the adoption of 5G Internet in healthcare? How can healthcare providers plan and actualize business value from protecting patient data (e.g., innovation, agility)? How can ‘privacy concerns’ and ‘lack of data protection by design’ trigger individuals' resistance to adopt and use contact tracing apps in the pandemic context such as COVID-19? 4. Conclusions Healthcare organizations and users are moving towards digital health to enhance their performance and co-create value (e.g., in improved management of chronic diseases). However, unlocking the net benefit of digital health technologies requires attention to and practice of safeguarding health data and protecting users’ privacy. This bibliometric study reported on the hot topics in health data privacy literature. Furthermore, our study illustrated that the research streams are moving towards new trends such as blockchain and IoT, which show opportunities for health data privacy researches. Also, beyond the emerging trends, we proposed new directions for privacy researches, i.e., GDPR and privacy in the context of contact tracing apps in pandemics. Theses research opportunities are worth exploring to inform research, policy, and data protection practices. While future research can update the hot research topics in the five identified clusters of health data privacy, we encourage scholars to set a high priority for emerging topics and delve deeper into boundary-breaking alternatives. References [1] [2] [3] [4] [5] [6] CISCO. From Privacy to Profit: Achieving Positive Returns on Privacy Investments. USA: Cisco and/or its affiliates, 2020. Ponemon-Institute. Cost of a Data Breach. USA: Ponemon Institute LLC, Results sponsored, analyzed and reported by IBM Security, 2019. Choi SJ, Johnson ME, Lehmann CU. Data breach remediation efforts and their implications for hospital quality. Health services research. 2019;54(5):971-80. Kim SH, Kwon J. How Do EHRs and a Meaningful Use Initiative Affect Breaches of Patient Information? Information Systems Research. 2019;30(4):1184-202. Prasser F, Spengler H, Bild R, Eicher J, Kuhn KA. Privacy-enhancing ETL-processes for biomedical data. International journal of medical informatics. 2019;126:72-81. Van Eck NJ, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping. scientometrics. 2010;84(2):523-38. 172 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200717 Multicriteria Decision Support Would Avoid Overdiagnosis and Overtreatment Vije Kumar RAJPUTa, Jack DOWIEb,c,1 and Mette Kjer KALTOFTc a Stonydelph Health Centre, Tamworth, UK b London School of Hygiene and Tropical Medicine c University of Southern Denmark Abstract. Population-level studies confirm the existence of significant rates of overdiagnosis and overtreatment in a number of conditions, particularly those for which the screening of asymptomatic individuals is routine. The implication is that the possibility of being overdiagnosed and/or overtreated must be mentioned as a possible harm in generating informed consent and participation from the individual invited to be screened. But how should the rates of such preference-insensitive population-level phenomena be introduced into preference-sensitive individual decision making? Three possible strategies are rejected, including the currently dominant one that involves presenting the rates relevant to overdiagnosis and overtreatment as discrete pieces of information about a single criterion (typically condition-specific mortality). Extensive quotation from a review of cancer decision aids confirms that processing this complex and isolated information is not a practical approach. However, the task is unnecessary, since an outcome-focused multicriteria decision support tool will incorporate the effects of overdiagnosis and overtreatment - along with the effects of any underdiagnosis and undertreatment. Keywords: Overdiagnosis, overtreatment, multi-criteria decision support 1. Introduction There has been growing recognition of the possibility, indeed likelihood, that healthcare delivery is characterised by phenomena variously labelled Over-Detection/ OverTesting/Over-Diagnosis (OD), with resulting Over-Treatment (OT), increasingly referred to as ‘too much medicine’. The possibility of the obverse Under-phenomena (UD, UT) is well-recognised, but not currently regarded with such concern, and, interestingly, by implication, seen as a separable issue. Interest in Decision Support Tools (DSTs) has simultaneously expanded in recent years, largely as a result of the growing commitment to ‘shared decision making’ by healthcare professionals and the consequent need for more effective and detailed communication with autonomous patients regarding the decision being made. Legal changes in relation to the obtaining of informed and preference-based consent have been another stimulus and this is one likely to become more important in the increasingly digital age that patients inhabit. 1 Corresponding author, Jack Dowie, LSHTM, 15-17 Tavistock Place, London, UK WC1H 9SH; E-mail: jack.dowie@lshtm.ac.uk V.K. Rajput et al. / Multicriteria Decision Support 173 These two trends are related, but not easily integrated. Clinical decision making, whether shared or not, takes place at the individual level ex ante the rest of their life. In contrast, OD/OT are group/population level constructs that can only be measured at that level and ex post – as the percentage of those who had a disease or condition detected, diagnosed, and treated, from which they would not have died or experienced lifeaffecting symptoms. The earlier detection and diagnosis may well have been correct, in the sense that a tumour was present in the individual and was correctly identified as ‘cancer’ according to standard definitions. But it would have - with probability OD/OT - remained ‘indolent/benign’ and not affected the individual’s length of life or health. Whether the overdiagnosis rate established by follow-up studies at a population level is 22%, one finding for breast cancer screening, or 42%, the parallel finding for prostate cancer screening [1], the question arises: how should the phenomenon, and its extent, be introduced into a clinical process committed to making decisions for which preferencesensitive informed consent has to be obtained? The question is relevant to the ‘empowered physician’ of futurist Bertalan Mesko [2], as well as to the digitallyempowered citizen with whom that future physician will be engaging. Brodersen [3] has rightly emphasised the conceptual complexity of the issue and communication task: “All health professionals, politicians, health authorities, patients, and citizens, in general, have a stake in the answer to the question: what is the risk of being overdiagnosed? However, to answer this question, the denominator or the comparator must be defined. The risk of being overdiagnosed in cancer screening could be split into numerous questions, for example, (1) how many in a cohort invited to screening are overdiagnosed with cancer? (2) How many of the screening participants are overdiagnosed with cancer? (3) How many of the screening-detected cancers are overdiagnosed? (4) How many deaths from cancer are prevented compared to how many screening participants are overdiagnosed with cancer? “[3] (p81) Housten and colleagues have recently reviewed how OD/OT is treated - or not - in 85 cancer screening patient decision aids [4]. Their systematic review embraces the various verbalisations of the concept/s which avoid using the specific OD/OT terms, such as ‘experiencing testing or treatment which would turn out to have been unnecessary’. They emphasise the need for improved understanding of the phenomena at both individual and collective levels, especially via decision support. “The trade-offs regarding cancer screening and how to communicate them persist and warrant the development of effective communication strategies to support decision making. Moreover, there is a strong ethical need to include the potential harms of cancer screening, including overdiagnosis, that can be understood by a broad population.”[4] (p9). 2. Method How should overdiagnosis and overtreating be dealt with in point of care decision making, where the clinical task is to identify the preference-sensitive optimal option for the person? From the Housten systematic review we identified four possible strategies for introducing OD/OT considerations into an individual decision process, within or outside a clinical consultation. The four strategies uncovered were then assessed in terms of their communication complexity and ability to meet the requirements of informed and preference-based consent. 174 V.K. Rajput et al. / Multicriteria Decision Support 3. Result 3.1. Strategy 1 Inform the person that there is a possibility of OD/OT, perhaps with some qualitative verbal quantification (‘small chance’, ‘moderate risk’), but no numerical rate/s, even in the form of uncertainty ranges. This strategy fails to meet the minimum requirements for informed consent, let alone preference-based informed consent. It is often adopted where the aim is to increase the uptake of an intervention believed to be in the interests of the patient. 3.2. Strategy 2 Provide relevant numerical rate/s of OD/OT as discrete information to be input into a verbal deliberative process, either without a decision aid or incorporated within one. This is the current dominant strategy, so we quote extensively from four of the 67 breast cancer screening aids that introduced OD/OT according to the Housten systematic review [4]. (The remaining 18 aids did not mention this possibility.) x ‘Breast screening’ leaflet [Available from: https://patient.info/cancer/breast-cancer-leaflet/breastscreening] “An independent review in the UK in 2012 concluded that breast screening does save lives. If 10,000 women are screened from when they are 50 to when they are 70, around 43 deaths would be prevented… it was concluded that for the 10,000 women screened from when they are 50 to 70, 129 women would be over-diagnosed. The Cochrane review found that for the 2,000 women screened over 10 years, 10 women would have unnecessary treatment. In this analysis, for every life saved, ten women would have treatment which was not necessary. In the UK, the NHS screening programme estimates that for every life saved, three women have treatment that they didn't need.” x ‘It’s your Choice’ [Available from: https://bit.ly/SydneyBCAid] “Out of 1000 women who have breast screening for 25 years: 5 women avoid dying from breast cancer because of screening and 14 women still die from breast cancer; 103 women are diagnosed with breast cancer. Of these, 30 women experience over-detection: they are diagnosed and treated for a cancer that would not have caused any trouble and 73 women are diagnosed with breast cancer that is not over-detection….More women experience over-detection than avoid dying from breast cancer.“ (For a report on the trial of this aid see [5]). x Health Decision [Available from: https://www.healthdecision.org/tool#/tool/mammo] “Studies show that 10-30% of tumors found on a screening mammogram will not grow or spread fast enough to affect a woman’s life. Case example: Patient is 50 years old; no family history of breast cancer; no previous breast biopsy; V.K. Rajput et al. / Multicriteria Decision Support 175 breast density is "unknown"; race/ethnicity White. For 1000 such women aged 50 for 10 years No mammogram: 29 are diagnosed with BC; 971 are not diagnosed with and will not have BC; 24 survive BC with or without screening. Biennial mammogram: 33 are diagnosed with BC: 24 survive BC with or without screening, 1 saved from a BC death, 4 die from BC, 4 extra are overdiagnosed from screening. 967 are not diagnosed with BC: 587 no BC, recalls or biopsies, 380 recalled for one or more false alarms, 63 undergo a biopsy which is normal.” x ‘Is a mammogram right for me’ Canadian Cancer Society [Available from: http://www.mybreastsmytest.ca/en/] “In Canada, about 1 out of 215 women aged 50-69 who go for a mammogram as part of a provincial screening program will be diagnosed with breast cancer. Of the 215, 199 will get the ‘all clear’, 16 are called for more tests, 15 get the ‘all clear’, 1 will have breast cancer… For every breast cancer found, approximately 1-10% are non-life threatening.” These examples confirm that valiant attempts to communicate about OD/OT, even when accompanied by pictograms, are likely to result in confusion, misinterpretation, or simple abandonment of any attempt to absorb. Our cognitive competencies are not up to the task of dealing with the complex results produced, however attractively communicated, and certainly not in the time likely to be allocated at the point of decision. In our view the task is not one that can be addressed without support which embeds the relevant numbers in a decision framework. Simply ‘being informed’ about them without knowing how to process them in decision making is of dubious value. 3.3. Strategy 3 Include ‘Being OD/OT’ (or something similar) as a separate criterion in a Multi-Criteria Decision Analysis (MCDA)-based decision support tool (DST). In this sort of decision support tool the criterion ‘avoiding being OD/OT’ would be assigned a rate from the literature. The individual concerned would then assign a weight to ‘being OD/OT’ relative to the other criteria in the tool, such as the length and quality of life and treatment burden. However, as soon as this strategy is spelled out, it is clear that this is not a valid one, since it will be the consequences of OD/OT not being OD/OT which is of concern. These consequences will be incorporated in the other criteria and. while there may be annoyance attached to simply being OD/OT, we assume the weight attached to this, as opposed to the consequences, will be negligible. 3.4. Strategy 4 Ignore any consideration of OD/OT phenomena in an MCDA-based DST 176 V.K. Rajput et al. / Multicriteria Decision Support This strategy emerges as dominant for three reasons. One, entering in the DST the best individualised performance rates of options on the key outcome criteria - all-cause and condition-specific mortality, and various forms of all-cause and condition-specific morbidity, will incorporate the effect of any OD/OT. (To be clear, being diagnosed with a condition is not an outcome criterion.) Two, this strategy will simultaneously address any UD/UT, which will also be of major, if not more, importance to the person. Three, the weighting of these criteria by the person will overcome the preference insensitivity of the population rates of OD/OT and complete the process of meeting the requirements of informed and preference-based consent. 4. Conclusion Debates about OD/OT are useful, but in seeking to establish, classify and modify the sources of these phenomena located outside the individual decision, they represent a distraction from the central question: how do we enable, for the individual, the ‘duallypersonalised’ care that is optimal for them, i.e. combines their personalised preferences and individualised evidence into an evaluation of each option. Not only is any attempt to introduce OD/OT rates as discrete information on a single criterion unlikely to be helpful, it also fails to address UD/UT simultaneously, or at all. Fortunately, it is unnecessary to do so, given our vision of the future citizen empowered by MCDA-based DSTs. In these, the best available estimates of the performance rates of the available options on all-cause and condition-specific outcomes will implicitly incorporate group/population rates of OD/OT – and indeed of UD/UT as well. We do not need to burden clinician and/or person with having to ‘take into account and bear in mind’ these phenomena. The problem at the individual level is not of possible over- or under-treatment, but of possible mis-treatment, i.e. management which is out of line with the optimal decision for the person, either because the relevant performance rates are not available, or are not drawn on, if available. References [1] [2] [3] [4] [5] Glasziou P, Jones MA, Pathirana T, Barratt AL, Bell KJL. Estimating the magnitude of Cancer Overdiagnosis in Australia. Med J Aust. 2020 212(4):163–68. Mesko B, Győrffy Z. The rise of the empowered physician in the digital health era: Viewpoint. J Med Internet Res. 2019 21(3):e12490. Brodersen J. How to conduct research on overdiagnosis. Eur J Gen Pract 2017 23(1):78-82. Housten AJ, Lowenstein LM, Hoffman A, Jacobs LE, Zirari Z, Hoover DS, et al. A Review of the Presentation of Overdiagnosis in Cancer Screening Patient Decision Aids. MDM Policy Pract. 2019 4(2). Hersch J, Barratt A, Jansen J, Irwig L, McGeechan K, Jacklyn G, et al. Use of a decision aid including information on overdetection to support informed choice about breast cancer screening: A randomised controlled trial. Lancet. 2015 385(9978):1642–52. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200718 177 Creating Synthetic Patients to Address Interoperability Issues: A Case Study with the Management of Breast Cancer Patients Akram REDJDALa1, Jacques BOUAUDb,a, Gilles GUÉZENNECa, Joseph GLIGOROVc,d and Brigitte SEROUSSIa,c a Sorbonne Université, Université Sorbonne Paris Nord, INSERM, UMR S_1142, LIMICS, Paris, France b AP-HP, DRCI, Paris, France c AP-HP, Hôpital Tenon, Paris, France d Sorbonne Université, Institut Universitaire de Cancérologie, Paris, France Abstract. Interoperability issues are common in biomedical informatics. Reusing data generated from a system in another system, or integrating an existing clinical decision support system (CDSS) in a new organization is a complex task due to recurrent problems of concept mapping and alignment. The GL-DSS of the DESIREE project is a guideline-based CDSS to support the management of breast cancer patients. The knowledge base is formalized as an ontology and decision rules. OncoDoc is another CDSS applied to breast cancer management. The knowledge base is structured as a decision tree. OncoDoc has been routinely used by the multidisciplinary tumor board physicians of the Tenon Hospital (Paris, France) for three years leading to the resolution of 1,861 exploitable decisions. Because we were lacking patient data to assess the DESIREE GL-DSS, we investigated the option of reusing OncoDoc patient data. Taking into account that we have two CDSSs with two formalisms to represent clinical practice guidelines and two knowledge representation models, we had to face semantic and structural interoperability issues. This paper reports how we created 10,681 synthetic patients to solve these issues and make OncoDoc data re-usable by the GL-DSS of DESIREE. Keywords. Health information interoperability, Knowledge representation, Clinical decision support systems, Breast cancer. 1. Introduction Today, it is common for health care to be delivered across multiple settings. Each stay generates a record, but due to the lack of interoperability between these records, quality of care can be put at risk when patients are transferred from one organization to another. Thus, cross-organizational healthcare data sharing is a major issue, and improving healthcare interoperability is a top priority for health organizations. Indeed, interoperability issues are currently common, and reusing data generated from a system by another system, for instance a clinical decision support system (CDSS), in a new organization is a complex task due to recurrent problems of alignment between data 1 Corresponding Author, Akram Redjdal, LIMICS UMRS_1142, 15 rue de l’Ecole de Médecine, Paris, France; E-mail: redjdalakram300@gmail.com 178 A. Redjdal et al. / Creating Synthetic Patients to Address Interoperability Issues models and semantics. Solutions have been proposed like the OMOP common data model or the FHIR exchange format, while sharing common reference terminologies (e.g., SNOMED-CT, ICD10, UMLS, etc.). But, the source and the target systems often share the same conceptual model. Thus, it remains complex to smoothly integrate existing data sources into other systems. DESIREE2 is a recent European-funded project which aimed at developing a webbased platform to improve the management of primary breast cancer patients. Among other services, DESIREE includes a guideline-based decision support system (GL-DSS) that the authors of this article have developed [1]. OncoDoc is another CDSS that the authors also developed previously for the management of breast cancer patients. OncoDoc has been routinely used by the multidisciplinary tumor boards (MTBs) of the Tenon hospital (Paris, France) during three years proposing guidance for 1,861 decisions [2]. As part of the final deliverable of the DESIREE project, we had to evaluate the GLDSS. Since we were lacking a large sample of clinical data, we decided to reuse the database of clinical cases resolved with OncoDoc. Given the two CDSSs use two different domain knowledge models and two different formalisms to represent breast cancer guidelines, the aim was to develop and implement a model transformation from OncoDoc to the GL-DSS of DESIREE that accounts for both semantic and structural interoperability issues. This paper reports the solution we implemented to deal with interoperability issues by creating synthetic patients. 2. Material and Methods 2.1. Two CDSSs, two knowledge models, two guideline representation formalisms OncoDoc has been developed in a documentary approach of decision support. The knowledge base is structured as a decision tree within which the user navigates while interactively answering questions that instantiate a patient clinical profile. Nodes represent decision variables and edges represent their modalities. OncoDoc data sample is made of clinical cases resolved when using OncoDoc during MTBs. Each recorded decision is attached to a “breast side” and includes a description of the patient profile as a list of instantiated clinical parameters corresponding to decision variables that are all qualitative (e.g., “tumor size” has three values, “less than 2 cm”, “between 2 and 4 cm”, or “more than 4 cm”), and the decision actually made by MTB physicians. The GL-DSS of DESIREE relies on a Breast Cancer Knowledge Model (BCKM) formalized as an ontology. The BCKM allows for rule-based and subsumption-based reasoning to provide best patient-centered therapeutic recommendations. It combines a data model based on the generic Entity-Attribute-Value (EAV) model [1], the main entities being the patient, the breast side, and the lesion, each entity having attributes, and each attribute having a value that can be primitive or hierarchical (e.g., the clinical T of the TNM classification is an attribute of the side entity, and has values among cT1, cT2, cT3, cT4, or cTx). 2 The DESIREE project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 690238. A. Redjdal et al. / Creating Synthetic Patients to Address Interoperability Issues 179 2.2. Model Transformation We started with the identification of correspondences between the two CDSS models, then we developed the mapping of concepts, and we finished with the comparison of the recommendations issued by both OncoDoc and the GL-DSS. 2.2.1. Identification of correspondences We identified three types of alignment between Oncodoc and BCKM concepts: x 1-to-1 correspondences when a variable in OncoDoc has a unique equivalent concept in the BCKM. Several distinctions can be made, as reported in Figure 1: o Exact matching: OncoDoc variables and BCKM concepts and their values are equivalent in both models o Partial matching: several OncoDoc variables are aligned with a unique BCKM concept but some values of OncoDoc variables do not have correspondence in the BCKM o Conditional matching: several OncoDoc variables are aligned with a unique BCKM concept and all values of OncoDoc variables do have correspondence in the BCKM Figure 1. The three types of 1-to-1 correspondences x x n-to-1 correspondences when a variable in OncoDoc is a macro variable that relies on different sub-variables. For instance, the variable “lumpectomy contraindicated” in OncoDoc is described by different subvariables (radiotherapy contra-indicated, widespread microcalcifications, local recurrence) that have to be taken into account in the correspondence with the concept of contra-indicated lumpectomy in the BCKM. 1-to-n correspondences when a value in OncoDoc has multiple correspondences in the BCKM (the tumor size & the lymph node invasion). For instance, the variable “tumor size” has three values in OncoDoc, while its BCKM equivalent concept is captured by the clinical T of the TNM classification, and correspondences are not exact as displayed in the Figure 2. For a patient with “tumor size” = “> 4 cm” in OncoDoc, there are two possible BCKM values, cT2 (which means the tumor size is more than 2cm but no more than 5cm) or cT3 (which means the tumor size is larger than 5cm). To address these semantic issues, we generated for each OncoDoc clinical case, several synthetic patients to represent all possible values of this kind of concepts in the BCKM. 180 A. Redjdal et al. / Creating Synthetic Patients to Address Interoperability Issues 2.2.2. Creation of synthetic patients The first step was to identify which variables in OncoDoc were involved in a 1-to-n correspondence. These variables were related in the BCKM either to the clinical and pathological T of TNM or the clinical and pathological N of TNM. Then we identified all patients that had at least one of these variables in their profile as recorded in the OncoDoc database and we implemented an algorithm to create synthetic patients for each of them depending on tumor size and lymph nodes invasion, e.g., if a patient had “tumor size” = “> 4cm” and “MoreThan2N” = “false” (false in OncoDoc is aligned with cN0 or cN1 in the BCKM), this patient would have 4 synthetic patients as displayed in Figure 2. The creation of synthetic patients is performed through the combinatory combinations of T and N values. Figure 2. Example of two 1-to-n correspondences generating the creation of four synthetic patients. 3. Results OncoDoc database included 1,861 resolved clinical cases described by a set of 61 variables. After identifying correspondences, 30 OncoDoc variables had an exact matching in the BCKM, eight had a partial matching, and five had a conditional matching. For these variables, there was no need to create synthetic patients. We identified 18 “1-to-n” correspondences leading to the creation of synthetic patients. They were related to four main concepts in the BCKM that were added as variables in OncoDoc to be used by the algorithm implemented: x Clinical T of TNM: this BCKM concept matches with eight OncoDoc variables related to the clinical size of the lesion. Besides, there is an additional Boolean variable “TUM-Operable” that specifies whether a tumor is operable or not. It corresponds to cT4 when the tumor is not operable, and other cT values when the tumor is operable. x Pathologic T of TNM: this BCKM concept matches with seven OncoDoc variables describing the pathologic size of the lesion (after surgery), and depending on the cancer type (ductal or lobular carcinoma). x Clinical N of TNM: as displayed in Figure 2, the OncoDoc variable “MoreThan2N” is related to the clinical N of TNM in the BCKM. x Pathologic N of TNM: this BCKM concept is matched with the OncoDoc variable “LymphNodesInvasion” which refers to the result of the axillary lymph node dissection (N-, 1-to-3N+, or >4N+). We finally created 12,542 synthetic patients, from 1,861 resolved clinical cases in OncoDoc. These BCKM-compliant patients represent all the possible representations of 181 A. Redjdal et al. / Creating Synthetic Patients to Address Interoperability Issues OncoDoc clinical cases. Table 1 displays the distribution of synthetic patients according to their referent OncoDoc clinical cases. The average number of synthetic patients is 206. The max number of synthetic patients created for a clinical case is 35 coming from the combination of seven pNt2 (pN2, pN2a, pN2b, pN3, pN3a, pN3b, pN3c), and five pT1 (pT1, pT1a, pT1b, pT1c, pT1mic). The category of patients with the most repetitions (766) corresponds to patients that have a unique N or no information about the N of TNM. In this case, synthetic patients are created only because of the T of TNM, with cT1, cT4, pT1 or pT4, values, thus leading to five synthetic patients (i.e., cT1, cT1a, cT1b, cT1c, cT1mic, for cT1). Table 1. Distribution of synthetic patients according to OncoDoc Clinical cases. # synthetic patients/clinical cases # clinical cases 1 207 2 274 4 12 5 766 7 74 10 379 14 14 25 132 35 3 4. Discussion and Conclusions We have developed an algorithm that creates synthetic patients to make the clinical cases resolved with one CDSS (OncOdoc) reused to be solved by another CDSS (GL-DSS of DESIREE). We first considered aligning OncoDoc data to OMOP [4], in order to use the common OMOP data model as a transient model, and then develop ETL tools to map concepts from OMOP to the BCKM ontology. However, matching OncoDoc to OMOP was complex because of semantic issues, and we decided to use synthetic patients to cover the missing matches. The lack of clinical data is a common problem in health information technology. It has hindered innovation and raised the barrier of entry into the industry which lags behind other industries involving information technology, data exchange, and interoperability. The main reason comes from data privacy and relies on the problem of re-identification. Approaches and tools have been proposed to generate synthetic data [4] and some tools were validated [5]. To evaluate the GL-DSS of DESIREE, the next step is to enrich the BCKM ontology and add all concepts related to OncoDoc as attributes with their values to be able to run the GL-DSS on all the cohort of synthetic patients, and compare the recommendations produced by the GL-DSS and the decision taken by MTB physicians with OncoDoc. References [1] [2] [3] [4] [5] Bouaud J, Guézennec G, Séroussi B. Combining the Generic Entity-Attribute-Value Model and Terminological Models into a Common Ontology to Enable Data Integration and Decision Support. Stud Health Technol Inform. 2018;247:541-545. Séroussi B, Laouénan C, Gligorov J, Uzan S, Mentré F, Bouaud J. Which breast cancer decisions remain non-compliant with guidelines despite the use of computerised decision support?. Br J Cancer. 2013;109(5):1147-1156. Chakrabarti S, Sen A, Huser V, et al. An Interoperable Similarity-based Cohort Identification Method Using the OMOP Common Data Model version 5.0. J Healthc Inform Res. 2017;1(1):1-18. Walonoski J, Kramer M, Nichols J, et al. Synthea: An approach, method, and software mechanism for generating synthetic patients and the synthetic electronic health care record [published correction appears in J Am Med Inform Assoc. 2018;25(7):921]. J Am Med Inform Assoc. 2018;25(3):230-238 Chen J, Chun D, Patel M, Chiang E, James J. The validity of synthetic clinical data: a validation study of a leading synthetic data generator (Synthea) using clinical quality measures. BMC Med Inform Decis Mak. 2019;19(1):44. doi:10.1186/s12911-019-0793-0 182 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200719 The New Smart-Meds: Redesign of a Gamified App to Improve Medication Adherence Using a Mixed Methods Design Arnaud RICCIa,1, Laetitia GOSETTOb, Katherine BLONDONa,b and Frédéric EHRLERa a University Hospitals of Geneva, Switzerland b University of Geneva, Switzerland Abstract. SMART-MEDS is a gamification-based mobile application to improve medication adherence. In its first version, it relied on storytelling to bolster user engagement. The feedback collected from users after one month testing revealed that although they appreciated the proposed story, they did not find it compelling enough. On the positive side they really appreciated to learn about their medications and disease through a dedicated quiz. In this paper, we present a new version of the app redesigned based on the collected feedback. We have based ourselves on the theories of gamification and self-efficiency to propose new mechanics such as minigames, and interactive dialogues with a chatbot. Everything is wrapped up inside a new story that takes us on a journey through Switzerland. We also tried to reinforce the app educational aspects by integrating documentation directly inside the new mechanics. This new app seems to address all the issues raised during the first user tests, and will be tested in the near future. Keywords. Mobile, application, gamification, health, medication adherence, coronary artery disease, game, elderly people, treatment 1. Introduction SMART-MEDS is a gamified application aimed at improving medication adherence. According to a 2003 WHO report, 50% of people taking medication adhere poorly or not at all [1]. Our target population are individuals with coronary heart disease who require daily medication. This disease mainly affects people over 60 years old [2]. The literature highlights several factors that reduce adherence. These are: lack of confidence in the ability to take treatment over the long term (self-efficacy); lack of knowledge and understanding of the risks of the disease, and the patient's inadequate expectations about the treatment. The mismatch between the expected and the perceived benefits of treatment may also play an important role in medication adherence [1]. As an attempt to act on these factors we developed a first application in 2018 with an editable medication plan, in which medication reporting was encouraged through gamification strategies (Figure 1). Indeed, gamification makes it possible to increase intrinsic motivation and encourage the practice of good behaviors [4]. In the original 1 Arnaud Ricci, University Hospitals of Geneva, Rue Gabrielle-Perret-Gentil 4, 1205 Geneva, Switzerland; E-mail: Arnaud.ricci@hcuge.ch. A. Ricci et al. / The New Smart-Meds: Redesign of a Gamified App 183 concept the mobile application that included a quiz as well as the use of storytelling and the visualization of progression [3]. Figure 1. Home screen, quiz and storytelling of the first smart-meds application In the original concept, a story was the central pillar to engage users. A realistic narrative was created to incite the user to adopt the goals and plans of the story's character, projecting themselves through the various stages of the change process. A new part of story is revealed daily when the patient records her medication intake [3]. In this article, we present the revised design of this gamified app, after taking into account the feedback received about the first version: the new concept integrates new features based on the theories of gamification [4] [5] and self-efficacy [6]. 2. Methodology We combined several inputs form users as well as from the literature in order to guide our design process. We collected the feedback from a group of 18 individuals with coronary heart disease that have tested the original app during a period of 30 days. The participants provided their feedback regarding the main functionalities and its gamification strategies (reported elsewhere, manuscript submitted). Since the objective of the new concept was to increase engagement, we browsed the literature to propose new gamification mechanisms than can support this goal. We also reflected on how to reinforce educational aspects of the app by integrating information directly in the new mechanics. 3. Results We learned from participants’ feedback that they appreciated learning about their disease and medication. They also like the quizzes, and found the story interesting but not compelling enough to drive app use or better medication adherence. 184 A. Ricci et al. / The New Smart-Meds: Redesign of a Gamified App Based on this first input we defined our new gaming application concept around a trip in Switzerland. The user embodies a person who wants to learn more about our country and to adopt a healthier behavior in relation to her disease. During her journey, she will discover several Swiss cities. In these cities, the app offers the possibility of playing mini-games. Our first concept highlighted the attractiveness of the quiz for our target audience. This interest was confirmed by studies showing that older people prefer puzzle-type casual games [7, 8]. Therefore, we complemented the quiz with additional mini-games such as crosswords and word searches. In order to integrate an educational aspect, we designed the mini-games so that their answers were linked to a health thematic. During her journey, she will also meet different people such as a nutritionist or a sports coach who will teach him about her health, her disease, and the treatments. All the new features are identified from theoretical models of behavioral change and gamification theories [4] [5] [7] such as Tondello's Hexad Scale player profile [8], flow theory [9] and Bandura's self-efficacy theory [6] to increase adherence and knowledge about the disease (Figure 2). These features are presented below. Figure 2. Home screen, dialog screen and mini-game screen 3.1. Putting the user at the center of the story The impact of the story on the reader's attitude depends on her involvement within the story [10]. It’s why it seemed essential to change the user's perspective, not only suggesting that she identifies with the main character, but involving him as the main character of the story. The aim is to increase her involvement with the story. In the new concept, the user creates their own avatar, which appears throughout the levels. For example, during the dialogues with the chatbot (see below), the face of the avatar is displayed next to our dialogues. This helps the user to be more involved in the virtual environment [11] and helps increase user identification [6]. A. Ricci et al. / The New Smart-Meds: Redesign of a Gamified App 185 3.2. Adapting to the user’s “player profile” Several scales have been developed to identify the profile of players, to know what their preferences are in terms of games and mechanics. One of these scales is the Hexad Scale developed by Tondello. Hexad Scale developed by Tondello. On this scale which defines 6 user profiles (Disruptor, Philanthropist, Socializer, Player, Free spirit, and Achiever) we kept three of these profiles to build our concept because it is difficult to integrate all the profils [8]. Among the profile defined in the Hexad framework, Achievers want to be competent in whatever they do. They like to complete all the tasks and levels and to take on the difficult challenges [8]. The visualization of progress through levels provides feedback about what she has already achieved [6]. In our new design, the progress is represented through a path in the map of Switzerland that shows the cities as levels. Other Hexad profile such as “Player” concerns users who are highly motivated by extrinsic rewards. They will strive to earn a reward, such as badges or points, regardless of the type of activity. [8] For these users, we devised a badge system, which is rewarded after playing 5 mini-games of a given theme. According to Bandura's theory of selfefficacy [6], in order to encourage the user, we included smaller sub-goals to help achieve the final goal. The badges therefore represent the sub-goals to be achieved. Additionally, these badges provide clear objectives and immediate feedback, which is one of the dimensions for achieving the state of flow [9]. This state of flow is a driver for intrinsic motivation. According to the flow theory, the player's skill must equal the difficulty of the game for the player to be in a state of flow (in a state of concentration that is entirely dedicated to the game). If the game is too simple, the player will see no interest in it, but if the game is too difficult, she will exceed her abilities and will discourage him from continuing to play it. Thus, the difficulty of the games increases every 5 games to adapt to the skills that the player has acquired while playing. 3.3. Improving interactions Chatbots have already proven their effectiveness in several medical fields such as neurological disorders or addictions [12]. In particular, they allow the collection of data on the patient's condition, facilitate the transmission of information, and can coach the patient to change their behaviors by motivating and counselling them [12]. In our concept, the user interacts with a chatbot in several contexts during the tour of Switzerland. The user participates in interactive dialogues with different characters that are used to construct the story during the tour. This is an important element for the screenplay, and allows more scripted transitions between the different stages of the story. Introducing several characters with the chatbot systems also allows the user to learn about their health, their treatments, or their disease. Through these different characters, the chatbot can also motivate the user in response to the answers she provides. The use of various characters also allows us to address some of the Hexad profiles such as the Socializer and Philanthropist profiles, who are driven by interactions and well-intended actions. 3.4. Reward system Rewards are important to motivate users to get involved in a real setting [4]. To encourage users to take their medication, we have devised a reward system where users 186 A. Ricci et al. / The New Smart-Meds: Redesign of a Gamified App receive health points when they document their intake of medication. These health points can be spent in several ways: they can help the user to complete the mini-games by providing clues. The user can only play 2 mini-games per day. Once the 2 daily minigames are finished, she will have to spend 1 health point to start a new mini-game. If the user fails at a mini-game, a health point can allow him to restart the mini-game, otherwise she will have to wait until the next day. Health points can also help the user advance faster in the story. Moreover, this health point reward system can help motivate those who appreciate to get rewards and collect points (i.e., Hexad Players and Achievers). [8] 4. Conclusion As demonstrated in numerous studies, gamification has a real potential to drive behavioural changes. However, defining a suitable concept is not simple. It is necessary to design solutions adapted to the preferences of users, which are heterogeneous. In this article, we propose a new concept supported by the main theories of gamification and behavioural change. The gamified concept we propose needs to be evaluated with the target audience, in terms of engagement and in improving medication adherence. Therefore, the next step will be to test this new design on the target users to see if it meets their needs. In the future we would like to generalize this gamified concept to users with other diseases; we also aim to target other age groups. Once again, further adaptations and user tests will be needed. References Schneider MP, Herzig L, Hugentobler and others. Adhésion thérapeutique du patient chronique : des concepts à la prise en charge ambulatoire. Rev Med Suisse. 2013; 9: 1032-1036. [2] Mozaffarian D, Benjamin EJ, Go AS, and others. Executive summary: heart disease and stroke statistics - 2015 update: a report from the American Heart Association. Circulation. 2015; 131(4): 434-441. [3] Ehrler F, Gschwind L, Meyer P, and others. SMART-MEDS: Development of a Medication Adherence App for Acute Coronary Syndrome Patients based on a Gamified Behaviour Change Model.Proceedings of the AMIA Annual Symposium; San Francisco; 2018. 413 p. [4] Nicholson S. A recipe for meaningful gamification. In Gamification in education and business. Springer, Cham, 2015. Chapter 1; p. 1-20. [5] Johnson D, Deterding S, Kuhn K. A and others. Gamification for health and wellbeing: A systematic review of the literature. Internet interventions. 2016; 6: 89-106. [6] Bandura A. Self-Efficacy The Exercise of Control. New York: W. H. Freeman; 1997. [7] Abraham, O., Thakur, T., & Brown, R. Developing a Theory-Driven Serious Game to Promote Prescription Opioid Safety Among Adolescents: Mixed Methods Study. JMIR Serious Games. 2020; 8(3), 1-13. [8] Tondello GF, Wehbe RR, Diamond L, and others. The gamification user types hexad scale. Proceedings of the 2016 annual symposium on computer-human interaction in play. 2016 Oct; Austin, TX. 229-243 p. [9] Csikszentmihalyi M. The Evolving Self: A Psychology for the Third Millennium. New York: HarperCollins; 1993. [10] Dal Cin S, Zanna MP, Fong GT. Narrative persuasion and overcoming resistance. Resistance and persuasion. 2004; 2, 175-191. [11] Lim S, Reeves B. Computer agents versus avatars: Responses to interactive game characters controlled by a computer or other player. International Journal of Human-Computer Studies. 2010; 68(1-2): 57-68. [12] Pereira J, Díaz Ó. Using health chatbots for behavior change: a mapping study. Journal of medical systems. 2019; 43(135), 1-13. [1] Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200720 187 New Scopes for Practice - Interdisciplinary Webinars for Emergency Medicine and Biomedical Informatics - Health Informatics Kaija SARANTOa,b,1, Catherine CHRONAKIa,c, Luis GARCIA-CASTRILLO RIESGOd, Louise B. PAPE-HAUGAARD a,e and John MANTASa,f a European Federation of Medical Informatics, Switzerland b Department of Health and Social Management, University of Eastern Finland, Finland c HL7 Europe, Brussels, Belgium d European Society for Emergency Medicine, Belgium e Department of Health Science and Technology, Aalborg University, Denmark f National and Kapodistrian University of Athens, Greece Abstract. This paper presents the early outcomes of the educational cooperation between two European academic associations, namely the European Federation of Medical Informatics (EFMI) and European Society of Emergency Medicine (EUSEM). Two webinars were organized in December 2019 and June 2020 to explore areas where mutual education would be beneficial for interdisciplinary cooperation to advance the digitization of emergency departments for the benefit of patients, health professionals and the health system as a whole. Preliminary findings from the analysis of these two webinars are presented and the steps for further cooperation are outlined. Keywords. education, webinars, evaluation, interdisciplinary cooperation, competencies 1. Introduction EUSEM (European Society for Emergency Medicine) and EFMI (European Federation Medical Informatics), signed an agreement to collaborate on research and education to advance the digital transformation of emergency departments throughout Europe. [1-2] As a part of the agreement members of EFMI Working Group of Education and EUSEM Research Committee decided to design educational offerings in the form of virtual seminars (i.e. webinars) on topics advancing the digital transformation of emergency departments through the paradigm of biomedical and health informatics science and technology. Interdisciplinary collaboration is a concept used to convey when two or more scientific fields integrate methods, knowledge and skills, theories, perspectives, and 1 Corresponding kaija.saranto@uef.fi author, Dr. Kaija Saranto, P.O. Box 1627, 70211 Kuopio, Finland, 188 K. Saranto et al. / New Scopes for Practice different disciplinary knowledge bodies, to realise innovative solutions and new insights in areas where new scopes or practices are needed. [3-5] Professional competencies are regulated based on national legislation and European level agreements. Literature also highlights the importance of continuous personal development in an expert area to release pressures in professional competencies. [6] Implementation of electronic health records is a key area of education needs. However, after implementation a lot of updates and extra tools need adoption to mainstream their use in daily practice. [5] Digitization of continuing education offers new ways of organizing education sessions. This trend has been further accelerated by the recent COVID-19 pandemic. Health care professionals working in hectic environments may have difficulties to participate in-service training that often is restricted to most important areas related to work practices. [7] Virtual seminars or Webinars are an effective way to organize education for a wide audience and maintain recordings for offline viewing. EFMI and EUSEM cooperated to advance the digital transformation of Emergency Departments, an area where the need for education, skills development and capacity building is recognised. The following research questions are addressed in this paper based on our experiences from the two interdisciplinary webinars co-organized by EFMI and EUSEM: (1) How do participants from two scientific fields assess their experiences from the joint webinars? (2) What are the areas for development for future webinars? (3) What are the opportunities for long term cooperation of EUSEM and EFMI? 2. Methodology In this section, the process of cooperation in organising the EUSEM-EFMI webinars is described with aims, structure, and content including an assessment plan. The committee defined aims of the webinars and more detailed objectives for the two webinars in the joint meeting at the EUSEM conference in autumn 2019. Under the leadership of President Luis Garcia-Castrillo Riesgo from EUSEM and Vice President Catherine Chronaki from EFMI two webinars were agreed to be organized jointly under the auspices of the Education WG of EFMI and the Research committee of EUSEM. The EUSEM Academy and the EFMI website were to host the recorded webinars for those unable to participate when the webinars are originally broadcasted. The aims of both webinars were set based on the title, “Structured data, Big data, Health analytics, and Clinical decision support”. A guiding question to focus on the content and stimulate the interest of the audience was added: “How does it change emergency medicine?” The specific aims of the first webinar were to help the participants: x to identify the special requirements of emergency care in terms of providing urgent medical care to patients x to identify the tools and knowledge needed for decision making in the demanding environment of the Emergency Departments (ED) x to familiarize with current evidence on structuring electronic health records and understand the purpose and role of clinicians in structured documentation development K. Saranto et al. / New Scopes for Practice 189 to demonstrate that it is possible to implement and successfully adopt centralized integrated and shared electronic prescription services for health care on national level based on the Finish experience. The aims for the second webinar were to help the participants: x to define the key concepts: Big Data, Artificial Intelligence, and eHealth in terms of achievements, opportunities, and benefits of their use in healthcare x to present how current use of data can be prepared and used for data analytics and AI in emergency care/emergency medicine x to understand the sources and management of diagnostic error in the ED x to evaluate the potential of computerized diagnostic decision support (CDDS) in the ED, while understanding the limitations of current CDDS. The speakers for the webinars were selected among the experts from both organisations. Each of them represented both national and clinical expertise in the field of health and medical informatics or emergency medicine. The length of the webinars were 60 minutes each and their structure was as follows: introduction, presentations, Questions and Answers (Q&A), and discussions. Polls were used to activate the participants and a moderator was monitoring the Q&A option. Chat and microphone were disabled for the viewers during the presentation. An assessment tool was designed to collect feedback from the participants. The tool had questions about the background of participants, about the usefulness and satisfaction of the webinar content (Likert scale from 1 to 5), and the intent of participants to participate in future webinars on specific topics of joint interest. The second webinar also gathered information about the pandemic situation in terms of their intention to join webinars or other online educational events more often. The assessment tool was available after the webinars. x 3. Results 3.1. Participant experiences The number of unique on-line viewers was 45 in the first and 74 in the second webinar. The participants represented both scientific fields equally in the first webinar. They were experienced viewers in 40% (first webinar = FW) and 26% (second webinar = SW) having attended webinars more than 6 times already. For more than 30% (FW) and 46% (SW) of viewers this was their first webinar. In terms of their intention to join webinars or other online educational events more often due to pandemic situation all viewers agreed. Professional interest was the main reason to participate in the webinars. The next most reported reason ‘because informatics is a hot topic within my institute’. Viewers also argued their participation ‘because I feel I need to develop myself on this topic or out of personal interest’. The attendees represented Clinical Emergency Medicine and Medical Informatics equally (44%) in the FW. Most of the attendees (73%) in the SW came from Clinical Emergency Medicine and 13 percent of viewers were from Biomedical and Health Informatics (BMHI). They regarded that each speaker gave high quality information, the presentations were useful (mean 4) and the level of information was adequate (mean 4). The viewers were overall satisfied of the webinar (mean 4). The number of speakers was assessed as proper. 190 K. Saranto et al. / New Scopes for Practice 3.2. Future development The viewers expressed their interest to participate in the next webinars: 80 % (first webinar = FW) and 90% (second webinar = SW). Regarding future topics the viewers would like to have are presented in the figure 1. Figure 1. The topics of interest among the webinar viewers 4. Discussion and Conclusion Interdisciplinary education is challenging not only due to difficulty to choose content of high relevance for specific audience, but also due to new insights in areas where new scopes or practices are emerging [4]. BMHI is not an unknown discipline for professionals in Emergency Medicine or vice versa. However, each discipline has its own concepts terms, and theories, which need to be introduced to establish baseline understanding and be able to elicit rewarding outcomes in continuing education [3, 5]. Based on our experience, the interdisciplinary cooperation worked extremely well. The speakers were invited based on carefully selected topics and they all received excellent K. Saranto et al. / New Scopes for Practice 191 feedback from the viewers. Online seminars or webinars are demanding to plan in terms of content, structure, and cooperation with the viewers, partly because it is difficult to maintain the attention of viewers for a long time [7]. We used polls to activate them and stimulate their interest. The speakers answered one or two questions, a number typical for conference venues. The viewers represented equally both scientific fields in the first webinars but in the second most of the viewers were from emergency medicine. They were also less experienced viewers than those presented in the first webinar. The topics suggested for future webinars indicate the appetite of emergency medicine professionals to gain knowledge about recent trends in the digitization of emergency medicine and its impact in their work. The number of registrations for the webinars was almost double for both webinars compared to those who actually participated. However, the number of viewers on the on-line webinars doubled in the second webinar. This is probably the case because the professionals may have other commitments late afternoon which was the on-line time for the webinars. The webinars are available on the EUSEM Academy site. For the time being (July 2020) the first webinar was viewed 99 times, and the second one 53 times on the EUSEM Academy site. The webinars were the first actions in the joint agreement between EFMI and EUSEM. The success of these first webinars are encouraging and EUSEM and EFMI plan to continue their cooperation for the years to come. Acknowledgements The authors want to thank the distinguished speakers of the two webinars and Mrs. Willemijn van Hees, Project Manager at EUSEM for her extensive support in preparing and launching the webinars. References [1] [2] [3] [4] [5] [6] [7] European Federation of Medical Informatics (EFMI). https://efmi.org/ Accessed July 2nd, 2020 European Society of Emergency Medicine (EUSEM). https://eusem.org/ Accessed July 2nd, 2020 Menken S, Keestra M. An introduction to interdisciplinary research: theory and practice. Amsterdam: Amsterdam University Press, 2016. Klaassen RG. Interdisciplinary education: a case study. European Journal of Engineering Education. 2018; 43(6): 842–859. https://doi.org/10.1080/03043797.2018.1442417 Patel V, Yoskowitz NA, Arocha JE, Shortliffe EH. Cognitive and learning sciences in biomedical and health instructional design: A review with lessons for biomedical informatics education. Journal of Biomedical Informatics. 2009; 42(1): 176-197. Mantas J, Ammenwerth E, Demiris G, Hasman A, Haux R, Hersh W, Hovenga E, Lun KC, Marin H, Martin-Sanchez F, Wright G. IMIA Recommendations on Education Task Force. Recommendations of the International Medical Informatics Association (IMIA) on Education in Biomedical and Health Informatics. First Rev. Methods Inf Med. 2010; 49(2): 105–120. Nadama1 HH, Tennyson M, Khajuria A. Evaluating the usefulness and utility of a webinar as a platform to educate students on a UK clinical academic programme. J R Coll Physicians Ed 2019; 49: 317–22. 192 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200721 Description of Data Breaches Notifications in France and Lessons Learned for the Healthcare Stakeholders Marie SIMONa and Vincent LOOTENb,1 Université Paris-Est Créteil, Créteil, France b UMRS 1138, Centre de Recherche des Cordeliers, Université de Paris, Paris, France a Abstract. Although the consequences of the General Data Protection Regulation (GDPR) have been widely discussed, the violations have not been described in medical literature. In this study, we focus our analyses on the data breach notifications, in France, defined in the article 4 of GDPR as “a breach of security resulting, accidentally or unlawfully, in the destruction, loss, alteration, unauthorized disclosure of personal data transmitted, stored or otherwise processed, or unauthorized access to such data.” Among 3,824 data breach notifications reported between May 2018 and February 2020, 244 (6.4%) is related to the health sector. Loss of confidentiality is the most important breach (80.7%) in this sector, followed by the loss of availability (27.5%). Malicious cause occurred in 58.2% of them. We hypothesized a phenomenon of underreported data breach incidents in health due to a mismatch between cybersecurity and data privacy issues. Keywords. Policy, Data Privacy, Cybersecurity 1. Introduction In 2017, the WannaCry cyberthreat affected more than 600 organizations as the National Health Service (NHS) in England; in 2018 the Singapore Health System reported a major breach of over one million of patient records: cybersecurity attacks are a growing threat to healthcare. Included in the General Data Protection Regulation (GDPR), cybersecurity in health is a major issue for the next decade. Although the consequences of GDPR have been widely discussed, the violations have not been described in medical literature. Since May 2018, the GDPR provides the mandatory legal framework for all data processing including European citizens’ personal data[1]. National authorities across the European Union can sanction any company or administration performing non-conform data processing regarding to the GDPR. From the researcher's perspective, Peloquin et al.[2] exposed some technical challenges for data reuse: the anonymization or the pseudonymization of personal data, the management of consent, the cross-border transfers of personal data and the right limitations in the research context. Furthermore, Bernd Blobel and Pekka Ruotsalainen[3] proposed a model to implement data governance and data access management into a medical information systems. However, a description of the GDPR violations recorded by the national authorities in Europe could 1 Corresponding Author, Dr Vincent Looten, Université de Paris, Paris, France; E-mail: lootenv@gmail.com M. Simon and V. Looten / Description of Data Breaches Notifications in France 193 provide essential information about the legal practice of this regulation and the impact on its implementation. In this study, we focus our analyses on the data breach notifications defined in the article 4 of GDPR as “a breach of security resulting, accidentally or unlawfully, in the destruction, loss, alteration, unauthorized disclosure of personal data transmitted, stored or otherwise processed, or unauthorized access to such data”. The aim is to describe data breach notifications in France. 2. Methods Definitions. The French national authority for data privacy is the CNIL (“Commission nationale de l’informatique et des libertés” in French). The GDPR have made mandatory to notify the CNIL of any personal data breach that poses a risk to the rights and freedoms of personal data. This notification to the CNIL must be made within 72 hours, by the responsible for processing or by its representative. Data sources. We extracted data breach notifications reported to the CNIL from May 2018 to February 2020. The code and the data used in this study are available at www.github.com/vlooten/databreach, while more recent data can be downloaded from the open data governmental website (www.data.gouv.fr). Outcomes. Three types of violation were described: the loss of confidentiality, the loss of integrity and the loss of availability. These categories are not exclusive. We described the number of people impacted by the breaches according to the same categories proposed in the original dataset. We described the cause of the breach (accidental, malicious or unknown) and the origin (Internal, external or unknown). Data breaches included individual identifiers are at higher risk regarding GDPR regulation. Thereby, we performed a focus in the health notification to compare data breaches included or not the national identification number (“numéro d’inscription au répertoire national des personnes physiques“ or NIR in French), which is a permanent identifier throughout the individual’s lifetime. Statistical analyses. Data were expressed as numbers (%). Chi2 tests (for categorical data) was used to compare groups. All tests involved use of R 3.6.1(R Foundation, Vienna, Austria). 3. Results Among 3,824 data breach notifications reported between May 2018 and February 2020, 675 (17.7%) are related to the administration, 660 (17.3%) to science and education activities, 485 (12.7%) to the financial and insurance activities, 326 (8.5%) to the Information and communication sectors and 244 (6.4%) to the Health sector. Table 1 presents a description of the whole dataset and a comparison between Health sector and the other activities. Among the 503 notifications included the national identification number (NIR), 121 (24.1%) are related to the administration, 112 (22.3%) to the science and education, 87 (17.3%) to the commercial and industrial sectors, 71 (14.1%) to the financial and insurance activities, 36 (7.2%) to the health sector 28 (5.6%) to the information and communication sectors, and 48 (9.5%) to other sectors. Table 2 proposed a description of the data breach notification for the health sector and a comparison between notification included NIR and the others. 194 M. Simon and V. Looten / Description of Data Breaches Notifications in France Table 1. Description of the data breach notification and comparison between Health sector and the other sectors All notifications (N=3824) Year of accident 2018 2019 2020 Type of violation Loss of confidentiality Loss of integrity Loss of availability Number of people impacted <=5 [6-50] [51-300] [301-5000] >=5000 Cause of accident Accidental Malicious Unknown Origin of accident Internal External Unknown Health sector (N=244) Other activity (N=3580) p value 1170 (30.6%) 2287 (59.8%) 367 (9.6%) 41 (16.8%) 174 (71.3%) 29 (11.9%) 1129 (31.5%) 2113 (59.0%) 338 (9.44%) <0.001 3450 (90.2%) 197 (80.7%) 3253 (90.9%) <0.001 406 (10.6%) 659 (17.2%) 27 (11.1%) 67 (27.5%) 379 (10.6%) 592 (16.5%) 0.898 <0.001 0.009 919 (24.0%) 652 (17.1%) 746 (19.5%) 1010 (26.4%) 497 (13.0%) 69 (28.3%) 50 (20.5%) 56 (23.0%) 45 (18.4%) 24 (9.84%) 850 (23.7%) 602 (16.8%) 690 (19.3%) 965 (27.0%) 473 (13.2%) 0.516 1151 (30.1%) 2138 (55.9%) 535 (14.0%) 62 (25.4%) 142 (58.2%) 40 (16.4%) 1089 (30.4%) 1996 (55.8%) 495 (13.8%) 0.200 1060 (27.7%) 2229 (58.3%) 535 (14.0%) 64 (26.2%) 140 (57.4%) 40 (16.4%) 996 (27.8%) 2089 (58.4%) 495 (13.8%) Table 2. Comparison between data breach notifications with NIR and without NIR in the health sector Health sector (N=244) Year of accident 2018 2019 2020 Type of violation Loss of confidentiality Loss of integrity Loss of availability Number of people impacted <=5 [6-50] [51-300] [301-5000] >=5000 Included NIR (N=36) Without NIR (N=208) p value 41 (16.8%) 174 (71.3%) 29 (11.9%) 5 (13.9%) 26 (72.2%) 5 (13.9%) 36 (17.3%) 148 (71.2%) 24 (11.5%) 0.809 197 (80.7%) 29 (80.6%) 168 (80.8%) 1.000 27 (11.1%) 67 (27.5%) 7 (19.4%) 13 (36.1%) 20 (9.62%) 54 (26.0%) 0.090 0.290 69 (28.3%) 50 (20.5%) 56 (23.0%) 45 (18.4%) 24 (9.84%) 4 (11.1%) 3 (8.33%) 14 (38.9%) 10 (27.8%) 5 (13.9%) 65 (31.2%) 47 (22.6%) 42 (20.2%) 35 (16.8%) 19 (9.13%) 0.003 M. Simon and V. Looten / Description of Data Breaches Notifications in France Cause of accident Accidental Malicious Unknown Origin of accident Internal External Unknown 4. 62 (25.4%) 142 (58.2%) 40 (16.4%) 10 (27.8%) 24 (66.7%) 2 (5.56%) 52 (25.0%) 118 (56.7%) 38 (18.3%) 0.162 64 (26.2%) 140 (57.4%) 40 (16.4%) 9 (25.0%) 25 (69.4%) 2 (5.56%) 55 (26.4%) 115 (55.3%) 38 (18.3%) 0.127 195 Discussion Main results. Among 3,824 data breach notifications reported between May 2018 and February 2020, 244 (6.4%) is related to the health sector, increasing by a factor four between 2018 and 2019. Data breach characteristics of the health sector were similar to data breach characteristics of the other sectors. Loss of confidentiality is the most important breach (80.7%) in health sector, followed by the loss of availability (27.5%), some data breaches are mixed. 175 (71.7%) notifications reported fewer than 300 people impacted. Malicious cause occurred in 58.2% of them, accidental cause accounted for 25%. Technical significance. Firstly, we didn’t find important differences between data breach notifications in health and the other sectors but may lead to higher threat for citizens regarding to international experience [4]. Secondly, the French ministry of health and the French digital health agency have reported 327 incidents in 2018 and 392 in 2019, included respectively 276 and 333 hospitals. The rates of malicious incidents were 41% in 2018 and 43% in 2019[5]. Authorities hypothesized a phenomenon of underreported incidents: “The total number of reports is still low compared to the number of structures concerned by the reporting obligation (more than 3,000) and the probability that at least half of the structures concerned have had to deal with an incident that has impacted its normal operation during the year.” Our results are similar with a lower amount of data breaches notification. Worldwide, healthcare lead in number of incidents (27%), as described in 2018 by the European Union Agency for Network and Information Security report [6], which is much more than the 6,4% notified in France based on our results. Thirdly, regarding the increase of data reuse for research purposes in France [7], the data processing included national identification numbers are regulated by the French law [8]. Nevertheless, only 36 (14.8%) notifications in the health sector included NIR, with 4 (11.1%) data breaches impacted 5 people or lower, which is nonrealistic. Therefore, we hypothesized a phenomenon of underreported data breach incidents due to a mismatch between cybersecurity and data privacy issues. This underreported is likely a violation of GDPR. We could explain this underreporting by the distinction between data privacy and cybersecurity in the hospitals’ organization in France. Data privacy is managed by the chief information officers with the data protection officer as advisors; they are focus on users’ community and data processing purposes. Cybersecurity is leads by the chief information security officers focus on the data infrastructure integrity. We hypothesized that all data breaches cannot be detected by the chief information security officers (e.g. breaches with 5 people or less or internal breaches). Perspectives and recommendations. Dean F. Sittig and Hardeep Singh[9] proposed a four steps socio-technical approach that organizations can undertake to secure an 196 M. Simon and V. Looten / Description of Data Breaches Notifications in France electronic health record system: (1) To ensure adequate system protection by correctly installing and configuring computers and networks (2) To ensure more reliable system defense by implementing user focused strategies (3) To ensure a comprehensive system monitoring of suspicious activities, and (4) To respond, to recover, to investigate, and to learn from ransomware attacks. For practical implementation, we recommend: (1) to plan seasonal assessments of information security management systems and to try to meet the international standards for information security with long-term and comprehensive perspectives as recommended by W.-S. Park at al[10], (2) to reduce the end point complexity (due to a technology saturated environment) and improving internal stakeholder alignment as recommended by M.S. Jalali, and J.P. Kaiser[11]. Finally, to improve the completeness of data breaches notification database, an electronic declaration system could be proposed to all users of the information system included physicians and patients. Conclusion. We highlight a phenomenon of underreported data breach incidents in health possibly due to a mismatch between cybersecurity and data privacy issues. References Demotes-Mainard J, Cornu C, Guérin A, et al. How the new European data protection regulation affects clinical research and recommendations? Therapies. 2019; 74: 31–42. doi:10.1016/j.therap.2018.12.004. [2] Peloquin D, DiMaio M, Bierer B, and Barnes M. Disruptive and avoidable: GDPR challenges to secondary research uses of data. Eur. J. Hum. Genet. 2020. doi:10.1038/s41431-020-0596-x. [3] Blobel B and Ruotsalainen P. How Does GDPR Support Healthcare Transformation to 5P Medicine?, Stud. Health Technol. Inform. 2019; 264: 1135–1139. doi:10.3233/SHTI190403. [4] Ghafur S, Kristensen S, Honeyford K, Martin G, Darzi A, and Aylin P. A retrospective impact analysis of the WannaCry cyberattack on the NHS. Npj Digit. Med. 2019; 2: 98. doi:10.1038/s41746-019-01616. [5] Observatoire des signalements d’incidents de sécurité des systèmes d’information pour le secteur santé. 2019. Ministère des solidarités et de la santé et Agence du Numérique en Santé. esante.gouv.fr/sites/default/files/media_entity/documents/ans_acss_rapport_public_observatoire_signal ements_issis_2019_v0.10.pdf. [6] ENISA Threat Landscape Report 2018 15 Top Cyberthreats and Trends. European Union Agency for Network and Information Security. doi:10.2824/622757 [7] Looten V and Simon M. Impact Analysis of the Policy for Access of Administrative Data in France: A Before-After Study. Stud. Health Technol. Inform. 2020; 270: 1133–1137. doi:10.3233/SHTI200339. [8] Tout savoir sur le décret « cadre NIR » dans le champ de la protection sociale. Commission Nationale de l’Informatique et des Libertés, (2020). https://www.cnil.fr/fr/tout-savoir-sur-le-decret-cadre-nir-dans-lechamp-de-la-protection-sociale. [9] Sittig D and Singh H. A Socio-technical Approach to Preventing, Mitigating, and Recovering from Ransomware Attacks. Appl. Clin. Inform. 2016; 7: 624–632. doi:10.4338/ACI-2016-04-SOA-0064. [10] Park W-S, Seo S-W, Son S-S, Lee M-J, Kim S-H, Choi E-M, Bang J-E, Kim Y-E, and Kim O-N. Analysis of Information Security Management Systems at 5 Domestic Hospitals with More than 500 Beds. Healthc. Inform. Res. 2010; 16: 89. doi:10.4258/hir.2010.16.2.89. [11] Jalali MS and Kaiser JP. Cybersecurity in Hospitals: A Systematic, Organizational Perspective. J. Med. Internet Res. 2018; 20: e10059. doi:10.2196/10059. [1] Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200722 197 User-Centred Design with a Remote Approach: Experiences from the Chronic Pain Project Berglind F. SMARADOTTIRa,b,1, Johan Gustav BELLIKA a, Aina FREDENG a and Asbjørn J. FAGERLUNDa a Norwegian Centre for E-health Research, University Hospital of North Norway, Tromsø, Norway b Department of Information and Communication Technology, University of Agder, Grimstad, Norway Abstract. User-centred design involves end-users or user groups during all the parts of the development process. The research project Chronic Pain aims to develop a shared decision making application for patients and physicians, addressing individually adapted pain treatment. The project employs a user-centred design process, and in middle of it, Covid-19 pandemic social distancing restrictions were imposed. This paper presents how the user-centred design process together with a patient organisation was transformed to a digital approach and the experiences from performing a remote co-creation user workshop. The digital approximation had a satisfactory result and the main contribution lies in the sharing of recommendations for how to practically apply a remote user-centred design methodology. Keywords. User-centred design, video conference, chronic pain management, decision support, digital services 1. Introduction A user-centred design process means involvement of groups of users throughout the entire development cycle [1]. The tasks of the users are to contribute with descriptions of the context of use, elicitation of user needs and being test participants in user tests [2]. These are all contributions for designing and building health information technology through iterations. Workshops are a common way for the collection of user needs and context of use and where potential end-users, often recruited from patient organisations, are gathered together for a half or whole day [3]. A first workshop aims to familiarize with the goal of the development, the other participants, the development- and research team and the commitment of the participation in the user-centred design process. Further, in such a workshop the users are asked to, based on their own experiences, describe context of use, how the use of the technology could support their daily life and in what way to interact with it. The following workshops work as feedback sessions for conceptual design, wireframes or prototypes of technology. During iterative 1 Corresponding author, Department of Information and Communication Technology, Faculty of Engineering and Science, University of Agder, Jon Lilletuns vei 9, N-4879 Grimstad, Norway, E-mail: berglind.smaradottir@uia.no. 198 B.F. Smaradottir et al. / User-Centred Design with a Remote Approach development, individual user tests or paired testing are made to frequently evaluate the technology [4]. In the research project Chronic Pain -Decision support for personalised chronic pain care (2019-2021) user-centred design has been applied from the early project idea and is planned for the entire development process of a shared decision making application [5]. The application will provide patients and physicians with relevant and valid decision alternatives, also presenting realistic probabilities for outcomes, side effects and adverse events. Another requirement for the project is to address how to collect and share patientreported outcomes (PROMs) and experiences using a mobile application that utilises distributed data storage. The user-centred design procedure is made in collaboration with the Norwegian Fibromyalgia Association [6], and one user meeting and two user cocreation workshops to elicit context of use and user needs have previously been organised [7]. Four workshops are planned organised for each year, approximately every third month. However, in the middle of the user-centred design process, the Covid-19 pandemic escalated during the spring of 2020 and social distancing restrictions were imposed. All physical meetings were discouraged and cancelled, and the research team involved was obliged to home officing. This would imply a delay for the project, with negative effects on the progress and goals. However, the end-users from the patient organisation was an engaged, enthusiastic and active group, and the research team decided to continue the user-centred design process remotely. This paper reports from how the user-centred design process was transformed from physical to digital meetings, and shares the experiences from a remote co-creation user workshop. The research questions (RQs) stated were: RQ1: How can a user-centred design process be performed with a remote digital approach? RQ2: What are the benefits and constraints of performing a remote digital approach in a user-centred design process? 2. Methodology Qualitative methods were used in the Chronic Pain project to analyse the user-centred design process. The project has five steps: 1) user-centred design of the PROMs data collection tools, 2) design data storage and computation environment, 3) technical development of PROMs application, 4) building data storage and computation environment and 5) user evaluations. The project is in the middle of the first step, and earlier two in-house user workshops have been organised, with a duration of 4 hours including a lunch break. A third workshop was planned in June 2020. Due to the Covid19 pandemic restrictions, this workshop was converted to a digital event to remotely gather the project participants. Six participants from the Norwegian Fibromyalgia Association that had attended the previous physical workshops were invited and all accepted to join for the digital event. The workshop was hosted on the browser-based video conferencing platform Whereby (Video Communication Services AS, Maaloy, Norway). The workshop was scheduled with a duration of 2 hours, including a short break. The workshop addressed the topics: storage of research data, demonstration of a web-application and mobile system for pain registration and a session for user feedback. The feedback session targeted the first time user experience regarding the log-in procedure and the interactions for registering pain using a numeric rating scale (NRS). B.F. Smaradottir et al. / User-Centred Design with a Remote Approach 199 The six participants were all female pain patients, with an average age of 54 years and the average of 27.8 years since onset of pain and 15.1 years since diagnosed with fibromyalgia. On a scale from 0-10, they self-evaluated their technology skills at 6.0 and interest of technology at 6.7. The research team consisted of people with expertise in human factors in design, psychology, statistics, health science and medical informatics. After the remote workshop, there was a 30 minutes long debriefing session for the research team to summarise the outcome and the experiences. The data collection consisted of audio recordings from the workshop and annotations from the workshop and the debriefing session. The recording was made with an Olympus VN-3200PC audio recorder, physically located at the workshop host. Screen recording was not made due to privacy regulations for data storage. The Privacy Officer at the University Hospital of North Norway approved the study with project number 02147. The participation in the study was voluntary and all informants signed a consent form at the project start. Additionally, a specific consent for audio recording was obtained prior to the remote workshop. 3. Results The results target the 1) technical and practical issues and 2) experiences of the research team from performing the remote user workshop. The overall experience was that the execution of the digital workshop worked in a satisfactory way. There was an active dialogue throughout the entire session. It might have had a positive impact that the participants had met before and were familiar with each other, the project and the research team. A few days before the workshop user training on Whereby was conducted with each participant individually, testing how to connect, the sound quality, the mute function and the use of the video-camera, to avoid delaying technical issues at the workshop day. Only minor technical issues were experienced during the digital workshop, mainly regarding the sound quality. A useful feature of the videoconference solution was the simultaneous visualisation of all the participants in the screen view (up to 12 persons) and that provided a good overview for the dialogue and discussion between the participants. Even in presentation mode or the share screen function, the participants were shown with a small picture. Compared to other videoconference solutions that allow only a few participants’ video to be displayed simultaneously, this worked well and made it easier for each participant to interact when seeing not only the speaker or the shared screen, but all the faces simultaneously. All participants were asked to mute their microphone when not speaking to reduce noise, but no specific instructions regarding camera use was given. A couple of participants turned off their video-camera for some time during the session. The active program elements of the workshop were reduced from 2 hours and 30 minutes for in-house event to 1 hour and 45 minutes for the remote program, based on the fact that digital meetings are often shorter than in-house meetings and participants speak less freely. However, we experienced to slightly running over the time of the first remote program block. Another time constraint occurred due to higher than expected degree of active participation from the users in the user feedback session. Despite the delays, a full 15 minutes break was made in between the blocks, to allow participants to completely leave the screen and take some fresh air. At the summary of the workshop it was expressed that this was appreciated and of importance for the participants. 200 B.F. Smaradottir et al. / User-Centred Design with a Remote Approach The organisation and preparation of the workshop were made in digital meetings for the research team in home offices. First meeting was one month before the workshop, to prepare the invitations and the program, that was sent by post mail to the participants together with a printed version of the consent form to be filled in. Later meetings targeted the program content in details. The last meeting was organised the day before the workshop, for last minutes amendments. This was experienced as an efficient way for the preparations, with meetings lasting 30-60 minutes. 4. Discussion and Conclusion The main contribution of this paper lies on how to remotely apply user-centred design methodology with active contribution of end-users. The research questions (RQs) are answered based on the results. RQ1 addressed how to perform remote user-centred design. Traditionally, in-house workshops are used for co-creation purposes and user meetings. Such arrangements can successfully be carried out with a remote digital approach, but a crucial precondition is stable internet-connectivity at home for the participants. The platform has to be carefully chosen regarding user-friendliness and the features of the screen view during the event. A collective or individual session with each participant to test-run the video conference platform, familiarise with the functionality as well as to resolve technical issues should be considered. We recommend all participants, both end-users and researchers, are instructed to mute their microphone when not speaking, but keeping the camera transmitting during the entire session as it might impact negatively on the discussion not knowing if the person is joining actively or doing other things. When entering the videoconference solution, each participant could write their name, to be visualised in the screen view. This is recommended to do for all workshop participants, also the research team, to ease the following of the discussion as all faces might not be familiar to all participants and it is easier to lead the discussion in a structured way by using the first names. Recording is recommended to ease the retrospective analysis, however attention must be paid to the storage of recordings with reference to the European privacy regulations [8]. When organising meetings with chronic pain patients, attention has to be paid on the health and safety of the participants. Physical breaks in the program schedule is such a consideration, also of importance for digital meetings. We recommend at least 10 minutes break per hour, allowing the participants to stretch the legs or leaving meeting for a while. RQ2 asked about benefits and constraints of a digital approach. An apparent benefit of remote workshops is the participation from home, with no applied travelling time or costs. With this reduced time consumption for all parts, this approach might be used to organise user meetings with a more frequent schedule. In terms of iterative development and evaluation, the remote approach might facilitate increased user-involvement in the collection of user requirements and later on in assessment of user interface design, interactions and usability. Moreover, taking into consideration the Covid-19 pandemic, the remote approach contributes to social distancing and thereby less risk of contact spread, particularly relevant when working with users in elevated risk demographic. Regarding constraints, digital meetings offer limited social interaction between the participants, particularly if the group is unfamiliar with each other. People might speak less freely, and for that reason, the moderator(s) must actively lead the discussion by using the first names to facilitate the active voice of each participant. The recommended B.F. Smaradottir et al. / User-Centred Design with a Remote Approach 201 number of end-users is 6-8 persons to endeavour contribution from everyone. Digital events tend to be shorter than in-house meetings. Nevertheless, it is important to allocate enough time particularly for active user sessions. We reduced the active program schedule of the digital meeting compared to an in-house event, with the expected constraints in mind, but experienced to run out of time due to more than expected active participation from the users. Next digital workshop will be extended with another block of time. It is likely that the relative ease to get participants to be active in the workshop can partially be attributed to the fact that they had previously met in physical meetings and were familiar with the research team and the other user group members. This study has some limitations, such as including one single workshop with a limited number of informants. However, the participants meaningfully represented the end-user group and contributed actively with their experiences. In addition, the research team has expertise in the user-centred design domain, and this paper is intended for sharing knowledge and reflections on remote procedures. This remote user-centred design methodology can be recommended for other digital health projects, and particularly usable for patients with rare diseases, as there might be large geographic distances between the participants. Future work of the project is associated with continuation of the remote approach for co-creation user workshops and the preparation and execution of remote user evaluations, with the individual participant performing the test at home with guidance from the research team located in the control room of a usability laboratory. References [1] Ergonomics of human system interaction. ISO 9241-210: 2019. Part 210: Standard for human-centred design for interactive systems. International Standardization Organization (ISO), Switzerland, [2] Gulliksen J, Göransson B, Boivie I, Blomkvist S, Persson J, Cajander Å. Key principles for user-centred systems design. Behav Inf Technol. 2003; 22:6, 397-409. [3] Smaradottir BF. The steps of user-centered design in health information technology development. In: Proceedings of International Conference on Computational Science and Computational Intelligence; 2016 Dec, Las Vegas (NV): IEEE; p. 116-121. [4] Bastien JC. Usability testing: a review of some methodological and technical aspects of the method. Int J Med Inform. 2010; 79(4), e18-e23. [5] Chronic Pain: decision support for personalized chronic pain care. [cited 2020 July 1]. Available from:https://ehealthresearch.no/en/projects/decision-support-for-personalized-chronic-pain-care [6] Norwegian Fibromyalgia Association. [cited 2020 July 1]. Available from: https://fibromyalgi.no/ [7] Smaradottir BF, Fagerlund AJ, Bellika JG. User-centred design of a mobile application for chronic pain management. Stud Health Technol Inform. 2020; 272:272-275. [8] General Data Protection Regulation. [cited 2020 July 1]. Available from: https://ec.europa.eu/info/law/lawtopic/data-protection/data-protection-eu_en 202 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200723 Analysis of ISO/TS 21526 Towards the Extension of a Standardized Query API Hannes ULRICHa,1, Ann-Kristin KOCK-SCHOPPENHAUERa, Cora DRENKHAHNa, Matthias LÖBEb and Josef INGENERFa,c a IT Center for Clinical Research (ITCR-L), University of Lübeck, Germany b Institute for Medical Informatics, Statistics and Epidemiology (IMISE), University of Leipzig, Germany c Institute of Medical Informatics, University of Lübeck, Germany Abstract. Metadata is often used for different tasks in the field of medical informatics: semantic description of data, quality validation, data integration, or information retrieval. Metadata definitions are captured and curated in timeconsuming tasks and stored in metadata repositories that manage and preserve the metadata. Due to technical and legal restrictions, metadata is rarely as easily accessible and interoperable as it is necessary for modern information systems. In a previous study, a uniform interface based on the widely used ISO/IEC 11179 and the Facebook data retrieval language GraphQL was introduced as a solution to these technical obstacles. In the meantime, the ISO standard 21526 has been published, a recent version designed with a strong focus on health informatics. While it is conceptually oriented on the metamodel in ISO 11179, a number of extensions but also restructurings have been introduced. In this study, the authors investigated the difference between ISO 11179 and ISO 21526 and extended the unified metadata query interface to be future-proof and in particular, to support the semantic extensions of ISO 21526. Keywords. Metadata, ISO 11179, ISO 21526, Metadata Repository, GraphQL 1. Introduction Metadata – in our definition machine-readable descriptions of items of data - is increasingly applied in the field of medical informatics and is often used for different tasks, e.g. semantic characterization, quality validation, or data integration. Metadata definitions are captured and curated in time-consuming tasks, involving experts and data stewards to ensure reliability. The information is stored in metadata repositories (MDR) that manage and preserve the metadata. Due to technical and structural obstacles, metadata is rarely interoperable. This hampers aggregation and management of (meta-)data sets in order to answer research questions (1,2). One reason is, that the leading metadata standard ISO 11179 (3) does not constrain implementations, so existing interfaces of MDR systems differ technically. In earlier studies, Ngouongo et al. (4) and Park et al. (5) showed structural problems of ISO/IEC 11179. So, the recently published ISO 21526 standard, successor of ISO 11179 with focus on medical applications, 1 Corresponding Author, Hannes Ulrich, IT Center for Clinical Research, Lübeck; Telephone: +49 (0) 451 3101 5607; Fax: +49 451 3101 5604; E-mail: h.ulrich@uni-luebeck.de H. Ulrich et al. / Analysis of ISO/TS 21526 Towards the Extension of a Standardized Query API 203 introduces new concepts, but also restructures existing concepts and aims to overcome the structural problems. This raises two new research questions: (1) does the restructuring create incompatibility between the two standards and (2) are the extended possibilities offered by the new standard profitable enough to integrate them into existing systems? In this study, the authors investigate the difference between ISO 11179 and its successor ISO 21526. If the comparison shows remarkable enhancements, an extension of our standardized metadata interface will be present as part of this study to support the ISO 21526 and to be adaptable to upcoming systems. 2. Background The ISO/IEC 11179 (3) is a much-used metadata norm in the field of medical informatics. The defined metamodel separates the representation of structural information from the conceptual categorization of metadata. The central information object, called data element, is defined by definitions and value domains that restrict the value represented, and by a link to data element concepts to describe its information in a semanticpreserving manner. Various MDR systems use the standard to constrain and harmonize their information: caDSR (6), METeOR (7), Aristotle (8), and USHIK (9). Since ISO 11179 does not constrain implementation, existing MDR systems differ technically in the provided interfaces. In a prior study, we designed QL4MDR (10) as a new approach to overcome technical heterogeneity of the existing MDRs. Inspired by HL7 FHIR and its uniform interface concept (11), QL4MDR and the underlying schema is an interface definition that constrains the exchanged data based on the ISO 11179-3 metadata model. Figure 1. The metadata standards ISO 11179 and 19763 are combined in the new ISO 21526 to define metadata repository requirements. Additionally, a mapping package is introduced (12). The successive ISO/TS 21526 Health informatics — Metadata repository requirements (MetaRep) is designed to be an extension and a clarification of 11179 (12). It combines the well-known metadata metamodel of 11179-3 with the metadata standard ISO 19763 and aims to simplify the definition of metadata despite its structural complexity, as shown in Figure 1. It is focused on capturing the interrelation between data models, which are used to exchange information in healthcare. The storage of these interrelations and their contextual information are necessary for the later interpretation and (re)use of the exchanged data. 204 H. Ulrich et al. / Analysis of ISO/TS 21526 Towards the Extension of a Standardized Query API The authors will examine both standards systematically focusing on the research questions mentioned before. The analysis checks the packages of both standards against each other to identify effective modifications. Newly introduced packages and classes will be examined towards their changes regarding their impact on the limitations (4,5). 3. Results 3.1. Comparison of ISO 11179 and ISO 21526 In ISO 21526, various previous packages are aggregated to refocus on metadata definition resulting in size-wise reduction regarding numbers of defined packages and classes. The data description package of 11179-3 is still the core model of the new standard. But the successor introduced a third conceptual definition axis between Conceptual Domain and Value Domain to link external concept systems. ISO 21526 favors HL7 FHIR CodeSystems (13), including LOINC and SNOMED CT, to be used in new Conceptual Domain Definitions. The concept package is simplified class-wise and modeled according to the Simple Knowledge Organization System (SKOS) (14) to make implementation easier. As a novelty, ISO 21526 introduces a mapping model to provide a uniform way to describe mappings between (artifacts of) data elements. A mapping is defined as an association between two different items, characterized by a type with a value set of elements like broader, narrow, related, same_as and derived_from. 3.2. Expansion of QL4MDR The schema below was derived from ISO 21526 to match the newly introduced mapping classes and furthermore the classes of concept package were included to enable enhanced querying. As shown in Figure 2, the first schema contained six (plus seven supportive) objects and was expanded by five additional classes: Concept, SemanticRelation, Map, MDRMapping and Conceptual Domain Definition. Concept and MDRMapping are introduced as new entry points to start a query at these objects. Figure 2. The newly added objects are shown in the dashed box. Besides the existing three entry points Conceptual Domain, Namespace, and Data Element, two new were added: MDRMapping and Concept. H. Ulrich et al. / Analysis of ISO/TS 21526 Towards the Extension of a Standardized Query API 205 4. Discussion ISO 21526 is a constructive extension to the commonly used ISO 11179 but also inherits some problems. The third conceptual definition axis is a beneficial addition to the standard. It enables the direct usage of ontological knowledge as a conceptual domain as well as the usage of predefined and externally managed value sets for the value domain. The emerging adoption of FHIR and thereby provided machine-readable concept systems (in FHIR code systems and value sets) are beneficial for metadata repositories. In earlier studies, Ngouongo et al. (4) and Park et al. (5) described and categorized the problems of ISO/IEC 11179: the absence of semantic or syntactic linkage of shared concepts between components (15) and the missing support of structure for either metadata extension mechanism (2) or usage model (16). Remodeling the concept package towards SKOS and the introduction of the mapping package opens the possibility to solve the problem of missing linkage between concepts. The structural mapping using the MDRMapping and semantic annotation using concepts with SKOS enables direct links between every administrated item. The missing technical extensibility, like the FHIR extensions, is a structural problem and should have been addressed in the new metadata standard. Machine-readable or machine-actionable extensions are highly useful as demonstrated by the often-used FHIR extensions and profiles and recommended by the renowned FAIR principles (17). A structure for a usage model of the metadata is not directly addressed, but the newly added semantic possibilities allow SKOS-based annotations. Context and corresponding usage should not be annotated using domain-specific ontologies like SNOMED CT (18) since they are describing the “what is”, whereas the usage is dependent on the situation and its context. Conceptual orientation thus requires that each term in the vocabulary has a single, coherent meaning, even though its meaning may vary depending on its occurrence in a context (19). On the contrary, SKOS is able to represent this contextual relationship pragmatically and thus to depict a usage model. The extension of the previously developed QL4MDR will enable data sharing between MDR of both ISO standards since the underlying metamodel is not altered. The introduction of the mapping class is beneficial for federated metadata processing. The standardized mapping between metadata items enables schema crosswalks between different items in different systems and promotes their reuse and sharing due to their findability. The upcoming implementations based on ISO 21526 will open up interesting possibilities, for example, for the consensus process of core data sets with preferred data elements based on multiple existing, possibly conflicting data set specifications in source systems. Additionally, an extension of QL4MDR does not break the current interface implementation due to the nature of GraphQL and can support semantic querying for metadata. 5. Conclusion The new metadata standard ISO 21526 is a qualified successor and solves some inherited problems of the leading ISO 11179 using a good combination of newly introduced and refined packages. The extension of QL4MDR will enable better queries using semantic identifiers, and the mapping classes will be beneficial for the metadata processing, especially in a federated context. 206 H. Ulrich et al. / Analysis of ISO/TS 21526 Towards the Extension of a Standardized Query API Acknowledgements This work was supported by the German Research Foundation (Deutsche Forschungsgemeinschaft) DFG grants IN 50/3-2 and WI 1605/10-2. References [1] Dugas M, Jöckel K-H, Friede T, Gefeller O, Kieser M, Marschollek M, et al. Memorandum “Open Metadata.” Methods Inf Med. 2015;54(4):376–8. [2] Nadkarni PM, Brandt CA. The Common Data Elements for cancer research: remarks on functions and structure. Methods Inf Med. 2006;45(6):594–601. [3] ISO/IEC. ISO/IEC 11179-3:2013 - Information technology -- Metadata registries (MDR) -- Part 3: Registry metamodel and basic attributes [Internet]. 2013. Available from: https://www.iso.org/standard/50340.html [4] Ngouongo SMN, Löbe M, Stausberg J. The ISO/IEC 11179 norm for metadata registries: Does it cover healthcare standards in empirical research? J Biomed Inform. 2013 Apr;46(2):318–327. [5] Park YR, Kim JH. Achieving interoperability for metadata registries using comparative object modeling. In: Studies in Health Technology and Informatics [Internet]. 2010. p. 1136–1139. Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-78649520866&doi=10.3233%2F978-1-60750588-4-1136&partnerID=40&md5=f3da6f5ef8c6517837565b072c067fac [6] Warzel DB, Andonyadis C, McCurry B, Chilukuri R, Ishmukhamedov S, Covitz P. Common Data Element (CDE) Management and Deployment in Clinical Trials. AMIA Annu Symp Proc. 2003;2003:1048. [7] Australien Institute of Health and Welfare. METeOR home [Internet]. [cited 2018 Jun 29]. Available from: http://meteor.aihw.gov.au/content/index.phtml/itemId/181162 [8] Aristotle Metadata Registry - [cited 2019 Feb 20]. Available from: https://www.aristotlemetadata.com/ [9] Penoza C, Barnes R, Chaparro M, Carney J, Levin B, Chudy M, et al. United States Health Information Knowledgebase (USHIK). Int J Funct Inform Pers Med. 2010;3(4):292–313. [10] Ulrich H, Kern J, Tas D, Kock-Schoppenhauer A-K, Ückert F, Ingenerf J, et al. QL4MDR: a GraphQL query language for ISO 11179-based metadata repositories. BMC Med Inform Decis Mak. 2019 Mar 18;19(1):45. [11] Bender D, Sartipi K. HL7 FHIR: An Agile and RESTful approach to healthcare information exchange. In: Proceedings of the 26th IEEE international symposium on computer-based medical systems. IEEE; 2013. p. 326–31. [12] ISO/IEC. ISO/TS 21526:2019 - Health informatics — Metadata repository requirements (MetaRep) [Internet]. ISO. 2019. Available from: https://www.iso.org/cms/render/live/en/sites/isoorg/contents/data/standard/07/10/71041.html [13] Benson T, Grieve G. Principles of Health Interoperability - SNOMED CT, HL7 and FHIR. 3rd ed. London: Springer International Publishing; 2016. [14] Miles A, Bechhofer S. SKOS simple knowledge organization system reference. W3C Recomm. 2009;18:W3C. [15] Davies J, Harris S, Crichton C, Shukla A, Gibbons J. Metadata standards for semantic interoperability in electronic government. In: Proceedings of the 2nd international conference on theory and practice of electronic governance. 2008. p. 67–75. [16] Solbrig HR. Metadata and the reintegration of clinical information: ISO 11179. MD Comput Comput Med Pract. 2000;17(3):25. [17] Wilkinson et al. MD, Dumontier M, Aalbersberg IJJ, Appleton G, Axton M, Baak A, et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data. 2016;3:160018. [18] Schulz S, Stegwee R, Chronaki C. Standards in healthcare data. In: Fundamentals of Clinical Data Science. Springer; 2019. p. 19–36. [19] Cimino J. Desiderata for Controlled Medical Vocabularies in the Twenty-First Century. Methods Inf Med. 1998 Nov;37(4–5):394–403. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200724 207 Effects of User Participation in the Development of Health Information Systems on Their Evaluation Within Occupational Health Services a Anna VAHTERISTO a,1 and Virpi JYLHÄa Department of Health and Social Management, University of Eastern Finland, Finland Abstract. Information management and the usability of health information systems (HIS) are important for the development of HIS in occupational health services. User participation in the HIS development process has been shown to contribute to the success of an HIS. The purpose of this study was to analyze how user participation in HIS development affected evaluation of the success of HIS. The success was assessed on the basis of the DeLone and McLean Information Systems (IS) Success Model. The study was conducted within occupational health services and the data (n=210) was analyzed with quantitative methods. The results showed that users participating in the HIS development process assessed the success of the HIS as better than those that had not taken part in the development. This difference could be seen in all seven dimensions of the DeLone and McLean IS success model but was statistically significant only for System Quality and Intention to Use. The results also showed that the users that had participated in the HIS development process also used the HIS more often and more extensively than those that had not participated in the development. The results indicate that user participation in the development process positively influences their assessment of the HIS and increases their active use of the IS. However, more research is needed to determine the longterm effects of using participatory design in HIS development. Keywords. Occupational Health Services, Information Systems, Participatory Design Information Systems, Health 1. Introduction Progress in eHealth and health information systems (HIS) development has been prominent in the World Health Organization (WHO) member states. However, there are still barriers to overcome before eHealth and HIS can be fully integrated into healthcare. One of the barriers is that systems are mostly developed separately, causing an additional burden in data utilization, and poor quality of data. There is also a need to develop systems that better support the health professionals in their work [1]. According to health professionals, HIS do not support the users in their daily work and the users are concerned about their technical functionality (e.g. slowness and system crashes) [2-4]. 1 Corresponding Author, Anna Vahteristo, Department of Health and Social Management, University of Eastern Finland, P.O.Box 1627, 70211 Kuopio, Finland: E-mail: avahteri@student.uef.fi. 208 A. Vahteristo and V. Jylhä / Effects of User Participation in the Development of HIS Nurses also reported having to document the same information several times [4]. In a national Finnish survey, 55 % of nurses stated that they had not participated in HIS development, and only less than 10 % assessed that they had participated significantly in development processes. At the same time the nurses claimed that the development of the HIS did not meet their requirements [4]. The physicians agreed with this, as only 10 % of them reported that their suggestions on electronic patient records (EPR) development had been implemented [5]. Thus, it appears that users, either health professionals or citizens, should participate actively in the development of HIS. Participatory information system (IS) design aims to combine technical development processes and end users’ knowledge of the substance. Participation strengthens users’ positive attitudes towards the development process, as they can affect the development of the IS they are using. Participation also increases commitment to use the IS [6]. Internationally, health professionals have been interested in participating in the development of HIS [7-8]. The results of participatory HIS design have also been successful [7-11]. There is a clear need for participatory HIS design and for understanding its effect on the success of IS. The earlier literature describes research using participatory design in HIS development in hospitals and clinical contexts [7-11]. The objective of this study was to investigate how participation in the HIS development process affects the evaluation of the success of HIS in Finnish occupational health services. The research question was “How does the evaluation of the HIS success of the participating users differ from that of non-participating users?”. 2. Methods This cross-sectional study was conducted in June 2019 using an electronic survey to evaluate the success of HIS. The assessed HIS was developed for the use of occupational health services for information management and analysis. The information obtained is further used in occupational health care to promote health and work ability. The HIS was developed in cooperation with occupational health professionals, using participatory design in the development process. The data were collected from occupational health professionals (physicians, health nurses, physiotherapists, and psychologies) in Finland. A total of 252 of 1124 professionals returned the questionnaire, of whom 243 gave their informed consent. After excluding the responses of non-users, the data consisted of 210 completed questionnaires. In this study we used the DeLone and McLean IS Success Model as the framework to assess the HIS, as its seven dimensions describe the systems technical quality (system quality, information quality and service quality) as well as the user aspect (use, intention to use and user satisfaction) and the benefits of using the IS (net benefits) [12]. The DeLone and McLean IS Success model is also widely used in the assessment of HIS, mostly in hospitals [13-17]. The dimensions were operationalized in order to analyze users’ assessment of HIS used in occupational health services. The questionnaire consisted of fifteen statements, which were based on the dimensions of the DeLone and McLean IS success model [5] and assessed on a 5-point Likert-scale. In addition, there were basic background questions including a question about participation in the development process. In this study, we utilized validated statements from previous studies [13-17]. The evaluations of respondents that had participated in the development process were compared to the evaluations of the respondents that had not taken part in 209 A. Vahteristo and V. Jylhä / Effects of User Participation in the Development of HIS the development. The data were analyzed using the Independent samples Mann-Whitney U-test. U-values and p-values are presented along with mean, median and standard deviation. A p-value lower than 0.05 was considered statistically significant. The data was collected, maintained, and reported following the good research practices and ethical principles of the Finnish Advisory Board on Research Integrity [18]. 3. Results Half of the respondents (n=104) had worked from one to ten years in the occupational health services, 44 % (n=93) over 11 years, and only 6 % (n=13) for less than one year. Of the respondents, 70 % (n=146) were occupational health nurses and 13 % (n=28) occupational physicians. 11 % (n=24) of the respondents had participated in the development of the HIS. The users that had participated in the HIS development were more active users of the HIS, as 38 % of them used the HIS weekly and 25 % daily, whereas of the non-participating users 27 % used HIS weekly and only 1 % daily (Table 1). The participating users also used the HIS more extensively than the regular users, as they used on average five of the nine sections of the HIS compared to the four sections used by the non-participating users. Table 1. Effect of participation in development on the use of HIS Activity of use Extent of use Participating Users (n=24) Mean md SD 3.75 4.00 0.989 4.92 5.00 1.976 Non-participating users (n=186) Mean md SD 2.92 3.00 0.811 3.93 4.00 1.773 Mann Whitney U-test U-value p-value 1207.50 <0.001*** 1559.50 0.015* *p < 0.05, ***p<0.001 The users that had participated in the development of the HIS also assessed the success of the HIS as better than the non-participating users (Table 2). Overall, participating users assessed all seven dimensions of the DeLone and McLean IS Success Model as more successful than non-participating users. The differences in System Quality and Intention to Use were statistically significant. Table 2. Effect of participation in development on the success of HIS System Quality Information Quality Service Quality Use Intention to Use User Satisfaction Net benefits Participating Users (n=24) Mean md SD 2.96 3.00 0.78 3.18 3.67 1.03 3.54 3.75 1.03 2.96 3.00 1.12 4.25 4.00 0.90 2.81 2.50 1.14 4.25 3.33 1.06 *p < 0.050, ***p<0.001 Non-participating users (n=186) Mean md SD 2.58 2.33 0.76 3.00 3.00 0.84 3.22 3.00 0.95 2.51 2.00 1.14 3.84 4.00 0.97 2.41 2.50 0.90 3.02 3.00 0.90 Mann Whitney U-test U-value 1457.00 1947.00 1839.00 1727.00 1680.00 1801.00 1742.00 p-value 0.017* 0.306 0.152 0.062 0.028* 0.117 0.079 210 A. Vahteristo and V. Jylhä / Effects of User Participation in the Development of HIS 4. Discussion The objective of this study was to reveal how participation in the development process affects the evaluation of the success of IS in the Finnish occupational health care environment. The results indicate that participation in the development process of the HIS resulted in more active and extensive use of the HIS. This supports the basic assumption of the participatory development process, which aims to increase commitment to the IS by providing the opportunity to affect its development [6]. The results of this study also indicate that the users that participated in the development process of the HIS assessed its success as better than did the nonparticipating users. This supports the conclusion of Tubaishat, who stated that the users that used the HIS more actively were generally more satisfied with the system than users not using HIS as actively [17]. Furthermore, Saghaeiannejad-Isfahani et al. reported that the developers of HIS assessed its success better than the users [14]. Although earlier research on HIS development using participatory design has not used the DeLone and McLean IS Success Model in evaluation of the development process, similar results of increased satisfaction of the participating users can be seen [8]. Therefore, this study confirms the earlier conclusions of successful HIS development with participatory design [7,11]. The use of the DeLone and McLean IS Success Model yields a broad view of the IS success, as its dimensions provide information about the success on a technical level as well as about the users’ aspect and the benefits of the IS. Thus, the model appears to be suitable in assessing the success of an HIS development using participatory design. The results of this study indicate that the use of participatory design in the development of HIS improves the success of HIS. However, there is a need to use it more widely and for a longer period in the development of HIS in order to tackle the challenges identified in earlier studies [4-5]. There are also some limitations to this study. First, the study was cross-sectional and described the assessment of the HIS only after its implementation and only at one moment. A longitudinal study would provide more information on the effect of the participatory design on the HIS development, as the non-participating users gain more experience with the HIS. Secondly, a longitudinal study with a predevelopment assessment would provide more information about the assessments of both groups of users and their attitudes towards the HIS and its development. Thirdly, the group of participating users was rather small compared to the non-participating users, and therefore more data is still needed to verify the results. 5. Conclusions This study provides information on the effects of participatory design on HIS development in the context of occupational health services. The results indicate that participation of health professionals in the HIS development process helps to commit the users to the use of the HIS. The results also show that having the possibility to influence the development of the HIS results in better satisfaction with the success of the HIS, especially with regard to System Quality and Intention to Use the HIS. A. Vahteristo and V. Jylhä / Effects of User Participation in the Development of HIS 211 References [1] World Health Organization. Global diffusion of eHealth: Making universal health coverage achievable. Report of the third global survey of eHealth. World Health Organization, Geneve. 2016. Available from: http://apps.who.int/iris/bitstream/handle/10665/252529/9789241511780-eng.pdf?sequence=1 (accessed June 6, 2020). [2] Kaipio J. Usability of electronic patient report. In: Vehko T, Ruotsalainen S, Hyppönen H, editors. E-health and e-welfare of Finland. Checkpoint 2018.National Institute for Health and Welfare Report 7/2019. National Institute for Health and Welfare, Helsinki; 2019. p.109–15. Available from: http://urn.fi/URN:ISBN:978-952-343-326-7 (accessed June 27, 2020). [3] Hyppönen H. Usability of health information exchange and utility systems. In: Vehko T, Ruotsalainen S, Hyppönen H, editors. E-health and e-welfare of Finland. Checkpoint 2018.National Institute for Health and Welfare Report 7/2019. National Institute for Health and Welfare, Helsinki; 2019. p.115–21. Available from: http://urn.fi/URN:ISBN:978-952-343-326-7 (accessed June 27, 2020). [4] Kinnunen U-M, Hyppönen H, Liljamo P, Saranto K. Nurses’ experiences of health and social care information systems. In: Vehko T, Ruotsalainen S, Hyppönen H, editors. E-health and e-welfare of Finland. Checkpoint 2018.National Institute for Health and Welfare Report 7/2019. National Institute for Health and Welfare, Helsinki; 2019. p.130–47. Available from: http://urn.fi/URN:ISBN:978-952-343326-7 (accessed June 27, 2020). [5] Martikainen S, Kaipio J. Physicians’ participation in the IT system development. In: Vehko T, Ruotsalainen S, Hyppönen H, editors. E-health and e-welfare of Finland. Checkpoint 2018.National Institute for Health and Welfare Report 7/2019. National Institute for Health and Welfare, Helsinki; 2019. p.125–6. Available from: http://urn.fi/URN:ISBN:978-952-343-326-7 (accessed June 27, 2020). [6] Bansler J. Systems Development in Scandinavia: Three theoretical schools. Information Technology & People. 1988; 4(2), 117-33. doi:10.1108/eb022657. [7] Reeder B, Hills RA, Turner AM, Demiris G. Participatory Design of an Integrated Information System Design to Support Public Health Nurses and Nurse Managers. Public Health Nurs. 2014; 31(2), 183–92. doi: 10.1111/phn.12081. [8] Nelson SD, Del Fiol G, Hanseler H, Crouch BI, Cummins MR. 2016. A case report of refining user requirements for a health information exchange dashboard. Appl Clin Inform. 2016; 7, 22–32. http://dx.doi.org/10.4338/ACI-2015-07-CR-0091 [9] Urda D, Ribelles N, Subirats JL, Franco L, Alba E, Jerez JM. Addressing critical issues in the development of an Oncology Information System. Int. J. Med. Inform. 2013; 82, 398– 407http://dx.doi.org/10.1016/j.ijmedinf.2012.08.001 [10] Pollack AH, Miller A, Mishra SR, Pratt W. PD-atricians: Leveraging Physicians and Participatory Design to Develop Novel Clinical Information Tools. Proceedings of AMIA Annu Symp. 2017 Feb 10; 2016:1030-9. [11] Qin Y, Zhou R, Wu Q, Huang X, Chen X, Wang W, Wang X, Xu H, Zheng J, Qian S, Bai C, Yu P. The effect of nursing participation in the design of a critical care information system: a case study in a Chinese hospital. BMC Med. Inform. Decis. Mak. 2017 Dec; 17(1):165. doi: 10.1186/s12911-017-0569-3 [12] DeLone WH, McLean ER. The DeLone and McLean Model of Information Systems Success: A TenYear Update. J. Manag. Inf. Syst. 2003; 19(4), 9–30. https://doi.org/10.1080/07421222.2003.11045748 [13] Bossen C, Jensen LG, Udsen FW. Evaluation of a comprehensive EHR based on the DeLone and McLean model of IS success: Approach, results, and success factors. Int. J. Med. Inform. 2013; 82, 940–53. [14] Choi JS, Yun SH, Kim D, Park SW. Impact of Doctors’ Resistance on Success of Drug Utilization Review System. Healthc Inform Res. 2014; 20(2), 99–108. http://dx.doi.org/10.4258/hir.2014.20.2.99 [15] Saghaeiannejad-Isfahani S, Saeedbakhsh S, Jahanbakhsh M, Habibi M. Analysis of the quality of hospital information systems in Isfahan teaching hospitals based on the DeLone and McLean model. J Educ Health Promot. 2015 Feb; 4(5). doi: 10.4103/2277-9531.151883. [16] Gaardboe R, Sandalgaard N, Nyvang T. An assessment of business intelligence in public hospitals. Int. J. Inf. Syst. Proj. Manag. 2017; 5(4), 5–18. doi: 10.12821/ijispm050401. [17] Tubaishat A. Evaluation of Electronic Health Record Implementation in Hospitals. Comput Inform Nurs. 2017 Jul; 35(7):364-72. doi: 10.1097/CIN.0000000000000328. [18] Finnish Advisory Board on Research Integrity (TENK). Responsible conduct of research and procedures for handling allegations of misconduct in Finland. Guidelines of the Finnish Advisory Board on Research Integrity 2012. Available from: http://www.tenk.fi/sites/tenk.fi/files/HTK_ohje_2012.pdf (accessed 3.7.2020). 212 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200725 Typology-Based Analysis of Covid-19 Mobile Applications: Implications for Patient Empowerment Riikka VUOKKOa,1, Kaija SARANTOb and Sari PALOJOKIa a The Ministry of Social Affairs and Health, Finland b University of Eastern Finland, Finland Abstract. During COVID-19 pandemic, mobile technology is seen as potential tool for epidemic control and citizens’ empowerment. Based on literature, we explore, which are the currently known types of the mobile apps and what implications do the apps have for patient empowerment. There is a need for evidence and an assessment framework to ensure that COVID-19 apps deliver on their promises. Keywords. COVID-19, mobile application, tracing, remote technologies, empowerment 1. Introduction During COVID-19 pandemic, mobile technology is being envisioned as potential and ubiquitous tool for authorities’ epidemic control. At the same time, mobile technology has potential to provide easily accessible information for the citizens. Targeting those goals, COVID-19-related smartphone, and web-based health applications (later apps) are being rapidly developed, leading to a multitude of options, raising ethical and legal challenges and potentially confusing end users. [1, 2] The increasing presence of technology in health care has created new opportunities for patient engagement and with this, an emerging exploration of patient empowerment within the digital health context. Research gives evidence that there is a linkage between digital health solutions, and patient empowerment, but measurable health outcomes remains yet elusive [3]. Alarmingly, there is currently a lack of real-world evidence for potentially beneficial mobile applications used by citizens and patients during the COVID-19 pandemic for their need of information and support for coping. Health literacy and - in this context - the digital divide are important aspects of empowerment but remaining challenges in this are less discussed even though they may hinder maximizing the potential of mobile tools [4, 5]. Due to a diversity of COVID-19 apps with abundant objectives, it is important to support professionals and the public in identifying the varied types and functionalities of the apps. Additionally, taken that apps promote health-care intervention it is substantive to outline their impact on patient empowerment. Therefore, our research 1 Corresponding author, Dr. Riikka Vuokko, P.O. Box 33, 00023 Helsinki, Finland, riikka.vuokko@gmail.com. R. Vuokko et al. / Typology-Based Analysis of Covid-19 Mobile Applications 213 questions are: (1) What are the functionalities of currently known types of COVID-19 mobile apps? (2) What implications do the apps have in regard of patient empowerment? 2. Methods In this paper, we apply approach of typologies, similar to classifications as useful tools to classify and organise items based on common variables (attribute such as colour), where the types are mutually exclusive (e.g., red type) and the typology system complete, although in the real world, people tend to disagree of their nature. [6, 7] European Commission (later EC) identifies four types of COVID-19 applications based on their services: symptom checkers and self-diagnosis apps, apps for tracking the spread of the coronavirus, apps for delivering trustworthy information and guidelines to public, and apps for supporting homebound patients and enabling selfmanagement. [2] Alternative typology is suggested based on the outcomes of the apps: whether their goal is in societal impact, in personal impact or in density dependence [1].While there is yet little evidence of the apps’ outcomes, in this paper, we concentrate on the EC typology based exploration of the COVID-19 apps. Terms for literature searches were composed according to the typology: “Covid-19”, “apps”, “guideline”, “information”, “self-diagnosis”, “symptom checker”, “symptom”, “tracking”, “tracing”, “home”, “self-management”, “triage”, “coping”. In this context, we conceptualize patient empowerment to cover situations where citizens are encouraged to take an active role in the management of their own health [5, 9]. Patient empowerment is a meta-paradigm and it is a broader concept than patient participation and patient-centeredness. [8] PubMed search in the middle of July 2020 resulted in 28 peer-reviewed papers. When the concept of empowerment was composed with other search terms, it did not result in added papers. After removing duplicates and the first exclusion round based on two researcher reading the abstracts (out of scope, e.g., focus on professionals, not relevant e.g., focus on dark net activities, language) 20 papers were selected for further reading. After full paper reading, additional five of the research papers were excluded as they were out of scope, or focused on professionals. Total of 15 papers were analysed using the EC typology. 3. Results The first results based on PubMed searches indicate that there is yet little evidence for research of COVID-19 apps and that the terms describing these apps are not well established. Of the types, symptom checkers and self-diagnosis apps resulted in 3 papers, tracing apps 7, apps for information and guidelines 2, and monitoring apps 5 papers, when two papers covered several types of apps. Results of the apps for symptom checking and triage show that evidence on these kinds of apps is scarce. While numerous apps are available for professionals, patients’ perspective remains understudied. [10]. Devising personalized self-testing kits for COVID-19 virus is important because providing real-time testing will facilitate speedy prediagnosis to a large population [11]. Smartphone embedded software and highperformance computing have the potential to be deployed as self-test breathing 214 R. Vuokko et al. / Typology-Based Analysis of Covid-19 Mobile Applications monitoring apps. Those with higher risks of severe illness can check their breathing sound pattern frequently through the app [11]. Communication of health needs is of paramount importance when patients are isolated. Usage of alternative digital mental health options such as smartphone apps has increased, thus providing support for empowerment. The wide availability of these resources may promote resilience and well-being on a wider community level as mental health information is disseminated widely and potentially destigmatizes illness while promoting acceptance of digital tools. On turn, developing digital mental health resources without an evidence-based framework might be harmful. [12] Results of the tracing apps give evidence of potentially useful tools that may be employed to limit disease transmission [1, 13]. Several countries have now started to deploy apps capable of supporting COVID-19 contact tracing, but the efficacy of such apps has yet to be proved. Key functionalities include that apps inform people that they have spent a specific time near someone with the virus. The contacts should then respond according to local rules, for example by self-quarantining themselves immediately [1, 13-17]. These apps are not without concerns from a user perspective and consequently, they may cause limitations for patient empowerment. The topic of user adoption is presented align with privacy concerns, where some users may not be comfortable with an app that tracks their location or has otherwise negative effects on individual privacy. Users may become fatigued from procedures, e.g., scanning QR code, and choose to discontinue. [13, 17] False negative alarms could spur a false sense of safety in others. Moreover, many apps work only with certain phones causing uncertainties for availability. [15]. In turn, an emerging evidence suggests also the app may enable some patients to return more quickly to their lives [18]. In summary, literature evidences that these apps can contribute towards a more general, population level goal but a personal benefit and impact on empowerment is not as evident [1]. Results of the apps for information and guidance illustrate that during the pandemic people have a need for timely information and guidance when they seek for the latest news of the pandemic, check facts when encountering uncertainties, and want to obtain informational guidance for health management. [19] Typically information and guidance can be received autonomously, which supports citizens’ selfdetermination and control, which are close coupled with empowerment. The information content in an app should be reliable and based on current data. [1] Results of the apps for coping and monitoring at home emphasize necessity to avoid traditional face-to-face visits especially for patients with higher risks, such as elderly, without hampering the quality of care. During COVID-19 pandemic especially outpatient visits have been cancelled or postpone and digital technologies have become a way for accessing remote care. Advances in remote care and monitoring, e.g., via apps enable variety of possibilities for virtual visits, follow-ups, monitoring and consultation. [12, 20-23] At the same time, remote technologies, such as videoconferences, video monitoring and wearable devices, can provide electronic reminders and support in daily activities. Reported advantages of these kinds of apps are improved access and quality of care regardless of location or time, thus prompting full potential of empowerment. [12, 20-21] Reported limitations are technical issues, patients’ and caregivers’ skills with technology, and ethical concerns related to data privacy. [20] Consequently, while apps can increase agency in self-care and improvement in health, ability to share data captured with the devices back to caregivers remains a challenge, therefore limiting potential patient participation. [22] R. Vuokko et al. / Typology-Based Analysis of Covid-19 Mobile Applications 215 4. Discussion Having applied a structured approach of a typology-based analysis, four types of COVID-19 apps and related functions were identified. Our results show that current development concentrates on two types of apps, namely the apps for tracing and for remote care and monitoring. Taken that the development of the apps has been exceptionally rapid due to pressure set by the ongoing pandemic some compromising ways in developing these apps have inevitably been applied. It should be noted that when developing apps, methods should be backed by scientific evidence. [1] An indepth analysis of comparison and consideration of the relative benefits and possible harms is required. [18] Structured assessment of already deployed apps is needed [1]. As most apps’ use is still at initial stages, their full impact is yet unknown but scientific evidence and assessment would support recognizing their potential. This would illustrate which of the apps are effective and applicable for wider use which is a prerequisite for e.g., tracing apps [1,12,15,16]. To sum up our results, it is obvious that future evidence of COVID-19 digital interventions is urgently needed [19]. Plausible evidence of the types of apps and their implications to empowerment are yet scarce. Although empowerment is being articulated, structures emerging and supporting it are yet mostly unanalyzed. [9]. Research may give evidence how apps advance an emerging view of patient empowerment. Considering the nature of the pandemic as public health threat, we suggest exploring apps’ impact on preventive behaviour and empowerment. [19] Especially, as a result of our analysis, the apps as ubiquitous technology supporting equity in care needs further evidence [18]. In the context of empowerment, it is critical to raise the fact that the introduction of new technologies can cause discrimination. This can take the form of bias where technology is available to some but not all. Thus, it is crucial to recognize the importance of equity when deploying apps if patient empowerment is one of the goals. [1,18] Our approach is subject to some limitations. We wish to highlight a number of factors affecting reliability and validity of research, which deserve attention: the number of apps, the purpose of apps including a possible collection of functions, and an analysis framework for data should be clearly stated also in seminal research. Due to the ongoing situation, preliminary reporting is descriptive and may be selective or biased data. [23] While we applied EC typology for the current apps, different types of apps may dominate when the pandemic situation evolves. Moreover, no established frameworks or terminology is available for analyzing COVID-19 apps and patient empowerment, which is among the recognized development aims in the future. To conclude, there is a need for evidence of apps’ outcomes and their impact on empowerment. An assessment framework to evaluate how COVID-19 apps deliver on their promises should be established. Collaborative initiatives should harness both conventional and novel evidence-based tools to provide an effective and timely response to the COVID-19 pandemic on the global stage. [1, 2, 23] References [1] Vokinger KN, Nittas V, Witt CM, Fabrikant SI, von Wyl V. Digital health and the COVID-19 epidemic: an assessment framework for apps from an epidemiological and legal perspective. Swiss Med Wkly. 2020 May 17;150:w20282. doi: 10.4414/smw.2020.20282. PMID: 32418194. 216 R. Vuokko et al. / Typology-Based Analysis of Covid-19 Mobile Applications [2] European Commission. Mobile applications to support contact tracing in the EU’s fight against COVID19. Common EU Toolbox for Member States. Version 1.0, 15.04.2020 https://ec.europa.eu/health/sites/health/files/ehealth/docs/covid-19_apps_en.pdf. [3] Risling T, Martinez J, Young J, Thorp-Froslie N. Defining Empowerment and Supporting Engagement Using Patient Views From the Citizen Health Information Portal: Qualitative Study JMIR Med Inform 2018;6(3):e43 URL: https://medinform.jmir.org/2018/3/e43 doi: 10.2196/medinform.8828 PMID: 30201603 PMCID: 6231721. [4] Demiris G. Consumer Health Informatics: Past, Present, and Future of a Rapidly Evolving Domain. Yearb Med Inform. 2016 May 20;Suppl 1(Suppl 1):S42-7. doi: 10.15265/IYS-2016-s005. PMID: 27199196; PMCID: PMC5171509. [5] Calvillo J, Román I, Roa LM. How technology is empowering patients? A literature review. Health Expect. 2015 Oct;18(5):643-52. doi: 10.1111/hex.12089. Epub 2013 May 28. PMID: 23711169; PMCID: PMC5060865. [6] Chute CG. Clinical Classification and Terminology: Some History and Current Observations. J Am Med Inform Assoc. May-Jun 2000;7(3):298-303. doi: 10.1136/jamia.2000.0070298. [7] Bowker GC, Star SL. Sorting Things Out: Classification and Its Consequences. 4th ed. Cambridge, Mass: MIT Press; c1999. 377 p. [8] Castro EM, Van Regenmortel T, Vanhaecht K, Sermeus W, Van Hecke A. Patient empowerment, patient participation and patient-centeredness in hospital care: A concept analysis based on a literature review. Patient Educ Couns. 2016 Dec;99(12):1923-1939. doi: 10.1016/j.pec.2016.07.026. Epub 2016 Jul 18. PMID: 27450481. [9] Umar A, Mundy D. Re-thinking Models of Patient Empowerment. Stud Health Technol Inform. 2015;209:175-81. PMID: 25980722. [10] Cervino G, Oteri G. COVID-19 Pandemic and Telephone Triage before Attending Medical Office: Problem or Opportunity? Medicina (Kaunas). 2020 May 20;56(5):250. doi: 10.3390/medicina56050250. PMID: 32443828; PMCID: PMC7279364. [11] Faezipour M, Abuzneid A. Smartphone-Based Self-Testing of COVID-19 Using Breathing Sounds. Telemed J E Health. 2020 Jun 2. doi: 10.1089/tmj.2020.0114. Epub ahead of print. PMID: 32487005. [12] Marshall JM, Dunstan DA, Bartik W. The role of digital mental health resources to treat trauma symptoms in Australia during COVID-19. Psychol Trauma. 2020 Jun 4. doi: 10.1037/tra0000627. Epub ahead of print. PMID: 32496103. [13] Yasaka TM, Lehrich BM, Sahyouni R. Peer-to-Peer Contact Tracing: Development of a PrivacyPreserving Smartphone App. JMIR Mhealth Uhealth. 2020 Apr 7;8(4):e18936. doi: 10.2196/18936. PMID: 32240973; PMCID: PMC7144575. [14] Morley J, Cowls J, Taddeo M, Floridi L. Ethical guidelines for COVID-19 tracing apps. Nature. 2020 Jun;582(7810):29-31. doi: 10.1038/d41586-020-01578-0. PMID: 32467596. [15] Editorial: Show evidence that apps for COVID-19 contact-tracing are secure and effective. Nature. 2020 Apr;580(7805):563. doi: 10.1038/d41586-020-01264-1. PMID: 32350479. [16] Zastrow M. Coronavirus contact-tracing apps: can they slow the spread of COVID-19? Nature. 2020 May 19. doi: 10.1038/d41586-020-01514-2. Epub ahead of print. PMID: 32433633. [17] Bengio Y, Ippolito D, Janda R, Jarvie M, Prud'homme B, Rousseau JF, Sharma A, Yu YW. Inherent privacy limitations of decentralized contact tracing apps. J Am Med Inform Assoc. 2020 Jun 25:ocaa153. doi: 10.1093/jamia/ocaa153. Epub ahead of print. PMID: 32584990. [18] Parker MJ, Fraser C, Abeler-Dörner L, Bonsall D. Ethics of instantaneous contact tracing using mobile phone apps in the control of the COVID-19 pandemic. J Med Ethics. 2020 Jul;46(7):427-431. doi: 10.1136/medethics-2020-106314. Epub 2020 May 4. PMID: 32366705; PMCID: PMC7231546. [19] Liu PL. COVID-19 Information Seeking on Digital Media and Preventive Behaviors: The Me-diation Role of Worry. Cyberpsychology, Behavior, and Social Networking. 2020 Jun. https://doi.org/10.1089/cyber.2020.0250. Online ahead of Print. [20] Cuffaro L, Di Lorenzo F, Bonavita S, Tedeschi G, Leanica L, Lavargna L. Dementia care and COVID19 pandemic: a necessary digital revolution. Neurological Sciences Jun 2020 41(8): 1977-1979. doi: 10.1007/s10072-020-04512-4. [21] Greiwe J, Nyeanhuis SM. Wearable Technology and How This Can Be Implemented into Clinical Practice. Curr Allergy Asthma Rep. 2020 Jun 6;20(8):36. doi: 10.1007/s11882-020-00927-3. [22] Gamble, A, Pham Q, Goyal S, Cafazzo JA. The Challenges of COVID-19 for People Living With Diabetes: Considerations for Digital Health. JMIR Diabetes. 2020 May 15;5(2):e19581. doi: 10.2196/19581. [23] Boukhris M, Hillani A, Moroni F, Annabi MS, Addad F, Ribeiro MH, Mansour S, Zhao X, Ybarra LF, Abbate A, Vilca LM, Azzalini L. Cardiovascular Implications of the COVID-19 Pandemic: A Global Perspective. Can J Cardiol. 2020 May 16:S0828-282X(20)30464-5. doi: 10.1016/j.cjca.2020.05.018. Online ahead of print. Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200726 217 The Master Study in Telemedicine and Ehealth at the University of Tromsø, Norway, 2005-2018 a Rolf WYNNa,1 and Gunnar ELLINGSENa UiT The Arctic University of Norway, Tromsø, Norway Abstract. In this paper we describe the Master Study in Telemedicine and E-health at the University of Tromsø, Norway. The study enrolled its first students in 2005 and was closed in 2018. We describe and discuss the background of the programme, its development and accomplishments and why it was closed. Hopefully, this narrative will be of use to other programmes focusing on e-health. Keywords. Telemedicine, E-health, Master Programme, Norway 1. Introduction North Norway is a large region covering 112.951 km2, but it is sparsely populated with only approximately half a million inhabitants. Telemedicine and e-health are therefore especially important in North Norway. Telemedicine and e-health in the region can trace its roots back to 1988 when Telenor (a formerly state owned Norwegian telecom) launched the large-scale research project “Telemedicine in North Norway”. Initially, Telenor targeted its telemedicine activities towards two principal areas. The first was modem-based transmissions of laboratory results to GP practices, and the second was remote consultations through videoconferences. Very soon several other pilot projects were spawned within medical fields, such as teleradiology, teledermatology, telepsychiatry, teledialysis, telepathology, tele-ENT, telecardiology, teleophthalmology, etc. A Telemedicine Department at the University Hospital of North Norway was established to promote, coordinate and implement the services, and many of these were subsequently put into routine use. The health authorities delegated the Department the role as National Centre of Competence for Telemedicine and it was renamed the Norwegian Centre for Telemedicine. In recent years, the Centre has been renamed the Norwegian Centre for E-health Research (NSE). As part of the early activities, plans for a Master programme in Telemedicine and E-health at the University of Tromsø arose. The University of Tromsø (now named UiT Arctic University of Norway) is the world’s northernmost university. It has more than 3600 employees and in excess of 16000 students. The university has a range of studies within all major fields of study, including life and health sciences, finance, languages, natural sciences, law, etc. According to the Times Higher Education World Ranking [1] the university’s ‘…main 1 Corresponding Author, Rolf Wynn, Department of Clinical Medicine, Faculty of Health Sciences, UiT The Arctic University of Norway, N-9037 Tromsø, Norway; E-mail: rolf.wynn@uit.no. 218 R. Wynn and G. Ellingsen / The Master Study in Telemedicine and E-health teaching expertise lies in scientific fields such as polar environment, climate research, telemedicine, medical biology and fishery science’. In 2005, the first students at the Master programme in Telemedicine and E-health were admitted. The programme was closed in 2018. In this paper, we will describe and discuss the background of the programme, its development and accomplishments and why it was closed. Hopefully, this discussion may help others who are planning or already running a Master programme in e-health. 2. Methods Drawing on our experience as professors on the Programme and on evaluation reports [2], we briefly present and discuss the background, main contents, development, and closure of a Master Programme in Telemedicine and e-Health. 3. Results The programme was a cooperation between the Faculty of Health Sciences and the Faculty of Science and Technology. It also involved collaboration with the University Hospital of North Norway and especially the NSE. The collaboration with NSE occurred over a broad area, including supervision of Master students, internships, lectures by NSE personnel on current projects and evaluations, research collaboration and PhD student positions. There was also national and international collaboration in teaching, student exchange and research activities with other hospitals, universities and private companies. For instance, there was an international collaboration in terms of research and/or student activities with universities in Genova, Verona, Barcelona, Valencia, Graz, Krakow, San Diego, Copenhagen, Cambridge, Kathmandu, Tribhuvan and Khulna. It was a two-year Master of Science programme with two fields of study, ‘Technology’, focusing on the technological construction of systems, and ‘Health’, focusing on the implementation and use of e-health and telemedicine in health services. In the following, we will focus on the ‘Health’ field of the programme unless otherwise stated. The study was a 120 credits full-time programme. The first year of study consisted of 7 compulsory courses in different subjects: The topics of the courses were 1) Medical informatics, 2) Electronic patient records – theories, concepts and practice, 3) Telemedicine applications, 4) International health and environmental medicine, 5) Quantitative methodology, 6) Qualitative methodology, 7) Patients and the public as users of Net health services. The second year was fully devoted to the Master thesis. The ‘Health’ field part of the programme was served by two full professors (the authors of this paper) with background in informatics and medicine, respectively, in addition to several part-time professors, post-docs, PhD students, and external collaborators from other academic and health institutions. The students that participated in the Health-related Master had a wide variety of backgrounds, including nursing, medicine, pharmacy, physical therapy, dentistry, radiography, psychology, engineering, and public health. Among the ‘Health’ field students were also several leaders from hospitals and other parts of the Norwegian health services. R. Wynn and G. Ellingsen / The Master Study in Telemedicine and E-health 219 The students came from wide variety of countries, including the neighbouring areas of Denmark, Sweden, Iceland and Russia, Latvia, England, Germany, Czech Republic, Poland, Belgium, Austria, Slovakia, Greece, and the more distant countries such as Nepal, Bangladesh, the Philippines, India, Ghana, Nigeria, Eritrea, Ethiopia, Cameroon, and South Africa. There was great variability in the topics and methodology of the Master theses. There were case studies, interview-based studies, surveys, epidemiological studies, in addition to different types of reviews. In addition, the programme shifted its focus and scope to encompass technological and societal developments and the “e-health” component in programme got an increasingly prominent role. Along with the widespread digitalization of the health care services, both the NSE and the Master programme got involved in evaluations of regional and national digitalization projects, for instance the implementation and use of Electronic Health Record systems, laboratory systems, nursing documentation, electronic medication managements systems and so on. And as the use of technology for health changed, the Master programme increased its emphasis on new forms of e-health, including social media and video services [3,4]. Up to and including 2016, 63 students had graduated (40 in the ‘Health’ field and 23 in the ‘Technology’ field), approximately ¼ were Norwegian and ¾ from other countries. Following graduation, most of the international students returned to their home countries, providing valuable knowledge that could be implemented in their local health services, private companies, and academic institutions. Many Master students have published work in international peer-reviewed journals during or after their studies [517]. The research group in Telemedicine and E-health that was associated with the Master programme was for a period of several years the Department of Clinical Medicine’s most productive in terms of publications and completed PhDs. In the period 2007-2018, the group’s members produced altogether 486 publications including 259 peer-reviewed scientific articles (of which 42 were so-called ‘level 2’, i.e. published in the presumed top scientific journals). Altogether 18 PhD candidates completed their studies as group members. One major challenge for the programme was the recruitment of a sufficient number of students. The programme had 20 places for students each year, divided equally on the two fields of study. The total number of applicants to the programme in the years 20112016 was 716 (on average 119 students/year) of which 127 were students from the Nordic countries. While the number of applicants, and especially applicants from without the Nordic countries, was high, the number that actually started their studies was much lower. The ‘Health’ field recruited a mean of approximately 8 new students that started each year in the period 2011-2016 and the ‘Technology’ field a mean of less than 2 in the same period. In addition, some of the students that started their studies did not finish. The university decided to close the programme in 2018, and the last students were supervised in 2020. 4. Discussion The Master programme in Telemedicine and E-health was in a field with an increasing importance for the health services. The programme was a collaborative effort, representing different stakeholders and anchored in a strong and productive research 220 R. Wynn and G. Ellingsen / The Master Study in Telemedicine and E-health group. It was internationally acknowledged and recruited students from all over the world. Nevertheless, the programme was shut down in 2018. One central explanation for the closure of the programme was the insufficient recruitment of students. While the biggest recruitment problems were in the ‘Technology’ field, there was also a recruitment challenge in the ‘Health’ field. One explanation for this difference in qualified students in the two directions of the program was that the health-related Master could recruit students with a wide range of health-related backgrounds, while those choosing the technology-related Master were fewer as it was required that they had qualifications in computer science. In the end, the programme was assessed on the basis of the total number of enrolled students – which was considered insufficient. Another factor was the high representation of students from non-Nordic countries. While some considered the high degree of international participation a major strength of the programme, on the national level there was for some time a debate about whether students from outside the EU should have to pay for their studies [18]. However, today university courses in Norway still remain free for all. The programme had from its start some quota places for students from low income countries. The international students that were offered quota places were given extra benefits in terms of student scholarships and loans. The removal of the student quota places also negatively impacted the programme. Perhaps the “telemedicine” notion itself was a cause to the closure of the programme. As a visionary concept, it promises to provide easy access to health services by disregarding geographical and bureaucratic boundaries. This concept may have been particularly attractive to international students from developing countries. However, at the same time, the “telemedicine” concept may also have been its downfall for two reasons: a) It lost its visionary attraction by successfully transforming into routine use, and b) Due to its relatively narrow scope it did not manage to reflect many of the ongoing challenges in the health care sector, for instance the challenges related to large-scale digitalization of different areas, platformization, shared electronic management system etc. Unfortunately, this perception overlooks the increasingly prominent role of the ehealth component of “telemedicine and e-health”. While the notion of telemedicine comes around as static, the notion of e-health (as a rather malleable concept) reflects and responds to current challenges in the health care sector. The programme was repeatedly evaluated internally and in 2017 also externally. The external evaluation committee was positive to the continuation of the programme and had a range of suggestions regarding how to improve the quality of the study and its recruitment [2]. The students also evaluated the programme. While there naturally were different opinions, most students were satisfied with teaching, supervision and their work load. Some expressed a desire for better student facilities with better rooms for teaching and reading and improved digital equipment. Some expressed a wish for a broader selection of courses and more external collaboration, training in the health services and internships. In the final years of the programme, several strategies to increase enrollment of Nordic students were implemented, including increased marketing also with student ambassadors [19], admitting students with a wider range of backgrounds and increasing net-based teaching, and having preparatory courses for Master theses to increase completion rates. In 2017, the ‘Technology’ field of the programme was closed. This cooccurred with the start up in 2018 of a new field of health technology studies at the Department of Informatics at the university. In 2018, a revision process was initiated R. Wynn and G. Ellingsen / The Master Study in Telemedicine and E-health 221 with an aim to update the study program, now increasing the focus on Nordic students. However, also this process was terminated in 2019. 5. Conclusions The importance of e-health is increasing and there is a need for educational programmes focusing on e-health. In this paper we describe the Master Study in Telemedicine and eHealth at the University of Tromsø, Norway. We discuss the background of the programme, its development and significant accomplishments and why it nevertheless was closed. Future Master programmes in e-health may benefit from drawing on our experiences. References [1] The Times Higher Education World University Rankings 2020. https://www.timeshighereducation.com/world-university-rankings/uit-arctic-university-norway [2] Evalueringsrapport Master i Telemedisin og e-helse. Helsefak, UiT, 2017. [3] Wynn R, Oyeyemi SO, Budrionis A, Marco-Ruiz L, Yigzaw KY, Bellika JG. Electronic Health Use in a Representative Sample of 18,497 Respondents in Norway (The Seventh Tromsø Study - Part 1): Population-Based Questionnaire Study. JMIR Med Inform. 2020;8(3):e13106. [4] Wynn R, Gabarron E, Johnsen JK, Traver V. Special Issue on E-Health Services. Int J Environ Res Public Health. 2020;17(8):2885. [5] Oyeyemi SO, Wynn R. The use of cell phones and radio communication systems to reduce delays in getting help for pregnant women in low- and middle-income countries: a scoping review. Glob Health Action. 2015;8:28887. [6] Oyeyemi SO, Wynn R. Giving cell phones to pregnant women and improving services may increase primary health facility utilization: a case-control study of a Nigerian project. Reprod Health. 2014;11(1):8. [7] Wynn R, Hagen K, Friborg O. Videoconferencing at a centre for rare disorders: user satisfaction and user participation. Acta Paediatr. 2012;101(2):e83-e85. [8] Meum T, Wangensteen G, Soleng KS, Wynn R. How does nursing staff perceive the use of electronic handover reports? A questionnaire-based study. Int J Telemed Appl. 2011;2011:505426. [9] Adjorlolo S, Ellingsen G. Readiness Assessment for Implementation of Electronic Patient Record in Ghana: A Case of University of Ghana Hospital. J Health Inform Develop Countries. 2013;7(2): 128140. [10] Darko-Yawson S, Ellingsen G. Assessing and Improving EHRs Data Quality through a Socio-technical Approach. Procedia Computer Science. 2016;58: 243-250. [11] Birkemose M, Ellingsen G. How to evaluate telemedicine projects in clinical practice? Int J Integr Care. 2015;15:7. [12] Acharibasam JW, Wynn R. Telemental Health in Low- and Middle-Income Countries: A Systematic Review. Int J Telemed Appl. 2018;2018:9602821. [13] Acharibasam JW, Wynn R. The importance of cultural awareness when planning and implementing telepsychiatric services. Rural Remote Health. 2018;18(3):4724. [14] Oyeyemi SO, Gabarron E, Wynn R. Ebola, Twitter, and misinformation: a dangerous combination? BMJ. 2014;349:g6178. [15] Bäckström S, Wynn R, Sørlie T. Coronary bypass surgery patients' experiences with treatment and perioperative care - a qualitative interview-based study. J Nurs Manag. 2006;14(2):140-7. [16] Wynn R, Oyeyemi SO, Johnsen J-AK, Gabarron E. Tweets are not always supportive of patients with mental disorders. Int J Integr Care. 2017;17(3):A149. [17] Bhatta R, Ellingsen G. Opportunities and challenges of a rural telemedicine program in Nepal. J Nepal Health Res Council. 2015;13: 149-153. [18] Stranden AL. Vil at utenlandsstudenter skal betale. Forskning.no (webpage). Published 8 October, 2014. https://forskning.no/forskningsfinansiering-forskningspolitikk/vil-at-utenlandsstudenter-skalbetale/537659 [19] Ready for the challenge? Master in telemedicine and E-health. https://vimeo.com/40071825. 222 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200727 Health Informatics Solutions in Response to COVID-19: Preliminary Insights from an International Survey Seyedeh-Samin BARAKATI a, Maxim TOPAZ b, Laura-Maria PELTONEN c, James MITCHELL d, Dari ALHUWAIL e, Tracie RISLING f and Charlene RONQUILLO a,1 a Ryerson University b Columbia University, New York, USA c Department of Nursing Science, University of Turku, Finland d School of Computing and Mathematics, Keele University, UK e Information Science Department, Kuwait University, Kuwait f University of Saskatchewan, Saskatoon, Canada Keywords. medical informatics, COVID-19, pandemics, patient care technology 1. Introduction Organizations and governments have prioritized the implementation of health information technologies (HIT) as a key tool in addressing the impacts of the COVID19 pandemic. With organizations shifting focus on high priority procedures, little opportunity is left for documentation and tracking the nature and scope of these technologies; much of this information is only known anecdotally. The aim of this ongoing study is to facilitate information sharing in the international health informatics community by collecting, synthesizing, and sharing information about the nature of HIT use in response to the COVID-19 pandemic. 2. Methods This was a cross-sectional, exploratory, descriptive study that surveyed health informatics professionals internationally. A Web-based survey was developed using best practice guidance, in consultation with health informatics experts [1; 2]. The survey consists of 9 open-ended questions that cover the nature of HIT being used as part of the pandemic response. Ethical approval was obtained via Ryerson University Research Ethics Board. Convenience and snowball sampling were used. The sample comprised health informatics professionals who are involved with the development and deployment of HIT in healthcare settings. Thematic descriptive analyses of preliminary results were conducted to identify salient themes in the narrative survey responses [3]. In this paper, preliminary findings from data collected in May 2020 1 Charlene Ronquillo, Daphne Cockwell School of Nursing, Ryerson University, 350 Victoria Street, Toronto ON, M5B 2K3, Canada; E-mail: cronquillo@ryerson.ca. S.-S. Barakati et al. / Health Informatics Solutions in Response to COVID-19 223 are reported in response to the question: “What types of HIT are being used to address COVID-19 in the setting where you work?” 3. Results and Discussion Fifty responses from eleven countries were analyzed. The majority of participants were health informaticians (n=24), in IT related roles (n=8) and management/decision making positions (n=8). Three themes were identified in response to the types of HIT being used to address the COVID-19 pandemic. The first theme related to technologies for working remotely with patients and colleagues. For example, telehealth was reported to be deployed both within the care facility to minimize direct contact with patients and to connect with patients at home. The second theme pertained to technologies used in data collection, distribution, and analysis. Responses highlighted modifications made to electronic health record (EHR) systems to streamline data for reporting and pandemic planning. The third theme included technologies that were specifically developed to address COVID-19. This category included both expected (e.g. newly developed clinical decision support tools and standard terminology to be integrated into EHRs) and novel technologies (e.g. use of drones used for fever assessment). The types of HIT used by participants reflect necessary tools to respond to the challenges created by COVID-19 (e.g. remote working resulting from physical distancing requirements). The stated importance of data collection, management, and analytics tools solidify the power of data and the data-driven nature of healthcare as a key factor in planning and decision making in health care systems. The development of pandemic-specific solutions highlights the need to be nimble and innovative in order to mitigate the challenges posed by COVID19, although there is little overlap between the nature and use of HIT found in this study as compared to more visionary possibilities for HIT use as described elsewhere [4]. 4. Conclusion The preliminary results of this study provide insight into the diverse application of HIT for addressing COVID-19. This is consistent with the claims made by the World Health Organization which has identified HIT as “one of the most promising approaches to address this challenge in modern societies” [5]. Future research should advance the dialogue on HIT and pandemic planning with a focus on addressing concerns and creating clear actionable directives. References [1] [2] [3] [4] [5] Dillman DA, Smyth JD, and Christian LM. Internet, phone, mail, and mixed-mode surveys: the tailored design method, John Wiley & Sons, 2014. Sue VM and Ritter LA. Conducting online surveys, SAGE publications, 2011. Vaismoradi M, Turunen H, and Bondas T. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & Health Sciences 2013; 15: 398-405. Mahmood S, Hasan K, Colder Carras M, and Labrique A. Global Preparedness Against COVID-19: We Must Leverage the Power of Digital Health. JMIR Public Health Surveill 2020; 6, e18980. World Health Organization, Responding to community spread of COVID-19: interim guidance, 7 March 2020, in, World Health Organization, 2020. 224 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200728 Continuity of Health, Citizen Empowerment as Key Driver a Jacob HOFDIJKa, 1 and Felix CILLESSENb Casemix-CQT Zorg en Gezondheid, Utrecht, The Netherlands b Hospital Rivierenland Tiel, the Netherlands Abstract. Self-management for prevention and care will play a significant role in the transition to apply person-centered care. Interoperability requirements, an overarching care plan, integration of social determinants, and the focus on prevention are important ingredients in the vision on its implementation. Keywords. Care continuity, integrated health care systems, health maintenance 1. Introduction The global strategy on people-centered care (PCC) and integrated health services from the World Health Organization is challenging [1]. Citizens have to play a major role in the fundamental paradigm shift it requires. The Covid-19 pandemic stimulates citizens as co-creator of the overarching objectives of their health and wellness. An interoperable holistic problem list (HPL) with interoperable subjective and objective data is vital for patients and providers in our vision to properly manage all health and social issues [2]. The most important value of any health care system should be the maintenance of health [3]. What next step is needed to intensify the PCC implementation? 2. Methods Since the European Medical Informatics Conference 2012 in Pisa, various international workshops have been held in which the requirements for integrated care and the shift towards PCC were discussed. At the 2020 ICIMTH conference in Athens, the scope was broadened to prevention. To guarantee the continuity of care in combination with social distancing, health organizations are ramping up their telehealth. As the determinants of health are not only dependent on medical factors, but much more on factors like genetics, biology, life style, and social characteristics, a multifaceted less medical approach of prevention is required. From the concept of a HPL, the patient’s story plays a key role to PCC. 1 Corresponding Author, Jacob Hofdijk, Casemix-CQT Zorg en Gezondheid, Utrecht, The Netherlands; Email: jacob.hofdijk@casemix.nl J. Hofdijk and F. Cillessen / Continuity of Health, Citizen Empowerment as Key Driver 225 3. Results It takes technical and semantical interoperability of data and empathic design of systems, to shift to a technology enabled continuity of care concept. To combine them with the overarching care plan and the societal incentive program does form a solid foundation. The available subjective and objective data can be linked to an active PCC plan to reach and document common agreed health and/or social objectives. This approach helps the patient safeguard their health and with its providers better manage their active problems. The Blue Line Statement of the 2015 The Hague PCSI Conference recommends to continuously develop and formalize these principles as requirements for holistic, person-centered, integrated care systems. 4. Discussion To manage the social determinants of health, sensitive personal data on social, lifestyle, and genetics issues have to be collected by citizens. The patient’s story contributes to the continuity and transparency of the provider–patient partnership [4]. Full participation of the citizen will be dependent on the added value it brings to their health and care [5]. Policymakers and health system leaders need to adopt the necessary requirements by creating a societal incentive framework, like innovative funding to enable this vision. It will provide a base for regionally arranging the integrated service delivery as proposed by the WHO strategy for the benefit of both health maintenance and health and social care. 5. Conclusion Applying the transition to PCC we promote the concept of continuity of health, which could start at birth and should be maintained all along the life path of the person. For each phase of life, the primary focus of the overarching holistic health and care plan should be on managing health and wellbeing, taking into account the actual risk factors. Continuity of health should thus be the driver to motivate citizens as data producers and service co-creators actively managing their health. Trust, participation, engagement, education, demonstrated benefits, and other sorts of incentives, associated with healthy life, will support this vision and stimulate the transition to its implementation. References 1. WHO global strategy on people-centred and integrated health services 2015 [Available from: http://apps.who.int/iris/bitstream/10665/155002/1/WHO_HIS_SDS_2015.6_eng.pdf?ua=1&ua=1, Accessed July 9, 2020. 2. Cillessen FH, Hofdijk J. Transition Requirements from Problem List to an Overarching Care Plan for the Support of Person-Centered Care. Stud Health Technol Inform. 2020;272:292-5. 3. Porter ME. What is value in health care? N Engl J Med. 2010;363(26):2477-81. 4. Ekman I, Swedberg K, Taft C, Lindseth A, Norberg A, Brink E, et al. Person-centered care--ready for prime time. Eur J Cardiovasc Nurs. 2011;10(4):248-51. 5. Cantor MN, Thorpe L. Integrating Data On Social Determinants Of Health Into Electronic Health Records. Health Aff (Millwood). 2018;37(4):585-90. 226 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200729 Digital Allergy Card: Design and Users’ Perceptions Rhode Ghislaine NGUEWO NGASSAMa,b 1, Linnea UNGd, Roxana OLOGEANUTADDEIc, Pascal DEMOLYe , Jorick LARTIGAUc and Anca M. CHIRIACe a Montpellier Research Management, University of Montpellier, France b Pikcio services, France c Toulouse Business School, France d Montpellier Institute Alexander Grothendieck, France e University Hospital of Montpellier, France Abstract. This paper presents the design and the users’ perceptions of a Digital Allergy Card for recording, sharing and tracing information on drug allergies. Keywords. Digital allergy card, mobile application, healthcare, allergy, mhealth. 1. Introduction The reliability of the label "drug allergic" is important because it guides the physician for drug prescription and treatment administration [1]. Therefore, the documentation of a drug allergy should ideally be detailed and allow for proper classification. Currently, there is no solution to reliably report drug allergies in all European countries, despite the various existing solutions such as oral transmission from patient to physician which is limited when the patient is unconscious, paper cards that can easily get lost, and drug allergy reports held in Electronic Health Records (EHR) that are not accessible to other hospitals [2]. Therefore, the European Academy of Allergy and Clinical Immunology (EAACI) encourages development of digital solutions for reliable and accurate documentation on drug allergies [3]. Our paper aims to present 1) the design of such a digital solution as a Digital Allergy Card (DAC), 2) the users’ perceptions related to this DAC. 2. Design process According to the EAACI’s recommendation to design and implement a DAC to better manage drug allergy information we rely on the need to go through such a project. Firstly, needs were identified based on an analysis of the allergy information process, which highlighted the fact that at several levels of the process, information may be missing, unreliable or lost. Secondly, the modeling of the solution has resulted in the design of interactive mock-ups which were tested by six patients. The feedback from this evaluation was taken into account for the development of the first version of the app. The evaluation of this first version took place with five patients and five physicians. 1 Corresponding author, Montpellier Research Management, University of Montpellier, Place Eugène Bataillon – CC 19001, 34095 Montpellier cedex 5 – France ; E-mail :rhode-ghislaine.nguewongassam@etu.umontpellier.fr R.G. Nguewo Ngassam et al. / Digital Allergy Card: Design and Users’ Perceptions 227 At the end of this process, we obtained an app that allows to collect, access, secure and trace drug allergy information under the control of the patient, except in emergencies situations where the physician may exceptionally access the patient's account without his or her authorization. To achieve this level of security, control and decentralization of data management in this app, we used a permissioned blockchain [2]. We have provided for physicians the possibility to use the app directly or to access it from their usual working tool. On the other hand, the evaluation iterations were conducted though functional testing with eleven patients (six for mock-ups and five for the app) and five physicians only for the app. Users testing was performed through interviews after the test of the app. We started with open questions on the interviewees’ personal context concerning drug allergy and allergy card and their representation about a DAC, and then we asked them about their perceptions of the mock-ups or app that they have tested. Data analysis was performed using the grounded theory [4], as follows: data were first coded by a first code, closed to respondents’ words; then, a more general coding was performed. Hence the users’ perceptions presented below. 3. Users’ Perceptions At the end of the interviews, both patients and physicians reported insufficient usability and time consuming process for registration into the app. In addition, they stated several issues related to ease of use and usability. For example, a patient said that allergy information should be presented directly when the app was launched. The general practitioners outlined need of ease of use and interoperability with the clinical systems that they had already used. Furthermore, physicians reported the need that the app provides guidelines on drug allergy diagnosis while patients reported the need to be led by a physician in recording information on drug allergy suspicion. In other words, they both feared to make information errors which would led to diagnosis errors. Moreover, for patients, the app was considered useful especially in mobility situation (travelling) or emergency as their daily care was provided by the same practitioners who had known their clinical history. 4. Conclusions This paper presented the user centered design of DAC using a permissioned blockchain to record and share patients’ information on drug allergy with high traceability and reliability. We reported here the users’ perceptions on usability and usefulness. These perceptions are important both to improve the app and to target the communication for patients, i.e. especially for travelling and emergency situations. References [1] Ferner R. and McGettigan P. The patient who reports a drug allergy. BMJ, 2020. 368. [2] Nguewo Ngassam RG, Ologeanu-Taddei R, Lartigau J and Bourdon I., A Use Case of Blockchain in Healthcare: Allergy Card, in Blockchain and Distributed Ledger Technology Use Cases. 2020, Springer. p. 69-94. [3] Brockow K, et al. Drug allergy passport and other documentation for patients with drug hypersensitivity– An ENDA/EAACI Drug Allergy Interest Group Position Paper. Allergy, 2016; 71(11): 1533-1539. [4] Corbin JM and Strauss A. Grounded theory research: Procedures, canons, and evaluative criteria. Qual socio. 1990; 13(1): p. 3-21. 228 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200730 Physical Activity in Cardiac Rehabilitation: Towards Citizen-Centered Digital Evidence-Based Interventions Johanna GUTENBERGa,b1, Stefan Tino KULNIKa, Rada HUSSEINa, Thomas STÜTZa, Josef NIEBAUER a,c and Rik CRUTZENb a Ludwig Boltzmann Institute for Digital Health and Prevention, Salzburg, Austria b CAPHRI Department of Health Promotion, Maastricht University, Maastricht, The Netherlands c University Institute of Sports Medicine, Prevention and Rehabilitation, Paracelsus Medical University, Salzburg, Austria Abstract. Physical activity is a vital part of cardiac rehabilitation (CR). However, heart-healthy physical activity levels in people with cardiovascular disease drop significantly after CR. This exploratory study employs qualitative and survey methods within a co-creation approach. The aim is to understand the mechanisms of healthy behavior and habit formation in order to create a novel evidence-based (post-)rehabilitation approach that employs digital means to sustain long-term physical activity levels in people with cardiovascular disease. Keywords. Cardiac rehabilitation, physical activity, behavior change, citizencentered, empowering care 1. Introduction Physical activity is a central part of modern cardiac rehabilitation (CR) after a cardiac event in people with cardiovascular disease (CVD) [1]. However, evidence of secondary prevention in people with CVD consistently demonstrates challenges in maintaining improved physical activity behavior after completion of CR [2]. The current Coronavirus Disease 2019 (COVID-19) adds another layer of complexity to the problem due to widespread disruption and discontinuation of training and physical activity programs. In Austria and many other countries, evidence-based digital interventions, e.g., digital CR platforms that have been proven effective in mitigating adverse effects of CVD, are not widely available [3]. Our study aims to close this gap under consideration of the national (Austria) and local (Salzburg) healthcare context by exploring 1) how digital technologies can support citizens to stay physically active, 2) what types of technologies citizens use, and 3) how citizens recognize and use digital health in CR and after. The accumulated data will be used to co-create evidence-based digital interventions that eventually motivate and sustain heart-healthy physical activity behavior in people with CVD to actively support their rehabilitation and self-management process. 1 Corresponding Author, Johanna Gutenberg, Ludwig Boltzmann Institute for Digital Health and Prevention, Lindhofstrasse 22, 5020 Salzburg, Austria, E-mail: johanna.gutenberg@dhp.lbg.ac.at J. Gutenberg et al. / Physical Activity in Cardiac Rehabilitation 229 2. Method This study will use an exploratory mixed-methods approach. Data collected through semi-structured qualitative interviews and workshops (n = 75) will sensitize the work towards various experiences and views of local people with CVD. This will also inform a survey (n ≥ 250) to gain comprehensive insights from a larger cohort into their physical activity behavior and use of technologies to stay physically active. Once the data is collected, a digital platform prototype is designed. We will apply a co-creation approach [4] with citizens in order to place citizens in the foreground of the design process, not as passive recipients of a ready-made digital intervention, and to refine the prototype design, the interventions, test usability, and reveal usability flaws. The collected data will be kept anonymous and confidential. The data processing will comply with the European General Data Protection Regulations (GDPR). 3. Results Continuous access to (post-)rehabilitation resources is vital for maintaining heart-healthy levels of physical activity in people with CVD. This study, running from 2020-2024, aims to 1) understand the contextualized mechanisms of healthy behavior and sustainable healthy habit formation in CR, 2) create evidence-based digital interventions as compared to standard CR interventions, and ultimately 3) provide on-demand digital access and support, as well as participatory empowering care [5]. 4. Discussion and Conclusion The present study is a concrete example of a research study that supports a new (post-) rehabilitation approach through digital means that potentially increase access and improve services in healthcare. The study is at an early stage, and different facilitating and hindering factors such as structural barriers, levels of education, health literacy, techsavviness, and unknown-unknowns need to be considered. Involving citizens actively as co-creators in CR is integral in addressing these factors. References [1] [2] [3] [4] [5] Niebauer J, et al. Outpatient cardiac rehabilitation: The Austrian model. Eur J Prev Cardiol. 2013;20(3):112-6. Alves AJ, Viana JL, et al. Physical activity in primary and secondary prevention of cardiovascular disease: Overview updated. World J Cardiol. 2016;8(10):575. Khera A, et al. Continuity of care and outpatient management for patients with and at high risk for cardiovascular disease during the COVID-19 pandemic: A scientific statement from the American Society for Preventive Cardiology. Am J Prev Cardiol. 2020;1:100009. Dowie J, Kaltoft M. The Future of Health Is Self-Production and Co-Creation Based on Apomediative Decision Support. Med Sci. 2018;6(3):66 Lupton D. The digitally engaged patient: Self-monitoring and self-care in the digital health era. Soc Theory Heal. 2013;11(3):256-70. 230 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200731 Citizens’ Opinions About a Digital Health Insurance Record George KOSTIKIDISa,1, Parisis GALLOSa, Ioannis S. TRIANTAFYLLOUa and Vassilis PLAGIANAKOSa a Department of Computer Science and Biomedical Informatics, University of Thessaly, Lamia, Greece Abstract. An Electronic Health Insurance Record (EHIR) could give all the information needed to the insured citizens, informing them about the history of benefits and the health expenses. The aim of this work is to evaluate a Digital Health Insurance Record system as well as to explore the benefits of using this system, both for society and for each citizen individually. A quantitative survey was carried out using a questionnaire shared among 180 people in Greece in 2019. The questionnaire consisted of 25 closed-ended questions, 3 of which related to demographics and the remaining 22 related to the use and benefits of use of the EHIR system. Most of all people who took part in this study believe that EHIR can contribute positively giving both social benefits and benefits for the patients. An important finding of the study is the concern expressed by respondents about the security of the system in the management of sensitive personal data. Based on citizens’ opinions a Digital Health Insurance Record can provide a lot of benefits to citizens and to the society as well as to the national health insurance system. Keywords. Health Insurance Record, Evaluation Study 1. Introduction Most of the times, the access to Health Insurance data can be done through the Electronic Health Records [1], providing sufficient but not all the data regarding the healthcare provision [2]. An Electronic Health Insurance Record (EHIR) could give all the information needed to the insured citizens, informing them about the history of benefits and the health expenses for which they have received compensation [3]. The purpose of this work is to evaluate a Digital Health Insurance Record system as well as to explore the benefits of using this system, both for society and for each citizen individually. 2. Methods In order to achieve this objective, a quantitative pilot survey was carried out using a selfdeveloped questionnaire based on previous related studies [4]. The questionnaire was shared, in paper form, among 180 randomly selected people who had visited healthcare services in 3 cities (Kozani, Thessaloniki and Lamia) in Greece in 2019. The 1 Corresponding Author, George Kostikidis, Student at the Department of Computer Science and Biomedical Informatics, University of Thessaly, Lamia, Greece; E-mail: kostikidis1@gmail.com. G. Kostikidis et al. / Citizens’ Opinions About a Digital Health Insurance Record 231 questionnaire consisted of 25 closed-ended questions, 3 of which related to demographics and the remaining 22 related to the use and benefits of use of the EHIR system. More specifically, 9 were bivalent and the 13 “Five Likert” scale. About the reliability of the tool Cronbach's A was 0.904. Descriptive statistics and corellations between the participants’ opinions about the EHIR and their personal characteristics were examined using SPSS. 3. Results and Discussion The results of this preliminary survey show that the largest proportion of participants were young adults and middle-aged (18-55 years old are the 84,8% of the sample). Their level of education was also quite high (71,1% completed high education studies), whereas in terms of their professional activity the majority consisted of public/private employees (54,4%) and students (22,2%). Regarding knowledge of the EHIR, respondents present an evenly distributed picture (yes=48,3%). Unfortunately, less than 20% of the participants use the system. Most participants (81,7%) seem to have all the necessary resources to access and use the EHIR. The majority of the participants believe that EHIR can make a positive contribution by both social benefits and benefits for the patient. An important finding is the concern expressed by respondents about the security level of the system in the personal data management. Comparing the responses about the security with participants’ age, it is found to be significant (p=0,005) with correlation coefficient -0,212. All the other tested hypotheses were not confirmed. 4. Conclusions According to the aforementioned results, citizens need to be informed properly about the EHIR system and the data security of this system. Based on citizens’ opinions, a Digital Health Insurance Record can provide a lot of benefits to citizens and to the society as well as to the national health insurance system too. Under this frame, a dissemination of the system is recommended, because the majority of the citizens have positive opinions about the system but few are use it. Future work may include a wider scale survey, based on this pilot study, after the dissemination of the system, in order to record more accurately the citizens’ opinions about a digital health insurance record system. References [1] [2] [3] [4] Hatch B, Angier H, Marino M, Heintzman J, Nelson C, Gold R, et al. Using electronic health records to conduct children’s health insurance surveillance. Pediatrics 2013;132(6);e1584-e1591. Heintzman J, Marino M, Hoopes M, Bailey SR, et al. Supporting health insurance expansion: do electronic health records have valid insurance verification and enrollment data?. Journal of the American Medical Informatics Association 2015; 22(4): 909-913. Petmesidou M. Challenges to Healthcare Reform in Crisis-Hit Greece. e-cadernos CES 2019; 31. Tavares J, Oliveira T. Electronic health record patient portal adoption by health care consumers: an acceptance model and survey. Journal of medical Internet research 2016;18(3). 232 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200732 Semi-Automated Method to Generate Simulated Clinical Data from OpenEHR Platform – Think!EHR a Abdul Mateen RAJPUTa,1 Köln University Hospital, Köln, Germany Keywords. OpenEHR, Think!EHR, KNIME, ETL Process 1. Introduction OpenEHR is openly available and vendor independent technology based on two-level modelling; Reference Model and Archetype Model [1]. Think!EHR is an OpenEHR platform used by many HiGHmed partner sites, it offers clinical data storage, management, querying, retrieval and exchange. To work with Think!EHR, Templates have to be developed by combining different Archetypes. Examples of Archetypes are Blood pressure and Body temperature, a Template example would be Vital Signs. Think!EHR provides an example of data values, once a Template is uploaded to the platform or the cloud instance i.e. EHRScape.com. Since our aim was to generate data from all the Templates and of numerous patients, an automated approach was needed. Our tool of choice was KNIME (Konstanz Information Miner) [2], which is a freely available software. It has many dedicated nodes which have predefined functionalities and readily available nodes for different tasks. 2. Methods Our environment was Think!EHR (Ver. 2.45) running on a Windows machine. In our example we used Localhost because the system was installed locally. The same method can be applied by using Ehrscape REST API URL [3]. However, one needs to change the base URL accordingly as we used localhost:8081 for this paper. Following steps have to be performed to generate and retrieve data of a patient: 1. First step was to POST Operational Template in OPT format to Think!EHR: o Post https://localhost:8081/rest/v1/template 2. This Get method retrieves the complete list of Templates available Think!EHR: o Get http://localhost:8081/rest/v1/template 3. To retrieve example patient data based on template (step 1), following link was used. Actual OPT can also be retrieved, by replacing "example" with "opt": o Get http://localhost:8081/rest/v1/template/[TemplateId]/example 1 Corresponding Author, Abdul-Mateen Rajput, Universitätsklinikum Köln, Kerpener Str. 62, 50937 Köln, Germany; E-mail: Abdul.mateen@uni-koeln.de. A.M. Rajput / Semi-Automated Method to Generate Simulated Clinical Data 233 To avoid manual querying, repeating step 3, for hundreds of patients’ data, the following workflow (shown in figure 1) has been developed to automate the process. Figure 1. This workflow generates data of 10 patients based on the Template Pankreaskarzinom. From left, the first node creates an empty table. Second node starts the loop and numbers of loop can be set in the setting, in our case it was 10. Third node uses the link mentioned in text in step 3 and retrieve the data of a patient. Fourth node checks the conditions whether number of loops have been executed and collects the data. Once all the loops are executed, it ends the loop. Fifth node write the JSON files to given directory. 3. Results The following link was used to retrieve the data of 10 patients based on “Pankreaskarzinom” Template: - Get http://localhost:8081/rest/v1/template/Pankreaskarzinom/example Figure 2 shows the output of the workflow, where each line represents a new record based on the Template as discussed earlier. The same can be done with all other Templates with just an additional node which loops over the name of Templates. Figure 2. The output of the workflow, where each row contains a record of a patient with the Template "Laborbefund_Pankreaskarzinom". 4. Discussion and Conclusions The approach presented in this paper shows the possibility to generate clinical data from Think!EHR. The approach is based on REST API methods and the analytical tool KNIME was used to automated the data retrieval process. Data generated with this approach is not only compatible to all OpenEHR system but it has also appropriate data types and values representing the real patient data. This would reduce the need of real patient data if one needs to test the health IT systems. Acknowledgment The project is funded by the German Federal Ministry of Education and Research (BMBF, grant id: 01ZZ1802U). References [1] [2] [3] Min L, Tian Q, Lu X, Duan H. Modeling EHR with the openEHR approach: an exploratory study in China. BMC Medical Informatics and Decision Making. 2018;18:75. KNIME | Open for Innovation. https://www.knime.com/. Accessed 4 Mar 2020. API Explorer. https://www.ehrscape.com/api-explorer.html. Accessed 31 Mar 2020. 234 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200733 Standardizing the Unit of Measurements in LOINC-Coded Laboratory Tests Can Significantly Improve Semantic Interoperability Abdul Mateen RAJPUTa,1, Sarah BALLOUT b and Cora DRENKHAHN c,d a Köln University Hospital, Köln, Germany b Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, Hannover, Germany c IT Center for Clinical Research (ITCR-L), University of Lübeck, Germany d Institute of Medical Informatics (IMI), University of Lübeck, Germany Keywords. Semantic Interoperability, LOINC, UCUM 1. Introduction Within healthcare environments, diagnostic and clinical data comes from many different systems, which frequently leads to an inconsistent presentation of important information. Controlled standard terminologies such as Logical Observation Identifiers Names and Codes (LOINC), allow some of the data inconsistency problems to be resolved. In the German Medical Informatics Initiative (MI-I), LOINC is used for sharing laboratory data across different departments and different university hospitals [1], [2]. Therefore, a list of 300 frequently used LOINC codes was composed, enabling a common basis for the mapping of site-specific terms and measurements to LOINC at all participating sites. Despite being a standardized coding system LOINC has been shown to leave some ambiguity in the tests coded with it, particularly by defining a “kind of quantity/property” instead of the unit of measurement itself [3], as shown in Table 1. In this short paper inter-mapping variability arising from the ambiguous property definition is investigated. Table 1. Two different LOINC Terms for similar measurements, differing only in their specified property. LOINC Code 2345-7 14749-6 LOINC Term Glucose [Mass/volume] in Serum or Plasma Glucose [Moles/volume] in Serum or Plasma Example Unit mg/dL mmol/L 2. Methods In the HiGHmed consortium participating university hospitals are required to map their local laboratory terms to LOINC based on the agreed TOP300 list. The mapping was 1 Corresponding Author, Abdul-Mateen Rajput, Universitätsklinikum Köln, Kerpener Str. 62, 50937 Köln, Germany; E-mail: Abdul.mateen@uni-koeln.de. A.M. Rajput et al. / Standardizing the Unit of Measurements in LOINC-Coded Laboratory Tests 235 done by domain experts, either using RELMA [4] or the LOINC web interface. Afterwards, site-specific mapping tables, with different locally used naming conventions, were joined by using Inner Join so only matching rows were included in further work. The unique identifiers were LOINC codes, so we could identify different test names and additional information including the unit of measurement that were associated with the same concepts. Entries were evaluated for discrepancies which were further analyzed. 3. Results Conflicts in the resulting table were found both for the name and also for the unit of measurements assigned to the same code. For 118 out of 186 entries the same unit was defined at both sites, whereas 67 disparities could be divided into two categories: 1) Different laboratories used two slightly different versions of the same unit of measurements. Table 2 shows “sec” and “sek” being used for time points. 2) The unit is reported in different granularities e.g. gram per liter versus milligram per milliliter. Examples can be found in rows two and three of Table 2. Table 2. Examples of disparate site-specific annotations mapped to the same LOINC LOINC Code 3243-3 3013-0 19113-0 Name (Site 1) Thrombinzeit Thyreoglob., hTG IgE Name (Site 2) Thrombinzeit (CP) Thyreoglobulin (S) Immunglobulin E (HP) Unit (Site 1) sek µg/l IU/ml Unit (Site 2) sec ng/ml kU/l 4. Discussion and Conclusion Differences in site-specific reporting are expected but can’t be eliminated solely by mapping to LOINC. Therefore, using Unified Code for Units of Measure (UCUM) can significantly improve semantic interoperability. Employing UCUM would not only eliminate minor disparities as described in category 1) but could also enable the automated conversion between related units differing in granularity [5]. Acknowledgment The project is funded by the German Federal Ministry of Education and Research (BMBF, grant id: 01ZZ1802U). References [1] [2] [3] [4] [5] Semler SC, Wissing F, and Heyder R. German Medical Informatics Initiative. Methods Inf. Med., 2018; 57(1); e50–e56, 2018. Semler SC. LOINC: Origin, development of and perspectives for medical research and biobanking – 20 years on the way to implementation in Germany. J. Lab. Med. 2019; 43(6): 359–382, 2019, doi: 10.1515/labmed-2019-0193. Drenkhahn C and Ingenerf J. The LOINC Content Model and Its Limitations of Usage in the Laboratory Domain, Stud. Health Technol. Inform. 2020; 270: 437–442, Jun. 2020, doi: 10.3233/SHTI200198. “RELMA,” LOINC. https://loinc.org/relma/ (accessed Jul. 15, 2020). Hauser RG, Quine DB, Ryder A, Campbell S. Unit conversions between LOINC codes. J. Am. Med. Inform. Assoc. JAMIA. 2017; 25(2): 192–196, Jun. 2017, doi: 10.1093/jamia/ocx056. 236 Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI200734 The Drug Addicts’ Usage of Information and Communication Technologies a Christos MIMIGIANNIS1a, Parisis GALLOSa and John MANTASa Health Informatics Laboratory, School of Health Sciences, National and Kapodistrian University of Athens, Greece Abstract. Information and Communication Technologies (ICT) are broadly used to support people’s daily needs. Individuals addicted to psychoactive drugs sometimes present social exclusion as well as, limitations to the usage of ICT such as Internet, devices and applications. The aim of this paper is to present the findings of a pilot study related to the use of Information and Communication Technologies by Drug Addicts. A survey was conducted on 204 users of psychoactive substances. According to the results, the majority of the drug addicts seem to use ICT on a daily basis, showing their preference on Smartphones compared to other devices. The Internet access and the usage of Social Media and Communication Networks by addicted individuals is quite high, probably because they are willing to reintegrate into the society through Social Networks. Age is often related to the usage of ICT on Drug Addicts. Keywords. Addicted Individuals, ICT Usage 1. Introduction Information and Communication Technologies (ICT) are broadly used to support people’s daily needs. Internet and electronic devices have become a part of the everyday life for the majority of the people in society. Individuals addicted to psychoactive drugs sometimes present social exclusion as well as, limitations to the usage of ICT such as Internet, devices and applications [1,2]. The aim of this paper is to present the findings of a pilot study related to the use of Information and Communication Technologies by Drug Addicts. 2. Methods To investigate the usage of ICT by addicted individuals, a self-developed questionnaire was constructed based on previous surveys [3] and was distributed among 204 users of psychoactive substances, who were in the reception and treatment rooms of 12 KETHEA (Therapy Centre for Dependent Individuals) centres (after KETHEA’s research study permit) in three-month period in Greece. The questionnaire was anonymous and it was in Greek language. It included questions related to demographics, some personal characteristics, and the usage of the current communication devices and technologies. 1 Corresponding Author, Christos Mimigiannis, MSc; E-mail: christos.mimigiannis@hotmail.com. C. Mimigiannis et al. / The Drug Addicts’ Usage of Information and Communication Technologies 237 All participants were over 18 years old. The data analysis included Descriptive Statistics and Correlations, and was conducted using the SPSS. 3. Results and Discussion The 89,2% (N=182) of the sample were males. The average age was 34,72 years old. The main substance used was heroin (N=86 / 42,4%), followed by cannabis (N=56 / 27,6%) and cocaine (N=42 / 20,8%). About the device usage, Smartphones was 79,4%, Laptop Computer was 41,7%, Desktop Computer was 36,8%, and Tablet was only 17,2%. Specifically, the 73,6% of the Smartphone users were using their device daily. 186 (91,6%) participants had access to the Internet and 82,2% were using it for Social Networks and Media. The majority of them (85,5%) were using Smartphones to access the above services. Comparing the age with the internet access was found to has a significant relation (p<0.01). The mean age of internet users were 33,2 years old and for non-users 45,7 years old. Additionally, Smartphone usage found to be related with age (p<0.01). The mean age of Smartphone users were 33 years old and for non-users 38,2 years old. No significant relations have been found between the above usages and the substances or gender. Also, neither age, gender, nor substances were related to the usage of Internet for Social Media and Networking. 4. Conclusions Based on the aforementioned results, the majority of the drug addicts seem to use ICT on a daily basis, showing their preference on Smartphones compared to other devices. The Internet access and the usage of Social Media and Communication Networks by addicted individuals is quite high, probably because some of them are socially excluded and they are willing to reintegrate into the society through Social Networks. Only the age, from personal characteristics of the addicts, is often related with the usage of ICT. A limitation of this study is that the sample was collected at therapy centres and does not include drug addicts who are not having any support. Future work may include the further investigation about the reasons of Internet use by drug addicts and their opinions whether ICT can be a valuable tool on the rehabilitation process. References [1] [2] [3] Baroni S, Marazziti D, Mucci F, Diadema E, and Dell’Osso L. Problematic Internet use in drug addicts under treatment in public rehab centers. World journal of psychiatry 2019; 9(3): 55. McClure EA, Acquavita SP, Harding E, and Stitzer ML. Utilization of communication technology by patients enrolled in substance abuse treatment. Drug and Alcohol Dependence 2013; 129(1-2): 145-150. Widyanto L, Griffiths M. An empirical study of problematic Internet use and self-esteem. International Journal of Cyber Behavior, Psychology and Learning (IJCBPL) 2011; 1(1): 13-24. This page intentionally left blank Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). 239 Subject Index addicted individuals 236 adoption 82 allergy 226 anonymization 37 application 77, 182 arrhythmia 127 audio segmentation 132 Bayesian networks 62 behavior change 228 behavior change theory 77 behavioral therapy 42 biostatistics 137 blockchain 17 breast cancer 107, 177 cardiac rehabilitation 228 care continuity 224 chronic lower respiratory diseases 32 chronic pain management 197 citizen 112 citizen-centered 228 classification 157 clinical decision support system(s) 1, 177 clinical deterioration 152 clinical practice guidelines 1, 107 cluster analysis 32 competencies 187 conflict of interest 47, 52 content analysis 67 coronary artery disease 182 COVID-19 22, 32, 112, 117, 212, 222 COVID-19 risk factors 6 cross-correlation 132 cybersecurity 167, 192 dashboard 27 data analytics 17 data collection 57 data integration 22 data lake 17 data privacy 137, 192 data protection 167 data visualization 12 de-identification 37 decision aid 47, 52 decision conflict scale 52 decision quality instrument 52 decision support 197 decision support systems 107 definition 67 device-to-device protocol 92 diagnostic data 87 diagnostic images 87 digital allergy card 226 digital health 67, 87, 167 digital services 147, 197 digitalization 127 disease 147 documentation 122 e-health 217 education 187 eHealth 127 EHR 87 elderly people 182 electronic health record system 157 electronic medical record 12 emergency department 152, 162 emergency medical service 27 empirical evaluation 52 empowering care 228 empowerment 212 error 157 ETL process 232 evaluation 187 evaluation study 230 exercise capacity 72 expectations 57 FAIR 37 FAIRification 37 feature extraction 117 federated learning 137 fitness trackers 142 game 182 gamification 77, 182 genomics 37 geriatrics 1 GraphQL 202 240 health 77, 182 health care 147 health care informatics 102 health data 167 health informatics 152 health information exchange 92 health information interoperability 177 health information systems 207 health insurance record 230 health maintenance 224 health service research 162 health technology 147 health-enabling technologies 57 healthcare 17, 226 HL7 FHIR 92 homecare 97 ICT Usage 236 imputation of data 117 incident reporting 157 information display 1 information system(s) 27, 207 insomnia 42 integrated health care systems 224 integration 17 interdisciplinary cooperation 187 interpretive reports 87 interview 57 IPDAS 52 IPDASi 47 ISO 11179 202 ISO 21526 202 k-means clustering 162 KNIME 232 knowledge graphs 6 knowledge representation 177 license attribution 37 LOINC 234 LSTM 152 machine learning algorithms 152 master programme 217 medical informatics 222 medication adherence 182 mental disorders 57 mentally ill persons 57 metadata 202 metadata repository 202 mHealth 42, 62, 77, 226 mobile 77, 182 mobile access 82 mobile application 212, 226 multi-criteria decision support 172 natural language processing 6, 112 normative 47 Norway 217 nursing homes 1 nursing informatics 122 occupational health services 207 OpenEHR 232 overdiagnosis 172 overtreatment 172 PACS 87 pain management 122 pandemic surveillance 22 pandemics 222 participatory design 207 patient acceptance of health care 57 patient care planning 107 patient care technology 222 patient guidance 127 patient incident reporting 102 patient portal(s) 82, 87 patient safety 157 patient-generated health data (PGHD) 12 patient-reported outcomes 12 personalised prediction 62 personalization 77 PHR 87 physical activity 228 policy 192 population trends 6 precision public health 22 preferences 47 pressure ulcer 1 principal component analysis 117 privacy 37, 142, 167 pseudo-anonymous data 117 pulmonary rehabilitation 72 qualitative content analysis 162 recurrent neural network 152 relation extraction 6 remote technologies 212 rheumatoid arthritis 62 rules 37 security 142 semantic interoperability 234 sleep disorders 42 social determinants of health 6 241 SSD 132 standardized nursing terminology 122 technology 97 telemedicine 72, 217 term mapping 67 the theory of planned behaviour 102 Think!EHR 232 time series 152 tracing 212 transfer delay 162 treatment 182 UCUM UHR urban health observatory user-centred design UTAUT video conference visualization wearable devices webinars welfare YouTube 234 87 22 197 97 197 27 142 187 97 112 This page intentionally left blank Integrated Citizen Centered Digital Health and Social Care A. Värri et al. (Eds.) © 2020 The European Federation for Medical Informatics (EFMI) and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). 243 Author Index Abdellatif, A. Akhlaghpour, S. Alhuwail, D. Ammar, N. Andrey, P. Ballout, S. Barakati, S.-S. Bellika, J.G. Belmin, J. Bettencourt-Silva, J.H. Blackledge, E. Blease, C. Blondon, K. Bouaud, J. Bowles, J.K.F. Brakefield, W.S. Cabrera, M. Chiriac, A.M. Christen, O.M. Chronaki, C. Cillessen, F. Crutzen, R. Cui, W. Curzon, P. Davis, R.L. Delgado, J. Demoly, P. Denecke, K. Di Biagio, A. Diehl, A. Doerstling, M. Dowie, J. Drenkhahn, C. Droegemueller, B. Ehrler, F. Ellingsen, G. Fagerlund, A.J. Fahmi, A. Falquet, G. Fatehi, F. Ficheur, G. Filiot, A. Finkelstein, J. 1 167 222 22 137 234 222 197 1 6 17 82 12, 182 1, 107, 177 17 22 32 226 27 187 224 228 32 62 22 37 226 27, 42 117 57 72 47, 52, 172 202, 234 57 12, 77, 182 217 197 62 77 67, 167 137 137 32, 72 Fredeng, A. 197 Gallos, P. 230, 236 Garcia-Castrillo Riesgo, L. 187 Giacomini, M. 117 Gleize, M. 6 Gligorov, J. 177 Gosetto, L. 77, 182 Gouvas, P. 92 Guézennec, G. 107, 177 Guinhouya, B. 112 Gutenberg, J. 228 Hägglund, M. 82 Hassandoust, F. 167 Haux, R. 57 Henner, A. 127 Hertenstein, E. 42 Hofdijk, J. 224 Humby, F. 62 Hussein, R. 228 Ingenerf, J. 202 Jeong, I.c. 72 Jochim, C. 6 Johnson, K.C. 22 Jylhä, V. 207 Kagan, J.S. 87 Kaltoft, M.K. 47, 52, 172 Karpatkin, H. 72 Kazemi, A. 67 Kazlouski, A. 142 Kemppi, A. 147 Kinnunen, U.-M. 122, 147 Kiourtis, A. 92 Kivekäs, E. 97 Kock-Schoppenhauer, A.-K. 202 Koivunen, K. 127 Koponen, S. 97, 102 Kostikidis, G. 230 Kouvo, J. 102 Kouz, H. 107 Kulnik, S.T. 228 Kuusisto, H. 102, 147 Kyriazis, D. 92 Lamer, A. 112, 137 244 Lartigau, J. Laurent, G. Lazarova, E. Liljamo, P. Llorente, S. Löbe, M. Looten, V. Lopez, V. MacBrayne, A. Mahmut, E.-E. Mangold, P. Manifavas, H. Mansourvar, M. Mantas, J. Marchioro, T. Markatos, E. Marsh, W. Mavrogiorgou, A. Mendoza-Santana, J. Menesidou, S.-A. Mielke, C. Mielonen, J. Mikkonen, S. Mimigiannis, C. Mitchell, J. Moen, H. Mora, S. Mösching, Y. Moussa, M. Müller, P. Mulligan, N. Naemi, A. Nguewo Ngassam, R.G. Nicola, S. Niebauer, J. Nissen, C. Nüssli, S. Ologeanu-Taddei, R. Olusanya, O. Ozdenerol, E. Palojoki, S. Pape-Haugaard, L.B. Peltonen, L.-M. Plagianakos, V. 226 112 117 122, 127 37 202 192 6 62 132 137 142 152 187, 236 142 142 62 92 17 92 57 147 97 236 222 162 117 27 137 27 6 152 226 132 228 42 27 226 22 22 157, 212 187 162, 222 230 Pool, J. 167 Rajput, A.M. 232, 234 Rajput, V.K. 47, 52, 172 Redjdal, A. 177 Ricci, A. 182 Risling, T. 222 Ronquillo, C. 222 Säilynoja, H. 127 Salanterä, S. 162 Samadbeik, M. 67 Saranto, K. 97, 102, 147, 157, 187, 212 Scandurra, I. 82 Schmidt, T. 152 Schneider, C.L. 42 Schwartz, D.L. 22 Sedlazek, W. 6 Seroussi, B. 1, 107, 177 Shaban-Nejad, A. 22 Shen, Y. 72 Simon, M. 192 Smaradottir, B.F. 197 Sobanski, V. 137 Soyel, H. 62 Stewart, A.J. 22 Stoicu-Tivadar, V. 132 Stütz, T. 228 Thomas, F. 22 Topaz, M. 222 Triantafyllou, I.S. 230 Tuomikoski, K. 127 Ulrich, H. 202 Ung, L. 226 Vahteristo, A. 207 Vakkuri, A. 157 Värri, A. v Vena, A. 117 Vermeulen, A.F. 17 Vuokko, R. 157, 212 Webber, T. 17 Wei, C. 72 Whatelet, M. 112 Wiil, U.K. 152 Wynn, R. 217 Yadav, N. 6

RELATED PAPERS

RELATED TOPICS

Log In

Integrated Citizen Centered Digital Health and Social Care

Integrated Citizen Centered Digital Health and Social Care

Related Papers

RELATED PAPERS

RELATED TOPICS