Professional Documents
Culture Documents
Edited by
Om Prakash Jena, Bharat Bhushan,
and Utku Kose
First edition published 2022
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742
and by CRC Press
2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
CRC Press is an imprint of Taylor & Francis Group, LLC
© 2022 selection and editorial matter, Om Prakash Jena, Bharat Bhushan, Utku Kose; individual chapters,
the contributors
Reasonable efforts have been made to publish reliable data and information, but the author and publisher
cannot assume responsibility for the validity of all materials or the consequences of their use. The authors
and publishers have attempted to trace the copyright holders of all material reproduced in this publication and
apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright
material has not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or here-
after invented, including photocopying, microfilming, and recording, or in any information storage or
retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, access www.copyright.com
or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923,
978-750-8400. For works that are not available on CCC please contact mpkbookspermissions@tandf.co.uk
Trademark notice: Product or corporate names may be trademarks or registered trademarks and are
used only for identification and explanation without intent to infringe.
Names: Jena, Om Prakash, editor. | Bhushan, Bharat, editor. | Kose, Utku, 1985- editor.
Title: Machine learning and deep learning in medical data analytics and
healthcare applications / edited by Om Prakash Jena, Bharat Bhushan, Utku Kose.
Description: First edition. | Boca Raton : CRC Press, 2022. | Series: Biomedical and robotics
healthcare | Includes bibliographical references and index. | Summary: “The book incorporates
the many facets of computational intelligence, such as machine learning and deep learning, to
provide groundbreaking developments in healthcare applications. It discusses theory, analytical
methods, numerical simulation, scientific techniques, analytical outcomes, and computational
structuring”-- Provided by publisher.
Identifiers: LCCN 2021043580 (print) | LCCN 2021043581 (ebook) | ISBN 9781032126876 (hardback) |
ISBN 9781032127644 (paperback) | ISBN 9781003226147 (ebook)
Subjects: LCSH: Medical technology. | Medical care--Technological innovations. |
Robotics in medicine.
Classification: LCC R855.3 .M33 2022 (print) | LCC R855.3 (ebook) |
DDC 610.285--dc23/eng/20211103
LC record available at https://lccn.loc.gov/2021043580
LC ebook record available at https://lccn.loc.gov/2021043581
DOI: 10.1201/9781003226147
v
vi Contents
vii
Editors
Dr. Om Prakash Jena (PhD) is currently work-
ing as an Assistant Professor in the Department of
Computer Science, Ravenshaw University, Cuttack,
Odisha, India. He has ten years of teaching and
research experience in the undergraduate and post-
graduate levels. He has published several technical
papers in international journals, conferences, and
edited book chapters of reputed publications. He
is a member of IEEE, IETA, IAAC, IRED, IAENG, and WACAMLDS. His cur-
rent research interest includes database, pattern recognition, cryptography, network
security, artificial intelligence, machine learning, soft computing, natural language
processing, data science, compiler design, data analytics, and machine automation.
He has many edited books to his credit, published by Wiley, CRC Press, Bentham
Publication, and is also the author of two textbooks under Kalyani Publisher. He also
serves as a reviewer committee member and editor of many international journals.
ix
x Editors
CONTENTS
1.1 Introduction....................................................................................................... 2
1.2 Related Work..................................................................................................... 3
1.2.1 Semantic Interoperability and Semantic Interoperability in EHR........ 3
1.2.2 Fast Health Interoperability Resources.................................................4
1.2.3 Existing Systems.................................................................................... 5
1.3 Terminologies in the Healthcare Ecosystem..................................................... 6
1.3.1 Standardizing Healthcare Data..............................................................6
1.3.1.1 Referencing Terminology....................................................... 6
1.3.1.2 Syntax.....................................................................................6
1.3.1.3 Semantics................................................................................7
1.3.1.4 Pragmatics...............................................................................7
1.3.2 Fast Health Interoperability Resources.................................................7
1.3.3 FHIR as API..........................................................................................7
1.3.4 Common Drug Codes for India (CDCI)................................................8
1.3.5 EHR Standards...................................................................................... 8
1.4 Methodology......................................................................................................8
1.4.1 High-Level Architecture...................................................................... 11
1.4.2 High-Level Representation of CDI Layer............................................ 11
1.4.3 Common Data Interface Functionality................................................ 12
1.4.3.1 New Patient Registration...................................................... 13
1.4.3.2 Search for an Existing Patient in Application Database....... 14
1.4.3.3 Existing Patient Information Extraction Process.................. 14
1.4.3.4 Searching for Patient Data in a Federated System................ 14
1.4.3.5 Use Case................................................................................ 15
1.4.4 CDI Subspace Creation........................................................................ 16
1.4.5 CDI ID Process and Reference Model................................................. 17
1.4.6 CDI ID Format..................................................................................... 17
1.4.6.1 Novel Unique ID Design....................................................... 17
1.5 Discussion and Conclusion.............................................................................. 19
1.6 Future Enhancement........................................................................................ 19
Acknowledgments.....................................................................................................20
References.................................................................................................................20
DOI: 10.1201/9781003226147-1 1
2 Machine Learning and Deep Learning in Medical Data Analytics
1.1 INTRODUCTION
People are well connected in more places than ever and actively participate in digi-
tal healthcare activities in the current generation. Healthcare professionals look for
deeper health insights and actionable information like making better decisions and
efficiently improving patient record information at lower costs. The current health-
care scenario is expected to be digitized. People regularly connect to their differ-
ent health gadgets and regularly monitor their health activities. Digital healthcare
enables the healthcare ecosystem to have a huge amount of connected data for
regular monitoring. The proposed high-level integration architecture is the open
architecture that provides services in compliance with data standards and has a
capability that inspires new healthcare application developers to design next-gen
connected health and wellness systems like digital gadgets that connect across
the healthcare devices, unlike other open architecture. In the current scenario,
we see a patient present at the emergency department. We don’t know anything
about them even though they might have a lot of electronic records (R. Bayer et al.
2015). Knowing them as digital citizens, we take care of them on time and at a
lesser cost. The connected healthcare system supports these features and enables
the individual’s data or health record to be accessible at any point in time for the
health ecosystem’s concerned stakeholders. So there is a necessity that automation
and integration are necessary. We have found that our automated systems are often
much siloed (T. Benson et al. 2016).
It is a long-term challenge to make the health entities interoperable. When
health data and related information are standalone systems, they need to be con-
nected to serve the purpose for which they had been originally designed or intended.
To achieve interoperability, the databases are gradually deployed in a distributed
architecture and the subsequent federation by reusing resources to build a knowl-
edge-based system. But this can be well implemented by incorporating standards
in healthcare systems. With the wide opportunity of semantic technology, we can
incorporate interoperability (B. Hu et al. 2006).
In the federated system approach, various databases are put together to exchange
and communicate the data. But this can be achieved with certain constraints in
design and usage. In a semantic-based approach, the meaning of the information
that needs to be integrated is considered when integrating the database schema with
respective row and column names. Thus, it is the syntactic approach of integration.
The amount of data in the healthcare ecosystem keeps growing. Patient data
is generated by various health stakeholders like physicians, laboratories, medical
devices, research facilities, and now even from in-home data sources such as per-
sonal fitness devices. Sharing this data across so many disparate systems is critical
to ensuring the successful care of individual patients and improving overall popula-
tion health.
In G. Alterovitz et al.’s (2015) study, the authors report a typical case system from
ontology mapping. A similar integration algorithm and data will produce differ-
ent results. The results are arbitrary, as evidenced in the formatting and annotation,
which make the result difficult to reuse. And, this makes semantic integration dif-
ficult to apply.
CDI for Sustainable Healthcare System 3
using ontology (A. Kiourtis et al. 2019). It can automatically integrate information
between multiple users and systems to improve feedback efficiency to query terms
and ensure that the feedback is true and clear regardless of the data representation
(J. D. Heflin et al. 2000). Web ontologies can be used to integrate data and seman-
tic interoperability from medical data because they use existing health standards to
access patient records. In addition, the location of data instances is consistent with
medical terminology (D. Teodoro et al. 2011).
EHR is a digital representation of a patient’s health record, including medi-
cal treatment, diagnosis, treatment plan, and medical history. This is a system-
atic way to store this information and provide it to all parties under each party’s
authorization. Therefore, EHR adopted a prototype-based approach that enables
clinical decision support system (CDSS) tools to make decisions about patient care
(R. Bayer et al. 2015).
Assuming that the EHR is patient-centric, it is not an institution. It has a long-
term care record, including the various medical care that the patient has received,
and the medical treatments, plans, and prognostic instructions followed. The EHR
prototype provides a simplified process for the flow of information between clini-
cians without interpreting information in the existing system. The terms used in the
system are not universally defined; they may be specific to a particular system devel-
oper’s specific prototype. As a result, integrating this information between different
prototypes constitutes a limitation (S. Garde et al. 2007).
depression. When patient X experiences moderate chest pain, he visits the cardi-
ologist. The concerned healthcare provider asks patient X to provide the health
conditions, insurance, and basic demographic details of patient X. This continues
as and when patient X visits all types of care providers. The problem is that while
patient X’s information is already stored at the cardiologist center, it is not acces-
sible or shareable with the diabetic care provider. Interoperability is the approach
to solve this problem to avoid delay and provide the necessary information at the
right time to the right.
1.2.3 Existing Systems
Semantic interoperability can be achieved by incorporating healthcare standards (U.
Batr et al. 2014). The author has compared various standards and their implications
to achieve semantic interoperability. Choosing the best adoptable standard is very
important. HL7 is used widely as a messaging model.
In Pijush Kanti et al.’s (2019) study the author discusses the “V’s” of healthcare
big data where volume, velocity, variability, validity, variety, veracity, viability, vul-
nerability, and visualization of data are described. To achieve the efficient use of big
data in healthcare, incorporating standards is very much required. EHR require data
across healthcare applications using standards-based methods by which seamless
data exchange can be done.
HealthSuite (Philips, USA, 2018) is a cloud-based open digital platform that offers
users continuous, personalized health care. The kit includes functions for analy-
sis, sharing, and processing. Healthcare service coordination. The analysis section
employs ML algorithms as well as various predictive analysis technologies. Shared
functionality is essentially multi-device platform interoperability. Orchestration, in
essence, achieves workflow synchronization, such as Tasks and so on (D. P. Pijush
Kanti et al. 2019).
Watson health (IBM) is a complete software package developed by IBM that
can help all aspects of health. It has AI and ML capabilities that can help diag-
nose and treat diseases effectively and reduce hospital staff and patient care staff.
Watson can understand the patient’s medical history and ask for all possible new
drugs or technologies on the market, thus saving the doctor’s time checking all the
literature.
A fully integrated system with the Internet of Healthcare Things (IoHT) frame-
work can be used remotely to assist medical experts in diagnosing and treating
skin cancer (A. Khamparia et al. 2020). According to the performance index
evaluation, the proposed framework outperforms other pre-trained architectures
regarding accuracy, recall, and accuracy of detecting and classifying skin cancer
from skin lesion images.
Even though internet of health things (IoHT) has a very complex architecture due
to the connectivity of a wide range of devices and services in the system, it can be
incorporated into the healthcare system for data collection and real-time monitoring.
This paper presents a brief overview of urban IoT systems designed to support smart
cities and advanced communication technologies (A. K. Rana et al. 2019).
6 Machine Learning and Deep Learning in Medical Data Analytics
1.3.1.2 Syntax
Syntax is the standard that specifies the required order of composition when ana-
tomical terms and various restrictions, such as “acute,” “distal,” “left and right,” are
used in clinical narratives.
CDI for Sustainable Healthcare System 7
1.3.1.3 Semantics
Semantics is the study of meaning. Semantic standards are concerned with the real-
world meanings of various terminology codes.
1.3.1.4 Pragmatics
Pragmatics is the study of how things work; it makes use of terminology in spe-
cific clinical settings or contexts. Although various resources may contain medical
terminology, such as “asthma,” context is required to differentiate resources and
descriptions of “suspicious” asthma. For example, asthma is classified as “severe.”
Furthermore, resources containing the term asthma may be a laboratory test for
asthma, so pragmatics is important. Therefore, it is necessary to specify that the
resource is an asthma test.
These are useful in understanding the healthcare terminology system. The
most common and standard terminology is ontology. The healthcare system’s
widely used ontology is Systematized Nomenclature of Medicine – Clinical Terms
(SNOMED–CT).
provide lightweight applications that use FHIR data, and FHIR provides this data.
Security, threads, multiple representations, search and indexing, and persistence are
all addressed by the server. FHIR-compliant clients and servers use FHIR resources
for data transmission and exchange data following FHIR API specifications.
The CDCI Terminology Integrated Package is a set of files that integrate with
the standard SNOMED–CT terminology files and content for use in any data entry,
analysis, or record exchange systems that adhere to certain EHR standards (HER
Standards).
This enables standardized coding of medicinal products for clinical care and
enables linkages to terminology and use in clinical data, data retrieval, data analyt-
ics, etc.
Common Drug Codes for India (Flat Files Package) is introduced to support
standardized coding and sharing of drug codes without the need to integrate the
complete terminology. The existing system integration can be incorporated by
using this option and provide their data such as batch code or packages for custom-
ized use.
1.3.5 EHR Standards
The EHR standards, indicated in Table 1.1, is the standards that is openly available
at national resource center repository. The standard digital healthcare system should
incorporate the following EHR standard for interoperability.
1.4 METHODOLOGY
Healthcare architecture has been proposed over time by many researchers in our
methodology we propose an innovative middleware that is responsible for achieving
interoperability in adherence to the existing system architecture of healthcare appli-
cations. The adherence to the existing system may vary with minor configurations
and add-ons. This section proposes the novel CDI layer, which can be implemented
as middleware architecture to the existing system. The methodology is illustrated
considering the case study of patient registration and data access mechanism.
The novel CDI layer generates a 12-digit unique number that follows the standard
approach. The health unique number is illustrated in Figure 1.6. With the unique
CDI ID, the system can identify the patient, the health facility, and all stakeholders
of the healthcare ecosystem. This flexibility of identification makes the system more
efficient and easier to adopt.
CDI for Sustainable Healthcare System 9
TABLE 1.1
EHR Standards by NRC
Figure 1.3 shows the new user registration process via a CDI. The registration
process follows the regular data flow. However, the CDI algorithm via API fetches
the user demographic details and process the demographic details to create the data
subspaces. The figure illustrates the data subspaces creation and mapping process.
Further, the data subspaces are assigned with a unique CDI ID as defined in the
section. Finally, the CDI ID, data subspaces, and application-generated user ID are
stored in the CDI server for further processing.
The assumption of a new and existing user is as follows:
The detailed process of the CDI algorithm is indicated considering new and exist-
ing patient scenarios.
The data subspaces are generated as shown in Figure 1.5. The patient demographic
details are fetched using the API interface between the health applications and the
CDI server.
With the demographic details, using the combination technique, the data is cat-
egorized into subspaces. Further, these subspaces are assigned with the unique
system-generated CDI ID number. The data subspaces are the reference data that
are generated considering the patient demographic details. Further, as indicated
in Figure 1.5, patient data is processed. The patient data extraction starts with the
user query that is further processed to generate data subspace based on the query’s
parameters. Then, the match for the auto-generated subspaces from the user query
is searched with the CDI server repository with the metadata information. Once the
match is found, the data is extracted based on the application selected by the same.
A detailed illustration of the same is indicated in Figure 1.5.
Now consider the power set, that is, all subspaces of the given set A, denoted
as P(A).
Further P(A) is given as the input to the semantic algorithm for finding the best
match of patient record.
The semantic search is very useful for retrieving the most relevant result from
the data repository. The approach of NLP makes this task easier. The context-based
analysis in NLP verifies the unique CDI ID code and accordingly processes the
search result. For instance, if we need to search for a hospital in a particular district,
the search query first identifies the district code by checking for the facility code and
then processing the query request. Once the facility is identified as the hospital, the
search is done only for the repository about the hospital databases.
General workflow:
Step1: Identify the district and taluk code.
Step2: Verify the facility identification code – “understand the query
intent.”
Step3: Search operation based on the intent – patient/hospital/lab etc.
Step4: Understand the conceptual similarity.
Step5: Validate the data to match.
CDI for Sustainable Healthcare System 17
TABLE 1.2
Indicating the Metadata Table Format of CDI Server DB
1.4.6 CDI ID Format
Generating a unique ID is an important aspect of any application integration.
Unfortunately, having the common ID across the application is not possible consid-
ering the current healthcare system architecture and design. So we propose a novel
method for generating the unique ID for operating the CDI layer. The CDI ID is used
as a unique ID for making applications interoperable.
ACKNOWLEDGMENTS
This work is supported in part by the Department of Health and Family Welfare
Services, Government of Karnataka, India. We also thank the E-Health section of
KHFWS for continuous support and encouragement in this work. Finally, we would
like to thank two anonymous reviewers and editors of this chapter for commenting
on earlier versions of this chapter.
REFERENCES
Aggarwal, M., and M. Madhukar (2017) IBM’s Watson analytics for health care: A miracle
made true. In: Cloud Computing Systems and Applications in Healthcare. IGI Global,
117–134. 10.4018/978-1-5225-1002-4.ch007
Alterovitz, G., D. Dean, C. Goble, M. R. Crusoe, S. Soiland-Reyes, A. Bell et al. (2018)
Enabling precision medicine via standard communication of HTS provenance, analysis,
and results. PLoS Biol 16(12): e3000099. https://doi.org/10.1371/journal.pbio.3000099
Alterovitz, G., B. Heale, J. Jones, et al. (2020) FHIR Genomics: Enabling standardization for
precision medicine use cases. NPJ Genomic Medicine 5(13). https://doi.org/10.1038/
s41525-020-0115-6
Alterovitz, G., J. Warner, P. Zhang, Y. Chen, M. Ullman-Cullere, D. Kreda, I.S. Kohane
(2015) SMART on FHIR Genomics: Facilitating standardized clinico-genomic apps.
Journal of the American Medical Informatics Association 22(6):1173–78. https://doi.
org/10.1093/jamia/ocv045
Batra, U., S. Mukherjee, A. Nandal, S. Sachdeva (2014) Explorative study of healthcare data
interchange standards. Journal of Theoretical and Applied Information Technology
63(2): 420–430.
Bayer, R., J. Santelli, R. Klitzman (2015) New challenges for electronic health records:
Confidentiality and access to sensitive health information about parents and adoles-
cents. JAMA 313(1):29–30. https://doi.org/10.1001/jama.2014.15391
Benson, T, G. Grieve (2016) Principles of health interoperability. Health Information Technology
Standards. London, Springer-Verlag. https://doi.org/10.1007/978-3-319-30370-3_2
Dutta Pramanik, P. K., S. Pal, M. Mukhopadhyay (2019) Healthcare big data: A compre-
hensive overview. In: Intelligent Systems for Healthcare Management and Delivery
2019:72–100. https://doi.org/10.4018/978-1-5225-7071-4.ch004
Garde, S., P. Knaup, E.J. Hovenga, and S. Heard (2007) Towards semantic interoperability for
electronic health records. Methods of Information in Medicine 46(03):332–43.
Heflin, J. D., and J. Hendler (2000) Semantic interoperability on the web. Proceedings of
Extreme Markup Languages 2000.
Hu, B., Y. Kalfoglou, H. Alani, D. Dupplaw, P. Lewis, N. Shadbolt (2006) Semantic metrics.
In: Staab S., Svátek V. (eds) Managing knowledge in a world of networks. EKAW 2006.
Lecture Notes in Computer Science, vol 4248. Berlin, Heidelberg, Springer. https://doi.
org/10.1007/11891451_17
ISO/TR 20514:2005 (2005) Health informatics – Electronic health record – Definition,
scope, and context by ISO/TC 215, Multiple. Distributed through American National
Standards Institute, 1–27.
Khamparia, A., P. K. Singh, P. Rani, D. Samanta, A. Khanna, B. Bhushan (2020) An inter-
net of health things-driven deep learning framework for detection and classification
of skin cancer using transfer learning. Transactions on Emerging Telecommunication
Technologies: e3963. https://doi.org/10.1002/ett.3963
Kiourtis, A., S. Nifakos, A. Mavrogiorgou, and D. Kyriazis (2019) Aggregating the syntactic
and semantic similarity of healthcare data towards their transformation to HL7 FHIR
CDI for Sustainable Healthcare System 21
CONTENTS
2.1 Introduction..................................................................................................... 23
2.1.1 Phases of a BCI....................................................................................25
2.1.1.1 Signal Acquisition.................................................................25
2.1.1.2 Feature Extraction.................................................................26
2.1.1.3 Signal Classification..............................................................26
2.2 Review Work...................................................................................................26
2.3 Applications..................................................................................................... 32
2.4 Issues and Challenges...................................................................................... 33
2.5 Conclusion.......................................................................................................34
References.................................................................................................................34
2.1 INTRODUCTION
A BCI provides an interface between the brain and the computer. Also called a
mind–machine interface or brain–machine interface, it does not require the use of
muscles to interact with a computer. It instead employs sensors attached directly
to the brain to transfer those signals to a computer. BCI has found its application
in medical fields where these devices are used by patients in locked-in state to
communicate with their caretakers. A number of BCI devices are also used to aug-
ment and/or assist motor functions including the use of prosthesis [1]. The main
advantage of using BCI-based devices is that they work even if the patient faces
nervous damage due to the fact that the signals are not transmitted through nerves
to the prosthetic implant, but rather through the BCI device directly from the
brain. Recently, BCI is finding its use in various other non-medical fields including
authentication, controlling robots or other moving devices, and also communica-
tion [2–4]. It also faces challenges in protecting the privacy of users and reducing
the cost of hardware used.
The natural way for humans to communicate is by using their muscles by either
making gestures, creating sound using their throat and other muscles, or other
means. The person communicated to watches the actions or listens to the sounds and
DOI: 10.1201/9781003226147-2 23
24 Machine Learning and Deep Learning in Medical Data Analytics
deciphers the message communicated. The process consists of the brain generating
electrical signals which are then sent to the muscle group which has to perform the
action, via the nervous system.
A similar process is followed when a person wants to communicate with a com-
puter using a keyboard or a mouse or another input device. This can be called human–
computer interaction. This is improved using other input devices like a microphone
and camera through which a person can speak or issue commands using their voice
or facial expressions. This can further be improved if we send the brain’s electrical
signals directly to the computer. This communication process is referred to as the
brain–computer interface.
One of the first definitions of BCI, given by Jonathon R. Wolpaw in 1999, is [5]:
A BCI is a system that measures central nervous system (CNS) activity and converts it
into artificial out-put that replaces, restores, enhances, supplements, improves natural
CNS output and thereby changes the ongoing interactions between the CNS and its
external or internal environment.
BCI works by first acquiring the brain signals, then extracting the intentions of
those signals and then translating those signals to commands to the BCI application.
An illustration of a BCI system is shown in Figure 2.1. This was presented by Jack
Vidal in 1973.
2.1.1 Phases of a BCI
The following sections can be said as the phases of a BCI.
The most common method is recording the electrical signals from the brain
called electroencephalography. Since electrical fields also produce magnetic
fields, it is possible to measure the magnetic fields similarly by a process called
magnetoencephalography.
27
CNN common spatial patterns EEG dataset
(Continued)
28
TABLE 2.1 (Continued)
Literature Review on BCI
Publication
Year & BCI Setup
Reference Objective Methods Data and Software Results
In their study about identifying hand movement using BCI, Pattnaik et al. [11]
demonstrated how one could implement left- and right-hand movement classification
from the EEG signals recorded. They showed that these waves can be divided into
constituent alpha, beta, and delta waves by various sampling methods such as dis-
crete wavelet transform, and these could be used as the feature vector for classifiers
in support vector machines (SVMs).
Seo-Hyun Lee et al. [12] in their work on imagined speech and visual imagery in
2019 tried to create an application that could detect what the user wanted to speak
using BCI. They compared imagined speech (imagining the literals of the word to be
spoken) and visual imagery (imagining the object when trying to speak that word).
They used a 64-channel EEG cap to measure the brain signals and employed multi-
class classification of more than 10 classes in both paradigms.
In the study based on authenticating users in a virtual reality (VR)-based envi-
ronment, Sukun Li et al. [13] studied whether active portions of the brain are influ-
enced by the presence of VR and use it for authentication. In the study, 32 participants
were shown a video in a VR and non-VR setup and their brain signals were recorded
and analyzed to authenticate them. The best classification accuracy was achieved up
to 80.91%. It was demonstrated that a BCI could be used to recognize a user even in
a virtual environment.
In the study based on a gaming system using a steady state visually evoked potential
BCI (SSVEP-BCI), Nayak et al. [14] designed an SSVEP-based computer game based on
Jewel Quest. They used a 32-channel Neuroscan system for recording brain signals and
tried to improve rare target classifications characterized by class imbalance and overlap.
Bahman et al. [15] in their research to control a robot in 2019 designed a P300-
based BCI system using a low-priced EEG headset. They recorded brain signals
using a 14-channel Emotiv Epoc-EEG headset (shown in Figure 2.2) and created a
graphical user interface (GUI) (shown in Figure 2.2) over BCI2000 to give direc-
tions. The study showed impressive results with 93.3% accuracy while detecting the
direction. This paves the way for low cost BCI devices for regular usage.
There was another study in 2015 by Arunkumar et al. [3]. They used a NeuroSky
BCI headset (shown in Figure 2.3) in their study to detect brain waves to control a
robot wirelessly. The study was aimed at locked-in patients to control a robot using
their attention level and eye blinks.
In their study on prosthetic arm control using BCI, Bright et al. [1] attempted to
create a BCI for controlling an artificial arm. They too used a NeuroSky headset
to record brain EEG waves, and used MATLAB®’s ThinkGear module for pro-
cessing. These commands are then sent over to the microcontroller which controls
the arm. They were able to control the movement of fingers by contracting and
expanding them. The researchers plan to upscale this system to support full range
of motion of the arm.
Kama et al. [9] in their work in 2018 compared EEG systems with dry and wet
electrodes. They worked with the same 27 participants with both dry and wet elec-
trodes on different days. They were able to conclude that both types of electrodes
give comparable results and that dry electrodes do not compromise the quality of
EEG signals when compared to wet electrodes. Thus, they could be used at most
of the places instead of wet electrodes. They are also preferred for the low prepara-
tion effort (Figure 2.4).
In their study to control a smart home using eye movements, Putze et al. [16]
worked on a BCI-based AR system. They used an SSVEP-based BCI, along with a
binocular eye tracker, where the participants selected an object by gazing at it. BCI
tracking had lower accuracy than when it was combined with eye tracking, which
had an accuracy of 89.3%.
Renton et al. [17] in their study on human communication using BCI introduced
a system for free communication between participants without any fixed phrases or
cues. They created an asynchronous two-user messaging interface that allowed the
users to freely communicate. With high accuracy ranging from 88% to 96%, the
researchers mention that the focus of communication-based BCI setups should be on
usability rather than speed.
In the study to compare BCI-based systems with augmented BCI to control a
synthetic arm, Xu et al. [18] implemented a system that uses computer vision coupled
with BCI for controlling a robot arm. The BCI system was based on motor-imagery
and it had a good success rate of more than 70%. They noted that the unsuccessful
trials were due to poor guidance while using BCI. The researchers also mention that
one of the reasons for such high accuracy was that they had only one object in the
workspace.
In the study to control a prosthetic arm using EEG-based BCI, Baxter et al. [19]
tried to create a convenient system for controlling an artificial arm. The participants
imagined moving their left and right hands to move the respective arms. Similarly,
other such visualized movements were used by the BCI system to map a specific
hand movement. They had significant success with the system having an accuracy of
24.7% and 12% in vertical and horizontal arm claw movement, respectively.
Sakhavi et al. [20], in their work to classify BCI information, utilized convolu-
tional neural networks to create a new classification framework. Most of the pre-
vious classification frameworks were solely based on energy values and neglected
temporal information which was utilized by the researchers in this study. They used
filter-bank common spatial patterns for feature extraction and achieved an accuracy
of 74.46%.
In their study of BCI systems with audio-visual (AV) feedback, Brumberg et al.
[21] researched multimodal feedback on real-time speech synthesis using BCI. Three
different groups received only one of visual, audio, or both audio and video feedback
of their BCI synthesized speech. They noticed that the AV feedback had the high-
est accuracy of 68.3% compared to 47.2% and 50.1% of video only and audio only
feedback respectively.
In their study on a P300 based BCI speller, Fernández et al. [22] assess P300 based
spellers which work with complex visual stimuli. Participants were flashed with visual
stimuli which included pictures which were emotionally pleasing or unpleasing.
Emotionally pleasant pictures were detached in P300 BCI with an accuracy of 99.64%.
Such picture sets could thus be used for visual P300-based BCI applications.
In their study to classify chessboard EEG images, Fadel et al. [23] used a deep
convolutional neural network (DCNN) along with long short-term memory. They
extracted spatial, frequency, and temporal features to classify them into one of either
four different motor imagery classes or rest class. They had a classification accuracy
of 68.72% compared to SVM’s 64.64% accuracy.
32 Machine Learning and Deep Learning in Medical Data Analytics
2.3 APPLICATIONS
BCI has been useful in a lot of medical cases from allowing patients in locked-in
state to communicate with their caretakers, to controlling wheelchairs and robots.
A lot of applications and games have also been developed to induce relaxation and
lower stress levels of the person wearing the BCI device. There are a lot of non-med-
ical uses of BCI too, including authentication, communication, and entertainment.
There has been a lot of work to create BCI devices to control the movements of
robots, including those based on P3000 potentials using an EEG headset [15]. This
work by Bahman et al. [15] was attempted at high real-time accuracy using an EEG
headset which was not very expensive. It can be useful for patients who cannot use a
remote control for next-gen robot caretakers. Another such application was devised
where the applicants tried to use their brain to control a robot arm. It was based on
motor imagery and used a small setup [19].
A hybrid BCI (hBCI) system combines a BCI with other physiological or techni-
cal signals. hBCI systems have been used as a prosthesis in patients with spinal cord
injury. The prosthesis was shown to be used for moving hands, and grasping and
releasing objects. It is majorly suited for patients who have a damaged hand but with
shoulder and elbow capable of movement. [27].
A lot of use cases of BCI from the medical field also find their place in various
non-medical use cases. One of these is the attention-enhancing ball game [28]. The
authors propose an EEG-based game where the user uses his/her attention to play.
It is shown that it is able to increase the attention levels of the player using the feed-
back mechanism, taking advantage of the plastic nature of the brain. The game also
features multiple difficulty levels for the user to enhance his/her attention level by
training over a number of days.
There have also been attempts to create games using BCI as one of the input
modalities along with keyboard, mouse, and joystick. Little similar to [28], Pope
et al. in [29] also looked at the application of BCI to help kids with ADHD to improve
their attention levels by using neuro feedback.
Another attempt to use hybrid BCI in a well-known game is found in [30] where
a game similar to Jewel Quest is designed using BCI. It used SSVEP-based target
classification without a cascading menu, thus, decreasing the time required to play.
Using other devices along with BCI has improved the classification accuracy as
seen with [31] having an aim to control a home based on augment reality (AR) using
Brain–Computer Interface 33
BCI. The system presented in the study does not require training and is easy to set
up as well.
Another such application was seen in [18] where the researchers used a computer-
guided system along with motor imagery-based BCI which allow the participants to
move a robotic arm and grab an object.
One of the most interesting uses of BCI in non-medical fields is its use in authen-
tication. Since the brain EEG activities of each person are unique, it can be used as
an authentication method for unlocking devices and at other places where a pin or
a password is required [13]. [20] has shown that it is possible to detect brain waves
even in VR environments as in physical environments and hence it is possible both
to confirm the user’s authenticity and that he/she is in a virtual environment. It was
also trialed to authenticate people in VR applications and it turned out to be very
successful [13].
Another exciting prospect is controlling a computer without any peripheral device
except the BCI. There have been successful attempts earlier to control a specific
application using BCI such as the P300 speller which created a new application for
typing using brain signals. Martin Spuler [10] has already shown a method to control
any arbitrary application using brain signals. An application was developed to con-
trol the mouse and keyboard using the BCI based EEG signals. Other attempts such
as [31] have created complete user systems integrating a browser, file manager, and
other applications.
Another application of BCI is communication. This is complex, exciting, and
unnerving at the same time. Various methods have been proposed for brain-to-brain
communication, including connecting multiple animal brains to create an organic
brain network [32]. It has also been shown that both visual imagery and imagined
speech can be used in BCI applications to communicate using brain signals [12]. In
the early stages, it is employed for patients with a disabled neurological system for
communicating with their caretakers with a fixed set of words. Even for non-medical
usage, it can be a game-changing idea. Another such method is explored in [21]
where the user is given feedback on their synthesized BCI speech in real time to
improve the user speech.
A lot of BCI spellers do not actually test on free communication but on guided
cues and fixed phrases or words. In [17], the researchers tried to challenge the status
quo by experimenting with free communication between different participants. They
have also applied cross-classification, where the trained template from other users’
experience could be used. This would completely eliminate training and users would
be able to start communicating using BCI without any extra setup time.
Another challenge that the field faces and which prevents a more widespread
usage is the variable nature of the brain waves. Each person’s brain waves are unique.
Although the identifying features are the same, people have different mental abilities
and attention spans. The data set for most of the studies is small with only a small
number of participants are a part of them. Extensive testing is necessary to improve
the techniques and to study them on a wider audience.
There is also the issue of privacy with BCI applications. BCI applications are
allowed to read all the brain signals read by the headset. This could lead to the leak
of information to authentic-looking, malicious applications which could later lead to
awful consequences. Improving this aspect of BCI is also essential with its studies in
the field of authentication. With newer applications every day, it is becoming easier
to exploit someone who is not wary of such attacks [33].
There are also ethical considerations when trying to communicate using brain
signals or when researchers try to create an organic computer by connecting multiple
brains through the means of BCI. These questions need detailed discussion to reach
a consensus.
2.5 CONCLUSION
BCI is an exciting field. With regular new advancements in machine learning, we
are close to achieving the reality of communicating directly with the brain and doing
many more such tasks without utilizing any muscles [34–47].
It has found applications in remote-controlling robots, wheelchairs, and prosthetic
arms and legs thus helping immensely those who for any reason are not able to move
their complete body or some specific parts of the body. BCI spellers and applications
which allow the user to control the computer without peripheral devices also find use
for many such patients suffering from diseases affecting their motor skills.
Other than applications specifically targeted for patients, there has been work
lately on studies for other uses. Using brain waves for authentication and communi-
cation demands huge efforts in the future for these to be brought in publicly. The use
of templates lifted from other users’ experiences is also being attempted so that there
is no training required before using the application. This brings in the issue of ease
of use which is an important aspect when we talk about have widely accepted BCI.
Many such applications have already materialized, although in simpler forms,
as seen above, but researchers keep on improving existing works. Along with the
advancements, we also face the challenges that plague the field. One of the big-
gest issues that challenges the adoption of BCI is the price of the hardware used.
Similarly, there is hesitancy in using BCI devices because of the privacy challenges.
There are also the ethical considerations which must be kept in mind while working
in the field.
REFERENCES
[1] Bright, Dany, Amrita Nair, Devashish Salvekar, and Swati Bhisikar. “EEG-based
Brain Controlled Prosthetic Arm.” 2016 Conference on Advances in Signal Processing
(CASP), 2016. doi:10.1109/casp.2016.7746219
Brain–Computer Interface 35
[2] Yousefi, Fares, Hoshang Kolivand, and Thar Baker. “SaS-BCI: A New Strategy to Predict
Image Memorability and Use Mental Imagery as a Brain-based Biometric Authentication.”
Neural Computing and Applications, 2020. doi:10.1007/s00521-020-05247-1
[3] Stephygraph, L R, N Arunkumar, and V. Venkatraman. “Wireless Mobile Robot
Control through Human Machine Interface Using Brain Signals.” 2015 International
Conference on Smart Technologies and Management for Computing, Communication,
Controls, Energy and Materials (ICSTM), 2015. doi:10.1109/icstm.2015.7225484
[4] Soman, Sumit, and B K Murthy. “Using Brain Computer Interface for Synthesized
Speech Communication for the Physically Disabled.” Procedia Computer Science 46
(2015): 292–8. doi:10.1016/j.procs.2015.02.023
[5] Wolpaw, Jonathan R, Niels Birbaumer, Dennis J McFarland, Gert Pfurtscheller,
and Theresa M Vaughan. “Brain–Computer Interfaces for Communication and
Control.” Clinical Neurophysiology 113, no. 6 (June 2002): 767–91. doi:10.1016/
s1388-2457(02)00057-3
[6] Wolpaw, Jonathan R, and Elizabeth Winter Wolpaw. “Brain–Computer Interfaces:
Something New under the Sun.” In Brain–Computer InterfacesPrinciples and Practice,
3–12. Oxford University Press, 2012. doi:10.1093/acprof:oso/9780195388855.003.0001
[7] Vidal, J.. “Real-time detection of brain events in EEG.” In Special Issue on BiolSignal
Processing and Analysis. IEEE Proc, 1977. 633–664.
[8] Ramadan, Rabie A., and Athanasios V. Vasilakos. “Brain Computer Interface: Control
Signals Review.” Neurocomputing 223 (2017): 26–44. doi:10.1016/j.neucom.2016.10.024
[9] Kam, Julia W Y, Sandon Griffin, Alan Shen, Shawn Patel, Hermann Hinrichs, Hans-
Jochen Heinze, Leon Y. Deouell, and Robert T. Knight. “Systematic Comparison between a
Wireless EEG System with Dry Electrodes and a Wired EEG System with Wet Electrodes.”
NeuroImage 184 (January 2019): 119–29. doi:10.1016/j.neuroimage.2018.09.012
[10] Spuler, Martin. “A Brain-Computer Interface (BCI) System to Use Arbitrary Windows
Applications by Directly Controlling Mouse and Keyboard.” In 2015 37th Annual
International Conference of the IEEE Engineering in Medicine and Biology Society
(EMBC). IEEE, 2015. doi:10.1109/embc.2015.7318554
[11] Pattnaik, Prasant Kumar, and Jay Sarraf. “Brain Computer Interface Issues on Hand
Movement.” Journal of King Saud University – Computer and Information Sciences
30, no. 1 (January 2018): 18–24. doi:10.1016/j.jksuci.2016.09.006
[12] Lee, Seo-Hyun, Minji Lee, Ji-Hoon Jeong, and Seong-Whan Lee. “Towards an EEG-
Based Intuitive BCI Communication System Using Imagined Speech and Visual
Imagery.” 2019 IEEE International Conference on Systems, Man and Cybernetics
(SMC). IEEE, 2019. doi:10.1109/smc.2019.8914645
[13] Li, Sukun, Sonal Savaliya, Leonard Marino, Avery M. Leider, and Charles C. Tappert.
“Brain Signal Authentication for Human-Computer Interaction in Virtual Reality.”
2019 IEEE International Conference on Computational Science and Engineering
(CSE) and IEEE International Conference on Embedded and Ubiquitous Computing
(EUC). IEEE, 2019. doi:10.1109/cse/euc.2019.00031
[14] Nayak, Tapsya, Li-Wei Ko, Tzyy-Ping Jung, and Yufei Huang. “Target Classification
in a Novel SSVEP-RSVP Based BCI Gaming System.” 2019 IEEE International
Conference on Systems, Man and Cybernetics (SMC). IEEE, 2019. doi:10.1109/
smc.2019.8914174
[15] Bahman, Shafie, and Mohammad B. Shamsollahi. “Robot Control Using an Inexpensive
P300 Based BCI.” 2019 26th National and 4th International Iranian Conference on
Biomedical Engineering (ICBME). IEEE, 2019. doi:10.1109/icbme49163.2019.9030408
[16] Putze, Felix, Dennis Weiss, Lisa-Marie Vortmann, and Tanja Schultz. “Augmented
Reality Interface for Smart Home Control Using SSVEP-BCI and Eye Gaze.” 2019
IEEE International Conference on Systems, Man and Cybernetics (SMC). IEEE, 2019.
doi:10.1109/smc.2019.8914390
36 Machine Learning and Deep Learning in Medical Data Analytics
[17] Renton, Angela I., Jason B. Mattingley, and David R. Painter. “Optimising Non-
Invasive Brain-Computer Interface Systems for Free Communication between Naïve
Human Participants.” Scientific Reports 9, no. 1 (December 2019). doi:10.1038/
s41598-019-55166-y
[18] Xu, Yang, Cheng Ding, Xiaokang Shu, Kai Gui, Yulia Bezsudnova, Xinjun Sheng,
and Dingguo Zhang. “Shared Control of a Robotic Arm Using Non-Invasive Brain–
Computer Interface and Computer Vision Guidance.” Robotics and Autonomous
Systems 115 (May 2019): 121–29. doi:10.1016/j.robot.2019.02.014
[19] Baxter, Bryan S., Andrew Decker, and Bin He. “Noninvasive Control of a Robotic Arm
in Multiple Dimensions Using Scalp Electroencephalogram.” 2013 6th International
IEEE/EMBS Conference on Neural Engineering (NER). IEEE, 2013. doi:10.1109/
ner.2013.6695867
[20] Sakhavi, Siavash, Cuntai Guan, and Shuicheng Yan. “Learning Temporal Information
for Brain-Computer Interface Using Convolutional Neural Networks.” IEEE
Transactions on Neural Networks and Learning Systems 29, no. 11 (November 2018):
5619–29. doi:10.1109/tnnls.2018.2789927
[21] Brumberg, Jonathan S., Kevin M. Pitt, and Jeremy D. Burnison. “A Noninvasive Brain-
Computer Interface for Real-Time Speech Synthesis: The Importance of Multimodal
Feedback.” IEEE Transactions on Neural Systems and Rehabilitation Engineering 26,
no. 4 (April 2018): 874–81. doi:10.1109/tnsre.2018.2808425
[22] Fernández-Rodríguez, Álvaro, Francisco Velasco-Álvarez, María Teresa Medina-Juliá,
and Ricardo Ron-Angevin. “Evaluation of Emotional and Neutral Pictures as Flashing
Stimuli Using a P300 Brain–Computer Interface Speller.” Journal of Neural Engineering
16, no. 5 (September 11, 2019): 056024. doi:10.1088/1741-2552/ab386d
[23] Fadel, Ward, et al. “Chessboard EEG Images Classification for BCI Systems Using
Deep Neural Network.” Bio-Inspired Information and Communication Technologies
Lecture Notes of the Institute for Computer Sciences, Social Informatics and
Telecommunications Engineering, 2020, 97–104. doi:10.1007/978-3-030-57115-3\_8
[24] Jeong, Ji-Hoon, et al. “Brain-Controlled Robotic Arm System Based on Multi-
Directional CNN-BiLSTM Network Using EEG Signals.” IEEE Transactions on
Neural Systems and Rehabilitation Engineering, 28, no. 5 (2020): 1226–38. doi:10.1109/
tnsre.2020.2981659
[25] Wu, Di, Huayan Wan, Siping Liu, Weiren Yu, Zhanpeng Jin, and Dakuo Wang.
“DeepBrain: Towards Personalized EEG Interaction through Attentional and Embedded
LSTM Learning.” arXiv.org, April 14, 2021. https://arxiv.org/abs/2002.02086.
[26] Chai, Rifai, Ganesh R. Naik, Sai Ho Ling, and Hung T. Nguyen. “Hybrid Brain–
Computer Interface for Biomedical Cyber-Physical System Application Using Wireless
Embedded EEG Systems.” BioMedical Engineering OnLine 16, no. 1 (January 7, 2017).
doi:10.1186/s12938-016-0303-x
[27] Muller-Putz, G., Leeb, R., Tangermann, M., Hohne, J., Kubler, A., Cincotti, F., …
Millan, J. del R. (2015). “Towards Noninvasive Hybrid Brain–Computer Interfaces:
Framework, Practice, Clinical Application, and Beyond.” Proceedings of the IEEE, 10,
no. 6, 926–43. doi:10.1109/jproc.2015.2411333
[28] Shenjie, Sun, Kavitha P. Thomas, K.G. Smitha, and A.P. Vinod. “Two Player EEG-
Based Neurofeedback Ball Game for Attention Enhancement.” 2014 IEEE International
Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2014. doi:10.1109/
smc.2014.6974412
[29] Pope, A. T. and Palsson, O. S. (2001), “Helping Video Games Rewire ‘Our Minds’”,
Langley Research Center Research Report, Hampton, VA, http://ntrs.nasa.gov/archive/
nasa/casi.ntrs.nasa.gov/20040086464\_2004090499.pdf. Accessed 14 April 2011.
[30] Nijholt, Anton, Danny Plass-Oude Bos, and Boris Reuderink. “Turning Shortcomings
into Challenges: Brain–Computer Interfaces for Games.” Entertainment Computing 1,
no. 2 (April 2009): 85–94. doi:10.1016/j.entcom.2009.09.007
Brain–Computer Interface 37
[31] He, Shenghong, Yajun Zhou, Tianyou Yu, Rui Zhang, Qiyun Huang, Lin Chuai,
Madah-Ul- Mustafa, et al. “EEG- and EOG-Based Asynchronous Hybrid BCI: A
System Integrating a Speller, a Web Browser, an E-Mail Client, and a File Explorer.”
IEEE Transactions on Neural Systems and Rehabilitation Engineering 28, no. 2
(February 2020): 519–30. doi:10.1109/tnsre.2019.2961309
[32] Pais-Vieira, Miguel, Gabriela Chiuffa, Mikhail Lebedev, Amol Yadav, and Miguel A
L Nicolelis. “Building an Organic Computing Device with Multiple Interconnected
Brains.” Scientific Reports 5, no. 1 (July 9, 2015). doi:10.1038/srep11869
[33] Landau, Ofir, Aviad Cohen, Shirley Gordon, and Nir Nissim. “Mind Your Privacy:
Privacy Leakage through BCI Applications Using Machine Learning Methods.”
Knowledge-Based Systems 198 (June 2020): 105932. doi:10.1016/j.knosys.2020.105932
[34] Patra, Sudhansu Shekhar, Om Praksah Jena, Gaurav Kumar, Sreyashi Pramanik,
Chinmaya Misra, and Kamakhya Narain Singh. “Random Forest Algorithm in
Imbalance Genomics Classification.” Data Analytics in Bioinformatics: A Machine
Learning Perspective (2021): 173–90.
[35] Pattnayak, Parthasarathi, and Om Prakash Jena. “Innovation on Machine Learning in
Healthcare Services–An Introduction.” Machine Learning for Healthcare Applications
(2021): 1–15.
[36] Panigrahi, Niranjan, Ishan Ayus, and Om Prakash Jena. “An Expert System-Based
Clinical Decision Support System for Hepatitis-B Prediction & Diagnosis.” Machine
Learning for Healthcare Applications (2021): 57–75.
[37] Paramesha, K., H. L. Gururaj, and Om Prakash Jena. “Applications of Machine
Learning in Biomedical Text Processing and Food Industry.” Machine Learning for
Healthcare Applications (2021): 151–167.
[38] Bardhan, Shawli, Mrinal Kanti Bhowmik, Tathagata Debnath, and Debotosh Bhattacharjee.
“RASIT: Region shrinking based Accurate Segmentation of Inflammatory areas from
Thermograms.” Biocybernetics and Biomedical Engineering 38, no. 4 (2018): 903–917.
[39] Bardhan, Shawli, and Mrinal Kanti Bhowmik. “2-Stage classification of knee joint
thermograms for rheumatoid arthritis prediction in subclinical inflammation.”
Australasian Physical & Engineering Sciences in Medicine 42, no. 1 (2019): 259–77.
[40] Shawli Bardhan, Satyabrata Nath, Tathagata Debnath, Debotosh Bhattacharjee, and
Mrinal Kanti Bhowmik, “Designing of an Inflammatory Knee Joint Thermogram
Dataset for Arthritis Classification Using Deep Convolution Neural Network”,
Quantitative InfraRed Thermography Journal (to be published by Taylor & Francis).
doi:10.1080/17686733.2020.1855390
[41] Shawli Bardhan, and Sukanta Roga. “Feature Based Automated Detection of COVID-
19 from Chest X-Ray Images.” In Emerging Technologies During the Era of COVID-19
Pandemic, 348 (2021): 115–31, Springer, doi:10.1007/978-3-030-67716-9_8
[42] Shawli Bardhan, and Sukanta Roga. “Edge Feature Based Classification of Breast
Thermogram for Abnormality Detection.” In Advances in Mechanical Engineering,
511–17. Springer, Singapore, 2020. doi:10.1007/978-981-15-3639-7_61
[43] Sukanta Roga, Shawli Bardhan, Dilip H. Lataye. “Air Pollution Monitoring using Blue
Channel Texture Features of Image.” International Conference on Deep Learning,
Artificial Intelligence and Robotics Applications (ICDLAIR-2019), Springer, MNIT,
Jaipur, 2020. doi:10.1007/978-3-030-67187-7_15
[44] Shawli Bardhan, Samarpita Debnath, and Mrinal Kanti Bhowmik. “Classification
of IR Image Based Inflammatory Pain Diseases Using Statistical Pattern Analysis
Approach.” International Journal of Computational Intelligence & IoT 1, no. 2 (2018).
https://ssrn.com/abstract=3354550
[45] Shawli Bardhan, Satyabrata Nath, and Mrinal Kanti Bhowmik. “Evaluation of back-
ground subtraction effect on classification and segmentation of knee thermogram.” In
2017 8th International Conference on Computing, Communication and Networking
Technologies (ICCCNT), 1–7. IEEE, 2017. doi:10.1109/ICCCNT.2017.8204011
38 Machine Learning and Deep Learning in Medical Data Analytics
[46] Mrinal Kanti Bhowmik, Shawli Bardhan, Kakali Das, Debotosh Bhattacharjee, and
Satyabrata Nath. “Pain related inflammation analysis using infrared images.” In
Thermosense: Thermal Infrared Applications XXXVIII, 9861:986116. International
Society for Optics and Photonics, SPIE, 2016. doi:10.1117/12.2223425
[47] Shawli Bardhan, Mrinal Kanti Bhowmik, Satyabrata Nath, and Debotosh Bhattacharjee.
“A review on inflammatory pain detection in human body through infrared image anal-
ysis.” In 2015 international symposium on advanced computing and communication
(ISACC), 251–257. IEEE, 2015. doi:10.1109/ISACC.2015.7377350
3 Three-Dimensional
Reconstruction and
Digital Printing
of Medical Objects
in Purview of Clinical
Applications
Sushitha Susan Joseph and Aju D
CONTENTS
3.1 Introduction.....................................................................................................40
3.2 Literature Review............................................................................................ 41
3.3 Medical 3D Printing Workflow....................................................................... 43
3.4 Three-Dimensional Printing Technologies......................................................46
3.4.1 Selective Laser Sintering.....................................................................46
3.4.2 Selective Laser Melting....................................................................... 47
3.4.3 Selective Electron Beam Melting........................................................ 47
3.4.4 Laser-Induced Forward Transfer......................................................... 47
3.4.5 Nanoparticle Jetting............................................................................. 48
3.4.6 Binder Jetting....................................................................................... 48
3.5 Biometals For 3D Printing............................................................................... 48
3.5.1 Stainless Steel...................................................................................... 49
3.5.2 Titanium Alloy.................................................................................... 49
3.5.3 Tantalum.............................................................................................. 50
3.5.4 Cobalt-Chromium Alloy...................................................................... 50
3.5.5 Magnesium.......................................................................................... 51
3.5.6 Iron....................................................................................................... 51
3.5.7 Zinc...................................................................................................... 51
3.6 Applications of 3D Printing in Healthcare...................................................... 52
3.6.1 Medical Education............................................................................... 52
3.6.2 Surgical Training and Practice............................................................ 53
3.6.2.1 Cardiac Surgery.................................................................... 53
3.6.2.2 Neurosurgery........................................................................ 53
3.6.2.3 Craniomaxillofacial Surgery................................................ 53
3.6.3 Personalized Implants, Prostheses, Surgical Tools,
and Bioprinting.................................................................................... 54
DOI: 10.1201/9781003226147-3 39
40 Machine Learning and Deep Learning in Medical Data Analytics
3.1 INTRODUCTION
Three-dimensional printing has made a significant impact on the manufacturing
industry. It is a quickly developing revolutionary technology that is receiving con-
siderable interest from both the scientific community and academicians with users
from various domains like aerospace, military, engineering, architecture, chemi-
cal industry, and automotive and medical fields. Three-dimensional printing, also
known as additive manufacturing technology, easily produces designs of composite
internal structure and architecture when compared to conventional methods. The
automobile and aerospace industries have been using 3D printing technology for
more than 30 years. Medical and pharmaceutical fields started using this technology
on the recent development of novel biodegradable materials. Today the technology
is rapidly expanding and has extensive applications in the clinical field ranging from
personalized implants to protective equipment.
The history of medical 3D printing begins with the invention of stereolithog-
raphy by Chuck Hull in 1983. In 1988 the first bioprinting was performed using
the technique of micro positioning of cells. In the early 1990s dental implants and
custom prosthetics were 3D printed for medical purposes. In 1999 first laser-assisted
bioprinting came into existence. In due course, technological advancement sup-
ported the development of organs from human cells which required the support of
3D printed scaffolds in 2001. In 2002 the 3D Bioplotter, using extrusion-based bio-
printing, came into reality followed by the development of the inkjet printer in 2003.
Further evolution of technology aimed complete functioning organ without scaffold
and this became a reality in 2004. The year 2008 witnessed the first 3D printed
prosthetic leg. Vascular constructs without scaffold were fabricated in 2009. In 2012
articular cartilage and artificial liver were bioprinted. Fabrication of tubular struc-
tures using coaxial technology was performed in 2015. In 2016 a cartilage model
was produced using a tissue integrated organ printer. Eventually in 2019 Brazilian
researchers successfully 3D printed a mini-liver which performs all the functions of
a liver. There is a rapid increase in the speed of development of 3D printing technolo-
gies in the medical field.
Doctors mostly rely on two-dimensional (2D) X-ray, CT, and MRI images to acquire
the perception of pathologies. There is a need to view and understand the pathology
and structural relationship prior to surgery. Three-dimensional reconstruction helps by
enhancing understanding and visualization which means surgery can be planned more
accurately. The emergence of 3D printing provides haptic qualities to models. Complex
surgical procedures require guidance to obtain esthetical results. When compared to
conventional learning, a 3D printed model can be used to analyze complex cases, prac-
tice surgical procedures, and teach medical students and patients.
3D Reconstruction and Digital Printing of Medical Objects 41
There has been an increasing demand for organ transplantation in the medical
field which resulted in the emergence of tissue engineering. In tissue engineering,
scaffold fabrication using conventional techniques, like solvent casting and particle
leaching, electrospinning, and gas foaming, results in restrictions in design flexibil-
ity. When compared to traditional techniques, 3D organ printing provides design
flexibility and automation. The necessity for enhanced visualization and tactile
properties has given rise to 3D printed anatomical models, patient-specific guides,
and custom-made prosthetics. Three-dimensionally printed personalized drugs
have the capability of providing accurate dosage suitable for each patient, boosting
drug absorption, and controlling the cell distribution and extracellular matrix for
drug testing.
This chapter provides detailed summary of the utilization of 3D reconstruction
and 3D printing technologies in the medical field. A brief overview of printing tech-
niques such as selective laser sintering (SLS), selective laser melting (SLM), selective
electron beam melting (SEBM), laser-induced forward transfer (LIFT), nanoparticle
jetting (NPJ), and binder jetting (BJ) used for manufacturing is provided. The spe-
cific characteristics and clinical applications of biometals such as stainless steel,
titanium, tantalum, cobalt chromium alloy, magnesium, iron, and zinc used in 3D
printing is discussed. Moreover, the application of 3D printing in medical education,
surgical training and practice, personalized implants, prostheses, surgical tools and
bioprinting, pharmaceutical industry, personal protective equipment is brought into
focus. Furthermore, the software, advantages, disadvantages, and limitations associ-
ated with 3D reconstruction and 3D printing are explored.
The organization of this chapter is as follows. Section 3.2 provides informa-
tion on the techniques used for 3D reconstruction of different organs. Further, Section
3.3 explains the workflow of medical 3D printing process. Current 3D printing
technologies used for manufacturing various components are discussed in
Section 3.4. Section 3.5 provides an overview of the biometals used in 3D print-
ing process. Section 3.6 gives a brief description of how 3D printing is being
utilized in medical applications. The challenges and future perspectives of 3D
reconstruction and printing are discussed in Section 3.7, followed by concluding
remarks in Section 3.8.
twins (Inserra et al. 2020) used 3D printed ribcage modeling, bone modeling, color-
coded skin and internal organs which substantially reduced operating times along
with the risk for the patients. Moreover, preoperative planning using the 3D recon-
structed anatomical models revealed the connection between two hearts and aided in
the surgeons’ technical preparations. In 2017 3D printed ovaries restored the ovarian
function in infertile mice when the scaffolds are follicle seeded with vascularization
(Laronda et al. 2017). Cyborg beast (Zuniga et al. 2015) is the 3D printed prosthetic
hand developed for children which aims at reducing the cost of prosthetics, making it
available to children from low income families. The Food and Drug Administration
(FDA) perspective of additively manufactured products (Di Prima et al. 2016) dis-
cusses 3D printing in medicine. The FDA centers handle different products. The
Center for Device and Radiological Health (CDRH) reviews medical devices, drugs,
and biologics. The Center for Drug Evaluation and Research (CDER) reviews 3D
printed drugs. The Center for Biologics Evaluation and Research (CBER) reviews
bioprinting products.
The power of Internet of Things (IoT) can be combined with 3D printing to com-
pensate for the limitations and challenges in the medical field (Haleem, Javaid, and
Vaishya 2020; Khamparia et al.; Goyal et al. 2021). Three-dimensional printing pro-
vides solutions to organ transplantation and the 3D printed organs increase patients’
longevity. In order to transmit and store data, sensors and actuators can be integrated
with these organs which can trace the organ’s lifeline. This gives doctors informa-
tion regarding the time to transplant the organ again (Sharma et al. 2020). Similarly,
3D printed pacemakers integrated with IoT can be used to track patient conditions.
Surgeries performed with the help of surgical robots provide freedom of hand move-
ment and customized 3D printed surgical tools reduce risks by providing more accu-
rate results (Jindal, Gupta, and Bhushan 2019; Kumar et al. 2020).
Artificial intelligence (AI) has the potential to influence healthcare. AI together
with 3D printing contributes to hyperrealism by obtaining realistic renderings dur-
ing surgical training (Engelhardt et al. 2018). During the design stage, 3D printing,
when combined with machine learning (ML) approaches of hierarchical clustering
and support vector machines (SVM), helps the novice designers to enhance the deci-
sion making process (Maidin, Campbell, and Pei 2012; Yao, Moon, and Bi 2017). AI
provides solutions to the process planning problem in 3D printing through improving
the slicing acceleration (A. Wang et al. 2017) and path optimization (Fok et al. 2016).
The technique of using conformal geometric algebra with ML (Pillai and Megalingam
2020) and the Random Forest algorithm based on decision trees (Patra et al. 2021)
successfully identified and reconstructed a tumor. ML techniques highly influence the
healthcare field (Pattnayak and Jena 2021) and organizations benefit from these expert
systems (Panigrahi, Ayus, and Jena 2021; Paramesha, Gururaj, and Jena 2021).
∫
M×N
f ( a, b ) d ( a, b ) < ∞ (3.1)
where the integral is taken with respect to a product measure on the space over
M × N , then
∫∫
f ( a, b ) db da =
M N
∫∫ f ( a, b ) da db =
N M
∫M×N
f ( a, b ) d ( a, b ) (3.2)
The first and second integrals are iterated integrals with respect to M , N respec-
tively and the third one is an integral with respect to the product of M , N.
The process of 3D printing an object from medical images consists of several steps.
The initial step is the data acquisition in the form of diagnostic scans, followed by seg-
mentation on the scans to extract the desired region of interest. Then 3D reconstruction
provides the volume representation of the segmented part in 3D display. Optimization
of anatomical geometry data is performed with computer-aided design (CAD) software.
Then this model is transformed into a standard tessellation or triangle language (STL)
file, after which it is sliced into digital layers. The output file is then imported to the 3D
printer and the suitable biomaterials and printing parameters of the printer are deter-
mined. The printer builds the model by depositing material first on the bottom layer
and then depositing layers one after the other on the bottom layer. The printed model
undergoes further refinement such as cleaning and polishing to obtain the desired part.
Figure 3.1 depicts the various steps involved in developing a 3D printed object.
Digital data and technology have transformed the imaging field. Medical imaging
is at the heart of the healthcare field as it helps in the reliable early detection and diag-
nosis of disease as well as in medical communication. The commonly used imaging
modalities for obtaining diagnostic scans are MRI, CT, positron emission tomogra-
phy (PET), and ultrasound. The acquired images are in the standard DICOM format.
Two-dimensional segmentation extracts the particular region of interest from the
whole scan and the 3D reconstruction creates a reference to form a physical model.
The most commonly used technologies for 3D reconstruction are ray casting,
splatting, shear warp, and texture mapping. Ray casting was developed in 1980s
and 1990s and involves rays being cast from the eye or viewpoint across the image
plane and dataset. Instead of selecting the closest point, the ray splits the volume for
the interpolation of color and opacity. Merging of the interpolated values produces
the visibility on the image plane. Consider a model which has volume with density
D ( a, b, c ) and is penetrated by ray S. For each point along the ray, there is an illu-
mination I ( a, b, c ) reaching the point ( a, b, c ) from the source of light. The intensity
of the ray depends on this value, phase function H , and the local density D ( a, b, c ).
The density function is given along the ray as D ( a ( t ) , b ( t ) , c ( t )) = D ( t ) and the
illumination from light source as I ( a ( t ) , b ( t ) , c ( t )) = I ( t ). The illumination spread
along S from a point distance t along the ray can be defined as I ( t ) D ( t ) H ( cos θ )
where θ is the angle between S and M , where M is the light vector. Ray casting
computes volume rendering integral Vλ ( x , r ), the amount of light of wavelength λ ,
coming from ray direction r, at the point of location x as in equation (3.3). Lλ is the
light of wavelength λ reflected at position s in the direction of r. The color L ( si ) and
opacity o ( si ) are calculated in the interval i, and the interval width ∆s.
H / ∆s i −1
Vλ ( x , r ) = ∑ i=0
∏ (1 − α ( s )) (3.3)
Lλ ( si ) o ( si ) .
j=0
j
where
Splatting is faster than ray casting and works on the principle of the feed for-
ward method. The idea behind splatting is to project the voxel onto the image plane
and the projection depends on the color and opacity of the voxel. Further computa-
tion determines the part of the image plane to be included for each thrown voxel.
Splatting computes Vλ ( x , r ) as in (3.3) with the values of Lλ ( si ) and o ( si ) computed
as follows:
(i +1)∆s (i +1)∆s
Lλ ( si ) =
∫ i∆s
Lλ ( s ) ds
and o ( si ) =
∫ i∆s
α ( s ) ds
(3.5)
∆s ∆s
slice, obtaining the intermediate image by combining the slices in fore end to rear
end sequence, transforming the intermediate image to the final image by warping it.
Shear warp computes Vλ ( x , r ) as in (3.3) with Lλ ( si ) and o ( si ) computed as in ray
casting but with an extra constraint ∆s, which depends on view direction as follows.
2 2
dx dy
∆s = + + 1 (3.6)
dz dz
of material onto the receiver layer. LIFT is capable of printing inks in a broad range
of viscosities and of any particle size, preserving all the functionality of the materi-
als. The strength of LIFT lies in its capability to print metals and oxides, ceramics,
polymers, biomolecules, and alive cells. Based on LIFT, mesenchymal stem cells
were printed for the construction of scaffold-free autologous grafts. DNA strands,
antigens, immunoglobulins, enzymes, human osteosarcoma cells, and stem cells
have also been 3D printed using LIFT.
3.4.5 Nanoparticle Jetting
NPJ is a newly evolved technology that uses liquid ink for wrapping metal powders.
The nanoparticles obtained by smashing large metal pieces are given as input into
a binder that creates a regular printing ink. The suspensions of powdered material
in the ink avoids the need for sieving as in powder-based techniques and the ink
released through the nozzle enables fine detail printing. Extreme temperatures inside
the system cause the liquid to vaporize leaving behind smooth parts made from the
building material. The advantage of NPJ is that it provides a good surface finish
and precision with simple operations which are safe too. High resolution of parts is
obtained at low cost. In comparison with traditional 3D printing, the parts obtained
by NPJ have a low temperature tolerance. Hearing aids, surgical tools, crowns, and
bridges are the commonly manufactured objects using NPJ.
3.4.6 Binder Jetting
BJ is a quickly developing additive manufacturing technology in which powder
material is deposited into a layer and the required shape of the layer is obtained
by selective joining using a polymeric liquid binder. Then the printed metal part
undergoes subsequent sintering in order to achieve the vital mechanical strength.
This technology uses a thermally controlled sintering process and involves
low-cost operations. Metallic materials, ceramics, and composite materials are
commonly used materials in the BJ process. BJ finds application in the manu-
facturing of medical models such as the heart, ankle, backbone, knee, and pelvis
(Salmi 2016).
which do not initiate an adverse response from the host and are suitable for long-term
implantations. Biodegradable materials are designed to degrade in the body over a
specific implantation period such that degradation performs a particular function. A
perfect biomaterial should be biologically compatible, easily printable, and mimic
real tissue.
Metals and their alloys are broadly used in the 3D printing of hard tissues as
implants and fixtures due to their high strength and ductility (Frazier 2014).
Permanent metallic implants obtained from surgical stainless steel (316L), cobalt-
chromium (CoCr) alloys, titanium (Ti) alloys, and tantalum (Ta) have applications in
fracture fixation, angioplasty, and bone remodeling (Saini et al. 2015). This is owing
to their good mechanical properties and long-term stability. Although low corrosion,
friction, and wear are observed in these materials, there are possibilities of metal
degradation resulting in the liberation of unwanted metallic ions leading to local
tissue damage, inflammatory reactions such as osteolysis, or systemic damage such
as metal hypersensitivity (Farahani, Dubé, and Therriault 2016). Moreover, perma-
nent metallic implants used in orthopedic applications require a costly and invasive
second surgery for removal or adjustment. In order to overcome these problems,
implants made of degradable biometals were developed. Biodegradable biometals
such as magnesium (Mg), iron (Fe), and zinc (Zn) alloys are used in orthopedic and
cardiovascular applications.
3.5.2 Titanium Alloy
Titanium and Ti-based alloys are superior to stainless steel for their high ratio
strength. Major amounts of orthopedic implants use Ti6Al4V (Ti-64) and commer-
cially pure titanium (CP–Ti). Three-dimensional printing technology is successful in
manufacturing implants with porous microstructures. Porous titanium possesses the
advantages of low weight and lower mechanical modulus compared to solid titanium
(<20 GPa vs. 110 GPa), resistance to corrosion, high surface area, and relatively
high mechanical strength. Porous titanium alloy implants reduce the stress shielding
effect, initiate human bone tissue development, and build a powerful contact with
tissues and implants. Three-dimensionally printed Ti-based porous implants develop
an enhanced osteointegration effect which provides favorable stability and long life
50 Machine Learning and Deep Learning in Medical Data Analytics
to the implants (Shi et al. 2019). There are many surgical resections that success-
fully place 3D printed titanium implants, including mandibular prosthesis (Gadia,
Shah, and Nene 2018), heel prosthesis (Imanishi and Choong 2015), cervical cage for
spinal fusion (Spetzger et al. 2017), metacarpal prosthesis (Punyaratabandhu et al.
2017), and vertebral body (Wei et al. 2020).
3.5.3 Tantalum
Tantalum (Ta) has been widely used in 3D printing due to its ideal biocompatibility
and anti-corrosion property. Tantalum substantially stimulates bone ingrowth, cell
proliferation, and osteointegration. The widespread acceptance of tantalum prod-
ucts was previously hindered by its costly manufacturing process and impotence
to build modular implants. In several studies, porous tantalum has been shown to
have the benefits of possessing less elastic modulus, substantial surface area, and
good pore connectivity which makes it apt for orthopedic and dental applications.
Three-dimensionally printed porous tantalum implants are employed as joint
prosthesis in total knee arthroplasty (Wang et al. 2020), hip arthroplasty (Hailer
2018), spine fusion surgery (Patel et al. 2020), trabecular metal material in dental
implants (Bencharit et al. 2014), and knee joint cushions. However, tantalum’s high
melting point of 3017°C makes it difficult for the 3D printing equipment to work
with tantalum powder. Moreover, porous scaffolds exhibit low fatigue resistance.
In order to overcome these difficulties, many studies have been conducted to fabri-
cate solid or porous Ta using new techniques such as laser engineered net shaping
(Balla et al. 2010), spark plasma sintering (Dudina, Bokhonov, and Olevsky 2019),
and selective laser melting (Thijs et al. 2013). Coating of Ta on Ti using laser
engineered net shaping improved osteointegration properties while strength and
ductility was improved using spark plasma sintering. Selective laser melting was
successful in improving the ductility and osteoconductive properties, and normal-
ized fatigue strength.
3.5.4 Cobalt-Chromium Alloy
Cobalt-chromium (Co-Cr) alloys are superalloys with cobalt and chromium as major
constituents. These biomaterials have biocompatibility with superior mechanical
properties such as good wear-corrosion resistance and fatigue resistance (Niinomi,
Narushima, and Nakai 2015). They have high strength and can be exposed to high
temperature conditions making it feasible for 3D printing of orthopedic implants,
dental prosthetics, and cardiovascular stents. Furthermore, exhaustive studies on the
mechanical property, biocompatibility, and microstructure of SLM-fabricated Co-Cr
alloys have proved their capability as promising substances for manufacturing 3D
printed dental and maxillofacial prosthetics (Pillai et al. 2021). Porous Co-Cr scaf-
folds reduce stress shielding, minimize elastic modulus, and provide implant longev-
ity by good osteointegration stability (L. Wang et al. 2017). Co-Cr alloy exhibits
increased ceramic–metal bond strength, excellent electrochemical stability, and
microstructure homogeneity. Three-dimensionally printed Co-Cr alloy shows the
3D Reconstruction and Digital Printing of Medical Objects 51
3.5.5 Magnesium
Biodegradable magnesium (Mg) and its alloys are the most promising candidates for
use in orthopedic, cardiology, respirology, urology implants as they eliminate the
effects of stress shielding and second implant removal surgery. Magnesium and its
alloys have similar density, stiffness, compressive yield strength, and elastic modu-
lus to human bone making it ideal to be used in load-bearing implants. Magnesium
alloys are biosafe and biocompatible, and they accelerate growth and healing of
bone by osteoblastic cell proliferation and differentiation. The orthopedic applica-
tions of magnesium are bone screws, rods, and plates. Magnesium alloys are used
as cardiovascular stents (Erbel et al. 2007), tracheal stents (Luffy et al. 2014), and
urinary implants (Zhang et al. 2016). In spite of the challenges involved in process-
ing magnesium alloys due to their flammability, the requisite of 3D printed products
composed of Mg alloys is increasing. Research studies showed that wire arc addi-
tive manufacturing can extract the good mechanical properties of magnesium (Han
et al. 2018; Gneiger et al. 2020). Porous magnesium and magnesium alloy scaffold
meet the functional requirement of the ideal bone substitute (Yazdimamaghani et al.
2017).
3.5.6 Iron
Iron (Fe) and Fe-based materials are favorable candidates for producing biodegrad-
able implants due to their strength, good mechanical characteristics, and medium
corrosion process. Iron is a non-toxic metal and the ions discharged during degrada-
tion can be held by the body which allow pure Fe stents to be fixed into porcine aorta
(Peuster et al. 2006).The first metal used in 3D printing of bio functional scaffolds
was an iron-based metal (Do et al. 2015). Development of iron-manganese (Fe-Mn)
alloys with enhanced degradation rate and good mechanical properties offers high
efficiency in biomedical applications (Schinhammer et al. 2010). Studies show that
Fe-Mn, Fe-Mn-Ca, and Fe-Mn-Mg alloys all exhibit cytocompatibility. Porous iron
scaffolds prepared by topological design using direct metal printing (DMP) has the
potential to be used in orthopedic applications as they promote bone regeneration
(Li et al. 2018).
3.5.7 Zinc
Zinc is a vital trace element in the human body. Zinc possesses the properties such
as good antibacterial activity and negligible toxicity, and the degradation rate is in
the middle of iron and magnesium. These properties along with biocompatibility
and biodegradability make zinc and its alloys promising materials to serve as dental
implants and vascular stents. Studies have been carried out on fabricating 3D printed
zinc using SLM (Demir, Monguzzi, and Previtali 2017).
52 Machine Learning and Deep Learning in Medical Data Analytics
3.6.1 Medical Education
Standard anatomy training of medical students is carried out using cadavers which
involves difficulties such as cost, availability of cadaver, adding formalin preserva-
tives, and ethical issues (Hsieh et al. 2018). The advent of 3D printing has paved the
way for using models in anatomy training with adequate haptic feedback and moder-
ate cost. They provide better interpretation of anatomical features of a disease state
preoperatively (Matthews et al. 2009). Three dimensional printed models of the ner-
vous system (Tam et al. 2018), heart (Z. Wang et al. 2017), skull (Chen et al. 2017),
spine (Li et al. 2015), ventricular system (Yi et al. 2019), and thoracic aorta (Garcia
et al. 2018) have been used in training of medical students. Multicolored 3D printed
models have the power to better interpret the normal and diseased anatomy, are
durable and can be reproduced. The 3D printed models can improve doctor–patient
communication by giving a detailed explanation regarding the diseased organ,
pathology, and surgical risks. Recent studies illustrate the significance and requisite
of integrating a course on 3D reconstruction and printing for undergraduate medical
students in medical schools (Z. Wang et al. 2017).
3.6.2.2 Neurosurgery
In 1999, a 3D printed stereolithographic model was developed to assist in the neu-
rosurgical planning of orbital brachytherapy (Poulsen et al. 1999). The planning
of skull-based surgeries (Müller et al. 2003), surgery for lesion near motor cortex
(Spottiswoode et al. 2013), cerebral aneurysm surgery (Stadie et al. 2008), and simu-
lation of neurosurgical procedures for cerebral tumors (Waran et al. 2014; Waran
et al. 2015) were performed using the 3D printed models. These models improve
confidence in the candidates as they can repeat the procedures on the model which
results in improving the hands-on experience in resection.
provide correct defect dimensions and positions to decide the common morphology
of the implant. The studies show that 3D printing shortens the surgery time and
improves the results for craniofacial surgeries (D’Urso et al. 2000). Orbital recon-
struction (Rohner et al. 2013) and surgery on malformed ears (Longfield, Brickman,
and Jeyakumar 2015) performed using 3D printed models show that it enhances the
learning experience of trainees.
3.6.4 Pharmaceutical Industry
Three-dimensional printing helps to overcome the challenges involved in tradi-
tional pharmaceutical unit operations. It provides flexibility in design and produc-
tion of personalized medicines. It includes unique dosage forms, personalized drug
dosing, and complex drug release profiles. Three-dimensional printing technology
is capable to produce loose and porous tablets which reduces swallowing diffi-
culties (Fu et al. 2004). Microneedles for delivering drugs through the skin can
be manufactured using 3D printing in a single step fabrication (Pere et al. 2018).
Using 3D printing technology, manufacturing of complex drugs in various colors,
shapes and sizes is possible. The shape of the drug influences the rate of drug
release and drug dissolution. Three-dimensional printing creates drugs with the
same dosage in different shapes. Spritam, the first 3D printed drug approved by the
FDA, was prepared by powder bed fusion (Jamroz et al. 2017). Three-dimensional
printing provides the power to decide the ideal amount of drug administered to
a patient based on the age, weight, gender, and metabolism. These personalized
3D Reconstruction and Digital Printing of Medical Objects 55
medicines can be further adapted to patient’s clinical feedback. Polypills are pills
that combine many drugs into a single tablet. The whole medication for a patient
with multiple chronic disease can be incorporated into a single pill using three-
dimensional printing. Moreover, it facilitates on-spot printing for fewer stability
drugs and on-demand manufacturing of drugs for natural disasters and military
operations. Progress in technology can introduce a chance of on-demand printing
of pills in hospitals and pharmacies.
directly affects the cellular growth in fabricated bone tissues. In order to interface
the scaffolds with the biological system, the materials should have good mechani-
cal properties. Another challenge is the color and texture similarity or dissimi-
larity of the 3D printed biomedical products with the organs. Multi-extruder 3D
printers are also not able to produce realistic results in terms of color and texture.
The challenges related to printers involve low dimensional accuracy, as in fused
deposition modeling, which depends on the software, screw movements, and the
firmware control. Powder agglomeration is another difficulty faced in 3D printing,
in which bigger pore agglomeration leads to poor densification. The size of the
nozzle is the challenge for achieving a nanoscale design for biomedical products.
Cost associated with the materials, investment, utility, and technical servicing is
another challenge associated with 3D printing. The lack of guidelines demands a
trial and error method to obtain the product, which is also challenging. Finally,
there is a risk of cyber-attack, as 3D printing technology uses an internet connec-
tion, so data confidentiality and integrity are essential. Further advancements in
technologies are required for improving the resolution without surrendering the
structure of scaffolds. There is a need to remove the powder particles stuck in
small channels. One possible strategy is to create powder particles in spherical
form for easy removal. Future research should concentrate on the nanoarchitecture
for the direct integration of molecules into scaffolds.
3.8 CONCLUSION
Current developments in mechanical systems as well as in the software field have
immensely boosted the resolution, precision, and speed of the 3D printing technol-
ogy. It has been widely used in medical education, the surgical field, pharmaceu-
tical industry, personal protective equipment, personalized implants, prosthesis,
and surgical tools. Some of the aspects that have to be considered for 3D print-
ing applications are the type of software and printer, production time, mechanical
characteristics and haptic feedback of the material, costs involved. There is an
insufficiency in the precision and effectiveness of the 3D printing of metals, par-
ticularly in obtaining a good surface finish. There exists an imbalance between the
speed and printing accuracy also. Upon finding solutions to obtain high accuracy,
good surface finish, and quick manufacturing, 3D printing can be extensively used
in the medical field. There exist only a few well-established materials like titanium
alloys, stainless steel, and aluminum alloys, which leads to a lack of raw materi-
als for 3D printing. Research studies need to be carried out to face this limiting
factor. The 3D printing techniques such as laser-based, ink-based, extrusion-based
methods involve different kinds of factors which determines the properties of
printed parts. Therefore, these factors must be optimized relative to the character-
istics required, particularly where cells and biomolecules are involved. Although
3D printing in the medical field faces constraints, the continuous expansion and
advancement of the biomedical materials industry and 3D printing technology can
provide unparalleled growth and opportunities. In the coming years, integration
of 3D reconstruction and printing with AI and big data can bring about significant
change in the biomedical field.
3D Reconstruction and Digital Printing of Medical Objects 57
REFERENCES
Abdel-Sayed, Philippe, and Ludwig Karl von Segesser. 2011. “Rapid Prototyping for
Training Purposes in Cardiovascular Surgery.” In Md Enamul Hoque, Ed. Advanced
Applications of Rapid Prototyping Technology in Modern Engineering. 61–74, Books
on Demand.
Abdelazeem, Rania M, Doaa Youssef, Jala El-Azab, Salah Hassab-Elnaby, and Mostafa
Agour. 2020. “Three-Dimensional Visualization of Brain Tumor Progression Based
Accurate Segmentation via Comparative Holographic Projection.” Plos One 15 (7):
e0236835.
Anastasiou, Athanasios, Charalambos Tsirmpas, Alexandros Rompas, Kostas Giokas,
and Dimitris Koutsouris. 2013. “3D Printing: Basic Concepts Mathematics and
Technologies.” In 13th IEEE International Conference on BioInformatics and
BioEngineering, 1–4.
Arakeri, Megha P, G Ram Mohana Reddy. 2013. “An Effective and Efficient Approach to 3D
Reconstruction and Quantification of Brain Tumor on Magnetic Resonance Images.”
International Journal of Signal Processing, Image Processing and Pattern Recognition
6 (3): 111.
Bahraminasab, Marjan. 2020. “Challenges on Optimization of 3D-Printed Bone Scaffolds.”
BioMedical Engineering OnLine 19 (1): 1–33.
Balla, Vamsi Krishna, Shashwat Banerjee, Susmita Bose, and Amit Bandyopadhyay. 2010.
“Direct Laser Processing of a Tantalum Coating on Titanium for Bone Replacement
Structures.” Acta Biomaterialia 6 (6): 2329–34.
Bencharit, Sompop, Warren C Byrd, Sandra Altarawneh, Bashir Hosseini, Austin Leong,
Glenn Reside, Thiago Morelli, and Steven Offenbacher. 2014. “Development and
Applications of Porous Tantalum Trabecular Metal-Enhanced Titanium Dental
Implants.” Clinical Implant Dentistry and Related Research 16 (6): 817–26.
Binder, Johannes S, Michael Scholz, Stephan Ellmann, Michael Uder, Robert Grützmann,
Georg F Weber, and Christian Krautz. 2021. “Cinematic Rendering in Anatomy: A
Crossover Study Comparing a Novel 3D Reconstruction Technique to Conventional
Computed Tomography.” Anatomical Sciences Education 14 (1): 22–31.
Chen, Hao, Yuchen He, and Weidong Jia. 2020. “Precise Hepatectomy in the Intelligent
Digital Era.” International Journal of Biological Sciences 16 (3): 365.
Chen, Shi, Zhouxian Pan, Yanyan Wu, Zhaoqi Gu, Man Li, Ze Liang, Huijuan Zhu, et al.
2017. “The Role of Three-Dimensional Printed Models of Skull in Anatomy Education:
A Randomized Controlled Trail.” Scientific Reports 7 (1): 1–11.
Choong, Yu Ying Clarrisa, Hong Wei Tan, Deven C Patel, Wan Ting Natalie Choong, Chun-
Hsien Chen, Hong Yee Low, Ming Jen Tan, Chandrakant D Patel, and Chee Kai Chua.
2020. “The Global Rise of 3D Printing during the COVID-19 Pandemic.” Nature
Reviews Materials 5 (9): 637–39.
D’Urso, Paul S, David J Effeney, W John Earwaker, Timothy M Barker, Michael J Redmond,
Robert G Thompson, and Francis H Tomlinson. 2000. “Custom Cranioplasty Using
Stereolithography and Acrylic.” British Journal of Plastic Surgery 53 (3): 200–204.
Demir, Ali Gökhan, Lorenzo Monguzzi, and Barbara Previtali. 2017. “Selective Laser
Melting of Pure Zn with High Density for Biodegradable Implant Manufacturing.”
Additive Manufacturing 15: 20–28.
Do, Anh-Vu, Behnoush Khorsand, Sean M Geary, and Aliasger K Salem. 2015. “3D Printing
of Scaffolds for Tissue Regeneration Applications.” Advanced Healthcare Materials 4
(12): 1742–62.
Duan, Fuqing, Donghua Huang, Yun Tian, Ke Lu, Zhongke Wu, and Mingquan Zhou. 2015.
“3d Face Reconstruction from Skull by Regression Modeling in Shape Parameter
Spaces.” Neurocomputing 151: 674–82.
58 Machine Learning and Deep Learning in Medical Data Analytics
Dudina, Dina V, Boris B Bokhonov, and Eugene A Olevsky. 2019. “Fabrication of Porous
Materials by Spark Plasma Sintering: A Review.” Materials 12 (3): 541.
Dworzak, Jalda, Hans Lamecker, Jens von Berg, Tobias Klinder, Cristian Lorenz, Dagmar
Kainmüller, Heiko Seim, Hans-Christian Hege, and Stefan Zachow. 2010. “3D
Reconstruction of the Human Rib Cage from 2D Projection Images Using a Statistical Shape
Model.” International Journal of Computer Assisted Radiology and Surgery 5 (2): 111–24.
Engelhardt, Sandy, Raffaele De Simone, Peter M Full, Matthias Karck, and Ivo Wolf. 2018.
“Improving Surgical Training Phantoms by Hyperrealism: Deep Unpaired Image-
to-Image Translation from Real Surgeries.” In International Conference on Medical
Image Computing and Computer-Assisted Intervention, 747–55.
Erbel, Raimund, Carlo Di Mario, Jozef Bartunek, Johann Bonnier, Bernard de Bruyne, Franz
R Eberli, Paul Erne, et al. 2007. “Temporary Scaffolding of Coronary Arteries with
Bioabsorbable Magnesium Stents: A Prospective, Non-Randomised Multicentre Trial.”
The Lancet 369 (9576): 1869–75.
Farahani, Rouhollah D, Martine Dubé, and Daniel Therriault. 2016. “Three-Dimensional
Printing of Multifunctional Nanocomposites: Manufacturing Techniques and
Applications.” Advanced Materials 28 (28): 5794–821.
Fok, Kai-Yin, Chi-Tsun Cheng, K Tse Chi, and Nuwan Ganganath. 2016. “A Relaxation Scheme
for TSP-Based 3D Printing Path Optimizer.” In 2016 International Conference on Cyber-
Enabled Distributed Computing and Knowledge Discovery (CyberC), 382–85.
Frazier, William E. 2014. “Metal Additive Manufacturing: A Review.” Journal of Materials
Engineering and Performance 23 (6): 1917–28.
Fu, Yourong, Shicheng Yang, Seong Hoon Jeong, Susumu Kimura, and Kinam Park. 2004.
“Orally Fast Disintegrating Tablets: Developments, Technologies, Taste-Masking and
Clinical Studies.” Critical ReviewsTM in Therapeutic Drug Carrier Systems 21 (6): 433–475.
Gadia, Akshay, Kunal Shah, and Abhay Nene. 2018. “Emergence of Three-Dimensional
Printing Technology and Its Utility in Spine Surgery.” Asian Spine Journal 12 (2): 365.
Garcia, Justine, ZhiLin Yang, Rosaire Mongrain, Richard L Leask, and Kevin Lachapelle.
2018. “3D Printing Materials and Their Use in Medical Education: A Review of Current
Technology and Trends for the Future.” BMJ Simulation and Technology Enhanced
Learning 4 (1): 27–40.
George, Mitchell, Kevin R Aroom, Harvey G Hawes, Brijesh S Gill, and Joseph Love. 2017.
“3D Printed Surgical Instruments: The Design and Fabrication Process.” World Journal
of Surgery 41 (1): 314–19.
Gneiger, Stefan, Johannes A Österreicher, Aurel R Arnoldt, Alois Birgmann, and Martin
Fehlbier. 2020. “Development of a High Strength Magnesium Alloy for Wire Arc
Additive Manufacturing.” Metals 10 (6): 778.
Gnonnou, Christo, and Nadia Smaoui. 2014. “Segmentation and 3D Reconstruction of MRI
Images for Breast Cancer Detection.” In Image Processing, Applications and Systems
Conference (IPAS), 2014 First International, 1–6.
González Izard, Santiago, Ramiro Sánchez Torres, Oscar Alonso Plaza, Juan Antonio Juanes
Mendez, and Francisco José Garcia-Peñalvo. 2020. “Nextmed: Automatic Imaging
Segmentation, 3D Reconstruction, and 3D Model Visualization Platform Using
Augmented and Virtual Reality.” Sensors 20 (10): 2962.
Goyal, Sukriti, Nikhil Sharma, Bharat Bhushan, Achyut Shankar, and Martin Sagayam. 2021.
“IoT Enabled Technology in Secured Healthcare: Applications, Challenges and Future
Directions.” In: Hassanien A.E., Khamparia A., Gupta D., Shankar K., Slowik A. (Eds.)
Cognitive Internet of Medical Things for Smart Healthcare, 25–48. Springer, Cham.
Greil, Gerald F, Ivo Wolf, Axel Kuettner, Michael Fenchel, Stephan Miller, Petros Martirosian,
Fritz Schick, Matthias Oppitz, Hans-Peter Meinzer, and Ludger Sieverding. 2007.
“Stereolithographic Reproduction of Complex Cardiac Morphology Based on High
Spatial Resolution Imaging.” Clinical Research in Cardiology 96 (3): 176–85.
3D Reconstruction and Digital Printing of Medical Objects 59
Guo, Lei, Ying Li, Dongbo Miao, Lei Zhao, Weili Yan, and Xueqin Shen. 2011. “3-D
Reconstruction of Encephalic Tissue in MR Images Using Immune Sphere-Shaped
SVMs.” IEEE Transactions on Magnetics 47 (5): 870–73.
Hailer, Nils. 2018. “20 Years of Porous Tantalum in Primary and Revision Hip Arthroplasty—
Time for a Critical Appraisal.” Acta Orthopaedica 89 (3): 254.
Haleem, Abid, Mohd Javaid, and Raju Vaishya. 2020. “3D Printing Applications for the
Treatment of Cancer.” Clinical Epidemiology and Global Health 8 (4): 1072–76.
Han, Seungkyu, Matthew Zielewski, David Martinez Holguin, Monica Michel Parra, and
Namsoo Kim. 2018. “Optimization of AZ91D Process and Corrosion Resistance Using
Wire Arc Additive Manufacturing.” Applied Sciences 8 (8): 1306.
Hsieh, Tsung-yen, Brian Cervenka, Raj Dedhia, Edward Bradley Strong, and Toby Steele.
2018. “Assessment of a Patient-Specific, 3-Dimensionally Printed Endoscopic Sinus
and Skull Base Surgical Model.” JAMA Otolaryngology–Head & Neck Surgery 144
(7): 574–79.
Ige, Oladeji O, Lasisi E Umoru, and Sunday Aribo. 2012. “Natural Products: A Minefield of
Biomaterials.” International Scholarly Research Notices 2012.
Imanishi, Jungo, and Peter F M Choong. 2015. “Three-Dimensional Printed Calcaneal
Prosthesis Following Total Calcanectomy.” International Journal of Surgery Case
Reports 10: 83–87.
Inserra, Alessandro, Luca Borro, Marco Spada, Simone Frediani, and Aurelio Secinaro. 2020.
“Advanced 3D ‘Modeling’ and ‘Printing’ for the Surgical Planning of a Successful Case
of Thoraco-Omphalopagus Conjoined Twins Separation.” Frontiers in Physiology 11.
Jacobs, Stephan, Ronny Grunert, Friedrich W Mohr, and Volkmar Falk. 2008. “3D-Imaging
of Cardiac Structures Using 3D Heart Models for Planning in Heart Surgery: A
Preliminary Study.” Interactive Cardiovascular and Thoracic Surgery 7 (1): 6–9.
Jamroz, Witold, Mateusz Kurek, Ewelina Łyszczarz, Witold Brniak, and Renata Jachowicz.
2017. “Printing Techniques: Recent Developments in Pharmaceutical Technology.”
Acta Poloniae Pharmaceutica 74 (3): 753–763.
Jindal, Mansi, Jatin Gupta, and Bharat Bhushan. 2019. “Machine Learning Methods for IoT
and Their Future Applications.” In 2019 International Conference on Computing,
Communication, and Intelligent Systems (ICCCIS), 430–34.
Karayegen, Gökay, and Mehmet Feyzi Aksahin. 2021. “Brain Tumor Prediction on MR
Images with Semantic Segmentation by Using Deep Learning Network and 3D Imaging
of Tumor Region.” Biomedical Signal Processing and Control 66: 102458.
Kermer, Christian, Andreas Lindner, Ingrid Friede, Arne Wagner, and Werner Millesi.
1998. “Preoperative Stereolithographic Model Planning for Primary Reconstruction
in Craniomaxillofacial Trauma Surgery.” Journal of Cranio-Maxillofacial Surgery 26
(3): 136–39.
Khamparia, Aditya, Prakash Kumar Singh, Poonam Rani, Debabrata Samanta, Ashish
Khanna, and Bharat Bhushan. 2020. “An Internet of Health Things-Driven Deep
Learning Framework for Detection and Classification of Skin Cancer Using Transfer
Learning.” Transactions on Emerging Telecommunications Technologies, 32: e3963.
Kigka, Vassiliki I, George Rigas, Antonis Sakellarios, Panagiotis Siogkas, Ioannis O
Andrikos, Themis P Exarchos, Dimitra Loggitsi, et al. 2018. “3D Reconstruction of
Coronary Arteries and Atherosclerotic Plaques Based on Computed Tomography
Angiography Images.” Biomedical Signal Processing and Control 40: 286–94.
Kumar, Santosh, Bharat Bhusan, Debabrata Singh, and Dilip kumar Choubey. 2020.
“Classification of Diabetes Using Deep Learning.” In 2020 International Conference
on Communication and Signal Processing (ICCSP), 651–55.
Lamadé, Wolfram, Gerald Glombitza, Lars Fischer, Peter Chiu, Carlos E Cárdenas Sr, M
Thorn, Hans-Peter Meinzer, et al. 2000. “The Impact of 3-Dimensional Reconstructions
on Operation Planning in Liver Surgery.” Archives of Surgery 135 (11): 1256–61.
60 Machine Learning and Deep Learning in Medical Data Analytics
Laronda, Monica M, Alexandra L Rutz, Shuo Xiao, Kelly A Whelan, Francesca E Duncan,
Eric W Roth, Teresa K Woodruff, and Ramille N Shah. 2017. “A Bioprosthetic Ovary
Created Using 3D Printed Microporous Scaffolds Restores Ovarian Function in
Sterilized Mice.” Nature Communications 8 (1): 1–10.
Lecron, Fabian, Jonathan Boisvert, Saïd Mahmoudi, Hubert Labelle, and Mohammed
Benjelloun. 2013. “Three-Dimensional Spine Model Reconstruction Using One-
Class SVM Regularization.” IEEE Transactions on Biomedical Engineering 60 (11):
3256–64.
Li, Y, H Jahr, K Lietaert, P Pavanram, A Yilmaz, L I Fockaert, M A Leeflang, et al. 2018.
“Additively Manufactured Biodegradable Porous Iron.” Acta Biomaterialia 77: 380–93.
Li, Zhenzhu, Zefu Li, Ruiyu Xu, Meng Li, Jianmin Li, Yongliang Liu, Dehua Sui, Wensheng
Zhang, and Zheng Chen. 2015. “Three-Dimensional Printing Models Improve
Understanding of Spinal Fracture—A Randomized Controlled Study in China.”
Scientific Reports 5 (1): 1–9.
Longfield, Evan A, Todd M Brickman, and Anita Jeyakumar. 2015. “3D Printed Pediatric
Temporal Bone: A Novel Training Model.” Otology & Neurotology 36 (5): 793–95.
Luffy, Sarah A, Da-Tren Chou, Jenora Waterman, Peter D Wearden, Prashant N Kumta, and
Thomas W Gilbert. 2014. “Evaluation of Magnesium-Yttrium Alloy as an Extraluminal
Tracheal Stent.” Journal of Biomedical Materials Research Part A 102 (3): 611–20.
Maidin, Shajahan Bin, Ian Campbell, and Eujin Pei. 2012. “Development of a Design Feature
Database to Support Design for Additive Manufacturing.” Assembly Automation 32
(3): 235–244.
Matthew, Di Prima, James Coburn, David Hwang, Jennifer Kelly, Akm Khairuzzaman, and
Laura Ricles. 2016. “Additively Manufactured Medical Products–the FDA Perspective.”
3D Printing in Medicine 2 (1): 1–6.
Matthews, Felix, Peter Messmer, Vladislav Raikov, Guido A Wanner, Augustinus L Jacob,
Pietro Regazzoni, and Adrian Egli. 2009. “Patient-Specific Three-Dimensional
Composite Bone Models for Teaching and Operation Planning.” Journal of Digital
Imaging 22 (5): 473–82.
More, Sujeet, Jimmy Singla, Sahil Verma, Uttam Ghosh, Joel J P C Rodrigues, A S M Sanwar
Hosen, In-Ho Ra, et al. 2020. “Security Assured CNN-Based Model for Reconstruction
of Medical Images on the Internet of Healthcare Things.” IEEE Access 8: 126333–46.
Mottl-Link, Sibylle, Michael Hübler, Titus Kühne, Urte Rietdorf, Julia J Krueger, Bernhard
Schnackenburg, Raffaele De Simone, et al. 2008. “Physical Models Aiding in Complex
Congenital Heart Surgery.” The Annals of Thoracic Surgery 86 (1): 273–77.
Müller, Adolf, Kartik G Krishnan, Eberhard Uhl, and Gerson Mast. 2003. “The Application
of Rapid Prototyping Techniques in Cranial Reconstruction and Preoperative Planning
in Neurosurgery.” Journal of Craniofacial Surgery 14 (6): 899–914.
Murr, Lawrence E, Sara M Gaytan, Edwin Martinez, Frank Medina, and Ryan B Wicker.
2012. “Next Generation Orthopaedic Implants by Additive Manufacturing Using
Electron Beam Melting.” International Journal of Biomaterials 2012.
Ng, Wei Long, Alvin Chan, Yew Soon Ong, and Chee Kai Chua. 2020. “Deep Learning
for Fabrication and Maturation of 3D Bioprinted Tissues and Organs.” Virtual and
Physical Prototyping 15 (3): 340–58.
Ngan, Elizabeth M, Ivan M Rebeyka, David B Ross, Mohamed Hirji, Johan F Wolfaardt,
Rosemary Seelaus, Andrew Grosvenor, and Michelle L Noga. 2006. “The Rapid
Prototyping of Anatomic Models in Pulmonary Atresia.” The Journal of Thoracic and
Cardiovascular Surgery 132 (2): 264–69.
Niinomi, Mitsuo, Takayuki Narushima, and Masaaki Nakai. “Advances in metallic bioma-
terials: Tissues, Materials and Biological Reactions.” Heidelberg, DE: Springer (2015).
van Noort, Richard. 2012. “The Future of Dental Devices Is Digital.” Dental Materials 28
(1): 3–12.
3D Reconstruction and Digital Printing of Medical Objects 61
Nugroho, Pratomo Adhi, Dwi Kurnia Basuki, and Riyanto Sigit. 2016. “3D Heart Image
Reconstruction and Visualization with Marching Cubes Algorithm.” In 2016
International Conference on Knowledge Creation and Intelligent Computing
(KCIC), 35–41.
Otton, James M, Roberto Spina, Romina Sulas, Rajesh N Subbiah, Neil Jacobs, David W
M Muller, and Brendan Gunalingam. 2015. “Left Atrial Appendage Closure Guided
by Personalized 3D-Printed Cardiac Reconstruction.” JACC: Cardiovascular
Interventions 8 (7): 1004–6.
Panigrahi, Niranjan, Ishan Ayus, and Om Prakash Jena. 2021. “An Expert System-Based
Clinical Decision Support System for Hepatitis-B Prediction & Diagnosis.”In Sachi
Nandan Mohanty, G. Nalinipriya, Om Prakash Jena, Achyuth Sarkar, Eds. Machine
Learning for Healthcare Applications, 57–75. Wiley Online Library.
Paramesha, K, H L Gururaj, and Om Prakash Jena. 2021. “Applications of Machine Learning
in Biomedical Text Processing and Food Industry.” In Sachi Nandan Mohanty, G.
Nalinipriya, Om Prakash Jena, Achyuth Sarkar, Eds. Machine Learning for Healthcare
Applications, 151–67. Wiley Online Library.
Patel, Manan Sunil, Johnathon R McCormick, Alexander Ghasem, Samuel R Huntley, and
Joseph P Gjolaj. 2020. “Tantalum: The next Biomaterial in Spine Surgery?” Journal of
Spine Surgery 6 (1): 72.
Patra, Sudhansu Shekhar, Om Praksah Jena, Gaurav Kumar, Sreyashi Pramanik, Chinmaya
Misra, and Kamakhya Narain Singh. 2021. “Random Forest Algorithm in Imbalance
Genomics Classification.” In Rabinarayan Satpathy, Tanupriya Choudhury, Suneeta
Satpathy, Sachi Nandan Mohanty, Xiaobo Zhang, Eds. Data Analytics in Bioinformatics:
A Machine Learning Perspective, 173–90. Wiley Online Library..
Pattanayak, Deepak K, A Fukuda, T Matsushita, M Takemoto, S Fujibayashi, K Sasaki, N
Nishida, T Nakamura, and T Kokubo. 2011. “Bioactive Ti Metal Analogous to Human
Cancellous Bone: Fabrication by Selective Laser Melting and Chemical Treatments.”
Acta Biomaterialia 7 (3): 1398–1406.
Pattnayak, Parthasarathi, and Om Prakash Jena. 2021. “Innovation on Machine Learning in
Healthcare Services–An Introduction.” In Sachi Nandan Mohanty, G. Nalinipriya, Om
Prakash Jena, Achyuth Sarkar, Eds. Machine Learning for Healthcare Applications,
1–15. Wiley Online Library..
Pennarossa, Georgia, Sharon Arcuri, Teresina De Iorio, Fulvio Gandolfi, and Tiziana
A L Brevini. 2021. “Current Advances in 3D Tissue and Organ Reconstruction.”
International Journal of Molecular Sciences 22 (2): 830.
Pere, Cristiane Patricia Pissinato, Sophia N Economidou, Gurprit Lall, Clémentine Ziraud,
Joshua S Boateng, Bruce D Alexander, Dimitrios A Lamprou, and Dennis Douroumis.
2018. “3D Printed Microneedles for Insulin Skin Delivery.” International Journal of
Pharmaceutics 544 (2): 425–32.
Peuster, Matthias, Carola Hesse, Tirza Schloo, Christoph Fink, Philipp Beerbaum, and
Christian von Schnakenburg. 2006. “Long-Term Biocompatibility of a Corrodible
Peripheral Iron Stent in the Porcine Descending Aorta.” Biomaterials 27 (28):
4955–62.
Pilipchuk, Sophia P, Alexandra B Plonka, Alberto Monje, Andrei D Taut, Alejandro Lanis,
Benjamin Kang, and William V Giannobile. 2015. “Tissue Engineering for Bone
Regeneration and Osseointegration in the Oral Cavity.” Dental Materials 31 (4): 317–38.
Pillai, Sangeeth, Akshaya Upadhyay, Parisa Khayambashi, Imran Farooq, Hisham Sabri,
Maryam Tarar, Kyungjun T Lee, et al. 2021. “Dental 3D-Printing: Transferring Art
from the Laboratories to the Clinics.” Polymers 13 (1): 157.
Pillai, Soumya S, and Rajesh Kannan Megalingam. 2020. “Detection and 3D Modeling of
Brain Tumor Using Machine Learning and Conformal Geometric Algebra.” In 2020
International Conference on Communication and Signal Processing (ICCSP), 257–61.
62 Machine Learning and Deep Learning in Medical Data Analytics
Poulsen, Michael, Cathy Lindsay, Tim Sullivan, and Paul D’Urso. 1999. “Stereolithographic
Modelling as an Aid to Orbital Brachytherapy.” International Journal of Radiation
Oncology* Biology* Physics 44 (3): 731–35.
Pradhan, Nitesh, Vijaypal Singh Dhaka, Geeta Rani, and Himanshu Chaudhary. 2020.
“Transforming View of Medical Images Using Deep Learning.” Neural Computing
and Applications, 1–12.
Punyaratabandhu, Thipachart, Boonrat Lohwongwatana, Chedtha Puncreobutr, Arkaphat
Kosiyatrakul, Puwadon Veerapan, and Suriya Luenam. 2017. “A Patient-Matched
Entire First Metacarpal Prosthesis in Treatment of Giant Cell Tumor of Bone.” Case
Reports in Orthopedics 2017.
Rohner, Dennis, Raquel Guijarro-Martinez, Peter Bucher, and Beat Hammer. 2013.
“Importance of Patient-Specific Intraoperative Guides in Complex Maxillofacial
Reconstruction.” Journal of Cranio-Maxillofacial Surgery 41 (5): 382–90.
Saini, Monika, Yashpal Singh, Pooja Arora, Vipin Arora, and Krati Jain. 2015. “Implant
Biomaterials: A Comprehensive Review.” World Journal of Clinical Cases: WJCC 3 (1): 52.
Salmi, Mika. 2016. “Possibilities of Preoperative Medical Models Made by 3D Printing or
Additive Manufacturing.” Journal of Medical Engineering 2016: 1–6.
Schievano, Silvia, Francesco Migliavacca, Louise Coats, Sachin Khambadkone, Mario
Carminati, Neil Wilson, John E Deanfield, Philipp Bonhoeffer, and Andrew M Taylor.
2007. “Percutaneous Pulmonary Valve Implantation Based on Rapid Prototyping of
Right Ventricular Outflow Tract and Pulmonary Trunk from MR Data.” Radiology 242
(2): 490–97.
Schinhammer, Michael, Anja C Hänzi, Jörg F Löffler, and Peter J Uggowitzer. 2010.
“Design Strategy for Biodegradable Fe-Based Alloys for Medical Applications.” Acta
Biomaterialia 6 (5): 1705–13.
Schmauss, Daniel, Christoph Schmitz, Amir Koshrow Bigdeli, Stefan Weber, Nicholas Gerber,
Andres Beiras-Fernandez, Florian Schwarz, Christoph Becker, Christian Kupatt, and
Ralf Sodian. 2012. “Three-Dimensional Printing of Models for Preoperative Planning
and Simulation of Transcatheter Valve Replacement.” The Annals of Thoracic Surgery
93 (2): e31–e33.
Sharma, Nikhil, Ila Kaushik, Bharat Bhushan, Siddharth Gautam, and Aditya Khamparia.
2020. “Applicability of WSN and Biometric Models in the Field of Healthcare.”
In: K. Martin Sagayam, Bharat Bhushan, A. Diana Andrushia, Victor Hugo C. de
Albuquerque (Eds.) Deep Learning Strategies for Security Enhancement in Wireless
Sensor Networks, 304–29. IGI Global.
Shi, Jianping, Huixin Liang, Jie Jiang, Wenlai Tang, and Jiquan Yang. 2019. “Design and
Performance Evaluation of Porous Titanium Alloy Structures for Bone Implantation.”
Mathematical Problems in Engineering 2019.
Sodian, Ralf, Daniel Schmauss, Mathias Markert, Stefan Weber, Konstantin Nikolaou,
Sandra Haeberle, Ferdinand Vogt, et al. 2008a. “Three-Dimensional Printing Creates
Models for Surgical Planning of Aortic Valve Replacement after Previous Coronary
Bypass Grafting.” The Annals of Thoracic Surgery 85 (6): 2105–8.
Sodian, Ralf, Stefan Weber, Mathias Markert, Markus Loeff, Tim Lueth, Florian C Weis,
Sabine Daebritz, Edward Malec, Christoph Schmitz, and Bruno Reichart. 2008b.
“Pediatric Cardiac Transplantation: Three-Dimensional Printing of Anatomic Models
for Surgical Planning of Heart Transplantation in Patients with Univentricular Heart.”
The Journal of Thoracic and Cardiovascular Surgery 136 (4): 1098–99.
Spetzger, Uwe, Alexander S Koenig, and others. 2017. “Individualized Three-Dimensional
Printed Cage for Spinal Cervical Fusion.” Digital Medicine 3 (1): 1.
Spottiswoode, B S, D J den Heever, Y Chang, S Engelhardt, S Du Plessis, F Nicolls, H B
Hartzenberg, and A Gretschel. 2013. “Preoperative Three-Dimensional Model Creation
of Magnetic Resonance Brain Images as a Tool to Assist Neurosurgical Planning.”
Stereotactic and Functional Neurosurgery 91 (3): 162–69.
3D Reconstruction and Digital Printing of Medical Objects 63
Stadie, Axel Thomas, Ralf Alfons Kockro, Robert Reisch, Andrei Tropine, Stephan Boor,
Peter Stoeter, and Axel Perneczky. 2008. “Virtual Reality System for Planning
Minimally Invasive Neurosurgery.” Journal of Neurosurgery 108 (2): 382–94.
Tam, Chun Hei Adrian, Yiu Che Chan, Yuk Law, and Stephen Wing Keung Cheng. 2018.
“The Role of Three-Dimensional Printing in Contemporary Vascular and Endovascular
Surgery: A Systematic Review.” Annals of Vascular Surgery 53: 243–54.
Tawbe, Khalil, François Cotton, and Laurent Vuillon. 2008. “Evolution of Brain Tumor
and Stability of Geometric Invariants.” International Journal of Telemedicine and
Applications 2008.
Thijs, Lore, Maria Luz Montero Sistiaga, Ruben Wauthle, Qingge Xie, Jean-Pierre Kruth,
and Jan Van Humbeeck. 2013. “Strong Morphological and Crystallographic Texture
and Resulting Yield Strength Anisotropy in Selective Laser Melted Tantalum.” Acta
Materialia 61 (12): 4657–68.
Tino, Rance, Ryan Moore, Sam Antoline, Prashanth Ravi, Nicole Wake, Ciprian N Ionita,
Jonathan M Morris, et al. 2020. “COVID-19 and the Role of 3D Printing in Medicine.”
3D Print Med 6, 11(2020).
Vranicar, Mark, William Gregory, William I Douglas, Peter Di Sessa, and Thomas G Di
Sessa. 2008. “The Use of Stereolithographic Hand Held Models for Evaluation of
Congenital Anomalies of the Great Arteries.” Studies in Health Technology and
Informatics 132: 538.
Wang, Aosen, Chi Zhou, Zhanpeng Jin, and Wenyao Xu. 2017. “Towards Scalable and
Efficient GPU-Enabled Slicing Acceleration in Continuous 3D Printing.” In 2017 22nd
Asia and South Pacific Design Automation Conference (ASP-DAC), 623–28.
Wang, Di, Yimeng Wang, Jianhua Wang, Changhui Song, Yongqiang Yang, Zimian Zhang,
Hui Lin, Yongqiang Zhen, and Suixiang Liao. 2016. “Design and Fabrication of a
Precision Template for Spine Surgery Using Selective Laser Melting (SLM).” Materials
9 (7): 608.
Wang, Fuyou, Hao Chen, Pengfei Yang, Aikeremujiang Muheremu, Peng He, Haquan Fan,
and Liu Yang. 2020. “Three-Dimensional Printed Porous Tantalum Prosthesis for
Treating Inflammation after Total Knee Arthroplasty in One-Stage Surgery–a Case
Report.” Journal of International Medical Research 48 (3): 0300060519891280.
Wang, Ling, Jianfeng Kang, Changning Sun, Dichen Li, Yi Cao, and Zhongmin Jin. 2017.
“Mapping Porous Microstructures to Yield Desired Mechanical Properties for
Application in 3D Printed Bone Scaffolds and Orthopaedic Implants.” Materials &
Design 133: 62–68.
Wang, Zhongmin, Yuhao Liu, Hongxing Luo, Chuanyu Gao, Jing Zhang, and Yuya Dai. 2017. “Is
a Three-Dimensional Printing Model Better than a Traditional Cardiac Model for Medical
Education? A Pilot Randomized Controlled Study.” Acta Cardiologica Sinica 33 (6): 664.
Waran, Vicknes, Vairavan Narayanan, Ravindran Karuppiah, Devaraj Pancharatnam,
Hari Chandran, Rajagopalan Raman, Zainal Ariff Abdul Rahman, Sarah L F Owen,
and Tipu Z Aziz. 2014. “Injecting Realism in Surgical Training—Initial Simulation
Experience with Custom 3D Models.” Journal of Surgical Education 71 (2): 193–97.
Waran, Vicknes, Vairavan Narayanan, Ravindran Karuppiah, Hari Chandran Thambynayagam,
Kalai Arasu Muthusamy, Zainal Ariff Abdul Rahman, and Ramez Wadie Kirollos.
2015. “Neurosurgical Endoscopic Training via a Realistic 3-Dimensional Model with
Pathology.” Simulation in Healthcare 10 (1): 43–48.
Wei, Feng, Zhehuang Li, Zhongjun Liu, Xiaoguang Liu, Liang Jiang, Miao Yu, Nanfang
Xu, et al. 2020. “Upper Cervical Spine Reconstruction Using Customized 3D-Printed
Vertebral Body in 9 Patients with Primary Tumors Involving C2.” Annals of
Translational Medicine 8 (6).
Yao, Xiling, Seung Ki Moon, and Guijun Bi. 2017. “A Hybrid Machine Learning Approach
for Additive Manufacturing Design Feature Recommendation.” Rapid Prototyping
Journal 23 (6): 983–997.
64 Machine Learning and Deep Learning in Medical Data Analytics
Yap, Chor Yen, Chee Kai Chua, Zhi Li Dong, Zhong Hong Liu, Dan Qing Zhang, Loong Ee
Loh, and Swee Leong Sing. 2015. “Review of Selective Laser Melting: Materials and
Applications.” Applied Physics Reviews 2 (4): 41101.
Yazdimamaghani, Mostafa, Mehdi Razavi, Daryoosh Vashaee, Keyvan Moharamzadeh,
Aldo R Boccaccini, and Lobat Tayebi. 2017. “Porous Magnesium-Based Scaffolds for
Tissue Engineering.” Materials Science and Engineering: C 71: 1253–66.
Yi, Xuehan, Chenyu Ding, Hao Xu, Tingfeng Huang, Dezhi Kang, and Desheng Wang. 2019.
“Three-Dimensional Printed Models in Anatomy Education of the Ventricular System:
A Randomized Controlled Study.” World Neurosurgery 125: e891–e901.
Yin, Ziying, Xin Lu, Salomon Cohen Cohen, Yi Sui, Armando Manduca, Jamie J Van Gompel,
Richard L Ehman, and John Huston. 2021. “A New Method for Quantification and 3D
Visualization of Brain Tumor Adhesion Using Slip Interface Imaging in Patients with
Meningiomas.” European Radiology 31: 1–11.
Yoo, Dong-Jin. 2011. “Three-Dimensional Surface Reconstruction of Human Bone Using
a B-Spline Based Interpolation Approach.” Computer-Aided Design 43 (8): 934–47.
Zhang, Binkai, Xiang Wang, Xiao Liang, and Jinjin Zheng. 2017. “3D Reconstruction of
Human Bones Based on Dictionary Learning.” Medical Engineering & Physics 49:
163–70.
Zhang, Jing, Ling-Rui Gong, Keping Yu, Xin Qi, Zheng Wen, Qiaozhi Hua, and others. 2020.
“3D Reconstruction for Super-Resolution CT Images in the Internet of Health Things
Using Deep Learning.” IEEE Access 8: 121513–25.
Zhang, Shiying, Yang Zheng, Liming Zhang, Yanze Bi, Jianye Li, Jiao Liu, Youbin Yu,
Heqing Guo, and Yan Li. 2016. “In Vitro and in Vivo Corrosion and Histocompatibility
of Pure Mg and a Mg-6Zn Alloy as Urinary Implants in Rat Model.” Materials Science
and Engineering: C 68: 414–22.
Zuniga, Jorge, Dimitrios Katsavelis, Jean Peck, John Stollberg, Marc Petrykowski, Adam
Carson, and Cristina Fernandez. 2015. “Cyborg Beast: A Low-Cost 3d-Printed
Prosthetic Hand for Children with Upper-Limb Differences.” BMC Research Notes 8
(1): 1–9.
4 Medical Text and
Image Processing
Applications, Methods,
Issues, and Challenges
Behzad Soleimani Neysiani
and Hassan Homayoun
CONTENTS
4.1 Introduction..................................................................................................... 65
4.2 Medical Text Processing and Analysis............................................................ 70
4.2.1 Applications......................................................................................... 70
4.2.2 Methods............................................................................................... 72
4.2.3 Issues and Challenges.......................................................................... 74
4.3 Medical Image Processing and Analysis......................................................... 74
4.3.1 Applications......................................................................................... 76
4.3.1.1 Regression and Classification Applications.......................... 77
4.3.1.2 Detection Applications......................................................... 77
4.3.1.3 Segmentation Applications................................................... 77
4.3.2 Methods............................................................................................... 78
4.3.3 Issues and Challenges.......................................................................... 81
4.4 Conclusion....................................................................................................... 82
References................................................................................................................. 83
4.1 INTRODUCTION
Today, almost all industries save their data in computers and electronic devices,
making electronic medical records (EMR) a critical resource for medical analysis
and decision making. A Google Scholar search using Publish or Perish (Harzing
1997) tool shows more than 2000 articles about image and text processing since
1974, in which 1953 articles seem to be more related. The distribution of articles
by year of publication is shown in Figure 4.1 based on a logarithmic scale to depict
lower and higher values simultaneously. The year distribution indicates a progressive
study in this field, but its complexity indicates the necessity of this field in research
and practical markets.
DOI: 10.1201/9781003226147-4 65
66 Machine Learning and Deep Learning in Medical Data Analytics
FIGURE 4.1 Yearly distribution of published articles on medical text and image processing
based on Google Scholar results (logarithmic scale).
Famous publishers like Elsevier, Springer, Wiley, and IEEE publish a quarter of
selected articles, as shown in Figure 4.2. Moreover, there are more than fifty-five
books in this field, indicating good progress and maturity. Even though there are
many articles in this field, the challenges and issues are vast and need more efforts
such as in (AI Multiple 2020) (1) patient care and effective treatment, (2) medical
diagnostic, (3) management, and (4) research and development. Every domain has
many problems, and their objectives and visions are far from their current states
and achievements in most problems. In other words, there is a vast opportunity for
research and development on this topic.
A static search based on article titles demonstrates there were more than 200 vari-
ous review articles in the selected articles, excluding books. These review articles
FIGURE 4.2 Distribution of articles on medical text and image processing by publisher
based on Google Scholar results (logarithmic scale).
Medical Text and Image Processing 67
FIGURE 4.3 Distribution of various review articles and research articles on medical text
and image processing based on Google Scholar results (logarithmic scale).
can be helpful for beginners to learn about this field from scratch aside from a
good background and even can be taught in academies. These review articles can
specialize in comparative studies, survey articles, systematic review and simple
review. The distribution of various review articles and other research articles is
shown in Figure 4.3.
There are many data types in the medical field, but, generally, data can be catego-
rized as structural and non-structural (Manogaran et al. 2017). The structured data
are usually nominal or numerical and can be easily compared and used in math-
ematical or relational operations, essential basic operations for statistical analysis, or
machine learning (ML) algorithms (Pattnayak and Jena 2021). Texts and images are
unstructured data samples that cannot be processed using the usual mathematical or
relational operations, and they need to be converted to structural data, in a process
called feature extraction (Soleimani Neysiani, Babamir, and Aritsugi 2020).
There are many researches about both data types and their distribution can be
seen in Figure 4.4, which indicates some articles studied both texts and images and
FIGURE 4.4 Distribution of articles on medical text and image processing by data type
based on Google Scholar results (logarithmic scale).
68 Machine Learning and Deep Learning in Medical Data Analytics
FIGURE 4.5 Distribution of ML algorithms about medical text and image processing based
on Google Scholar results (logarithmic scale).
others used just one type, especially those for which our keywords can recognize
their data type. Interestingly, images are the most common data type used in stud-
ies, then structural data, and, finally, texts. It depends on their dataset and their
importance, too. There are fewer available textual data in the medical field, which
could be explained by the fact that privacy-preservation for medical images is more
accessible than for texts. It should be mentioned that there exists more than 80% of
unstructured data in hospital databases (Perera et al. 2013).
ML (Pattnayak and Jena 2021) algorithms primarily use data processing tech-
niques to learn and predict events, categorize data, and find knowledge behind the
data. More than 830 articles use various ML algorithms and artificial intelligence
techniques for medical data processing based on Figure 4.5, in which deep learning
is the most used algorithm. Graph-based algorithms are usually used for network
data as another kind of unstructured medical data primarily used in the genetic
domain. Natural language processing (NLP) is used to process texts appropriate
for prescriptions or social media comments. There are some proposed algorithms
for data processing that are categorized in the “Algorithm” method. Besides,
statistical analysis is a traditional method for data description and analysis. The
neural network refers to traditional artificial neural network methods before deep
learning, like a multi-layered feed-forward perceptron model, a primarily used
technique, especially in healthcare systems like Diabetes prediction (Kumar et al.
2020). These techniques can be used as keywords for further investigation of data
processing methods in this field. It should be considered that 940 articles’ data pro-
cessing methods cannot be determined based on our keywords, excluding books
and non-research-based articles.
The articles’ topic distribution is shown in Figure 4.6, which indicates that the
genetic, mental, and drug domains are the topics most widely studied. Many fields
like radiography and breast usually deal with image data, and other diseases usually
Medical Text and Image Processing 69
FIGURE 4.6 Distribution of articles on medical text and image processing by topics based
on Google Scholar results (logarithmic scale).
use structural data. Viruses, microbes, cancers, tumors, or mental health like stress
and psychology have many physical treatment efforts. Drug discovery is another
exciting problem based on animal or human experiments or finding their genetic
science relations. The privacy consideration is an essential issue for medical records
because if patients know their records will be used for other analysis and their case
potentially published globally, they will be afraid to visit doctors, discuss about
their symptoms, and, finally, voluntarily make noise in their treatments by wrong or
imperfect data providing. Thus privacy-preserving is another challenge for medical
data processing.
In the following sections, both text and image processing will be discussed to
determine their applications, methods, issues, and data processing challenges.
It should be considered that unstructured data processing is a complex and time-
consuming operation, so many other engineering fields like distributed computing,
including cloud, grid, and blockchain, are regarded to deal with big medical data
(Biswas et al. 2014). Moreover, unstructured data processing follows the steps out-
lined in Figure 4.7 (Sun et al. 2018). In the first step, the unstructured data should be
gathered from paper or electronic records and then integrated to build a database.
Then unstructured data should be pre-processed to clean its noise and prepare it
correctly for the primary processing. Central processing usually consists of infor-
mation extraction and selection and data analyzing. Information selection reduces
FIGURE 4.7 Unstructured data processing procedure. (See Sun et al. 2018.)
70 Machine Learning and Deep Learning in Medical Data Analytics
4.2.1 Applications
Information retrieval (IR) refers to searching documents and finding the most relevant
and similar documents (Popowich 2005); a general operation in medical notes and
documents is known as information extraction, too (Uramoto et al. 2004). IR tech-
niques usually deal with term frequencies, NLP, and text mining techniques. Disease
representation is an IR application to find those records, including diseases or not based
on disease symptoms, for further classification (Dreisbach et al. 2019). Fraud detection
Medical Text and Image Processing 71
based on healthcare bills for medical insurance claims is another text mining applica-
tion involving NLP and IR techniques (Popowich 2005). Question and answering is
another application of text processing using NLP and IR techniques, which usually
rank the IR results based on the results of the question asked (Iroju and Olaleke 2015).
Medical personnels need to command the computers and devices using speech
recognition, especially during critical operations, for searching questions and find-
ing their answers (Q&A), and documenting events. The speech recognition proce-
dure usually converts voice to text and processes texts to extract information (Goss
et al. 2019). Moreover, document translation is necessary for international medical
agencies to receive international reports (Zhu, Tu, and Huang 2021).
Categorizing and phenotyping clinical text data is another frequent operation,
especially with regard to legacy patients’ records or big medical data like genetic
information which is placed into hierarchical or distinct sets. Clustering is the main
technique used for this purpose (Raja et al. 2008). The clustering will find similar
documents and keep them in distinct sets, such as a high inter-similarity and low
intra-similarity between sets (Trivedi et al. 2018).
Text summarization is essential due to the large volume of text for almost every
text-processing application like IR or Q&A to optimize user reading time and find-
ing the most related documents at a glance (Moradi and Ghadiri 2018).
Semantic web improves search engine accuracy to find the more related results
using anthologies about objects and terms relationships. It is a symbolic processing
act like IR, but this approach extracts related concepts of texts and finds the relation-
ship between extracted concepts to generate its output. The Unified Medical Language
System (UMLS) is introduced as a standard knowledge representation for medical
purposes. Symbolic processing usually uses finite states and grammars for parsing the
input query and produces its output (Meystre and Haug 2005; Iroju and Olaleke 2015).
Medical text reports need de-identification before public publishing for researchers
and other studies. De-identification is a privacy-preserving operation to delete the names
of patients, doctors, nurses, and hospitals. It needs NLP and IR techniques to find and
eliminate sensitive information in EMR as the data integrity retained, for example, sub-
stitute patients names with random unique identity numbers (Neamatullah et al. 2008).
Nowadays, the web and social media applications are useful repositories for such
purposes as sharing people’s viewpoints about diseases, their symptoms, their heuris-
tic treatments, and their opinions about treatments, doctors, and hospitals. Therefore,
their comments can be processed to extract their opinion about new events and sub-
jects (Demner-Fushman and Elhadad 2016).
There are already many built tools for processing medical text data which can be
categorized based on their application as the following:
1 https://www.medlee.ca/
2 https://ctakes.apache.org/
72 Machine Learning and Deep Learning in Medical Data Analytics
4.2.2 Methods
NLP made texts ready to be used for processing. NLP steps are shown in Figure 4.8,
which indicate various processes, including (Iroju and Olaleke 2015):
FIGURE 4.8 NLP analysis steps. (See Iroju and Olaleke, 2015.)
Medical Text and Image Processing 73
6. Pragmatic analysis: Every term may have more than one meaning depend-
ing on the context, e.g., apple is a fruit and an international brand for elec-
tronic devices like cellphones or laptops. Thus, every term’s meaning should
be considered based on the context of its sentence and even paragraph or
entire document.
NLP techniques are usually used in the pre-processing phase of text processing.
The usual pre-processing operations for text consider the following steps:
The IR technique usually uses two famous techniques called term frequency
(TF) and inverse document frequency (IDF) which are the number a term is
repeated in the selected text and entire documents of the database, respectively.
Many other heuristic techniques like minimum, average, and maximum TF/IDF
improve IR technique validation performance (Soleimani Neysiani, Babamir, and
Aritsugi 2020). TF (equation 4.1) can be calculated for each t term with a specific
length in a document, including k textual fields like f as d[f] with wf weight field
importance and average_lengthf for all terms in d[f] and bf is a constant to pre-
vent division by zero.
K
w f × occurrences (d [ f ], t )
TFD (t , d ) = ∑1− b b f × length
(4.1)
f =1 f +
average _ length f
IDF (equation 4.2) is a logarithm in Euler number base for dividing N documents
by the number of d documents, including t in their f textual field as d[f].
N
IDF (t , D) = log (4.2)
{d ∈ D : t ∈ d[ f ]}
74 Machine Learning and Deep Learning in Medical Data Analytics
BM25F (equation 4.3) is a weighted average of TF and IDF for two documents
that can calculate text similarities (Soleimani Neysiani and Babamir 2019b).
BM 25Fext ( d , q ) = ∑
t ∈d [ f ] ∩ q[ f ]
IDF ( t , Total Text Fields of Bug Reports )
(4.3)
×
(
TFD t , d [ f ] )
K1 + TFD t , d [ f ]( )
Feature extraction is the most crucial procedure for predictive application. There
are many feature extraction methods like using TF/IDF for the most critical terms
in texts. The longest common subsequence (LCS) finds the largest similar substring
between two texts, a widespread operation in genetic analysis and text mining appli-
cations (Soleimani Neysiani and Babamir 2019e). Another recent and innovative
approach is word embedding, which uses a deep neural network to represent a term
using a vector representation like word2vec (Chiu and Baker 2020).
TP + TN
Accuracy = (4.4)
TP + TN + FP + FN
TABLE 4.1
Comparison of MMTDNN results for Segmentation and Detection
of Abnormal Tissues in Medical Images with Other Methods
Dice Similarity
Method Accuracy Sensitivity Specificity Coefficient Jaccard Index
Fully CNN using Jaccard 0.963 0.926 0.971 0.922 0.861
Distance (Yuan, Chao,
and Lo 2017)
Very Deep Residual 0.949 0.911 0.957 0.897 0.829
Networks (Yu et al. 2017)
Multi-stage fully CNN 0.955 0.922 0.965 0.912 0.846
(Bi et al. 2017)
Massive Training ANN 0.861 0.790 0.847 0.713 0.580
(MTANN) (Khastavaneh
and Ebrahimpour-Komleh
2017)
MMTDNN (Khastavaneh 0.973 0.912 0.986 0.931 0.876
and Ebrahimpour-Komleh
2020a)
4.3.1 Applications
Based on three primary types of algorithms, three main application areas of auto-
mated analysis and processing of medical images are available. These applications
are very diverse, with various degrees of importance in clinics. Some of these
Medical Text and Image Processing 77
applications play an essential role in the diagnosis and follow-up of the disease.
Sometimes, this role has a substantial degree of importance so continuing treat-
ment without it is impossible. In other words, these broad ranges of applications
offer optimal treatment to different diseases and assist physicians with a second
opinion. Besides optimal treatment, automated medical image analysis is critical
and essential to conduct medical research, such as drug development and large
screening programs. In these cases, large populations need to be considered for
various purposes, such as measuring the risk of diseases and delineating abnor-
malities. According to the main algorithm, these applications are categorized into
three categories: regression/classification-based, localization or detection-based,
and segmentation-based.
4.3.2 Methods
This section aims to discuss the primary methods for analyzing and processing of
medical images. Nowadays, various simple image processing methods for analyz-
ing medical images and many state-of-the-art based on ML and deep learning are
proposed. As shown in Figure 4.9, a typical ML-oriented pipeline for medical image
analysis potentially includes seven stages: image acquisition, pre-processing, can-
didate extraction, feature generation, feature selection, main analysis, and post-
processing. A digital image of a specific organ of interest is obtained using one of the
previously mentioned medical imaging technologies in the image acquisition stage.
The pre-processing, as the next stage, depends on the analysis type and also imag-
ing technique. For example, noise elimination, bias field correction, and histogram
normalization are common tasks in the pre-processing stage. The candidate extrac-
tion stage produces initial expected results using fast clustering and thresholding
Medical Text and Image Processing 79
images, but not suitable for capturing localization information. A U-shaped network
has been proposed to have a CNN to capture both context and localization informa-
tion (Noh, Hong, and Han 2015). A typical U-shaped network is an encoder-decoder
network with two contraction and expansion paths, as shown in Figure 4.11 The
contraction path attempts to extract context information, while the expansion path
attempts to generate a segmentation mask. In U-shaped networks, some shortcut
connections from the contraction path to the expansion path take care of localization
information.
U-shaped networks have been employed for different segmentation tasks, includ-
ing segmentation of lesions in brain MR images (Duong et al. 2019), segmentation of
breast microcalcifications (Hossain 2019) and abnormalities (Almajalid et al. 2018),
segmentation of skin lesions (Tang et al. 2019; Goyal et al. 2020), and many other
organs and abnormalities.
4.4 CONCLUSION
In this study, representative applications and methods of medical image and text
processing are introduced. These applications and methods that serve physicians as
a second opinion are used for different purposes, including diagnostic, follow-up,
prevention, and medical research. Image analysis applications categorize different
tissues as healthily or abnormal, detect tissues and organs of interest in different
medical modalities, and accurately delineate organs or pathologies. Text analysis
applications are used to detect drug side effects, automated interpretation of medical
reports, information extraction, and text summarization.
As medical text and images are inherently unstructured data, typical feature-
based ML methods cannot correctly process and analyze these data types. As a
result, more sophisticated ML methods and techniques are needed. Deep neural net-
works are very advanced to act as end-to-end learners for analyzing unstructured
data. Deep networks apply multiple layer-wise transformations to the input data to
reach the final decision. In other words, unstructured data are abstracted in different
layers to facilitate final inference.
Moreover, deep networks can tackle complexities existing in medical data
by suppressing irrelevant information such as noise and artifacts, and amplify-
ing relevant information. Therefore, the applications based on deep methods are
considered state of the art. Moreover, much literature reported that the methods
based on the deep learning family are very successful in medical image and text
analysis.
A word cloud is drawn in Figure 4.12 based on titles and abstracts of selected
articles in this chapter about images and text medical data processing, in which the
boldest words indicate the more frequent and essential terms. This word cloud can
be used as a guideline for future work in this field, which relieves this chapter. The
word cloud demonstrates that image data are primarily used data against text and
structural data. Besides, it refers to many processing methods like deep learning.
The MRI technique is one of the boldest techniques in this figure, indicating its
importance among research articles.
Medical Text and Image Processing 83
REFERENCES
Agrawal, Shweta, and Sanjiv Kumar Jain. 2020. “Medical text and image process-
ing: Applications, issues and challenges.” In Machine Learning with Health Care
Perspective, 237–62. Springer, Cham.
Akkus, Zeynettin, Alfiia Galimzianova, Assaf Hoogi, Daniel L. Rubin, and Bradley J.
Erickson. 2017. “Deep learning for brain MRI segmentation: State of the art and
future directions.” Journal of Digital Imaging 30 (4): 449–59. https://doi.org/10.1007/
s10278-017-9983-4
Almajalid, Rania, Juan Shan, Yaodong Du, and Ming Zhang. 2018. “Development of a deep-
learning-based method for breast ultrasound image segmentation.” 2018 17th IEEE
International Conference on Machine Learning and Applications (ICMLA). 17-20 Dec,
Orlando, FL, USA, https://doi.org/10.1109/ICMLA.2018.00179
Anbeek, Petronella, Koen L. Vincken, Glenda S. van Bochove, Matthias J.P. van Osch, Jeroen
van der Grond. 2005. “Probabilistic segmentation of brain tissue in MR imaging.”
NeuroImage 27 (4): 795–804. https://doi.org/10.1016/j.neuroimage.2005.05.046
84 Machine Learning and Deep Learning in Medical Data Analytics
Araujo, T., G. Aresta, E. Castro, J. Rouco, P. Aguiar, C. Eloy, A. Polonia, and A. Campilho.
2017. “Classification of breast cancer histology images using convolutional neural net-
works.” PLoS One 12 (6): 1–14. http://journals.plos.org/plosone/article/file?id=10.1371/
journal.pone.0177544\& type=printable
Aronson, Alan R. 2001. “Effective mapping of biomedical text to the UMLS Metathesaurus:
the MetaMap program.” Proceedings of the AMIA Symposium: 17–21. https://pubmed.
ncbi.nlm.nih.gov/11825149
Bahadorpour, Mojdeh, Behzad Soleimani Neysiani, and Mohammad Nadimi Shahraki. 2017.
“Determining optimal number of neighbors in item-based kNN collaborative filter-
ing algorithm for learning preferences of new users.” Journal of Telecommunication,
Electronic and Computer Engineering (JTEC) 9 (3): 163–7.
Bi, Lei, Jinman Kim, Euijoon Ahn, Ashnil Kumar, Michael Fulham, and Dagan Feng. 2017.
“Dermoscopic image segmentation via multistage fully convolutional networks.” IEEE
Transactions on Biomedical Engineering 64 (9): 2065–74. https://doi.org/10.1109/
TBME.2017.2712771
Biswas, Sumon, Tanjina Akhter, MS Kaiser, and SA Mamun. 2014. “Cloud based healthcare
application architecture and electronic medical record mining: an integrated approach
to improve healthcare system.” 2014 17th International Conference on Computer
and Information Technology (ICCIT). 22-23 Dec, Dhaka, Bangladesh, https://doi.
org/10.1109/ICCITechn.2014.7073139
Cabezas, Mariano, Arnau Oliver, Eloy Roura, Jordi Freixenet, Joan C. Vilanova, and Torrent
Rami. 2014. “Automatic multiple sclerosis lesion detection in brain MRI by FLAIR
thresholding.” Computer Methods and Programs in Biomedicine 115 (3): 147–161.
https://doi.org/10.1016/j.cmpb.2014.04.006
Chiu, Billy, and Simon Baker. 2020. “Word embeddings for biomedical natural language
processing: A survey.” Language and Linguistics Compass 14 (12): e12402. https://doi.
org/10.1111/lnc3.12402
Cruz-Roa, Angel, Ajay Basavanhally, Fabio Gonzlez, Hannah Gilmore, Michael Feldman,
Shridar Ganesan, Natalie Shih, John Tomaszewski, and Anant Madabhushi.
2014. “Automatic detection of invasive ductal carcinoma in whole slide images
with convolutional neural networks.” 9041 (216): 904103. https://doi.org/10.1117/
12.2043872
Demner-Fushman, Dina, and Noemie Elhadad. 2016. “Aspiring to unintended consequences
of natural language processing: a review of recent developments in clinical and con-
sumer-generated text processing.” Yearbook of medical informatics (1): 224.
Doostali, Saeed, Seyed Morteza Babamir, Mohammad Shiralizadeh Dezfoli, and Behzad
Soleimani Neysiani. 2020. “IoT-based model in smart urban traffic control: Graph
theory and genetic algorithm.” 2020 11th International Conference on Information
and Knowledge Technology (IKT), 22-23 Dec, Tehran, Iran, https://doi.org/10.1109/
IKT51791.2020.9345623
Dreisbach, Caitlin, Theresa A Koleck, Philip E Bourne, and Suzanne Bakken. 2019. “A sys-
tematic review of natural language processing and text mining of symptoms from elec-
tronic patient-authored text data.” International Journal of Medical Informatics 125:
37–46.
Duong, M. T., J. D. Rudie, J. Wang, L. Xie, S. Mohan, J. C. Gee, and A. M. Rauschecker.
2019. “Convolutional Neural Network for Automated FLAIR Lesion Segmentation on
Clinical Brain MR Imaging.” American Journal of Neuroradiology 40 (8): 1282–90.
https://doi.org/10.3174/ajnr.A6138
Friedman, Carol, Lyudmila Shagina, Socrates A. Socratous, and Xiao Zeng. 1996. “A WEB-
based version of MedLEE: A medical language extraction and encoding system.”
Proceedings of the AMIA Annual Fall Symposium: 938. https://www.ncbi.nlm.nih.
gov/pmc/articles/PMC2233000/
Medical Text and Image Processing 85
Geremia, Ezequiel, Olivier Clatz, Bjoern H. Menze, Ender Konukoglu, Antonio Criminisi,
and Nicholas Ayache. 2011. “Spatial decision forests for MS lesion segmentation in
multi-channel magnetic resonance images.” NeuroImage 57 (2): 378–390. https://doi.
org/10.1016/j.neuroimage.2011.03.080
Ghasemi, Foad, Behzad Soleimani Neysiani, and Naser Nematbakhsh. 2020. “Feature
Selection in Pre-Diagnosis Heart Coronary Artery Disease Detection: A heuristic
approach for feature selection based on Information Gain Ratio and Gini Index.” 2020
6th International Conference on Web Research (ICWR), 22-23 April, Tehran, Iran,
https://doi.org/10.1109/ICWR49608.2020.9122285
Goss, Foster R, Suzanne V Blackley, Carlos A Ortega, Leigh T Kowalski, Adam B Landman,
Chen-Tan Lin, Marie Meteer, Samantha Bakes, Stephen C Gradwohl, and David W
Bates. 2019. “A clinician survey of using speech recognition for clinical documentation
in the electronic health record.” International Journal of Medical Informatics 130:
103938. https://doi.org/10.1016/j.ijmedinf.2019.07.017
Goyal, Manu, Amanda Oakley, Priyanka Bansal, Darren Dancey, and Moi Hoon Yap. 2020.
“Skin Lesion Segmentation in Dermoscopic Images with Ensemble Deep Learning
Methods.” IEEE Access 8: 4171–81. https://doi.org/10.1109/ACCESS.2019.2960504
Goyal, Sukriti, Nikhil Sharma, Bharat Bhushan, Achyut Shankar, and Martin Sagayam. 2021.
“IoT Enabled Technology in Secured Healthcare: Applications, Challenges and Future
Directions.” In Cognitive Internet of Medical Things for Smart Healthcare: Services
and Applications, edited by Aboul Hassanien, Aditya Khamparia, Deepak Gupta, K.
Shankar and Adam Slowik, 25–48. Cham: Springer International Publishing.
Greenhalgh, Trisha. 1997. “How to read a paper: the Medline database.” BMJ 315 (7101):
180–3.
Harzing, Anne-Wil. 1997. Publish or Perish. Tarma Software Research Pty Limited,
Melbourne, Australia.
Hazlehurst, Brian, John Mullooly, Allison Naleway, and Brad Crane. 2005. “Detecting pos-
sible vaccination reactions in clinical notes.” AMIA Annual Symposium proceedings
2005: 306–10. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1560600/
Homayoun, Hassan and Ebrahimpour-komleh, Hossein. 2021. “Automated segmentation of
abnormal tissues in medical images.” Journal of Biomedical Physics and Engineering
11 (4). pp. 415-424. https://doi.org/10.31661/jbpe.v0i0.958
Hoseini, Masome Sadat, Mohammad Nadimi Shahraki, and Behzad Soleimani Neysiani.
2015. “A new algorithm for mining frequent patterns in can tree.” 2015 2nd International
Conference on Knowledge-Based Engineering and Innovation (KBEI), 5-6 Nov,
Tehran, Iran, https://doi.org/10.1109/KBEI.2015.7436153
Hosny, Khalid M., Mohamed A. Kassem, and Mohamed M. Fouad. 2020. “Classification of
skin lesions into seven classes using transfer learning with AlexNet.” Journal of Digital
Imaging 33 (5): 1325–34. https://doi.org/10.1007/s10278-020-00371-9
Hossain, Md Shamim. 2019. “Microcalcification segmentation using modified u-net seg-
mentation network from mammogram images.” Journal of King Saud University –
Computer and Information Sciences. https://doi.org/10.1016/j.jksuci.2019.10.014
Hua, Kai-Lung, Che-Hao Hsu, Shintami Chusnul Hidayati, Wen-Huang Cheng, and Yu-Jen
Chen. 2015. “Computer-aided classification of lung nodules on computed tomography
images via deep learning technique.” OncoTargets and Therapy 8: 2015–22. https://doi.
org/10.2147/OTT.S80733
Iroju, Olaronke G, and Janet O Olaleke. 2015. “A systematic review of natural language pro-
cessing in healthcare.” International Journal of Information Technology and Computer
Science 7 (8): 44–50.
Jindal, M, J Gupta, and B Bhushan. 2019. “Machine learning methods for IoT and their future
applications.” 2019 International Conference on Computing, Communication, and
Intelligent Systems (ICCCIS), October 18–19, 2019.
86 Machine Learning and Deep Learning in Medical Data Analytics
Khamparia, Aditya, Prakash Kumar Singh, Poonam Rani, Debabrata Samanta, Ashish Khanna,
and Bharat Bhushan. 2020. “An internet of health things-driven deep learning framework
for detection and classification of skin cancer using transfer learning.” Transactions on
Emerging Telecommunications Technologies. https://doi.org/10.1002/ett.3963
Khastavaneh, Hassan and Ebrahimpour-Komleh, Hossein. 2015a. “Brain extraction using iso-
data clustering algorithm aided by histogram analysis.” 2015. International Conference
on Knowledge-Based Engineering and Innovation (KBEI), pp. 847-852, https://doi.
org/10.1109/KBEI.2015.7436154
. 2015b. “Brain extraction: A region based histogram analysis strategy,” Signal
Processing and Intelligent Systems Conference (SPIS), pp. 20-24, 16-17 Dec, Tehran,
Iran, https://doi.org/10.1109/SPIS.2015.7422305.
. 2017. “Neural network-based learning kernel for automatic segmentation of multiple
sclerosis lesions on magnetic resonance images.” Journal of Biomedical Physics &
Engineering 7 (2): 155–62. https://pubmed.ncbi.nlm.nih.gov/28580337.
. 2019. “Segmentation of Diabetic Retinopathy Lesions in Retinal Fundus Images
Using Multi-View Convolutional Neural Networks”, Iran Journal of Radiology 16
(Special Issue). https://doi.org/10.5812/iranjradiol.99148.
. 2020a. “MMTDNN: Multi-view massive training deep neural network for segmen-
tation and detection of abnormal tissues in medical images.” Frontiers in Biomedical
Technologies 7 (1): 22–32. https://doi.org/10.18502/fbt.v7i1.2722.
. 2020b. “Representation Learning Techniques: An Overview.” In: Bohlouli M.,
Sadeghi Bigham B., Narimani Z., Vasighi M., Ansari E. (eds) Data Science: From
Research to Application. CiDaS 2019. Lecture Notes on Data Engineering and
Communications Technologies, vol 45. Pp. 89-104. Springer, Cham. https://doi.
org/10.1007/978-3-030-37309-2_8.
Khastavaneh, Hassan and Haron, Hassan. 2014. “A Conceptual Model for Segmentation of
Multiple Scleroses Lesions in Magnetic Resonance Images Using Massive Training
Artificial Neural Network,” International Conference on Intelligent Systems, Modelling
and Simulation, pp. 273-278, 27-19 Jan, Langkawi, Malaysia, https://doi.org/10.1109/
ISMS.2014.53.
Kido, Shoji, Yasushi Hirano, and Shingo Mabu. 2020. “Deep learning for pulmonary image
analysis: Classification, detection, and segmentation.” Advances in Experimental
Medicine and Biology 1213: 47–58. https://doi.org/10.1007/978-3-030-33128-3_3
Klein, Stefan, Marius Staring, Keelin Murphy, Max A Viergever, and Josien P W Pluim. 2010.
“Elastix: A toolbox for intensity-based medical image registration.” IEEE Transactions
on Medical Imaging 29 (1): 196–205. https://doi.org/10.1109/TMI.2009.2035616
Kumar, S, B Bhusan, D Singh, and D K Choubey. 2020. “Classification of Diabetes
using Deep Learning.” International Conference on Communication and Signal
Processing (ICCSP), 28–30 July 2020, Chennai, India, https://doi.org/10.1109/
ICCSP48568.2020.9182293
Manogaran, Gunasekaran, Chandu Thota, Daphne Lopez, V Vijayakumar, Kaja M Abbas,
and Revathi Sundarsekar. 2017. “Big data knowledge system in healthcare.” In Chintan
Bhatt, Nilanjan Dey, Amira S. Ashour, Internet of Things and Big Data Technologies
for Next Generation Healthcare, 133–157. Springer International Publishing, Cham,
https://doi.org/10.1007/978-3-319-49736-5_7.
Meystre, Stéphane M, and Peter J Haug. 2005. “Comparing natural language processing
tools to extract medical problems from narrative text.” AMIA Annual Symposium
Proceedings 2005: 525–29. https://pubmed.ncbi.nlm.nih.gov/16779095
Moradi, Milad, and Nasser Ghadiri. 2018. “Different approaches for identifying important
concepts in probabilistic biomedical text summarization.” Artificial Intelligence in
Medicine 84: 101–16. https://doi.org/10.1016/j.artmed.2017.11.004
Medical Text and Image Processing 87
Sætre, Rune. 2006. “GeneTUC: Natural Language Understanding in Medical Text.” Doctoral
Thesis, Norwegian University of Science and Technology (NTNU), Trondheim,
Norway, https://ntnuopen.ntnu.no/ntnu-xmlui/handle/11250/249975
Samala, Ravi K., Heang-Ping Chan, Lubomir Hadjiiski, Mark A. Helvie, Jun Wei, and Kenny
Cha. 2016. “Mass detection in digital breast tomosynthesis: Deep convolutional neural
network with transfer learning from mammography.” Medical Physics 43 (12): 6654–6.
https://doi.org/10.1118/1.4967345
Sathitratanacheewin, Seelwan, Panasun Sunanta, and Krit Pongpirul. 2020. “Deep learning
for automated classification of tuberculosis-related chest X-Ray: dataset distribution
shift limits diagnostic performance generalizability.” Heliyon 6 (8): e04614. https://doi.
org/10.1016/j.heliyon.2020.e04614
Savova, Guergana K, James J Masanz, Philip V Ogren, Jiaping Zheng, Sunghwan Sohn,
Karin C Kipper-Schuler, and Christopher G Chute. 2010. “Mayo clinical Text Analysis
and Knowledge Extraction System (cTAKES): Architecture, component evaluation
and applications.” Journal of the American Medical Informatics Association 17 (5):
507–13. https://doi.org/10.1136/jamia.2009.001560
Setio, Arnaud Arindra Adiyoso, Francesco Ciompi, Geert Litjens, Paul Gerke, and Colin
Jacobs. 2016. “Pulmonary nodule detection in CT images: False positive reduction
using multi-view convolutional networks.” IEEE Transactions on Medical Imaging 35
(5): 1160–9. https://doi.org/10.1109/TMI.2016.2536809
Shi, Peng, Jing Zhong, Andrik Rampun, and Hui Wang. 2018. “A hierarchical pipeline
for breast boundary segmentation and calcification detection in mammograms.”
Computers in Biology and Medicine 96 (February): 178–188. https://doi.org/10.1016/j.
compbiomed.2018.03.011
Soleimani Neysiani, Behzad, and Seyed Morteza Babamir. 2016. “Methods of feature extrac-
tion for detecting the duplicate bug reports in software triage systems.” International
Conference on Information Technology, Communications and Telecommunications
(IRICT), 2 March. Tehran, Iran, http://www.sid.ir/En/Seminar/ViewPaper.aspx?ID=7677
. 2018. “Automatic Typos Detection in Bug Reports.” IEEE 12th International Conference
Application of Information and Communication Technologies, 8 July, Almaty, Kazakhstan.
. 2019a. “Automatic Interconnected Lexical Typo Correction in Bug Reports of Software
Triage Systems.” International Conference on Contemporary Issues in Data Science, 8
March, Zanjan, Iran.
. 2019b. “Duplicate detection models for bug reports of software triage systems: A sur-
vey.” Current Trends in Computer Sciences & Applications 1 (5): 128–34. https://doi.
org/10.32474/CTCSA.2019.01.000123
. 2019c. “Effect of typos correction on the validation performance of duplicate bug reports
detection.” 10th International Conference on Information and Knowledge Technology
(IKT), Tehran, Iran, 2020-Jan-2.
. 2019d. “Fast language-independent correction of interconnected typos to finding longest
terms: Using Trie for typo detection and correction.” 24th International Conference on
Information Technology (IVUS), Lithuania. http://ceur-ws.org/Vol-2470/
. 2019e. “Improving Performance of Automatic Duplicate Bug Reports Detection Using
Longest Common Sequence.” IEEE 5th International Conference on Knowledge-
Based Engineering and Innovation (KBEI), 28 Feb-1 March, Tehran, Iran, https://doi.
org/10.1109/KBEI.2019.8735038
. 2019f. “New labeled dataset of interconnected lexical typos for automatic correction in
the bug reports.” SN Applied Sciences 1 (11): 1385.
. 2019g. “New methodology for contextual features usage in duplicate bug reports detec-
tion: Dimension expansion based on Manhattan distance similarity of topics.” IEEE
5th International Conference on Web Research (ICWR), Tehran, Iran, 28 Feb-1 March,
https://doi.org/10.1109/ICWR.2019.8765296
Medical Text and Image Processing 89
Varzaneh, Hossein Hatami, Behzad Soleimani Neysiani, Hassan Ziafat, and Nasim Soltani.
2018. “Recommendation systems based on association rule mining for a target object
by evolutionary algorithms.” Emerging Science Journal 2 (2): 100–107.
Wang, Hongyu, and Yong Xia. 2018. “ChestNet: A deep neural network for classification of
thoracic diseases on chest radiography.” 1–8. https://arxiv.org/abs/1807.03058v1.
Yasar, Huseyin, and Murat Ceylan. 2021. “A novel comparative study for detection
of Covid-19 on CT lung images using texture analysis, machine learning, and deep
learning methods.” Multimedia Tools and Applications 80 (4): 5423–47. https://doi.
org/10.1007/s11042-020-09894-3
Yu, Lequan, Hao Chen, Qi Dou, Jing Qin, and Pheng-Ann Heng. 2017. “Automated
Melanoma Recognition in Dermoscopy Images via Very Deep Residual Networks.”
IEEE Transactions on Medical Imaging 36 (4): 994–1004. https://doi.org/10.1109/
TMI.2016.2642839
Yuan, Yading, Ming Chao, and Yeh-Chi Lo. 2017. “Automatic Skin Lesion Segmentation
Using Deep Fully Convolutional Networks With Jaccard Distance.” IEEE Transactions
on Medical Imaging 36 (9): 1876–86. https://doi.org/10.1109/TMI.2017.2695227
Zhu, Runjie, Xinhui Tu, and Jimmy Xiangji Huang. 2021. “Utilizing BERT for biomedi-
cal and clinical text mining.” In Data Analytics in Biomedical Engineering and
Healthcare, 73–103. Elsevier, London, United Kingdom.
5 Usage of ML Techniques
for ASD Detection
A Comparative Analysis
of Various Classifiers
Ashima Sindhu Mohanty, Priyadarsan
Parida, and Krishna Chandra Patra
CONTENTS
5.1 Introduction..................................................................................................... 91
5.2 Literature Survey............................................................................................. 93
5.3 Data Collection................................................................................................94
5.4 Methodology ...................................................................................................97
5.4.1 Pre-Processing.....................................................................................97
5.4.1.1 Standardization..................................................................... 98
5.4.1.2 Dimension Reduction Methods............................................. 98
5.4.2 Machine Learning Models................................................................ 100
5.4.2.1 k-Nearest Neighbors............................................................ 100
5.4.2.2 Support Vector Machine..................................................... 101
5.4.2.3 Naive Bayes......................................................................... 102
5.4.2.4 Decision Tree...................................................................... 102
5.4.3 Performance Parameters.................................................................... 103
5.5 Results and Discussion.................................................................................. 103
5.6 Conclusion..................................................................................................... 109
References............................................................................................................... 109
5.1 INTRODUCTION
Autism spectrum disorder (ASD) is a neurodevelopment state characterized by the
presence of repetitive behaviors and impairments in socio-communicative skills
(Mohanty, Patra, and Parida 2021). When the individual is 6 to 18 months old,
the beginning indications of ASD are noticed. Following the initial symptoms,
the individual further suffers from social as well as communication disability as
a result of unusual motor development during the first 18 to 36 months of life
(Backer 2015). Some of the abnormal behaviors associated with the individual
include uncertain giggling, problems in making eye contact, no response to sound
DOI: 10.1201/9781003226147-5 91
92 Machine Learning and Deep Learning in Medical Data Analytics
as well as physical pain, no interest in cuddling with parents, repeating words and
sentences, no proper attachment towards object, and less considerate to sudden
light or noise.
Identifying behavioral alterations due to ASD in children is much simpler than
in adolescent and adult cases, the reason being overlapping of some ASD signs with
other mental health disorders with the increase in individual’s age. This research
highlights the classification of ASD class as “ASD” or “no ASD” in the following
individuals: toddler (up to 36 months), child (4–11 years), adolescent (12–16 years) as
well as adult (17 years and over).
The “Qualitative Checklist for Autism in Toddlers (Q-CHAT-10)” (Robins
et al. 2001) and “Autism Spectrum Quotient (AQ-10)” (Baron-Cohen et al. 2006;
Auyeung et al. 2008) based on ten screening questions form the basis of investiga-
tion in the proposed approach. The screening questionnaires also exist in the ASD
data sets upon which investigation is done (Autism Research Centre – University of
Cambridge 1998).
The data sets used in this study are developed by Fadi Fayez Thabtah. The
researcher developed a mobile-based ASDTest app which is a screening applica-
tion and formulated toddler, child, adolescent, and adult ASD data sets, which are
utilized in this research. The toddler data set has 18 attributes with an output ASD
class whereas there are 21 attributes with an output ASD class in the case of child,
adolescent as well as adult data sets.
Successful classification of ASD classes on child, adolescent and adult cases
were found but the toddler case was left out. In the toddler dataset, major number
of toddler instances is of no ASD class which unbalances the respective data set.
This study emphasizes ASD classification on all categories of individuals includ-
ing toddler by using a minimum number of features from the data sets. Utilizing
ML, classifier models are built (Patra et al. 2021) to classify ASD classes. The clas-
sifier model is trained by training data and then the trained models get tested by
the test data. On-going analysis on classification of ASD highlights feature selec-
tion and dimension reduction, evolution of latest ML approaches for classification
of ASD class, enhancement of evaluation parameters, and reducing the diagnosis
time for ASD.
Healthcare is not only confined to neurological disorders which can be techni-
cally handled by artificial intelligence (Pattnayak and Jena 2021). But in addition
to neurology, there are numerous healthcare areas like skin cancer (which can be
detected from skin images by the application of deep learning along with the frame-
work of Internet of Health and Things (Khamparia et al. 2020)) and prediction of
Hepatitis-B (Panigrahi, Ayus, and Jena 2021). The implementation of Internet of
Things (IoT) in the healthcare sector increases productivity as well as examination
of data in healthcare unit (Goyal et al. 2021) Furthermore, application of wireless
sensor network (WSN) and biometric models in verifying fingerprint operations has
enhanced security and helped to preserve privacy in the healthcare sector (Sharma
et al. 2020). Presently, urban IoT systems, with the implementation of ML, supports
smart cities as well as advanced communication technologies (Jindal, Gupta, and
Bhushan 2019) in addition to biomedical text processing (Paramesha, Gururaj, and
Usage of ML Techniques for ASD Detection 93
Jena 2021). In the domain of medical applications, deep learning is utilized for the
classification of diabetes (Kumar et al. 2020).
Medical professionals can be positively benefitted by the method used in this
paper in providing awareness to the individuals with ASD symptoms for further
evaluation.
In this chapter, Section 5.2 outlines the related research work with classifier mod-
els, Section 5.3 outlines the data sets collected for the investigation followed by the
description of the proposed methodology in Section 5.4. The results obtained for
each category of individual are discussed in Section 5.5 and, finally, the conclusion
is outlined in Section 5.6.
method and compared the result from VA against CHI-SQ (Liu and Setiono
1995), IG (Pratiwi and Adiwijaya 2018), CFS (Wosiak and Zakrzewska 2018) and
correlation attribute evaluation analysis (Quinlan 1986). The ML classifiers are
repeated incremental pruning to produce error reduction (RIPPER) and C4.5 (DT)
(Salzberg 1994) classifying VA features with evaluation of performance param-
eters. VA selected minimum features in case of all the datasets unlike the former
mentioned filtering approaches followed by ML classification with acceptable rate
of performance parameters. The adolescent dataset outperformed the rest of the
datasets in the investigation.
The author in Thabtah (2019) put forward a rule-based machine learning (RML)
(Thabtah and Peebles 2020). The research covered the datasets used in Thabtah
(2019) over the same category of individuals having 1100 instances with 21 features
thereby dropping the toddler dataset from the investigation. There was the com-
parison of RML performance with eight ML classifiers: RIDOR, RIPPER, Bagging,
Nnge, Boosting, C4.5, CART, and PRISM (Gaines and Compton 1995; Salzberg
1994). The research experienced standard performance of RML compared with
other models of evaluation parameters.
The investigators in Akter et al. (2019) identified and detected ASD upon
adult, adolescent, child as well as toddler datasets. The source once again being
the UCI ML repository having 2009 instances. Three feature transformation
(FT) approaches – log, Z-score, and sine – were implemented. Following the
feature transformation, nine ML classifier models – Adaboost (Zhang et al. 2019)
flexible discriminant analysis (FDA), LDA, penalized discriminant analysis
(PDA) (Kassambara 2018), C5.0, mixture discriminant analysis (MDA), boosted
generalized linear model (Glmboost) (Hofner et al. 2014), CART, and SVM –
were utilized for the purpose of ASD classification followed by evaluation of
performance parameters. The result of investigation showed that the SVM and
Glmboost classifiers got maximum performance for toddler as well as adolescent
datasets respectively and Adaboost classifier showed standard performance for
child as well as adult datasets.
Raj and Masood (2020) made use of LR, NB, SVM, and KNN classifiers in addi-
tion to artificial neural network (ANN), as well as convolution neural network (CNN)
and speculated the probability of ASD in adult, adolescent, and child data sets. The
data sets were gathered from UCI ML repository covering 1100 instances with 21
features where the toddler data set was excluded due to its unbalanced nature. CNN
yielded maximum performance in case of all data sets. The category of adult data set
yielded maximum results. Table 5.1 summarizes the ASD classification approaches
as discussed in the literature survey.
Note: Data gathered by ASDTest app are stored in UCI repository for experimental purposes.
95
96 Machine Learning and Deep Learning in Medical Data Analytics
TABLE 5.2
ASD Data Sets
Number of
Number of Cases with
Name Attribute Number of Number Cases with no ASD
Sl no. of Data Set Source Type Attributes of Cases ASD Class Class
1 “Toddler” Kaggle “Categorical, 18 1054 728 326
(Thabtah continous and
2018) binary”
2 “Child” UCI ML “Categorical, 21 292 141 151
repository continous and
(Thabtah binary”
2017a)
3 “Adolescent” UCI ML “Categorical, 21 104 63 41
repository continous and
(Thabtah binary”
2017b)
4 “Adult” UCI ML “Categorical, 21 704 189 515
repository continous and
(Thabtah binary”
2017c)
research is conducted on toddler, child, adolescent as well as adult data sets. The
details of all data sets are encapsulated in Table 5.2.
All the data sets, excluding the toddler one, are characterized by some missing
values in the attributes: “age”, “ethnicity” as well as “Who_completed_the_test”.
The detail of missing value is summarized in Table 5.3.
Table 5.4 illustrates the details of common attribute in all data sets.
TABLE 5.3
Missing Values Present in ASD Data Sets
TABLE 5.4
Common Attributes in all ASD Data Sets
5.4 METHODOLOGY
In the proposed work, the input ASD data for all categories of individuals collected
from Kaggle as well as the UCI ML repository are pre-processed before classify-
ing under ASD class. The missing data in the data sets are dropped before stan-
dardization. The first phase of pre-processing is characterized by conversion of the
input data sets into numeric data followed by standardization, which is performed
to fit the data into ML models within a specific range. During the second phase of
pre-processing, the standardized inputs are applied to different dimension reduction
models for reducing the number of attributes in the data sets. Then, after the train-
ing data trains the ML classifier models, the trained models are tested by test data
for classification. The predicted outputs are finally found and compared against the
targets to evaluate the parameters such as accuracy, specificity, sensitivity, precision,
recall, Dice as well as F-measure. The workflow of the proposed technique is shown
in Figure 5.1
5.4.1 Pre-Processing
The raw data collected from the mentioned repositories are pre-processed in the first
stage. In this research the steps involved in pre-processing are standardization and
dimension reduction.
98 Machine Learning and Deep Learning in Medical Data Analytics
FIGURE 5.1 Architecture of the proposed method indicating processing and classification
using ML.
5.4.1.1 Standardization
After analyzing the data sets, it is observed that the attributes present in the datasets are
not properly scaled. Hence in order to properly scale the input attributes within a proper
range, standardization (Andrade 2020) is implemeted. Mean and standard deviation
approach is utilized to carry out the process of standardization for every single attribute
in the data set. Mathematically the standardized data is represented by equation 5.1,
( x − x mean )
Sta _ X = (5.1)
( x std )
where, x is current value of input X, x mean is mean value of X, and x std is standard
deviation of input X.
[C ]ij = ∑a b (5.2)
k =1
ik kj
cos(θ ) − sin(θ )
A= (5.3)
sin(θ ) cos(θ )
This matrix spins a vector about the origin by an angle of θ. One more instance is
to stretch matrix B in a 2-D space, as defined in equation 5.4,
k 0
B= (5.4)
0 1
Along the x-axis, this matrix stretches one vector via k, which is a constant factor,
but somehow y-direction remains unaffected. The stretching matrix in y-direction is
shown in equation 5.5,
k 0
B= (5.5)
0 1
(LMNN) dimension reduction (Sun and Chen 2011). Mathematically, impostors are
explicated by a simple inequality. For input ai with the label bi and target neighbor
a j , imposer is any output with label bl ≠ bi such that,
2 2
L (ai − al ) ≤ L (ai − a j ) + 1 (5.6)
Pj|i =
(
exp − ai − a j /2σ i2
2
) (5.7)
∑ exp ( − a − a
k ≠i
i k
2
/2σ i2 )
where, σ i is the variance of the Gaussian that is centered on data point ai .
In case of low-dimensional data points bi as well as a j of high-dimensional data
points ai as well as a j , computation of a similar conditional probability is possible
which is denoted by q j|i
q j |i =
(
exp − bi − b j
2
) (5.8)
∑ (
k ≠i
exp − bi − bk
2
)
Reasonably, the conditional probabilities Pj|i and q j|i should be the same for represent-
ing the similarity of data points in distinct dimensional spaces so that difference between
Pj|i and q j|i is nil for which the plot can be perfectly replicated in different dimensions.
Through this concept, t-SNE minimizes the difference for conditional probabil-
ity. It also minimizes the sum of differences in case of conditional probabilites.
the similar class unclassified data. By determining following distances the nearest
neighbor is decided. Assuming k as the total sample, mi as the ith input, ni as the
output for the respective input then,
Euclidean distance,
d ( m, n ) = ∑( m − n ) (5.9)
i =1
i i
2
Manhattan distance,
k
d ( m, n ) = ∑ m − n (5.10)
i =1
i i
Minkowski distance,
1/ q
k
d ( m, n ) =
∑(m − n )
i =1
i i
q
, q = 1,2,3,….. (5.11)
( )
ni wT φ ( mi ) + b ≥ 1 − ξi , ξ ≥ 0,
( )
K mi , m j ≡ φ ( mi ) φ m j (5.12)
T
( )
Other form of kernel functions are:
Linear function,
( )
K mi , m j = miT m j (5.13)
Polynomial function:
( ) ( )
d
K mi , m j = γ miT m j + r ,γ > 0 (5.14)
102 Machine Learning and Deep Learning in Medical Data Analytics
(
K mi , m j (5.15))
Sigmoid function:
( )
K mi , m j = tanh (γ miT + r )
P ( y | c ) P (c )
P (c | y ) = (5.16)
P ( y)
where, P(c|y) is posterior probability of class ‘c’ with the given predictor y, P(c) is
prior probability of class ‘c’, P(y|c) is likelihood which is probability of predictor
given class, and P(y) is prior probability of predictor.
E (T ) = ∑ − p log
i =1
i 2 pi (5.17)
where E is the entropy, T is the output class, pi is probability of ith class, and c
is number of output class.
b. Evaluation of entropy with two attributes:
E (T , X ) = ∑P (c) E (c , c ) (5.18)
c ∈X
1 2
Usage of ML Techniques for ASD Detection 103
G (T , X ) = E (T ) − E (T , X ) (5.19)
d. Continuation of the same process until entropy reaches 0. At that point, the
leaf node is assessed where all data are classified.
5.4.3 Performance Parameters
The performance parameters such as accuracy (Acc), sensitivity (Sn), specificity (Sp),
Dice coefficient (DC), precision (Pre), and F-measure (F1) evaluated the efficacy
of the proposed approach (Mohanty, Parida, and Patra 2021). All the performance
parameters are calculated from true positive (TP), true negative (TN), false positive
(FP) and false negative (FN) values. The values are achieved from the confusion
matrix (Thabtah and Peebles 2020). Mathematically the performance parameters are
represented in equations 5.20–5.25.
(TP + TN )
Acc = (5.20)
(TP + TN ) + ( FP + FN )
(TP)
SN = (5.21)
(TP + FN )
(TN )
SPE = (5.22)
(TN + FP)
2(TP)
DC = (5.23)
(2TP + FP + FN )
TP
Pre = (5.24)
TP + FP
(TP)
F1 = (5.25)
1
TP + ( FP + FN )
2
For the category of toddler data set, KNN classifier model for SVD attributes
resulted in 100 percent performance for no wrongly classified individuals under
ASD class. Following the KNN model is the performance of DT classifier model
with more than 99 percent performance. For DT model, it is found that one tod-
dler was wrongly classified as ASD and two toddlers were wrongly classified as
no ASD. For the LMNN attributes, KNN and SVM classifier models produced
100 percent performance following which DT classifier model produced more than
99 percent performance with four and one toddlers wrongly classified as ASD
and no ASD classes, respectively. In the case of t-SNE attributes, KNN classi-
fier model suppressed all other classifier models with 100 percent performance.
Following the KNN model is the DT classifier model with more than 95 percent
performance. The performance of the DT model slightly dropped due to a minor
increase in the number of wrongly classified individuals under ASD class, that is,
14 and 20 toddlers were wrongly classified as ASD and no ASD classes, respec-
tively. Somehow the performance of SVM and NB classifier models for t-SNE
attributes is not found so appealing due to the misclassification of 326 toddlers as
ASD class. Figure 5.2 shows the performance of applied ML algorithms in terms
of performance parameters on the toddler dataset.
In the category of the child data set, KNN classifier model in case of SVD attri-
butes outperformed the rest of the classifier models with 100 percent performance
with no wrong classification of individuals under ASD class. Next to the KNN model
is the performance of the DT classifier model which is beyond 98 percent with two
children wrongly classified as ASD and no ASD classes respectively. For the LMNN
attributes, the KNN model resulted in 100 percent performance with not a single
instance of wrong classification of individuals under ASD class. Following the KNN
model, the DT classifier model is the next one for LMNN attributes to produce more
than 95 percent performance with a small increase in misclassification of individuals
as 1 and 5 child individuals got misclassified as ASD and no ASD classes, respec-
tively. In case of t-SNE attributes, the performance of the KNN model outperformed
the rest of the classifier models with 100 percent performance followed by the DT
model with more than 90 percent performance, as well as two and ten children were
misclassified as ASD and no ASD classes, respectively. Somehow the performance
of SVM and NB classifier models for t-SNE attributes was not up to mark with
t-SNE attributes because of a greatly increased number of individual misclassifica-
tions. In the case of the SVM model, 56 and 51 children were misclassified as ASD
and no ASD classes, respectively, and for the NB model, 28 and 47 children faced
misclassification as ASD and no ASD classes, respectively. Figure 5.3 shows the
performance of applied ML algorithms in terms of performance parameters on the
child dataset.
For the category of adolescent data set, the KNN classifier model for SVD attri-
butes resulted in 100 percent performance with no instance of individual misclas-
sification. Next to the KNN model is the performance of the DT classifier model
which reached more than 90 percent performance with one and three adolescent
individuals misclassified as ASDF and no ASD classes, respectively. In addition,
the performance of SVM and NB classifier models was also acceptable. In the case
of LMNN attributes, the performance of the KNN classifier model outperformed
the rest of the classifier models with 100 percent performance with no instance of
individual misclassification. Following the KNN model is the performance of the
DT classifier model whose performance is more than 94 percent with two adolescent
individuals misclassified as ASD and two as no ASD. Somehow, the performance
Usage of ML Techniques for ASD Detection 107
of SVM and NB is also acceptable up to far extent. Finally, for t-SNE attributes, the
KNN classifier model resulted in 100 percent performance with no instance of indi-
vidual misclassification. Following the KNN model is the DT classifier model with
more than 90 percent performance, except for specificity which is found to be 83.33
percent due to the misclassification of six and one adolescent individuals as ASD and
no ASD classes, respectively. The performance of SVM and NB is not found to be
very appealing due to the greatly increased number of individual misclassification.
Figure 5.4 shows the performance of applied ML algorithms in terms of perfor-
mance parameters on the adolescent dataset.
In the category of adult data set, the best outcome for SVD attributes is shown
by the KNN classifier model with 100 percent performance and no instance of
individual misclassification. Following the KNN model is the performance of the
DT classifier model whose evaluation rate is more than 97 percent, with four and
one misclassified individuals as ASD and no ASD classes, respectively. However,
a remarkable rate of acceptance is also found in the case of SVM and NB classi-
fier models. For LMNN attributes, the KNN and SVM classifier models resulted
in the best performance with 100 percent evaluation rates and no instances of
individual misclassification. Next to KNN and SVM models, is the performance
of the DT classifier model with more than 99 percent of evaluation rate and one
instance misclassified as ASD class. However, the result of the NB classifier model
is to a greater degree or extent very acceptable with more than 90 percent per-
formance. Finally, for the t-SNE attributes, the best performance is shown by the
KNN classifier model with no misclassified instance. Following the KNN model
is the performance of the DT classifier model which is more than 95 percent with
four and eight adult individuals being misclassified as ASD and no ASD classes,
respectively. But the performance of SVM and NB is not up to the mark for t-SNE
108 Machine Learning and Deep Learning in Medical Data Analytics
TABLE 5.6
Performance Comparison Analysis
Individual’s Classifiers
Research in ASD Dataset Utilized Acc Sn Sp DC Pre F1
(Thabtah 2019) Adult LR 0.998 0.999 0.997 - - -
(Al-diabat 2018) Child FURIA 0.913 0.914 - - -
JRIP 0.928 - - -
(Vaishali and Sasikala Child SVM, MLP 0.996 - - - - -
2018)
(Thabtah, Kamalov, Adolescent C4.5 0.905 - - - -
and Rajab 2018) Adult C4.5 0.890 0.925 - - -
(Thabtah and Peebles Adult RML 0.940 0.940 0.970 - - 0.945
2020)
(Akter et al. 2019) Toddler SVM 0.987 0.993 - - - -
Adaboost - - 0.995 - - -
(Raj and Masood 2020) Adult CNN 0.995 0.993 1.000 - - -
Proposed method Child, Adolescent KNN 1.000 1.000 1.000 1.000 1.000 1.000
Toddler, Adult KNN, SVM 1.000 1.000 1.000 1.000 1.000 1.000
Usage of ML Techniques for ASD Detection 109
5.6 CONCLUSION
ASD greatly hampers the quality of life an individual should lead. Individuals with
ASD suffer a lot and their standard of living deteriorates. To improve the quality
and standard of life for the people with ASD, advanced detection with treatment is
crucial. The proposed procedure focuses on prior detection of ASD. In this work,
using dimension reduction techniques – SVD, LMNN, and t-SNE – the number of
attributes in all data sets are reduced followed by 80 percent training data to train
the various ML classifier models and the remaining 20 percent is used as testing
data to carry out testing. Finally, the classifier models detected ASD classes for all
categories of individuals – adult, adolescent, child, and toddler. The distinct perfor-
mance parameters in this work such as accuracy, sensitivity, specificity, dice, preci-
sion, recall, and F-measure are evaluated by utilizing various ML classifiers like
KNN, SVM, NB, and DT and proved to be clinically acceptable. The performance
of KNN and DT classifiers on all dimension reduction techniques proved to be lead-
ing whereas that of SVM and NB classifiers somehow lagged in performance due to
some misclassification of individuals under ASD classes. Overall, it can be assured
that ML classifier models can be applied in detecting ASD. Generally, in most of the
related investigations, the toddler data set is dropped due to its unbalanced character-
istics. As a result of the unbalanced nature, researchers found the respective data set
difficult to investigate. In this research, the toddler data set is investigated in addition
to other categories of data sets with successful ASD detection in toddlers.
REFERENCES
Aha, David W., Dennis Kibler, and Marc K. Albert. 1991. “Instance-Based Learning
Algorithms.” Machine Learning 6 (1): 37–66. https://doi.org/10.1007/BF00153759
Akter, Tania, Md. Shahriare Satu, Md. Imran Khan, Mohammad Hanif Ali, Shahadat Uddin,
Pietro Lio, Julian M. W. Quinn, and Mohammad Ali Moni. 2019. “Machine Learning-
Based Models for Early Stage Detection of Autism Spectrum Disorders.” IEEE Access
7: 166509–27. https://doi.org/10.1109/ACCESS.2019.2952609
Al-diabat, Mofleh. 2018. “Fuzzy Data Mining for Autism Classification of Children.”
International Journal of Advanced Computer Science and Applications 9 (7). https://
doi.org/10.14569/IJACSA.2018.090702
Andrade, Chittaranjan. 2020. “Understanding the Difference Between Standard Deviation
and Standard Error of the Mean, and Knowing When to Use Which.” Indian Journal
of Psychological Medicine 42 (4): 409–10. https://doi.org/10.1177/0253717620933419
Autism Research Centre - University of Cambridge. 1998. https://www.autismresearchcentre.
com/
Auyeung, Bonnie, Simon Baron-Cohen, Sally Wheelwright, and Carrie Allison. 2008. “The
Autism Spectrum Quotient: Children’s Version (AQ-Child).” Journal of Autism and
Developmental Disorders 38 (7): 1230–40. https://doi.org/10.1007/s10803-007-0504-z
Backer, Nouf Backer Al. 2015. “Developmental Regression in Autism Spectrum Disorder.”
Sudanese Journal of Paediatrics 15 (1): 21–6. http://www.ncbi.nlm.nih.gov/
pubmed/27493417
Baron-Cohen, Simon, Rosa A Hoekstra, Rebecca Knickmeyer, and Sally Wheelwright. 2006.
“The Autism-Spectrum Quotient (AQ)—Adolescent Version.” Journal of Autism and
Developmental Disorders 36 (3): 343–50. https://doi.org/10.1007/s10803-006-0073-6
110 Machine Learning and Deep Learning in Medical Data Analytics
Caluza, Las Johansen B. 2019. “Fuzzy Unordered Rule Induction Algorithm Application
Basic Programming Language Competence: A Rule-Based Model.” Indian Journal
of Science and Technology 12 (12): 1–10. https://doi.org/10.17485/ijst/2019/
v12i12/142575
Cendrowska, Jadzia. 1987. “PRISM: An Algorithm for Inducing Modular Rules.”
International Journal of Man-Machine Studies 27 (4): 349–70. https://doi.org/10.1016/
S0020-7373(87)80003-2
Gaines, B. R., and P. Compton. 1995. “Induction of Ripple-down Rules Applied to Modeling
Large Databases.” Journal of Intelligent Information Systems 5 (3): 211–28. https://doi.
org/10.1007/BF00962234
Golub, G., and W. Kahan. 1965. “Calculating the Singular Values and Pseudo-Inverse of
a Matrix.” Journal of the Society for Industrial and Applied Mathematics Series B
Numerical Analysis 2 (2): 205–24. https://doi.org/10.1137/0702016
Goyal, Sukriti, Nikhil Sharma, Bharat Bhushan, Achyut Shankar, and Martin Sagayam.
2021. “IoT Enabled Technology in Secured Healthcare: Applications, Challenges and
Future Directions.” In Janusz Kacprzyk (Ed.) Cognitive Internet of Medical Things
for Smart Healthcare. 25–48. https://doi.org/10.1007/978-3-030-55833-8_2
Hofner, Benjamin, Andreas Mayr, Nikolay Robinzonov, and Matthias Schmid. 2014.
“Model-Based Boosting in R: A Hands-on Tutorial Using the R Package Mboost.”
Computational Statistics 29 (1–2): 3–35. https://doi.org/10.1007/s00180-012-0382-5
Liu, H. and R. Setiono. 1995 “Chi2: Feature Selection and Discretization of Numeric
Attributes.” In Proceedings of 7th IEEE International Conference on Tools with
Artificial Intelligence, 388–91. IEEE Comput. Soc. Press. https://doi.org/10.1109/
TAI.1995.479783
Jindal, Mansi, Jatin Gupta, and Bharat Bhushan. 2019. “Machine Learning Methods for IoT
and Their Future Applications.” In 2019 International Conference on Computing,
Communication, and Intelligent Systems (ICCCIS), 430–34. IEEE. https://doi.
org/10.1109/ICCCIS48478.2019.8974551
Kassambara, A. 2018. “Classification Methods Essentials.” Statistical Tools for High-Throughput
Data Analysis. http://www.sthda.com/english/articles/36-classification-methods-
essentials/146-discriminant-analysis-essentials-in-r/
Keerthi, S. S., S. K. Shevade, C. Bhattacharyya, and K. R. K. Murthy. 2001. “Improvements
to Platt’s SMO Algorithm for SVM Classifier Design.” Neural Computation 13 (3):
637–49. https://doi.org/10.1162/089976601300014493
Khamparia, Aditya, Prakash Kumar Singh, Poonam Rani, Debabrata Samanta, Ashish
Khanna, and Bharat Bhushan. 2020. “An Internet of Health Things-driven Deep
Learning Framework for Detection and Classification of Skin Cancer Using Transfer
Learning.” Transactions on Emerging Telecommunications Technologies, May. https://
doi.org/10.1002/ett.3963
Kumar, Santosh, Bharat Bhusan, Debabrata Singh, and Dilip kumar Choubey. 2020.
“Classification of Diabetes Using Deep Learning.” In 2020 International Conference
on Communication and Signal Processing (ICCSP), 0651–55. IEEE. https://doi.
org/10.1109/ICCSP48568.2020.9182293
Liu, Sanyang, Mingmin Zhu, and Youlong Yang. 2013. “A Bayesian Classifier Learning
Algorithm Based on Optimization Model.” Mathematical Problems in Engineering
2013: 1–9. https://doi.org/10.1155/2013/975953
Melit Devassy, Binu, and Sony George. 2020. “Dimensionality Reduction and Visualisation
of Hyperspectral Ink Data Using T-SNE.” Forensic Science International 311 (June):
110194. https://doi.org/10.1016/j.forsciint.2020.110194
Mohanty, Ashima Sindhu, Priyadarsan Parida, and K. C. Patra. 2021. “Identification of Autism
Spectrum Disorder Using Deep Neural Network.” Journal of Physics: Conference
Series 1921 (May): 012006. https://doi.org/10.1088/1742-6596/1921/1/012006
Usage of ML Techniques for ASD Detection 111
Mohanty, Ashima Sindhu, Krishna Chandra Patra, and Priyadarsan Parida. 2021. “Toddler
ASD Classification Using Machine Learning Techniques.” International Journal of
Online and Biomedical Engineering (IJOE) 17 (07): 156. https://doi.org/10.3991/ijoe.
v17i07.23497
Pal, S. K., and S. Mitra. 1992. “Multilayer Perceptron, Fuzzy Sets, and Classification.” IEEE
Transactions on Neural Networks 3 (5): 683–97. https://doi.org/10.1109/72.159058
Panigrahi, Niranjan, Ishan Ayus, and Om Prakash Jena. 2021. “An Expert System-Based
Clinical Decision Support System for Hepatitis-B Prediction & Diagnosis.” In
Om Prakash Jena, ed., Machine Learning for Healthcare Applications, 57–75. Wiley-
Scrivener Publisher. https://doi.org/10.1002/9781119792611.ch4
Paramesha, K., H. L. Gururaj, and Om Prakash Jena. 2021. “Applications of Machine
Learning in Biomedical Text Processing and Food Industry.” In Om Prakash Jena, ed.,
Machine Learning for Healthcare Applications, 151–67. Wiley-Scrivener Publisher.
https://doi.org/10.1002/9781119792611.ch10
Patra, Sudhansu Shekhar, Om Praksah Jena, Gaurav Kumar, Sreyashi Pramanik, Chinmaya
Misra, and Kamakhya Narain Singh. 2021. “Random Forest Algorithm in Imbalance
Genomics Classification.” In Rabinarayan Satpathy, Tanupriya Choudhury, Suneeta
Satpathy, Sachi Nandan Mohanty, Xiaobo Zhang, ed., Data Analytics in Bioinformatics,
173–90. Wiley-Scrivener Publisher. https://doi.org/10.1002/9781119785620.ch7
Pattnayak, Parthasarathi, and Om Prakash Jena. 2021. “Innovation on Machine Learning
in Healthcare Services–An Introduction.” In Om Prakash Jena, Machine Learning
for Healthcare Applications, 1–15. Wiley-Scrivener Publisher. https://doi.
org/10.1002/9781119792611.ch1
Pratiwi, Asriyanti Indah, and Adiwijaya. 2018. “On the Feature Selection and Classification
Based on Information Gain for Document Sentiment Analysis.” Applied Computational
Intelligence and Soft Computing 2018: 1–5. https://doi.org/10.1155/2018/1407817
Quinlan, J. Ross. 1986. “Induction of Decision Trees.” Machine Learning 1 (1): 81–106.
https://doi.org/10.1007/BF00116251
Raj, Suman, and Sarfaraz Masood. 2020. “Analysis and Detection of Autism Spectrum
Disorder Using Machine Learning Techniques.” Procedia Computer Science 167:
994–1004. https://doi.org/10.1016/j.procs.2020.03.399
Robins, D. L., D. Fein, M. L. Barton, and J. A. Green. 2001. “The Modified Checklist for
Autism in Toddlers: An Initial Study Investigating the Early Detection of Autism and
Pervasive Developmental Disorders.” Journal of Autism and Developmental Disorders
31 (2): 131–44. https://doi.org/10.1023/a:1010738829569
Salzberg, Steven L. 1994. “C4.5: Programs for Machine Learning by J. Ross Quinlan.
Morgan Kaufmann Publishers, Inc., 1993.” Machine Learning 16 (3): 235–40. https://
doi.org/10.1007/BF00993309
Sharma, Nikhil, Ila Kaushik, Bharat Bhushan, Siddharth Gautam, and Aditya Khamparia.
2020. “Applicability of WSN and Biometric Models in the Field of Healthcare.” In
Deep Learning Strategies for Security Enhancement in Wireless Sensor Networks, IGI
Global, 304–29. https://doi.org/10.4018/978-1-7998-5068-7.ch016
Sun, Shiliang, and Qiaona Chen. 2011. “Hirarchical Distance Metric Learning for
Large Margin Nearest Neighbor Classification.” International Journal of Pattern
Recognition and Artificial Intelligence 25 (07): 1073–87. https://doi.org/10.1142/
S021800141100897X
Thabtah, Fadi Fayez. 2017a. “Machine-Learning-Databases/00419.” https://archive.ics.uci.
edu/ml
Thabtah, Fadi. 2017b. “Machine-Learning-Databases/00420.” https://archive.ics.uci.edu/ml
Thabtah, Fadi. 2017c. “Machine-Learning-Databases/00426.” https://archive.ics.uci.edu/ml
Thabtah, Fadi. 2018. “Autism-Screening-for-Toddlers/Version/1.” https://www.kaggle.com/
fabdelja/
112 Machine Learning and Deep Learning in Medical Data Analytics
Thabtah, Fadi. 2019. “An Accessible and Efficient Autism Screening Method for Behavioural
Data and Predictive Analyses.” Health Informatics Journal 25 (4): 1739–55. https://doi.
org/10.1177/1460458218796636
Thabtah, Fadi, Firuz Kamalov, and Khairan Rajab. 2018. “A New Computational Intelligence
Approach to Detect Autistic Features for Autism Screening.” International
Journal of Medical Informatics 117 (September): 112–24. https://doi.org/10.1016/j.
ijmedinf.2018.06.009
Thabtah, Fadi, Neda Abdelhamid, and David Peebles. 2019. “A Machine Learning Autism
Classification Based on Logistic Regression Analysis.” Health Information Science
and Systems 7 (1): 1–11. https://doi.org/10.1007/s13755-019-0073-5
Thabtah, Fadi, and David Peebles. 2020. “A New Machine Learning Model Based on
Induction of Rules for Autism Detection.” Health Informatics Journal 26 (1): 264–86.
https://doi.org/10.1177/1460458218824711
Vaishali R., and R. Sasikala. 2018. “A Machine Learning Based Approach to Classify Autism
with Optimum Behavior Sets.” International Journal of Engineering and Technology
(IJET) 7 (4): 4216–19. https://doi.org/10.14419/ijet.v7i3.18.14907
Wosiak, Agnieszka, and Danuta Zakrzewska. 2018. “Integrating Correlation-Based Feature
Selection and Clustering for Improved Cardiovascular Disease Diagnosis.” Complexity
2018 (October): 1–11. https://doi.org/10.1155/2018/2520706
Zhang, Yanqiu, Ming Ni, Chengwu Zhang, Shuang Liang, Sheng Fang, Ruijie Li, and Zhouyu
Tan. 2019. “Research and Application of AdaBoost Algorithm Based on SVM.” In
2019 IEEE 8th Joint International Information Technology and Artificial Intelligence
Conference (ITAIC), 662–66. IEEE. https://doi.org/10.1109/ITAIC.2019.8785556
6 A Framework for Selection
of Machine Learning
Algorithms Based on
Performance Metrices
and Akaike Information
Criteria in Healthcare,
Telecommunication,
and Marketing Sector
A. K. Hamisu and K. Jasleen
CONTENTS
6.1 Introduction................................................................................................... 114
6.2 Machine Learning Applications.................................................................... 114
6.3 Design and Implementation of Framework for Model Selection................... 117
6.3.1 Phase 1: Input Analysis Phase........................................................... 117
6.3.1.1 Input Attributes................................................................... 117
6.3.1.2 Attribute Analysis............................................................... 117
6.3.2 Phase 2: Model Building Phase......................................................... 118
6.3.2.1 Data collection.................................................................... 118
6.3.2.2 Data Pre-Processing............................................................ 118
6.3.2.3 Features Extraction............................................................. 118
6.3.2.4 Model Building................................................................... 119
6.3.3 Phase 3: Model Evaluation Phase...................................................... 120
6.3.3.1 Model Analysis Module...................................................... 120
6.3.4 Phase 4: Model Recommendation Phase........................................... 120
6.4 Result and Analysis....................................................................................... 120
6.4.1 Selection of Model based on Accuracy, Precision, Recall,
and F-Measure................................................................................... 121
6.4.2 Selection of Model Based on Akaike Information Criteria............... 124
6.5 Conclusion .................................................................................................... 126
References............................................................................................................... 126
DOI: 10.1201/9781003226147-6 113
114 Machine Learning and Deep Learning in Medical Data Analytics
6.1 INTRODUCTION
The growth of the internet has seen a profusion of data and a surge in technol-
ogy for extracting information from big data for marketing strategy, adding value
to products and services, and personalizing the consumer experience. Recently,
there has been a remarkable increase in interest in the era of artificial intelli-
gence (AI), ML, and deep learning (DL), as more individuals become aware of the
breadth of new applications enabled by ML and DL methodologies. The applica-
tions of ML and DL range from home to hospital, domestic to enterprise, agri-
culture to military, and include all aspects of life. The main focus of this chapter
is on applications of ML methodologies in three separate sub-domains: health-
care, marketing, and telecommunications. In the healthcare sector, two significant
problems are considered for this research work. One is cardiovascular disease and
another one is fetal health. The reason for choosing both these diseases is the rate
at which they affect the people. Cardiovascular disease, also known as coronary
ailment, is one of the most serious ailments in India and around the world. Heart
disease is estimated to be the cause of 28.1% of deaths. It is also the leading cause
of death, accounting for more than 17.6 million fatalities in 2016 across the world
(Shan et al. 2017). As a result, accurate and early diagnosis and treatment of
such diseases necessitates a system that can forecast with pinpoint accuracy and
consistency. The second problem that is considered for this work is fetal health
classification which includes classification of fetal as healthy or unhealthy. A total
of three datasets (two cardiovascular datasets and one fetal health dataset) were
used under healthcare sector. In this chapter, a framework for the selection of ML
algorithm has been proposed. ML algorithm was selected based on dataset attri-
butes, performance metrices, and AIC score. For experimentation purposes, ML
algorithms were divided into eager, lazy, and hybrid learners. For the evaluation
of the proposed framework, a total of eight datasets from three sectors (health-
care, telecommunication, and marketing) were selected for experimentation. This
paper contributes in context of framework for recommendation of the best ML
algorithm/model according to the input attributes. Model recommendation was
based on performance evaluation parameters (accuracy, precision, and recall) as
well as on model selection parameter (AIC).
The rest of this chapter is organized as follows. Section 6.2 presents related work
carried out in proposed direction. Complete methodology followed for implementa-
tion of this work is presented in Section 6.3. Detailed results and analysis are pre-
sented in Section 6.4 followed by concluding remarks in Section 6.5.
and healthcare aided living. Pattnayak and Jena (2021) discussed and explained the
need of ML for healthcare systems. Potential application of ML in healthcare and
healthcare aided areas which includes from patient to doctor, from diagnosis to treat-
ment, from surgery to decision support system were well elaborated. Panigrahi et al.
(2021) developed an expert system-based clinical decision support system (CDSS) for
prediction and diagnosis of hepatitis-B. This system comprises 59 rules and imple-
mentation is done using web-based Expert System Shell. Paramesha et al. (2021)
discussed ML-based approach for sentiment analysis of narrated drug reviews and
engineering in food technology which are indirectly related to the healthcare sec-
tor. Mohapatra et al. (2021) experimented with convolutional neural network (CNN)
for early detection of skin cancer. They have also performed comparative analysis
of MobileNet and ResNet50 CNN architectures for skin cancer classification task.
Ramakrishnudu et al. (2021) proposed a system that predicts the overall health status
of a person using ML techniques. Various parameters such as person’s sleeping pat-
tern, his/her physical activity, and his/her eating habits were used for predicting the
overall health of the person. Panicker et al. (2021) proposed lightweight CNN model
for classifying tuberculosis bacilli from non-bacilli objects. The performance of the
proposed model in terms of accuracy is close to the existing ML models Panicker
et al. 2021). Islam et al. discussed the use of DL techniques for autonomous disease
diagnosis from symptoms. They proposed a graph convolution network (GCN) as
a disease–symptom network to link the disease and symptoms. GCN-based deep
neural network determines the most probable diseases associated with the given
symptoms with 98% accuracy (Islam et al. 2021). Khamparia et al. (2020) proposed
transfer learning based novel DL internet of health and things driven method for skin
cancer classification. The proposed method performed well as compared to earlier
reported techniques. Güldoğan et al. (2021) proposed a transfer learning-based tech-
nique for the detection and classification of breast cancer (benign or malignant) based
on the ultrasound images. Performance metrices such as accuracy, sensitivity, and
specificity with 95% confidence intervals were 0.974 (0.923–1.0), 0.957 (0.781–0.999),
and 1 (0.782–1.0), respectively (Güldoğan et al. 2021). Said et al. (2021) proposed a
new transfer learning-based approach for the classification of breast cancer in histo-
pathological images. Block wise fine tuning strategy has been employed to handle
CNN RESNET-18 (Said et al. 2021). Yang et al. (2021) explored the potential of DL
models in the identification of lung cancer subtypes and cancer mimics from whole
slide images. Irene et al. (2021) elaborated the ethics of ML in healthcare through
the lens of social justice. Recent developments, challenges, and solutions to address
those challenges were discussed in detail. Danton et al. (2020) proposed a system-
atic approach to identify the ethics in ML-based healthcare applications. Elements
such as conceptual model, development, implementation, and evaluation were con-
sidered while framing the approach (Danton et al. 2020). Muhammad et al. (2020)
discussed the challenges, requirements, and opportunities in the area of fairness in
healthcare AI and the various nuances associated with it. Liu et al. (2020) proposed
DL approaches for automatic diagnosis of Alzheimer’s disease (AD) and its prodro-
mal stage, that is, mild cognitive impairment (MCI). Baskaret al. (2020) proposed
a framework for wearable sensors (WS) so that it can be applicable as a part of
116 Machine Learning and Deep Learning in Medical Data Analytics
smart healthcare tracking applications. Andre et al. (2019) discussed the application
of computer vision, natural language processing in the context of medical domain.
Siddique and Chow (2021) discussed the application of ML/AI in healthcare com-
munication. This work includes chatbots for the COVID-19 health education, cancer
therapy, and medical imaging. The challenges, issues, and problems for the imple-
mentation of ML- and DL-based applications in healthcare and healthcare-aided
sector have been discussed (Riccardo et al. 2018). Mateen et al. (2020) presented a
framework for improving the accuracy of ML algorithms in healthcare by incorpo-
rating reporting guidelines such as SPIRIT-AI and CONSORT-AI in clinical and
health science in ML approaches. Ferdous et al. (2020) presented a review on ML
when applied to prediction of different diseases. The contribution of ML in health-
care is discussed with aim to provide the best suitable ML algorithm (Ferdous et al.
2020). Utsav et al. (2019) presented a technique to use ML algorithms for predict-
ing the probability of cardiac arrest based on various attributes. Zoabi et al. (2021)
proposed an ML-based technique to predict whether an individual is infected with
SARS-CoV-2 or not. The model takes different parameters such as age, gender, and
presence of various COVID symptoms. Faizal and Sultan (2020) explored the appli-
cation of AI and data analytics techniques for mobile health. These techniques can
be used for providing valuable insights to users and accordingly resources can be
planned for mobile health. AI-based models have been proposed for mobile health.
Futoma et al. (2020) emphasized for clinical utility and generalizability of ML algo-
rithms and answers the various questions (when, how, and why) on ML applicabil-
ity for both clinicians and for patients. Wang et al. (2020) proposed an alternative
COVID-19 diagnosis methodology based on COVID-19 radio graphical changes in
computerized tomography (CT) images. They experimented with DL methods to
extract the hidden features from CT scans and provide the diagnosis for COVID-19
(Wang et al. 2020). Song et al. (2020) proposed DeepPneumonia technique (as DL
based COVID detection from CT scans) to identify patients with COVID-19. Punn
et al. (2020) proposed ML- and DL-based model to analyze predictive behavior of
COVID-19 using a dataset published on the Johns Hopkins dashboard.
Authors proposed a technique to predict the customer churn rate (who are likely
to cancel the subscription). Various ML algorithms such as DT, Random Forest,
and XGBoost have been experimented (Kavitha et al. 2020). Researchers presented
analysis to leverage ML methods in marketing research. Comparison between ML
methods with statistical methods was also presented. A unified conceptual frame-
work for ML methods have been proposed in this work (Liye and Baohong 2020).
Dev et al. (2016) used ML to predict heart disease.
In Galván et al.’s (2009) study, a lazy learning strategy was proposed for building
classification learning models. In this work, authors compared the accuracy of SVM
and KNN algorithms on student performance data sets. SVM performed well as com-
pared to KNN with accuracy of 91.07% (Nuranisah et al. 2020). Thanh and Kappas
(2017) examined and compared the performance of ML algorithm for land use/cover
classification. The classification results showed a high overall accuracy of all the algo-
rithms In this paper, authors experimented with ML algorithms on healthcare datasets
(Raj and Sonia 2017). Zhenlong et al. (2017) explored the usefulness of ML algorithms
for driver drowsiness detection. The results revealed that SVM performed well. In this
Framework for Selection of ML Algorithms 117
paper, authors proposed the application of lazy learning techniques to Bayesian tree
induction and presents the resulting lazy Bayesian rule learning algorithm, called Lbr
(Zheng and Webb 2000). Solomon et al. (2014) presented evaluation of eager and lazy
classification algorithms using UCI Bank Marketing data set. Results revealed that
eager learners outperform the lazy learners with accuracy of 98%.
TABLE 6.1
Dataset Description
1 if w·x + b ≥ 1
f (x ) = (6.2)
−1 if w·x + b ≤ −1
Where w and b parameters are identified from training data.
NN is supervised ML algorithm which is based on backprogation where
weights in hidden layer and output layer are updated according to error in
estimation. For weights updation, the following equation is utilized.
w kj +1 = w kj + λ ( yi − y
i ) x ij (6.3)
k
2. Lazy learning: This category includes KNN algorithm and LNB algorithm.
KNN algorithm is a distance-based ML algorithm which has application
in classification as well as regression problems. In this research work, dis-
tance is calculated based on Euclidean distance measure. Distance between
test point (x) and existing training point (y) is given by,
For each dataset, hyper-parameter for KNN, that is, k, is tuned using
elbow method.
120 Machine Learning and Deep Learning in Medical Data Analytics
TABLE 6.2
Category-Wise Machine Learning Algorithms
TABLE 6.3
Result Obtained with Marketing Sector
TABLE 6.4
Result Obtained with Healthcare Sector
TABLE 6.5
Result Obtained with Telecommunication Sector
TABLE 6.6
Average Accuracy-Based Comparison of Machine Learning Algorithms
Marketing Telecommunication Healthcare
Dataset Average Dataset Average Dataset Average
Learning Methods Accuracy Accuracy Accuracy
Eager learner 94 90 90
Lazy learner 91 86 85
Hybrid learner 92 85 76
TABLE 6.7
Average Precision-Based Comparison of Machine Learning Algorithms
Marketing Telecommunication Healthcare
Dataset Average Dataset Average Dataset Average
Learning Methods Precision Precision Precision
Eager learner 0.92 0.89 0.88
Lazy learner 0.86 0.90 0.86
Hybrid learner 0.88 0.78 0.78
accuracy of eager learners ranges from 90% to 94%. Accuracy- and precision-based
comparative analysis is presented in Tables 6.6 and 6.7.
From Figures 6.2 and 6.3, it can be observed that the eager learner category of
algorithms performed well for the healthcare sector based on accuracy and precision.
For identification of the best ML algorithm in each sector, performance analysis of
ML algorithms in the eager learner category is carried out. From Figures 6.2 and 6.3,
it can be observed that the eager learner category of algorithms performed well for
the healthcare sector based on accuracy and precision. For identification of the best
ML algorithm in each sector, performance analysis of ML algorithms in the eager
learner category is carried out. In the case of the healthcare dataset, SVM is proven to
be the best ML algorithm, whereas in the case of the telecommunication and market-
ing dataset, DT comes out as the top performing one. NN was the worst performing
algorithm in each sector. Furthermore, ROC curve (refer to Figures 6.4–6.6) and ROC-
AUC score was analyzed for top performing algorithms. ROC-AUC score comes out to
be 1.0 for all top performing ones.
FIGURE 6.5 ROC curve for SVM algorithm in the healthcare sector.
TABLE 6.8
Results Based on Average AIC Score
where k indicates the number of independent variables used to build the model
and L indicates maximum likelihood estimate of model (Akaike 1973). The best
model is one which minimizes the information loss and has the minimum score
for AIC.
From Table 6.8 it can be observed that in the marketing dataset, the lowest AIC
score of 15.41 is reported by the eager learner category of ML algorithms, whereas
for the telecommunication and healthcare datasets, the lowest AIC score is reported
by the lazy learner category of ML algorithms with a score of 19.91 and 21.76,
respectively.
From Figure 6.7 it can be observed that for the marketing sector, the lowest
AIC score is reported by eager learners, whereas in the case of telecommunica-
tion and healthcare sectors, the lazy learner category reported the lowest AIC.
To find the best suitable algorithm for each sector, comparative analysis has been
TABLE 6.9
Accuracy- v/s AIC Score-Based Comparative Analysis of Algorithms
Accuracy-Based Analysis AIC-Based Analysis
6.5 CONCLUSION
In this research work, a framework for recommendation of ML algorithm has been
formulated. The purpose was to find the most suitable ML algorithm for three differ-
ent sectors. For experimentation purpose, ML algorithm were divided into three cat-
egories: eager, lazy, and hybrid learner. KNN, LNB, SVM, DT, NN, and the hybrid
classifier using stacking were used on eight different datasets (from three different
sectors: marketing, healthcare, and telecommunication). On the basis of accuracy,
results revealed that eager learner ML algorithms are the best performing ones in
all three sectors. Among eager learners, SVM is proven to be the top performing in
healthcare with precision of 0.98. DT is the best suited for the telecommunication
and marketing datasets with precision of 0.99 and 0.94, respectively. Whereas, on the
basis of AIC score, SVM is the best suited for the marketing dataset, whereas KNN
is the best suited for telecommunication and healthcare dataset.
REFERENCES
Akaike, H 1973. Information theory and an extension of the maximum likelihood principle.
In: Petrov, B. N. and Csáki, F. (eds.), 2nd International Symposium on Information
Theory, Tsahkadsor, Armenia, USSR, September 2–8, 1971. Budapest: Akadémiai
Kiadó, pp. 267–81.
Andre, E., Alexandre, R., Bharath, R., Volodymyr, K., Mark, D., Katherine, C., Claire, C.,
Greg, C., Sebastian, T., Jeff, D. 2019. A guide to deep learning in healthcare. Nature
Medicine, 25: 24–29. https://doi.org/10.1038/s41591-018-0316-z
Framework for Selection of ML Algorithms 127
Kavitha, V., Hemanth, K. G., Mohan, K. S. V., Harish, M. 2020. Churn prediction of customer in
telecom industry using machine learning algorithms. International Journal of Engineering
Research & Technology, 09(05). http://dx.doi.org/10.17577/IJERTV9IS050022
Khamparia, A., Singh, P. K., Rani, P., Samanta, D., Khanna, A., Bhushan, B. 2020. An inter-
net of health things-driven deep learning framework for detection and classification
of skin cancer using transfer learning. Transactions on Emerging Telecommunication
Technologies: 32, e3963. https://doi.org/10.1002/ett.3963
Liu, M., Lian, C., Shen, D. 2020. Anatomical-Landmark-Based Deep Learning for Alzheimer’s
Disease Diagnosis with Structural Magnetic Resonance Imaging. In: Chen, Y. W. and
Jain, L. (eds), Deep Learning in Healthcare. Intelligent Systems. Reference Library, vol
171. Cham: Springer. https://doi.org/10.1007/978-3-030-32606-7_8
Liye, M., Baohong, S. 2020. Machine learning and AI in marketing – Connecting comput-
ing power to human insight. International Journal of Research in Marketing, 37(3):
481–504. https://doi.org/10.1016/j.ijresmar.2020.04.005
Mateen, B. A., Liley, J., Denniston, A. K. 2020. Improving the quality of machine learning
in health applications and clinical research. Nature Machine Intelligence, 2: 554–6.
https://doi.org/10.1038/s42256-020-00239-1
Mohapatra, S., Abhishek, N., Bardhan, D., Ghosh, A. A., Mohanty, S. 2021. Comparison of
MobileNet and ResNet CNN Architectures in the CNN-Based Skin Cancer Classifier
Model. In: Mohanty, S. N., Nalinipriya, G., Jena, O. P. and Sarkar, A. (eds), Machine
Learning for Healthcare Applications, Wiley Online Library, 169–86. https://doi.
org/10.1002/9781119792611.ch11
Muhammad, A. A., Arpit, P., Carly, E., Vikas, K., Ankur, T. 2020. Fairness in Machine Learning
for Healthcare. In Proceedings of the 26th ACM SIGKDD International Conference
on Knowledge Discovery & Data Mining (KDD ‘20). New York, NY: Association for
Computing Machinery, 3529–3530. https://doi.org/10.1145/3394486.3406461
Nuranisah, Syahril E., Poltak, S. 2020. Analysis of algorithm support vector machine learn-
ing and k-nearest neighbor in data accuracy. IOP Conf. Series: Materials Science and
Engineering, 725: 1–7.
Panicker, R. O., Pawan, S., Rajan, J., Sabu, M. 2021. A Lightweight Convolutional Neural
Network Model for Tuberculosis Bacilli Detection From Microscopic Sputum Smear
Images. In: Mohanty, S. N., Nalinipriya, G., Jena, O. P. and Sarkar, A. (eds), Machine
Learning for Healthcare Applications, Wiley Online Library,343–51. https://doi.
org/10.1002/9781119792611.ch22
Panigrahi, N., Ayus, I., Jena, O. P. 2021. An Expert System-Based Clinical Decision Support
System for Hepatitis-B Prediction & Diagnosis. In: Mohanty, S. N., Nalinipriya, G.,
Jena, O. P. and Sarkar, A. (eds), Machine Learning for Healthcare Applications, Wiley
Online Library,57–75. https://doi.org/10.1002/9781119792611.ch4
Paramesha, K., Gururaj, H., Jena, O. P. 2021. Applications of Machine Learning in Biomedical
Text Processing and Food Industry. In: Mohanty, S. N., Nalinipriya, G., Jena, O. P.
and Sarkar, A. (eds), Machine Learning for Healthcare Applications:, Wiley Online
Library, 151–67. https://doi.org/10.1002/9781119792611.ch10
Pattnayak, P., Jena, O. P. 2021. Innovation on Machine Learning in Healthcare Services–
An Introduction. In: Mohanty, S. N., Nalinipriya, G., Jena, O. P. and Sarkar, A. (eds),
Machine Learning for Healthcare Applications, Wiley Online Library,1–15. https://
doi.org/10.1002/9781119792611.ch1
Punn, N. S., Sonbhadra, S. K., Agarwal, S. 2020. COVID-19 epidemic analysis using
machine learning and deep learning algorithms. medRxiv. https://doi.org/10.1101/
2020.04.08.20057679
Raj, K., Sonia. 2017. SVM & KNN based classification of healthcare data sets using weka.
IJCSC, 8(1): 50–54. https://doi.org/10.031206/IJCSC.2016.012
Framework for Selection of ML Algorithms 129
Ramakrishnudu, T., Prasen, T. S., Chakravarthy, V. T. 2021. A Framework for Health Status
Estimation Based on Daily Life Activities Data Using Machine Learning Techniques.
In: Mohanty, S. N., Nalinipriya, G., Jena, O. P. and Sarkar, A. (eds), Machine
Learning for Healthcare Applications, Wiley Online Library, 17–32. https://doi.
org/10.1002/9781119792611.ch2
Riccardo, M., Fei, W., Shuang, W., Xiaoqian, J., Joel, T. D. 2018. Deep learning for health-
care: review, opportunities and challenges. Briefings in Bioinformatics, 19(6): 1236–46,
https://doi.org/10.1093/bib/bbx044
Said, B., Xiabi, L., Zhongshu, Z., Xiaohong, M., Chokri, F. 2021. A new transfer learning
based approach to magnification dependent and independent classification of breast
cancer in histopathological images. Biomedical Signal Processing and Control, 63.
https://doi.org/10.1016/j.bspc.2020.102192
Shan, X., Tiangang, Z., Zhen, Z., Daoxian, W., Junfeng, H., Xiaohui, D. 2017. Cardiovascular
Risk Prediction Method Based on CFS Subset Evaluation and Random Forest
Classification Framework. 2017 IEEE 2nd International Conference on Big Data
Analysis (ICBDA), 228–32. https://doi.org/10.1109/ICBDA.2017.8078813
Siddique, S, Chow, J. C. L. 2021. Machine learning in healthcare communication.
Encyclopedia, 1(1): 220–39. https://doi.org/10.3390/encyclopedia1010021
Solomon, G. F., Abebe, D. A., Bhabani, S. D. M. 2014. A comparative study on performance
evaluation of eager versus lazy learning methods. International Journal of Computer
Science and Mobile Computing, 3(3): 562–8.
Song, Y., Shuangjia, Z., Liang, L., et al. 2020. Deep learning Enables Accurate
Diagnosis of Novel Coronavirus (COVID-19) with CT images. IEEE/ACM
Transactions on Computational Biology and Bioinformatics https://doi.org/10.1101/
2020.02.23.20026930
Telecom dataset accessed from https://www.kaggle.com/blastchar/telco-customer-churn in
Dec 2020.
Thanh, N. P., Kappas, M. 2017. Comparison of Random Forest, k-Nearest Neighbor, and
support vector machine classifiers for land cover classification using Sentinel-2
imagery. Sensors (Basel). 18(1). doi: 10.3390/s18010018. PMID: 29271909; PMCID:
PMC5796274.
Utsav, C., Vikas, K., Vipul, C., Sumit, T., Amit, K. 2019. Cardiac arrest prediction using
machine learning algorithms. 2019 2nd International Conference on Intelligent
Computing Instrumentation and Control Technologies (ICICICT), 1, 886–90. https://
doi.org/10.1109/ICICICT46008.2019.8993296
Wang, S., Kang, B., Ma, J., Zeng, X., Xiao, M., Guo, J., Cai, M., Yang, J., Li, Y., Meng, X., Xu,
B. 2020. A deep learning algorithm using CT images to screen for Corona Virus Disease
(COVID-19). European Radiology. https://doi.org/10.1007/s00330-021-07715-1
Yang, H., Chen, L., Cheng, Z., Yang, M., Wang, J., Lin, C., Wang, Y., Huang, L., Chen, Y.,
Peng, S., Ke, Z., Li.,W. 2021. Deep learning-based six-type classifier for lung cancer
and mimics from histopathological whole slide images: a retrospective study. BMC
Medicine. 19(80). https://doi.org/10.1186/s12916-021-01953-2
Zheng, Z., Webb, G. I. 2000. Lazy learning of Bayesian rules. Machine Learning, 41: 53–84.
https://doi.org/10.1023/A:1007613203719
Zhenlong, L., Qingzhou, Z., Xiaohua, Z. 2017. Performance analysis of K-Nearest neighbor,
support vector machine, and artificial neural network classifiers for driver drowsiness
detection with different road geometries. International Journal of Distributed Sensor
Systems, 13(9): 1–12. https://doi.org/10.1177/1550147717733391
Zoabi, Y., Deri-Rozov, S., Shomron, N. 2021. Machine learning-based prediction of
COVID-19 diagnosis based on symptoms. npj Digital Medicine, 4(3): 1–5. https://doi.
org/10.1038/s41746-020-00372-6
7 Hybrid Marine Predator
Algorithm with
Simulated Annealing
for Feature Selection
Utkarsh Mahadeo Khaire, R. Dhanalakshmi,
and K. Balakrishnan
CONTENTS
7.1 Introduction................................................................................................... 132
7.1.1 Feature Selection............................................................................... 132
7.1.2 Feature Selection Techniques............................................................ 132
7.1.3 Meta-Heuristic Algorithms................................................................ 133
7.1.4 Motivation.......................................................................................... 134
7.1.5 Contributions..................................................................................... 134
7.1.6 Organization of Paper........................................................................ 135
7.2 Related Work................................................................................................. 135
7.3 Marine Predators Algorithm......................................................................... 136
7.3.1 MPA Formulation.............................................................................. 136
7.3.2 MPA Optimization Scenarios............................................................ 137
7.3.2.1 High-velocity Ratio or Prey Is Moving
Faster than Predators.......................................................... 137
7.3.2.2 Unit Velocity Ratio or Prey and Predators
Moving at the Same Speed................................................. 137
7.3.2.3 Low-velocity Ratio or Predators Moving
Faster than Prey.................................................................. 138
7.3.3 Eddy Formation or FAD’s Effect....................................................... 138
7.3.4 Marine Memory................................................................................. 139
7.4 Simulated Annealing..................................................................................... 139
7.5 Improved Marine Predators Algorithm......................................................... 139
7.5.1 Dataset............................................................................................... 139
7.5.2 Implementation.................................................................................. 140
7.6 Results and Discussion.................................................................................. 141
7.7 Conclusion and Future Scope........................................................................ 146
Acknowledgment.................................................................................................... 147
References............................................................................................................... 147
DOI: 10.1201/9781003226147-7 131
132 Machine Learning and Deep Learning in Medical Data Analytics
7.1 INTRODUCTION
This section presents the rigorous overview of feature selection (FS), the meta-heuris-
tic (MH) algorithm, motivation, contributions, and organization of the research work.
prone to overfitting. The diversity of the search space in nature is very high, but it
is limited. When compared to random search, MH algorithms outperform random
search because each iteration yields more than one solution.
7.1.3 Meta-Heuristic Algorithms
The critical feature of MH algorithms is their impressive ability to prevent algorithms
from converging prematurely. Since algorithms are stochastic, MH operates as a
black box, avoiding local optima, and rapidly and effortlessly exploring the search
space. The balancing factor between exploration and exploitation must be satisfied
to achieve the optimal solution in MH algorithms. Population-based MH algorithms
are classified into four types: evolutionary-based (EA), swarm-based (SA), physics-
based (PA), and human-based (HA) algorithms (Balakrishnan, Dhanalakshmi, and
Khaire 2021). Population-based MH launches their optimization process by generat-
ing initial random solutions. Figure 7.2 shows the cataloging of MH algorithms.
7.1.4 Motivation
Over the years, MH algorithms have earned an unrivaled reputation in the fields of
rigorous problem solving and optimization. However, MH algorithms have their own
set of benefits and drawbacks when it comes to solving different problems. Balancing
the nature between exploration and exploitation evaluates the performance of the
MH algorithm. The researchers favor the hybridization of one or more MH algo-
rithms to improve exploration and extraction capabilities. The No Free Lunch (NFL)
rule suggests that when all solutions in a class are added together, the contribution
of each solution method is the same. No matter how much the outcomes of various
algorithms differ from one another, they’re all similar. MPA is a nature-inspired,
swarm-based MH algorithm proposed Faramarzi et al. (2020) based on the survival
of the fittest theory. The standard MPA outperforms other MH techniques in many
engineering applications but fails to satisfy the local optima. In the paper, a hybrid
approach of the MPA and SA is proposed to improve the intensification and diversi-
fication of conventional MPA. The iMPA effectively improves the predictive model’s
search capability and avoids the local minima stagnation. The SA technique is used
to speed up the search space exploration. The foraging technique, also known as
Levy and Brownian movements, deals with predator–prey biological interactions.
7.1.5 Contributions
The following are the critical contributions made in this work:
• The proposed model evaluates the selection subset using a k-nearest neigh-
bors (KNN) classifier based on various performance measures such as pre-
cision, recall, and F1-score.
7.1.6 Organization of Paper
The following is a summary of this chapter: Section 7.2 presents a comprehensive
overview of related work. Sections 7.3 and 7.4 present preliminaries of standard
MPA and SA. Section 7.5 deals with the detailed description of the datasets used in
this chapter and the implementation of the proposed iMPA. Simulation outcomes are
discussed in Section 7.6, and the conclusion and the future direction of the research
work are provided in Section 7.7.
well-known local search technique (SA) is used to boost the exploitation of the pro-
posed model. The suggested FS system is validated using 18 well-known UCI datasets.
Cui et al. (2020) proposed a hybrid variant of the improved dragonfly algorithm
combined with Maximum Relevance-Minimum Redundancy (mRMR). First, the
model has three enhancement features with a low weight having a slight chance of
being picked into a candidate subset with a small probability of mRMR generating a
promising subset. Second, to balance global and local capacity, dynamic swarming
variables are suggested. Finally, the position updating function incorporates quan-
tum local optimum and global optimum. Abdollahzadeh and Gharehchopogh (2021)
proposed a novel hybridized version of HHO and fruit fly optimization algorithms
for FS problems. The author has presented three different multi-objective feature
selection framework algorithms based on evolutionary algorithms. Al-Tashi et al.
(2019) suggested a modern hybrid optimization algorithm that takes advantage of
GWO and PSO’s capabilities. The KNN classifier with Euclidean separation matri-
ces is used as a wrapper-based approach to finding the best solutions.
To address the shortcomings of FS, Faris et al. (2018) suggested an improved
version of SSA. They have created a wrapper FS using SSA as a search technique
in the proposed approach. In addition to the transformation features, the crossover
operator replaces the average operator and boosts the system’s exploration behavior.
Zhang et al. (2018) suggested a combination of biogeography-based optimization
(BBO) and GWO to balance the learning model’s exploration and exploitation. The
differential mutation and multi-migration operators, and the opposition-based learn-
ing approach in BBO and GWO, respectively, are used to increase performance. The
suggested model effectively outperforms all single-objective and clustering optimi-
zation benchmark functions. Zheng et al. (2019) proposed a hybrid model named
maximum Pearson maximum distance improved WOA for FS problems.
7.3.1 MPA Formulation
The MH algorithms distribute the initial population uniformly across the search
space. In the first iteration, the top predator searches for the prey and restructures the
Hybrid MPA with SA for Feature Selection 137
Elite matrix when the top predator replaces a better predator. Predators reorganized
their positions based on the Prey matrix. This optimization method focuses on the
entire process of locating the best predators using these two matrix values.
Constructing an Elite matrix determines the top predators’ fitness that contains
information about the position of the prey. Equation 7.2 depicts the formation of the
Elite matrix.
Y1,1 Y1,d
Elite = (7.2)
Y … Yn ,d
n ,1 n* d
Where Y1 indicates the top predator. Equation 7.3 represents the Prey matrix.
Y1,1 Y1,d
Prey = (7.3)
Y
Yn,d
n,1 n X d
unit velocity V equals one, the target moves according to Lévy, whereas the predator
moves according to Brownian motion.
1 2
While Max _ iter < iter < Max _ iter
3 3
Lévy/Exploitation
( )
Stepsize i = R L ⊗ Elite i − R L ⊗ Prey i i = 1,…n/2
(7.5)
Prey i = Prey i + P. R ⊗ Stepsize i
Where R L is Lévy’s movements. The element-wise multiplication of RL and prey
mimics the prey movements based on the Lévy movement strategy.
Brownian/Exploration
( )
Stepsize i = R B ⊗ R B ⊗ Elite i − Prey i i = n/2,…n
Prey i = Elite i + P.CF ⊗ Stepsize i
(7.6)
2*t
Max _ iter
t
CF = 1 −
Max _ iter
According to the study, predators do not move in the first phase when the veloc-
ity is more significant than ten. In the second phase, predators update their position
using Brownian motion when the speed equals one. The third phase is where preda-
tors move using the Lévy motion.
Where initially, the 0.2 value of FADs represents the effects of FADs on the opti-
mization process. r1 and r2 subscript of the prey indicates the random index of the
Prey matrix.
7.3.4 Marine Memory
The MPA foraging process is handled by recalling where the predators previously
found food using their memories. The elite matrix is revised after the fitness of
the matrix is determined, taking into account the impact of prey updates and the
use of FADs. The fitness evaluation is completed by comparing the current itera-
tion to the previous iteration; the updated one is replaced when it is better than
the previous one.
7.5.1 Dataset
The proposed model’s effectiveness is evaluated using three publicly available high-
dimensional microarray datasets. The data for this study is obtained from http://csse.
szu.edu.cn/staff/zhuzx/Datasets.html (Khaire and Dhanalakshmi 2020). All three
datasets used in this study have two classes in the target variables. Table 7.1 provides
a detailed overview of the dataset used in the study.
TABLE 7.1
Overview of the Dataset
7.5.2 Implementation
This section goes over the steps involved in the proposed method. The SMOTE-
Tomek algorithm has been used to balance the input data. The Gaussian distribution
is used to generate the initial population of random search agents during initializa-
tion. Equation 7.3 shows the random population matrix. Figure 7.3 depicts a flow-
chart of the proposed iMPA.
For classification, the proposed model employs the sigmoid function. The follow-
ing equation is used to convert the continuous values of the random search agents to
binary values [1 – selected, 0 – not selected].
1
1 if ≥ 0.95
Yi , j = 1 + e − yi , j (7.9)
0 otherwise
Yi
fitness(Yi ) = β * Errxi + (1 − β ) * (7.10)
n
Where β is an arbitrary number in the range of [0, 1]. Then, on each iteration,
run the SA algorithm. The proposed model employs a KNN classifier to evaluate the
selected subset.
TABLE 7.2
Validation of Features Selected by iMPA
the most important predictors of lethal cancer. The higher ROC-AUC curve of
the proposed model shows that the features selected by it can provide significant
confidence in knowledge discovery and decision making. Moreover, a reduced
set of selected features can yield more accurate results than overall features pres-
ent in the input data.
Tables 7.2 and 7.3 summarize and compare the performance analysis of the
predictive models for the selected feature subsets to validate the feature subsets.
The predictive model’s accuracy, precision, recall, and F1-score measures are used
to validate essential features. The comparative analysis of the selected features
indicates that the features carefully chosen by the proposed iMPA provide higher
confidence and classify the unseen test sample with higher precision. The pro-
posed iMPA outperforms conventional MPA in terms of selecting features and
their accuracy.
Tables 7.4 and 7.5 contain a detailed description of the selected features. Both
tables include the count of features selected throughout 20 epochs. Comparative
analysis of both tables indicates that the proposed iMPA select important features
with higher stability, whereas conventional MPA appoints elements with higher
deviation. Except for the first features for all datasets, features selected by MPA
have a count of less than five. On the other hand, the proposed iMPA picks stan-
dard features in every iteration; therefore, it has more stability in selecting the
elements.
TABLE 7.3
Validation of Features Selected by Conventional MPA
TABLE 7.4
Selected Features by Conventional MPA
TABLE 7.5
Selected Features by Proposed iMPA
ACKNOWLEDGMENT
This research is funded by the Department of Science and Technology, Government
of India, under the Interdisciplinary Cyber-Physical Systems (ICPS) scheme (Grant
no. T-54).
REFERENCES
Abdel-Basset, Mohamed, Weiping Ding, and Doaa El-Shahat. 2021. “A Hybrid Harris Hawks
Optimization Algorithm with Simulated Annealing for Feature Selection.” Artificial
Intelligence Review 54 (1): 593–637. https://doi.org/10.1007/s10462-020-09860-3
Abdollahzadeh, Benyamin, and Farhad Soleimanian Gharehchopogh. 2021. “A Multi-
Objective Optimization Algorithm for Feature Selection Problems.” Engineering with
Computers. https://doi.org/10.1007/s00366-021-01369-9
Al-Tashi, Qasem, Said Jadid Abdul Kadir, Helmi Md Rais, Seyedali Mirjalili, and Hitham
Alhussian. 2019. “Binary Optimization Using Hybrid Grey Wolf Optimization
for Feature Selection.” IEEE Access 7: 39496–508. https://doi.org/10.1109/
ACCESS.2019.2906757
Balakrishnan, K., R. Dhanalakshmi, and Utkarsh Mahadeo Khaire. 2021. “Improved
Salp Swarm Algorithm Based on the Levy Flight for Feature Selection.” Journal of
Supercomputing. https://doi.org/10.1007/s11227-021-03773-w
Cui, Xueting, Ying Li, Jiahao Fan, Tan Wang, and Yuefeng Zheng. 2020. “A Hybrid Improved
Dragonfly Algorithm for Feature Selection.” IEEE Access 8: 155619–29. https://doi.
org/10.1109/ACCESS.2020.3012838
Diao, Ren, and Qiang Shen. 2012. “Feature Selection with Harmony Search.” IEEE
Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 42 (6): 1509–23.
https://doi.org/10.1109/TSMCB.2012.2193613
Eberhart, Russell, and James Kennedy. 1995. “New Optimizer Using Particle Swarm
Theory.” In Proceedings of the International Symposium on Micro Machine and
Human Science, 39–43. https://doi.org/10.1109/mhs.1995.494215
Elgamal, Zenab Mohamed, Norizan Binti Mohd Yasin, Mohammad Tubishat, Mohammed
Alswaitti, and Seyedali Mirjalili. 2020. “An Improved Harris Hawks Optimization
Algorithm with Simulated Annealing for Feature Selection in the Medical Field.” IEEE
Access 8: 186638–52. https://doi.org/10.1109/ACCESS.2020.3029728
Faramarzi, Afshin, Mohammad Heidarinejad, Seyedali Mirjalili, and Amir H. Gandomi.
2020. “Marine Predators Algorithm: A Nature-Inspired Metaheuristic.” Expert Systems
with Applications 152 (August): 113377. https://doi.org/10.1016/j.eswa.2020.113377
Faris, Hossam, Majdi M. Mafarja, Ali Asghar Heidari, Ibrahim Aljarah, Ala’ M. Al-Zoubi,
Seyedali Mirjalili, and Hamido Fujita. 2018. “An Efficient Binary Salp Swarm
Algorithm with Crossover Scheme for Feature Selection Problems.” Knowledge-Based
Systems 154: 43–67. https://doi.org/10.1016/j.knosys.2018.05.009
148 Machine Learning and Deep Learning in Medical Data Analytics
Gao, Yuanyuan, Yongquan Zhou, and Qifang Luo. 2020. “An Efficient Binary Equilibrium
Optimizer Algorithm for Feature Selection.” IEEE Access 8: 140936–63. https://doi.
org/10.1109/ACCESS.2020.3013617
Hancer, Emrah. 2018. “Differential Evolution for Feature Selection : A Fuzzy Wrapper – Filter
Approach.” Soft Computing 23 (13): 5233–48. https://doi.org/10.1007/s00500-018-3545-7
Jindal, Mansi, Jatin Gupta, and Bharat Bhushan. 2019. “Machine Learning Methods for IoT
and Their Future Applications.” In Proceedings - 2019 International Conference on
Computing, Communication, and Intelligent Systems, ICCCIS 2019, 430–34. https://
doi.org/10.1109/ICCCIS48478.2019.8974551
Paramesha, K., H.L. Gururaj, and Om Prakash Jena. 2021. “Applications of Machine
Learning in Biomedical Text Processing and Food Industry.” In Machine Learning
for Healthcare Applications, edited by Sachi Nandan Mohanty, G. Nalinipriya, Om
Prakash Jena, and Achyuth Sarkar. Hoboken: Scrivener Publisher-Wiley. https://doi.
org/10.1002/9781119792611.ch10
Khaire, Utkarsh Mahadeo, and R. Dhanalakshmi. 2020. “High-Dimensional Microarray
Dataset Classification Using an Improved Adam Optimizer (IAdam).” Journal
of Ambient Intelligence and Humanized Computing 11: 5187–5204. https://doi.
org/10.1007/s12652-020-01832-3
Khamparia, Aditya, Prakash Kumar Singh, Poonam Rani, Debabrata Samanta, Ashish
Khanna, and Bharat Bhushan. 2020. “An Internet of Health Things-Driven Deep
Learning Framework for Detection and Classification of Skin Cancer Using Transfer
Learning.” Transactions on Emerging Telecommunications Technologies 32(7): e3963.
https://doi.org/10.1002/ett.3963
Lei, Shang. 2012. “A Feature Selection Method Based on Information Gain and Genetic
Algorithm.” In Proceedings - 2012 International Conference on Computer Science and
Electronics Engineering, ICCSEE 2012, 355–58. https://doi.org/10.1109/ICCSEE.2012.97
Mafarja, Majdi M, and Seyedali Mirjalili. 2017. “Hybrid Whale Optimization Algorithm with
Simulated Annealing for Feature Selection.” Neurocomputing 260: 302–12. https://doi.
org/10.1016/j.neucom.2017.04.053
Mirjalili, Seyedali, and Andrew Lewis. 2016. “The Whale Optimization Algorithm.”
Advances in Engineering Software 95 (May): 51–67. https://doi.org/10.1016/j.
advengsoft.2016.01.008
Muthukrishnan, R., and R. Rohini. 2017. “LASSO: A Feature Selection Technique in
Predictive Modeling for Machine Learning.” In 2016 IEEE International Conference
on Advances in Computer Applications, ICACA 2016, 18–20. https://doi.org/10.1109/
ICACA.2016.7887916
Nag, Kaustuv, and Nikhil R. Pal. 2016. “A Multiobjective Genetic Programming-Based
Ensemble for Simultaneous Feature Selection and Classification.” IEEE Transactions
on Cybernetics 46 (2): 499–510. https://doi.org/10.1109/TCYB.2015.2404806
Niranjan Panigrahi, Ishan Ayus, and Om Prakash Jena. 2021. “An Expert System-Based
Clinical Decision Support System for Hepatitis-B Prediction & Diagnosis.” In
Machine Learning for Healthcare Applications, edited by Sachi Nandan Mohanty, G.
Nalinipriya, Om Prakash Jena, and Achyuth Sarkar. Hoboken: Scrivener Publisher-
Wiley. https://doi.org/10.1002/9781119792611.ch4
Oh, Il Seok, Jin Seon Lee, and Byung Ro Moon. 2004. “Hybrid Genetic Algorithms for
Feature Selection.” IEEE Transactions on Pattern Analysis and Machine Intelligence
26 (11): 1424–37. https://doi.org/10.1109/TPAMI.2004.105
Pattnayak, Parthasarathi, and Om Prakash Jena. 2021. “Innovation on Machine
Learning in Healthcare Services–An Introduction.” In Machine Learning for
Healthcare Applications, edited by Sachi Nandan Mohanty, G. Nalinipriya, Om
Prakash Jena, and Achyuth Sarkar. USA: Scrivener Publisher-Wiley. https://doi.
org/10.1002/9781119792611.ch1
Hybrid MPA with SA for Feature Selection 149
Paul, Saurabh, and Petros Drineas. 2016. “Feature Selection for Ridge Regression with
Provable Guarantees.” Neural Computation 28(4): 716–42. https://doi.org/10.1162/
NECO_a_00816
Sadollah, Ali, Ardeshir Bahreininejad, Hadi Eskandar, and Mohd Hamdi. 2012. “Mine Blast
Algorithm for Optimization of Truss Structures with Discrete Variables.” Computers
and Structures 102: 49–63. https://doi.org/10.1016/j.compstruc.2012.03.013
Sharma, Nikhil, Ila Kaushik, Bharat Bhushan, Siddharth Gautam, and Aditya Khamparia.
2020. “Applicability of WSN and Biometric Models in the Field of Healthcare.” Deep
Learning Strategies for Security Enhancement in Wireless Sensor Networks, 304–29.
https://doi.org/10.4018/978-1-7998-5068-7.ch016
Stochino, Flavio, and Fernando Lopez Gayarre. 2019. “Reinforced Concrete Slab
Optimization with Simulated Annealing.” Applied Sciences (Switzerland) 9 (15).
https://doi.org/10.3390/app9153161
Patra, Sudhansu Shekhar, Om Praksah Jena, Gaurav Kumar, Sreyashi Pramanik, Chinmaya
Misra, and Kamakhya Narain Singh. 2021. “Random Forest Algorithm in Imbalance
Genomics Classification.” In Data Analytics in Bioinformatics: A Machine Learning
Perspective, edited by Xiaobo Zhang, Rabinarayan Satpathy, Tanupriya Choudhury,
Suneeta Satpathy, and Sachi Nandan Mohanty. River Street, Hoboken: Wiley. https://
doi.org/10.1002/9781119785620.ch7
Sundarrajan, Sudharsana, and Mohanapriya Arumugam. 2016. “Weighted Gene Co-Expression
Based Biomarker Discovery for Psoriasis Detection List of Abbreviations.” Gene 593
(1): 225–34. https://doi.org/10.1016/j.gene.2016.08.021
Thaiyalnayaki, K. 2021. “Classification of Diabetes Using Deep Learning and Svm
Techniques.” International Journal of Current Research and Review 13 (1): 146–49.
https://doi.org/10.31782/IJCRR.2021.13127
Urbanowicz, Ryan J., Melissa Meeker, William La Cava, Randal S. Olson, and Jason H.
Moore. 2018. “Relief-Based Feature Selection: Introduction and Review.” Journal of
Biomedical Informatics 85: 189–203. https://doi.org/10.1016/j.jbi.2018.07.014
Vasant, Pandian. 2010. “Hybrid Simulated Annealing and Genetic Algorithms for Industrial
Production Management Problems.” International Journal of Computational Methods
7 (2): 279–97. https://doi.org/10.1142/S0219876210002209
Wu, Jiansheng, and Zusong Lu. 2012. “A Novel Hybrid Genetic Algorithm and Simulated
Annealing for Feature Selection and Kernel Optimization in Support Vector
Regression.” In 2012 IEEE 5th International Conference on Advanced Computational
Intelligence, ICACI 2012, 999–1003. https://doi.org/10.1109/ICACI.2012.6463321
Zhang, Xinming, Qiang Kang, Jinfeng Cheng, and Xia Wang. 2018. “A Novel Hybrid
Algorithm Based on Biogeography-Based Optimization and Grey Wolf Optimizer.”
Applied Soft Computing Journal 67: 197–214. https://doi.org/10.1016/j.asoc.2018.02.049
Zheng, Yu Jun, Xin Li Xu, Hai Feng Ling, and Sheng Yong Chen. 2015. “A Hybrid Fireworks
Optimization Method with Differential Evolution Operators.” Neurocomputing 148:
75–82. https://doi.org/10.1016/j.neucom.2012.08.075
Zheng, Yuefeng, Ying Li, Gang Wang, Yupeng Chen, Qian Xu, Jiahao Fan, and Xueting Cui.
2019. “A Novel Hybrid Algorithm for Feature Selection Based on Whale Optimization
Algorithm.” IEEE Access 7: 14908–23. https://doi.org/10.1109/ACCESS.2018.2879848
8 Survey of Deep Learning
Methods in Image
Recognition and Analysis
of Intrauterine Residues
Bhawna Swarnkar, Nilay Khare,
and Manasi Gyanchandani
CONTENTS
8.1 Introduction................................................................................................... 152
8.1.1 Medical Imaging Modalities in Gynecology and Obstetrics............ 153
8.2 Related Work in Medical Image Analysis and its History............................ 155
8.3 Machine Learning (Deep Learning) Architectures...................................... 156
8.3.1 Convolutional Neural Network.......................................................... 156
8.3.2 Transfer Learning (with CNN).......................................................... 159
8.3.3 Recurrent Neural Network................................................................ 160
8.3.4 Deep Belief Networks and Restricted Boltzmann Machine.............. 160
8.3.5 Auto-Encoders................................................................................... 160
8.3.6 Generative Adversarial Networks...................................................... 160
8.4 Medical Image Analysis (Technologies and Applications)........................... 161
8.4.1 Image Feature Description Based on Deep Learning....................... 161
8.4.2 Analysis of Generative Model........................................................... 161
8.4.3 Low-Density Separation Method in Semi-Supervised...................... 161
8.5 Applications................................................................................................... 161
8.5.1 Classification...................................................................................... 161
8.5.2 Segmentation..................................................................................... 162
8.5.3 Detection............................................................................................ 162
8.5.4 Localization....................................................................................... 162
8.5.5 Registration........................................................................................ 162
8.6 Recognition of Image Based on Feature Extraction in Deep Learning........ 162
8.6.1 Feature Extraction of Image by Using Wavelet Transform............... 162
8.6.1.1 Cyclic Network Design....................................................... 163
8.6.2 Feature Extraction of Image by Using Gray-level
Co-Occurrence Matrix...................................................................... 163
8.6.3 Space Pyramid Recurrent Module..................................................... 165
8.7 Conclusion..................................................................................................... 165
8.8 Future Scope and Challenges........................................................................ 165
References�������������������������������������������������������������������������������������������������������������� 166
DOI: 10.1201/9781003226147-8 151
152 Machine Learning and Deep Learning in Medical Data Analytics
8.1 INTRODUCTION
In childbirth, after the delivery of a fetus from the birth canal of a woman, the pla-
centa is expelled within 3–20 minutes. Placenta is the temporary organ which forms
during pregnancy in women’s uterus. Basically the functionality of placenta is to
supply sufficient oxygen and provide nutrients to fetus as it grows in the womb. It
also removes the waste products from the body of the fetus.
Now when it comes to the health of placenta it depends upon various factors, with
some of them are under mother’s control while others are not. These are as follows:
maternal age, water breaking before labor, high blood pressure, twin pregnancy, dis-
order of blood clotting, any previous uterine surgery, any problem with the placenta
in a previous pregnancy, and abdominal trauma.
Placental problems that are more common during pregnancy involve abruption of
placenta and low-lying placenta, which causes premature birth of fetus. It is a medi-
cal emergency when placenta detaches itself totally or partially from the uterus inner
wall, causing insufficient supply of oxygen and nutrients, which results in excessive
bleeding from the woman’s vagina. Another problem is placenta previa in which the
cervix gets covered completely or partially by placenta and is common in the early
stages of pregnancy; however, it may get resolved as the uterus grows, but if it does
not, then C-section is the only solution. In all these situations the patient suffers
vaginal bleeding which is potentially excessive.
Now these come under the conditions which cause complications during preg-
nancy, but after the delivery of the fetus, residual placenta can create a concern
regarding the health of the mother. Retained placenta is the term used for placenta
which remains inside the mother’s uterus and this postpartum retained placenta has
some tissues which are still remaining [1]. Another outlook on remnant placenta
is that it is the phenomenon in which placenta’s part remains inside the mother’s
uterus more than half an hour after the child is delivered and it is known as the
third stage of labor [2]. Retained placenta is a common complication. In most cases,
tiny fragments are the tissues of placenta retained in womb are not easily detected.
Postpartum hemorrhage, abdominal infection of uterine cavity, and endometriosis
are the main causes of remnant placental tissues. On-time clinical intervention is
required which can efficiently and thoroughly administered so as to prevent adher-
ence (state of adhering) which is responsible for painful sensations in the uterus [3].
Reproductive health can be adversely affected because of the risk of intrauterine
adhesions, which is also termed secondary amenorrhea and is responsible for risking
the quality of a woman’s life who is of child-bearing age [4]. Intrauterine adhesions
also affect the recovery process of postpartum and medicinal overload. Although, it
is not a tough task to detect the postpartum remnant placenta; it is easily diagnosed
at the time of child delivery as to whether the placenta is undamaged or not [5]. On
time diagnosis plays a significant role in the history of medical science.
There are different perspectives of retained placenta in scientific demonstration.
If it is diagnosed as a mild case of postpartum residual placenta, it is demonstrated
as secondary anemia, infrequency in vaginal bleeding after childbirth, or it happens
in a sudden, occasional way in smaller amounts [6]. It is found in medical check-ups
that excessive bleeding can be life-threatening in women with severe complications
Survey of DL in Imaging of Intrauterine Residues 153
like poor restoration of the uterus, when the cervix gets remodeled in a loose and soft
structure, the softening of a swollen/widened uterus, and patients who have infec-
tions in the inner part of the uterine cavity have recognizable delicacy in the lower
portion of abdomen. Sometimes proper examination is needed to perform a diagno-
sis of residual placenta if no such typical symptoms like those mentioned by clinical
experts are present, otherwise it can easily be left unnoticed which is likely to harm
the reproductive health of the woman. So in order to get rid of such problems, doctors
(gynecologists) make use of B-scan ultra-sonography or B-scan.
With the help of B-scan, even the tiny tissues present in the uterus can be scanned
clearly, so it is the most preferred imaging tool to scrutinize remnant placental tis-
sues [8]. Ultrasound also provides a clear scan of placental lobules, which are cir-
cular-or ring-shaped and have a strong echo [9]. There is a presence of light mass in
uterine cavity which does not breach the outer muscular layer of the uterus which
can be seen by B-scan and is known as myometrium, unless there is implantation of
placental part. Medium and low echoes (irregular ones) are common in women with
short-term disease. Necrosis, organization, enhancement of echoes, degeneration of
tissues, and rough spots of light are common in women with long-term disease [10].
Accumulation of blood, endometriosis, and inner uterine cavity involution of uterine
are also scanned by ultrasound [11].
In this chapter, we mainly discuss deep learning architectures and how these
architectures are helping in the diagnosis of disease without much human interven-
tion. This chapter mainly focuses on these deep learning algorithms which are very
efficient and accurate in analyzing medical images for classification, segmentation,
and detection, which are discussed in this chapter. Also, how features of images get
Survey of DL in Imaging of Intrauterine Residues 155
(PPV): 5.1% (95%, Cl, 4.0–6.3); and negative predictive value (NPV): 99.3% (95%
Cl, 99.1–99.5). And for MEWC characteristics for sPPH the results were sensitivity:
75.0% (95% Cl, 67.3–82.7); specificity: 66.3% (95% Cl, 65.2–67.3); PPV: 3.3% (95%
Cl, 2.7–4.0); and NPV 99.4% (95% Cl, 99.2–99.6). So the combined sensitivity of
two systems was 83.3% (95% Cl, 75.4–89.5). By this automated system they try to
improve the detection of severely morbid postpartum hemorrhage.
Man [37] used trending classification algorithms of machine learning for better
the prediction risk of PPH. Dataset was extracted from the electronic health record
system with 12 variables which are of high relevance in risk of occurrence of PPH.
Comparative analysis is done using logistic regression, decision trees, random forest,
k-nearest neighbouring algorithm (KNN), support vector machines (SVMs), ANN
results were obtained in terms of precision, recall, and accuracy. Out of all the algo-
rithms, random forest predictions were the most accurate result at89%.
O ( t ) = ( I * K ) ( t ) (8.1)
O (t ) = ∑ I ( x).K ( t − x ) (8.2)
Above equation shows one-dimensional convolution operation, now for
two-dimensional (2D) operation with input I (x, y) and kernel (m, n) is given
by equation 8.3.
O (t ) = ∑ ∑ I ( m, n ). K ( x − m, y − n ) . (8.3)
Now when the kernel is flipped by commutative law, above equation is
written as:
O (t ) = ∑ ∑ I ( x − m, y − n ). K ( m, n ). (8.4)
When cross-correlation function is implemented which is same as of
convolution but without flipping the kernel, equation becomes:
O (t ) = ∑ ∑ I ( x + m, y + n ). K ( m, n ). (8.5)
158 Machine Learning and Deep Learning in Medical Data Analytics
2. RELU layer: An activation function which sets all negative input values to
zero is known as RELU layer. Training and calculation become accelerated
and simplified. It is helpful in avoiding the vanishing gradient problem. Its
equation can be written as:
f ( a ) = max ( 0, a ) . (8.6)
Here a is input to the neuron. Some other activation functions are sig-
moid, tanh, leaky RELUs, randomized RELUs, and parametric RELUs.
3. Pooling layer: This is placed between the convolution and RELU layers.
Pooling layer’s functionality is to cut the data’s dimensionality and thus
reduce the number of parameters. It can help reduce the problem of over-
fitting. Pooling layers include max pooling, average pooling, and L2 nor-
malization pooling layers, spatial pyramid pooling, and spectral pooling.
Max pooling usually takes the largest value of input by discarding other
values within the filter. It produces a strongest activation function over a
neighborhood. The logic behind this is that the relative location of one fea-
ture to another is more important than that of exact location.
4. Fully connected layer: This is the last layer in CNN. In this layer, each
and every neuron in preceding layers is connected to every other neuron.
Depending on what level of feature extraction is needed, there can be one
or more connected layer just like convolution, RELU, pooling layers. Fully
connected layer computes the probability score for the classification task
into the different classes which are available as its input and takes the out-
put from the preceding layers (convolution, RELU, or pooling layers).
RELU layer works on the features that are most strongly activated so that its class
can be easily determined. For instance, on histological glass slide, cancerous cells
have a higher DNA-to-cytoplasm ratio when compared with non-cancerous cells.
CNN would be more likely to predict the presence of cancerous cells if features of
DNA were strongly detected from the previous layer. CNN has the ability to learn
significant associations from the training data, by using training methods of standard
neural network which uses back propagation [10] and stochastic gradient descent.
FIGURE 8.2 Deep learning architectures: 1. RNN, 2. Auto-encoder, 3.RBM, 4. DBNs, 5. GANs.
160 Machine Learning and Deep Learning in Medical Data Analytics
architectures in conjunction with transfer learning is explored by Shin et al. [23] for
detection of enlarged thoraco-abdominal lymph node presence. It also makes use of
other classifications of post-delivery intrauterine remnants.
8.3.5 Auto-Encoders
Auto-encoders are neural networks which learn data in unsupervised manner. Auto-
encoders aim to reduce dimensionality by training the network. In medical image
analysis these features are of great significance where the data to be trained is limited.
Long-Yi-Guo et al. [24] proposed the composite features of multi-omics data, which is
produced by de-noising auto-encoder being used for generation of low-dimensional fea-
tures which were fed as an input into k-means for clustering to classify ovarian cancer.
An auto-encoder consists of two parts: encoder (ф) and decoder (ƒ) which maps
input (X) to code space (F), which are mathematically expressed as, ф:X→F, ƒ:F→X
and we aim to achieve encoder and decoder parameter such that ф, ƒ = arg maxф, ƒ||X =
(ф∘ ƒ)X||. Code-space, which is also named latent space, can be defined as basic neural
network as, z = σ (Wx + b) and x can be generated from z as x = σ′ (W′z + b′) where,
σ′, W′, b′ differs from σ, W, b depending upon the design of the network.
examples from the original dataset. Although GNNs are unsupervised in approach,
they can train a generative model by framing the problem as a supervised learning
problem with two sub-models: generator model and discriminator model. They pro-
vide a path to sophisticated domain-specific data augmentation.
8.5 APPLICATIONS
The area of computer vision has the conclusive aim of using systems to mutate learn-
ing and human vision. Analysis of any image lies between processing and computer
vision.
8.5.1 Classification
Classification is one of the first contributions of deep learning to the analysis of med-
ical images. Classification requires a huge dataset with known ground truth to train
on different cases. Optimum classification involves the accuracy in classification,
162 Machine Learning and Deep Learning in Medical Data Analytics
8.5.2 Segmentation
It is a great challenge to differentiate the organs of interest and extract them from the
background in algorithm development. The segmented region is of great importance
because steps to be taken are guided by segmentation in the whole analysis. It can be
done in three ways: manual segmentation, semi-automatic segmentation, and fully
automatic segmentation.
8.5.3 Detection
Detection, also known as computer aided detection (CAD), refers to software used
for pattern recognition which distinguishes suspicious features on the image to
reduce false negative readings. The processes of classification and localization come
together in order to detect and this is known as detection.
8.5.4 Localization
Localization of anatomical structures such as organs or landmarks, has been an
essential task. Localization requires parsing of 3D volumes and to solve 3D data
parsing several approaches have been processed in deep learning. Localization is
most commonly used in fully automatic end-to-end applications using supervised
and unsupervised learning models.
8.5.5 Registration
Registration refers to a spatial alignment of images and is a common function in
determining one-to-one correspondence between the coordinates of two or more
images. In the process of registration, alignment to second implant placement is
made for a reference image. Registration in medical images has potential applica-
tions which are reviewed by researchers.
When we apply 2D-DWT, it gets decomposed into two region of interest (ROI)
levels, from which we get four sub-bands named as Low-Low, Low-High, High-
High, High-Low; represented by LL, LH, HH, and HL, respectively.
As a result of this decomposition of image, we are able to see in detail high- and
low-level content of frequency in image. In wavelet approximation, low-frequency
position of an image is represented by LL1, LL2 at first and second level, respec-
tively. Similarly, LH1, LH2, HH1, HL2 are representations of high-frequency levels
of image. In these frequency levels, representation of horizontal, vertical, and diago-
nal is given in detail. Here we have used image of low frequency. So LL1 shows the
original image approximation, it further undergoes decomposition of second level
approximation and image details. This process will be repeated until the desired
level of resolution is obtained.
Now from sub-band, components of spatial frequency were extracted in the pro-
cess of decomposition. For better analysis of images, we have used both LL and HH,
because HH has higher performance in comparison to LL, to obtain text features of
brain tumor images. The components of different frequency and each component
studied with resolution matched to its scale and expressed is as:
∑ d ( s ) h × i ( s − 2 ij )
Pij =
DWT d ( s ) =
∑ d ( s ) l × i ( s −2ij )
(8.7)
Pij =
Here signal d(s) has component attribute corresponding to which is represented
by Pi, j coefficients correspond to the wavelet function, and d i, and j represent the
approximated signal components. The h(s) and l(s) are functions referring to high-
pass and low-pass filters coefficient in the equation, respectively, while wavelet scale
and translation factor are represented by parameters i and j.
with event q represented by (p, q)th elements, and is used to calculate how often a
pixel with intensity p occurs in relation with another pixel q at a certain distance S
and orientation. By this method, GLCM helps in finding the textural features such
as homogeneity, entropy, correlation, energy, and contrast from low- and high-level
sub-bands of the first four levels of wavelet decomposition.
8.7 CONCLUSION
In this survey chapter, we discussed common placental problems and intrauterine
residues, and how they can affect the health of patients. We also discussed how
deep learning algorithms are helpful in on-time, efficient, accurate diagnosis. We
discussed various image modalities that are being used in the treatment of intra-
uterine residues and how they play a significant role in diagnosis. Then this chapter
briefly presented various deep learning architectures which are capable of medical
image recognition, classification, and segmentation tasks. It showed how deep learn-
ing method of generative modeling and low density-based methods are capable of
describing features in medical image analysis and their applications in various steps
of image analysis, such as in segmentation, classification, localization, and detection.
Lastly we discussed feature extraction in deep learning which are based on GLCM,
Wavelet transform, and space pyramid recurrent module, which provide solutions for
extracting semantic and categorical information.
REFERENCES
[1] Q. Zhu, B. Du, and P. Yan, “Boundary-weighted domain adaptive neural network for
prostate MR image segmentation,” IEEE Trans. Med. Imag., vol. 39, no. 3, pp. 753–63,
Mar. 2020.
[2] M. Mehdipour Ghazi, B. Yanikoglu, and E. Aptoula, “Plant identification using deep
neural networks via optimization of transfer learning parameters,” Neurocomputing,
vol. 235, pp. 228–35, Apr. 2017.
[3] J. Bi, H. Yuan, and M. Zhou, “Temporal prediction of multiapplication consoli-
dated workloads in distributed clouds,” IEEE Trans. Autom. Sci. Eng., vol. 16, no. 4,
pp. 1763–73, Oct. 2019.
[4] M. Gao, U. Bagci, L. Lu, A. Wu, M. Buty, H.-C. Shin, H. Roth, G. Z. Papadakis, A.
Depeursinge, R. M. Summers, Z. Xu, and D. J. Mollura, “Holistic classification of CT
attenuation patterns for interstitial lung diseases via deep convolutional neural net-
works,” Comput. Methods Biomech. Biomed. Eng., Imag. Vis., vol. 6, no. 1, pp. 1–6,
Jan. 2018.
[5] A. Karargyris, J. Siegelman, D. Tzortzis, S. Jaeger, S. Candemir, Z. Xue, K. C. Santosh,
S. Vajda, S. Antani, L. Folio, and G. R. Thoma, “Combination of texture and shape fea-
tures to detect pulmonary abnormalities in digital chest X-rays,” Int. J. Comput. Assist.
Radiol. Surg., vol. 11, no. 1, pp. 99–106, Jan. 2016.
[6] A. Nibali, Z. He, and D. Wollersheim, “Pulmonary nodule classification with deep
residual networks,” Int. J. Comput. Assist. Radiol. Surg., vol. 12, no. 10, pp. 1799–808,
Oct. 2017.
[7] C. Qin, D. Yao, Y. Shi, and Z. Song, “Computer-aided detection in chest radiography
based on artificial intelligence: A survey,” Biomed. Eng. OnLine, vol. 17, no. 1,
pp. 1–23, Dec. 2018.
[8] M. Anthimopoulos, S. Christodoulidis, L. Ebner, A. Christe, and S. Mougiakakou,
“Lung pattern classification for interstitial lung diseases using a deep convolutional
neural network,” IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1207–1216, May 2016.
[9] E. Elyan and M. M. Gaber, “A _ne-grained random forests using class decomposi-
tion: An application to medical diagnosis,” Neural Comput. Appl., vol. 27, no. 8,
pp. 2279–88, Nov. 2016.
[10] S. Shen, S. X. Han, D. R. Aberle, A. A. Bui, and W. Hsu, “An interpretable deep hier-
archical semantic convolutional neural network for lung nodule malignancy classifica-
tion,” Expert Syst. Appl., vol. 128, pp. 84–95, Aug. 2019.
[11] Y. Li, C. Huang, L. Ding, Z. Li, Y. Pan, and X. Gao, “Deep learning in bioinformatics:
Introduction, application, and perspective in the big data era,” Methods, vol. 166,
pp. 4–21, Aug. 2019.
[12] E. H. Shortliffe, Computer-Based Medical Consultations: MYCIN, vol. 2. New York,
NY: Elsevier, 1976.
[13] G. Litjens et al. (Jun. 2017). “A survey on deep learning in medical image analysis.”
[Online]. Available at: https://arxiv.org/abs/1702.05747
[14] W. S. McCulloch and W. Pitts, “A logical calculus of the ideas immanent in nervous
activity,” Bull. Math. Biol., vol. 5, no. 4, pp. 115–33, 1943.
[15] F. Rosenblatt, “The perceptron: A probabilistic model for information storage and orga-
nization in the brain,”Psychol. Rev., vol. 65, no. 6, pp. 365–86, 1958.
Survey of DL in Imaging of Intrauterine Residues 167
[16] D. H. Hubel, and T. N. Wiesel, “Receptive fields, binocular interaction and functional
architecture in the cat’s visual cortex,” J. Physiol., vol. 160, no. 1, pp. 106–54, 1962.
[17] K. Fukushima, and S. Miyake, “Neocognitron: A self-organizing neural network
model for a mechanism of visual pattern recognition,” in Competition and Cooperation
in Neural Nets. Berlin, Germany: Springer, pp. 267–85, 1982.
[18] Y. LeCun et al., “Backpropagation applied to handwritten zip code recognition,” Neural
Comput., vol. 1, no. 4, pp. 541–51, 1989.
[19] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-
propagating errors,” Nature, vol. 323, pp. 533–56, Oct. 1986.
[20] A. Krizhevshy, I. Sutskever, and G. E. Hinton, “ImageNet classi_cation with deep con-
volutional neural networks,” in Proc. Adv. Neural Inf. Process. Syst., pp. 1097–105,
2012.
[21] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA: MIT
Press, 2016.
[22] O. Russakovsky et al., “ImageNet large scale visual recognition challenge,” Int. J.
Comput. Vis., vol. 115, no. 3, pp. 211–52, Dec. 2015.
[23] H.-C. Shin et al., “Deep convolutional neural networks for computer aided detection:
CNN architectures, dataset characteristics and transfer learning,” IEEE Trans. Med.
Imag., vol. 35, no. 5, pp. 1285–98, May 2016.
[24] Long-Yi Guo, Ai-Hua Wu, Yong-xia Wang, Li-ping Zhang, Hua Chai, Xue-Fang Liang,
“Deep learning-based ovarian cancer subtypes identification using multi-omics data,”
BioData Mining, vol. 12, 2020.
[25] R. Ashraf, M. Ahmed, S. Jabbar, S. Khalid, A. Ahmad, S. Din, and G. Jeon, “Content
based image retrieval by using color descriptor and discrete wavelet transform,” J. Med.
Syst., vol. 42, no. 3, p. 44, Mar. 2018.
[26] H. Xie, D. Yang, N. Sun, Z. Chen, and Y. Zhang, “Automated pulmonary nodule detec-
tion in CT images using deep convolutional neural networks,” Pattern Recognit., vol.
85, pp. 109–19, Jan. 2019.
[27] Y. Li, S. Wang, R. Umarov, B. Xie, M. Fan, L. Li, and X. Gao, “DEEPre Sequence-
based enzyme EC number prediction by deep learning,” Bioinformatics, vol. 34, no. 5,
pp. 760–69, Mar. 2018.
[28] A. S. Qureshi, A. Khan, A. Zameer, and A. Usman, “Wind power prediction using
deep neural network based meta regression and transfer learning,” Appl. Soft Comput.,
vol. 58, pp. 742–55, Sep. 2017.
[29] E. Elyan and M. M. Gaber, “A genetic algorithm approach to optimizing random forests
applied to class engineered data,” Inf. Sci., vol. 384, pp. 220–34, Apr. 2017.
[30] J. Kawahara, C. J. Brown, S. P. Miller, B. G. Booth, V. Chau, R. E. Grunau, J. G.
Zwicker, and G. Hamarneh, “BrainNetCNN: Convolutional neural networks for brain
networks; towards predicting neurodevelopment,” NeuroImage, vol. 146, pp. 1038–49,
Feb. 2017.
[31] G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W.
M. van der Laak, B. van Ginneken, and C. I. Sánchez, “A survey on deep learning in
medical image analysis,” Med. Image Anal., vol. 42, pp. 60–88, Dec. 2017.
[32] S. Zhou, Q. Chen, and X. Wang, “Active deep learning method for semi-supervised
sentiment classification,” Neurocomputing, vol. 120, pp. 536–46, Nov. 2013.
[33] Z. Zou, S. Tian, X. Gao, and Y. Li, “MlDEEPre: Multi-functional enzyme function
prediction with hierarchical multi-label deep learning,” Frontiers Genet., vol. 9, p. 714,
Jan. 2019.
[34] K. Kamnitsas, C. Ledig, V. F. J. Newcombe, J. P. Simpson, A. D. Kane, D. K. Menon,
D. Rueckert, and B. Glocker, “Efficient multi-scale 3DCNN with fully connected
CRF for accurate brain lesion segmentation,” Med. Image Anal., vol. 36, pp. 61–78,
Feb. 2017.
168 Machine Learning and Deep Learning in Medical Data Analytics
CONTENTS
9.1 Introduction................................................................................................... 169
9.2 Background of Breast Cancer........................................................................ 170
9.3 Thermography............................................................................................... 172
9.3.1 Thermography in Breast Cancer Diagnosis....................................... 172
9.4 Breast Thermograms Using Deep Learning Classification........................... 173
9.4.1 Preprocessing Images Process........................................................... 175
9.4.2 Convolutional Neural Networks........................................................ 175
9.4.2.1 Feature Extraction Layer.................................................... 175
9.4.2.2 Fully Connected Layer........................................................ 176
9.4.3 Image Classification........................................................................... 176
9.5 Review on Breast Cancer Using Thermographs and Deep Learning........... 177
9.6 Conclusion..................................................................................................... 180
References�������������������������������������������������������������������������������������������������������������� 180
9.1 INTRODUCTION
Worldwide cancer statistics report that breast cancer disease is one of the most
deadly types of cancer, after cellular breakdown in the lungs [1]. In 2018, two
billion new breast cancer cases were accounted around the world and 627,000
deaths. According to a research [2], breast cancer patient endurance is heavily
linked to tumor size during the diagnosis phase, indicating a 98% chance of patient
survival if the size is lesser than 10 mm. According to a parallel study, 70% of
bosom disease cases are identified when the malignancy is 30 mm in size [3].
When a tumor is larger than 20 mm in diameter, it is usually detectable during
screening [4]. As a result, boosting early cancer disease detection is crucial for
encouraging early treatment. Early treatment might be useful after detection by
screening assessments, for example, clinical breast examination (CBE) and breast
DOI: 10.1201/9781003226147-9 169
170 Machine Learning and Deep Learning in Medical Data Analytics
9.3 THERMOGRAPHY
Thermography is a prescient method for observing the state of plant hardware con-
structions and frameworks not simply electrical gear. It utilizes instrumentation to
view infrared energy outflows (surface temperature) to decide working conditions.
Infrared thermography (IRT), thermal imaging, and thermal video are instances of
infrared imaging areas. Thermographic cameras are a tool for distinguishing radia-
tion in the long-infrared scope of the electromagnetic range (approximately 9,000–
14,000 nanometers or 9–14 μm) and create thermograms, which are photographs of
the radiation. The dark radiative cooling law states that anything with a temperature
above absolute zero emits infrared radiation, making it possible for thermography to
see human current circumstances with or without visible illumination.
The consignment of radiation generated by every organism increases with tem-
perature; consequently, thermography allows us to perceive seasonal variation.
Thermal entities stand out well against cooler substrates during screening using ther-
mal imaging cameras; people and certain other warm-blooded organisms become
easily visible against the climate, day or night. As a result, thermography is particu-
larly beneficial to the military and various users of espionage equipment.
extremely easy to utilize. The NN algorithm allows for the learning of subjective
medical image values. Consequently, it is suitable for interpretations of bosom ther-
mograms characterization.
A CNN is a DNN technique in which it processes the input pictures by allot-
ting various accessible weights and biases to locate significant features which sepa-
rate one picture from another. Thus, the classification output can be seen as the
yield. Figure 9.4 supports the overall CNNs building for the classification of the
bosom thermograms in two sets, normal and cancer. Three significant contempla-
tions should be focused: “dataset readiness in picture pre-preparing”, “feature learn-
ing”, and “classification”. The classification may be double (“normal and cancer”), or
many classes, for example, solid, generous, and harmful. In the accompanying part,
we survey the ideas and related endeavors in CNN usage for classification breast
thermogram.
FIGURE 9.4 Proposed CNN method for premature breast thermograms detection.
Comprehensive Survey on Breast Cancer Thermography Classification 175
FIGURE 9.5 Conception of convolutional phases of a cancerous breast thermograms [see [19]].
with seven diverse deep learning pre-prepared structures. The DNNs VGG16 dem-
onstrated the best execution, accomplishing an accuracy of 91.18%. The outcomes
recommend that bosom disease thermography related to DNN can be utilized as
subordinate to mammography for pre-screening, despite the fact that there are
still bogus positives. Information growth has appeared to expand exactness of
DNN with a restricted dataset, for example, for their situation. Fernández-Ovies
et al. [28] introduced a CNN-based technique for bosom disease analysis utilizing
warm pictures. They indicated that an all-around delimited dataset split strategy
is required to diminish the inclination and over-fitting through the preparation
stage; in this manner their exploratory outcomes affirm that. Furthermore, their
paper passes on the primary cutting-edge benchmark of CNN models, for exam-
ple, ResNet, SeResNet, VGG16, Inception, Inception ResNetV2, and Xception
for the dataset from Mastology Research with Infrared Image (DMR-IR) dataset.
Similarly, this investigation builds up the primary CNN hyper-boundaries stream-
lining in a thermography dataset for bosom disease, where the top CNN model
accomplished 92% exactness. They showed that the compromise between database
size and data enlargement procedures is critical in classification assignments lack-
ing adequate information.
Iqbal et al. [25] introduced the framework of thermography-based application-
explicit Advanced Back End processor for a shrewd screening tool. Thermal
pictures of the chest taken by infrared cameras are pre-prepared to specify the dis-
tricts of interest. To ensure productive equipment, texture features are deliberately
chosen, which are then taken off to a double classifier, dependent on trained linear
support vector machine and CNN to choose the decision boundary. The presented
framework accomplishes proficient equipment usage by misusing developed clas-
sifier. In 2018, Matheus and Lucas [26] built a supervised strategy for dissecting
infrared thermography of bosoms for exact classification utilizing CNN that does
not depend on selected features. They propose four procedures to decide how the
unique convention fits better in a CNN algorithm. The outcomes demonstrated
that their DL approach utilizing the shaded picture dataset gave great execution
not as grayscale dataset for “static protocol” and those CNNs got outcomes for
the two conventions: static and dynamic. They expect that since substantially
more data about temperature is put into the shading set and CNN features catch
patterns in a more productive way than selected feature selection. Following this
investigation, Dalmia et al. [29] indicated that the division of problem areas in a
thermal image is an intense issue generally because of the inaccessibility of large
thermography datasets on bosoms, the absence of standardized information, and
the reliance of caught warm pictures on mood, enthusiasm, and actual condition
of the subject. In this study, they investigate different CNN models for semantic
segmentation beginning with naive patch-based classifiers to more refined ones
including a few varieties of the encoder–decoder network for recognizing the
areas of interest in the warm pictures. The author suggested the importance of the
utilization of multi-layered CNN for identification of areas of interest in infrared
bosom warm pictures. Other works related to breast cancer thermography are
reported concisely in Table 9.1.
Comprehensive Survey on Breast Cancer Thermography Classification
TABLE 9.1
Review on Breast Cancer Thermography using Deep Learning
179
180 Machine Learning and Deep Learning in Medical Data Analytics
9.6 CONCLUSION
Early breast cancer recognition remains fundamental to bosom disease control.
Bosom self-assessment is prescribed by the World Health Organization to raise
awareness in women about bosom malignancy hazards. Thermography is designed
for performing an early localization via screening technique, and we accept that it
gives a promising improvement for a self-screening strategy which would identify
bosom malignancy at an early stage. An outline of breast thermograms may demon-
strate that the early indications of bosom cancer can be seen through distinguishing
the lopsided thermal conveyances between the bosoms.
The lopsided thermal distribution on bosom thermograms would be assessed
using computer-aided innovation, which can limit mistakes. Our review has demon-
strated that the present NN models have prompted an optimization in classification
exactness of breast cancer thermograms, especially in distinguishing among normal
and harmful cases. Nevertheless, the NNs model performance should be improved.
Future research would see more efforts in improving bosom thermogram classifi-
cation. This will need delegate datasets, preparing enlarged ROIs, allocating ker-
nels, and executing “lightweight” CNN models. Accomplishment of these goals will
abbreviate the time relating to convolution calculation and increment precision rates.
A free detection technique utilizing thermography can then be built for self-bosom
detection tool at a beginning phase without requiring actual inclusion of tissues.
REFERENCES
[1] IAFR Cancer. Global Cancer Observatory [Online]. Available: http://gco.iarc.fr/
Accessed: Feb. 3, 2021.
[2] C. Nickson and A.M. Kavanagh, “Tumour size at detection according to different mea-
sures of mammographic breast density,’’ J. Med. Screening, vol. 16, no. 3, pp. 140–46,
Sep. 2009.
[3] S.A. Narod, “Tumour size predicts long-term survival among women with lymph node-
positive breast cancer,’’ Current Oncol., vol. 19, no. 5, pp. 249–53, Sep. 2012.
[4] OncoLink Team, Jan. 2021. All About Breast Cancer. [Online]. Available: https://www.
oncolink.org/cancers/breast/all-about-breast-cancer
[5] Breast Cancer: Prevention and Control. World Health Organization, Geneva,
Switzerland, 2019.
[6] H. Chougrad, H. Zouaki, and O. Alheyane, “Deep convolutional neural networks for
breast cancer screening,’’ Comput. Methods Programs Biomed., vol. 157, pp. 19–30,
Apr. 2018.
[7] M.A. Al-Masni, M.A. Al-Antari, J.-M. Park, G. Gi, T.-Y. Kim, P. Rivera, E. Valarezo,
M.-T. Choi, S.-M. Han, and T.-S. Kim, “Simultaneous detection and classification of
breast masses in digital mammograms via a deep learning YOLO-based CAD system,’’
Comput. Methods Programs Biomed., vol. 157, pp. 85–94, Apr. 2018.
[8] J. Arevalo, F.A. González, R. Ramos-Pollán, J. L. Oliveira, and M.A.G. Lopez,
“Representation learning for mammography mass lesion classification with convolu-
tional neural networks,” Comput. Methods Programs Biomed., vol. 127, pp. 248–57,
Apr. 2016
[9] D. Bardou, K. Zhang, and S. M. Ahmad, “Classification of breast cancer based on his-
tology images using convolutional neural networks,’’ IEEE Access, vol. 6, pp. 24680–
93, 2018.
Comprehensive Survey on Breast Cancer Thermography Classification 181
[10] X. Zhou, T. Kano, H. Koyasu, S. Li, T. Hara, X. Zhou, M. Matsuo, and H. Fujita, “Automated
assessment of breast tissue density in non-contrast 3D CT images without image seg-
mentation based on a deep CNN,’’ Proc. SPIE, vol. 10134, Mar. 2017, Art. no. 101342Q.
[11] H. Li, J. Weng, Y. Shi, W. Gu, Y. Mao, Y. Wang, W. Liu, and J. Zhang, “An improved
deep learning approach for detection of thyroid papillary cancer in ultrasound images,’’
Sci. Rep., vol. 8, no. 1, pp. 1–12, Dec. 2018.
[12] S.S. Patra, O.P. Jena, G. Kumar, S. Pramanik, C. Misra, and K.N. Singh, “Random
Forest Algorithm in Imbalance Genomics Classification.” In: Data Analytics in
Bioinformatics: A Machine Learning Perspective, Wiley Scrivener Publication, 2021.
https://doi.org/10.1002/9781119785620.ch7.
[13] P. Pattnayak and O.P. Jena, “Innovation on Machine Learning in Healthcare Services-
An Introduction.” In: Machine Learning for Healthcare Applications, pp. 1–14, Wiley
Scrivener Publisher, 2021. https://doi.org/10.1002/9781119792611.ch1
[14] N. Panigrahi, I. Ayus and O.P. Jena, “An Expert System-Based Clinical Decision
Support System for Hepatitis-B Prediction and Diagnosis.” In: O. P. Jena, S.N. Mohanty,
G. Nalinipriya and A. Sarkar (Eds.)Machine Learning for Healthcare Applications,
pp. 57–75, Wiley Scrivener Publisher, 2021. https://doi.org/10.1002/9781119792611.ch4
[15] K. Paramesha, H.L Gururaj, and O.P. Jena, “Applications of Machine Learning in Biomedical
Text Processing and Food Industry.” In: Machine Learning for Healthcare Applications,
pp. 151–67, Wiley Scrivener Publisher, 2021. https://doi.org/10.1002/9781119792611.ch10
[16] A. Hassan, M. Nasr, and W. Saber, “The future of Internet of Things for anomalies
detection using thermography”, International Journal of Advanced Networking and
Applications (IJANA), vol. 11, no. 1, pp. 4142–49 (2019).
[17] G. Ahmad, Z. Iman, G. Hossein, and H. Javad, “A review of the dedicated studies to
breast cancer diagnosis by thermal imaging in the fields of medical and artificial intel-
ligence sciences,” Biomedical Research, vol. 27, no. 2, pp. 543–52, 2016.
[18] Thermobiological, 2016. Available at: http://www.breastthermography.com/breast_
thermography_ proc.htm
[19] R. C. Gonzalez, “Deep convolutional neural networks [lecture notes],’’ IEEE Signal
Process. Mag., vol. 35, no. 6, pp. 79–87, Nov. 2018.
[20] R. Yamashita, M. Nishio, R. K.G. Do, and K. Togashi, “Convolutional neural networks: An
overview and application in radiology,” Insights Image, vol. 9, no. 4, pp. 611–29, Aug. 2018.
[21] O. Simeone, “A brief introduction to machine learning for engineers,” Found. Trends
Signal Process, vol. 12, nos. 3–4, pp. 200–431, 2018.
[22] Y. Samir and J. Shivajirao, “Thermal infrared imaging based breast cancer diagnosis
using machine learning techniques,” Multimedia Tools and Applications, 2020. https://
doi.org/10.1007/s11042-020-09600-3
[23] S. Ekici and H. Jawzal, “Breast cancer diagnosis using thermography and convolu-
tional neural networks,” Medical Hypotheses, 2020 Apr, 137, pp. 109542. https://doi.
org/10.1007/s11042-020-09600-310.1016/j.mehy.2019.109542
[24] H. Zadeh, A. Fayazi, B. Binazir, and M. Yargholi, “Breast cancer diagnosis based on
feature extraction using dynamic models of thermal imaging and deep autoencoder
neural networks,” Journal of Testing and Evaluation 49, 2020, https://doi.org/10.1520/
JTE20200044
[25] H. Iqbal, B. Majeed, U. Khan and M. A. Bin Altaf, “An Infrared High Classification
Accuracy Hand-held Machine Learning based Breast-Cancer Detection System,” 2019
IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 1–4, Nara, Japan,
2019. https://doi.org/10.1109/BIOCAS.2019.8918687
[26] M. Oliveira and L. Grassano, “Convolutional Neural Networks for Static and Dynamic
Breast Infrared Imaging Classification,” 2018 31st SIBGRAPI Conference on
Graphics, Patterns and Images (SIBGRAPI), pp. 174–81, Parana, Brazil, 2018. https://
doi.org/10.1109/SIBGRAPI.2018.00029
182 Machine Learning and Deep Learning in Medical Data Analytics
[27] Z. Juan Pablo, A. Zein, B. Khaled, M. Safa, and Z. Noureddine, “A CNN-based meth-
odology for breast cancer diagnosis using thermal images.” Computer Methods in
Biomechanics and Biomedical Engineering: Imaging & Visualization, pp. 1–15, 2020.
https://doi.org/10.1080/21681163.2020.1824685
[28] F.J. Fernández-Ovies, S. Alférez-Baquero, E.J. de Andrés-Galiana, A. Cernea, Z.
Fernández-Muñiz, and J.L. Fernández-Martínez, Detection of Breast Cancer Using
Infrared Thermography and Deep Neural Networks. In: Rojas I., Valenzuela O., Rojas
F., and Ortuño F. (eds), Bioinformatics and Biomedical Engineering. IWBBIO 2019.
Lecture Notes in Computer Science, Pp 513-524 vol. 11466, 2019, Springer Publisher,
Cham. https://doi.org/10.1007/978-3-030-17935-9_46
[29] A. Dalmia, S.T. Kakileti, and M. Geetha, “Exploring Deep Learning Networks for
Tumour Segmentation in Infrared Images.” 14th Quantitative InfraRed Thermography
Conference, 25 – 29 June 2018, Berlin, Germany, 2018. https://doi.org/10.21611/
qirt.2018.052
[30] S. Mishra, A. Prakash, S. K. Roy, P. Sharan, and N. Mathur, “Breast Cancer Detection
using Thermal Images and Deep Learning,” 2020 7th International Conference on
Computing for Sustainable Global Development (INDIACom), pp. 211–216, New
Delhi, India, 2020. doi: https://doi.org/10.23919/INDIACom49435.2020.9083722
[31] M. Farooq and P. Corcoran, “Infrared Imaging for Human Thermography and
Breast Tumor Classification using Thermal Images,” 2020 31st Irish Signals and
Systems Conference (ISSC), pp. 1–6, Letterkenny, Ireland, 2020. doi: 10.1109/
ISSC49989.2020.9180164
[32] M. Sebastien., O. Krejcar, P. Maresova, A. Selamat, and K. Kuca, Novel Four Stages
Classification of Breast Cancer Using Infrared Thermal Imaging and a Deep Learning
Model. In: Rojas I., Valenzuela O., Rojas F., and Ortuño F. (eds), Bioinformatics and
Biomedical Engineering. IWBBIO 2019. Lecture Notes in Computer Science, vol.
11466, Springer, Cham, 2019. https://doi.org/10.1007/978-3-030-17935-9_7
[33] S. Tello-Mijares, F. Woo, and F. Flores, “Breast cancer identification via thermogra-
phy image segmentation with a gradient vector flow and a convolutional neural net-
work.” Journal of Healthcare Engineering, 2019, Article ID 9807619. https://doi.
org/10.1155/2019/9807619
[34] R. Roslidar, K. Saddami, F. Arnia, M. Syukri, and K. Munadi, “A Study of Fine-
Tuning CNN Models Based on Thermal Imaging for Breast Cancer Classification,”
2019 IEEE International Conference on Cybernetics and Computational Intelligence
(CyberneticsCom), pp. 77–81, Banda Aceh, Indonesia, 2019. https://doi.org/10.1109/
CYBERNETICSCOM.2019.8875661
[35] J. Torres-Galván, E. Guevara and F. J. González, “Comparison of Deep Learning
Architectures for Pre-Screening of Breast Cancer Thermograms,” 2019 Photonics
North (PN), pp. 1–2, Quebec City, QC, Canada, 2019. https://doi.org/10.1109/
PN.2019.8819587
10 Deep Learning
Frameworks for
Prediction, Classification
and Diagnosis of
Alzheimer’s Disease
Nitin Singh Rajput, Mithun Singh Rajput,
and Purnima Dey Sarkar
CONTENTS
10.1 Introduction.................................................................................................. 183
10.2 DL-Based Prediction of AD......................................................................... 185
10.2.1 Prediction of Amyloidogenicity...................................................... 186
10.2.2 Prediction of Tau Protein Abnormality.......................................... 187
10.2.3 Prediction of Transition from Mild Cognitive Impairment to AD......188
10.3 DL-Based Classification of AD................................................................... 189
10.3.1 Classification Using MRI Data....................................................... 189
10.3.2 Classification Using PET Data........................................................ 191
10.4 DL-Based Diagnosis of AD......................................................................... 191
10.4.1 Diagnosis Using MRI Data............................................................. 193
10.4.2 Diagnosis Using PET Data............................................................. 193
10.4.3 Multimodal Diagnosis Using MRI and PET Data.......................... 194
10.4.4 Diagnosis Using Other Datasets..................................................... 194
10.5 Limitations and Challenges in DL-Based Detection,
Classification, and Diagnosis of AD............................................................ 195
10.6 Conclusion.................................................................................................... 196
References............................................................................................................... 197
10.1 INTRODUCTION
Alzheimer’s disease (AD) is one of the significant and common appearances of pro-
gressive dementia. Pathogenesis of AD is represented by two attributes: development
of extra-cellular senile plaquette made up of insolvable amyloid beta peptide along
with intraneural gathering of neuro-fibrillary entangles of accumulated hyperphos-
phorylated tau [1]. It also causes gliosis, defined as a widespread modification of glial
cells [2]. These hallmarks are significant advents of AD; although, the ultimate situ-
ation of this disorder remains imperceptible. It is anticipated that worldwide one out
DOI: 10.1201/9781003226147-10 183
184 Machine Learning and Deep Learning in Medical Data Analytics
of eighty-five individuals will be touched by this disease by the year 2050 [3]. Thus,
in order to have an effective management and to retard or avert the disease progres-
sion, excessive efforts are underway to come up with strategies for early detection,
classification, and diagnosis of AD. Particularly progressive neuroimaging and other
techniques viz. magnetic-resonance-imaging (MRI), positron-emission-tomography
(PET), computed-tomography (CT), microscopy, X-ray, etc. are commonly used to
recognize anatomical as well as molecular biomarkers pertaining to this disease [4].
Speedy advancement in neuroimaging technologies has complicated the process of
amalgamating extensive, multi-dimensional multimodal data related to neuroimag-
ing. Consequently, attention has been shooting up rapidly in computer-based ML
techniques aiming at integrative investigations for AD.
ML comes under the umbrella of artificial intelligence (AI), which studies multi-
faceted associations among variables in data and these algorithms have been catego-
rized as supervised, unsupervised, and reinforcement learning [5]. So as to solicit ML
algorithms, pre-processing phases and proper architectural design must be ensured [6].
Sorting studies with the aid of ML usually requires feature mining, feature assortment,
reduction in dimensions of data, and choice of classification algorithm based on the
features. Such requirements could be fulfilled with focused knowledge and optimiza-
tion, which may lack in reproduction and may be time-consuming [6]. With purpose of
removing these limitations, DL, a special class of ML techniques, has been employed
in AD research. DL is a subclass of ML, which acquires features from a hierarchical
learning method where depictions are inevitably revealed from raw data [7]. DL algo-
rithms engage numerous, deep layers made up of perceptron algorithm that apprehends
both low-level and high-level data depictions, permitting them to acquire more affluent
notions of inputs [7]. This precludes the necessity of manually engineered structures
and permits DL structures to automatically discover formerly unknown models apart
from simplifying various types of data. Modified versions of such algorithms have
been effectively used in the area of medicine and other engineering fields [8].
In neurodegenerative disease research, DL uses first-hand data, acquired by neu-
roimaging, to produce features via on-the-fly learning and it has been gaining valu-
able consideration in the context of extensive, multidimensional analysis of medical
images. For instance, DL methods, like convolutional neural networks (CNN), are
used to show faster growth in comparison to prevailing ML approaches [9]. DL tech-
niques are performing a progressive vital part in neurological research, addressing
complications in various sub-domains. Initially, sorting the images and partitioning
was the focus point of DL advancement. Such functions are exclusively suited to
DL due to many neuroimaging data obtained by manual analysis. Gradually, DL
techniques are applied to efficient brain mapping along with related areas, with the
aid of neuroimaging data for various tasks like early detection and classification.
Lately, DL-based diagnostic studies taking select data types, laboratory outcomes,
and images into account are used to pinpoint AD risk. Considering these facts, in
this chapter, we discuss the various models for scrutiny of neuroimaging data using
DL approaches for the early prediction, classification, and diagnosis of AD.
Figure 10.1 illustrates the generalized scheme for early detection, classification,
and diagnosis of AD. This scheme is employed for various datasets such as MRI,
PET, CT, and cerebro-spinal fluid (CSF) to study the outcomes. In this chapter, select
studies, which are following the technique specified in Figure 10.1, have been taken
Deep Learning Frameworks for Alzheimer’s Disease 185
FIGURE 10.1 Illustration of DL framework for detection, classification, and diagnosis of AD.
into account. Here, images from various sources are pre-processed and then feature
extraction, selection, and classification are performed using DL methods and, after
that, the outcome is considered to detect, classify, and diagnose AD.
10.2.1 Prediction of Amyloidogenicity
Brain amyloids are a specific kind of protein mass that are typical features of AD and
exhibit a specific diffraction-like pattern during X-ray. Some experimental procedures
allow identifying these peptides; however, they are expensive and tedious and hence
unsuitable for genome-wide investigations and need alternative prediction models [2].
Wojciechowski and Kotulska [17] proposed a new structure-based tech-
nique combined with ML tactics for predicting amyloidogenicity, termed PATH
(Prediction of Amyloidogenicity by THreading). It has been demonstrated that
relating existing structures of amyloidogenic fragments improves prediction per-
formance of AD [17]. Several ML techniques are employed on various steady
classes of query peptide, using their energy terms. Also, these methods give
insights into possibly the most steady structural group of peptides when kept open
in a crystallizing environment [17].
Scoring schemes that are semi-quantitative in nature, like the Consortium to
Establish a Registry for Alzheimer’s Disease (CERAD) created on ML tactics,
have produced numerical results for whole slide images, which were linked with
semi-quantified results derived by people for AD pathology [27]. Vizcarra et al. [18]
Deep Learning Frameworks for Alzheimer’s Disease 187
TABLE 10.1
Deep Learning Models in Early Prediction of Dementia of the Alzheimer Type
Imaging/ Number of
Method Dataset Technique Subjects (n) Outcome Reference
NMF-based ADNI MRI 1870 Accuracy 5% higher than [11]
CMC existing method (WRMK)
PATH Protein Data Modeller 9.21 1080 AUC of 0.87 [17]
Bank hexapeptides
CNN Data from CERAD 40 Measurable differentiation [18]
individuals among groups (p < 0.05)
CNN ADNI MRI 509 Accuracy rate 0.84 [19]
SlideNet Data from PET 2 AUC greater than 0.85 for [20]
individuals all cases
CNN ADNI MRI 785 AUC of 0.92 [21]
DBN ADNI FDG-PET 109 Accuracy of 83.9% [22]
DCNN ADNI DTI 151 Prediction accuracy 0.7463 [13]
at 24 m
SNN ADNI and MRI 3566 Sensitivity 0.82 and [23]
BIOCARD specificity 0.97
MRNN ADNI MRI, PET and 1618 Accuracy 81% [24]
other biological
markers
ResNet ADNI sMRI-fMRI data 134 Significant outperform (p < [14]
fusion framework 0.001)
CNN ADNI MRI 2146 Concordance index greater [25]
and AIBL than 0.70
CTDE ADNI MRI 805 R value for MMSE at 36 m [16]
is 0.85
DNN Data from EEG 20 Correct classification rate [15]
individuals (CCR) more than 50%
MSH-ELM ADNI MRI, FDG- 202 Accuracy of 96.10% [26]
PET, and CSF
biomarkers
confirmed these ML algorithms by means of CNNs and showed that the pathological
heterogeneity might modify the outcome parameters of the algorithm. Their findings
authenticate CNN models as reliable and robust in scenarios like cohort disparities
and deliver additional concept proving for futuristic investigations to include ML
techniques into neuro-pathological applications [18].
help of the kernel normalized whole brain anatomical volumetric asymmetry encod-
ing features [23]. Lee et al. [24] showed the applicability of multimodal recurrent
neural network (MRNN) while predicting conversion of mild cognitive impairment
to AD. The developed scheme integrates longitudinal multi-domain data which com-
bines longitudinal CSF and cognitive performance biomarkers with cross-section of
neuro-imaging biomarkers gained from AD neuro-imaging initiative (ADNI). This
model yielded an accuracy of up to 75% [24].
TABLE 10.2
Deep Learning Models in Classification of Dementia of the Alzheimer Type
Imaging/ Number of
Method Dataset Technique Subjects (n) Outcome Reference
DL ADNI MRI 4099 Accuracy of 91.83% [37]
(Backpropagation and NIFD
algorithm)
XGBoost ADNI and MRI 2727 Sensitivity of [38]
OASIS 68.00% and
specificity of
70.00%
FCN ADNI, AIBL, MRI 1483 Accuracy of 75.00% [39]
FHS, NACC to 95.00% for
various dataset
SCNN inspired by OASIS MRI 382 Accuracy of 99.05% [40]
OxfordNet
architecture
CNN and 3D ADNI MRI 449 Accuracy of 88.90% [41]
DenseNet
CNN Data from MRI 48 Accuracy of 73.00% [42]
individuals
VCNN ADNI MRI 695 Accuracy of 86.60% [36]
DM2L ADNI MIRIAD, MRI 1984 AUC of 0.986 [43]
AIBL
ResNet-50 OASIS MRI 4139 Accuracy of 99.34% [44]
CNN Data from MRI 196 Accuracy of 97.65% [45]
individuals
3LHPM-ICA Data from rs-fMRI 34 Accuracy of 95.59% [46]
individuals
3D CNN ADNI PET 300 Accuracy of 90.80% [47]
2D deep CNN Data from PET 430 Accuracy of 97.90% [48]
individuals (18F-florbetaben
scans)
CNN Data from PET 160 Accuracy of 89.32% [50]
individuals
CNN ADNI MRI and 1884 Accuracy of 85.00% [49]
FDG-PET to 95.00% for both
dataset
CNN and RNN ADNI FDG-PET 339 Accuracy of 91.20% [51]
DL algorithm ADNI MRI and PET 202 Accuracy of 91.40% [52]
specificity around 70%. Moreover, Oh et al. [36] distinguished AD with light cog-
nitive impairment with the aid of volumetric CNN and transfer learning in ADNI
dataset using MRI with an accuracy of 86.60%.
A new DL-based framework has been developed by integrating a fully convolutional
network (FCN) and a conventional multi-layer perceptron. It produces visualizations
Deep Learning Frameworks for Alzheimer’s Disease 191
of AD risk, with higher resolutions. It could now be utilized to estimate the AD grade
[39]. Moreover, aiming at reducing the dependencies on big scale datasets, Mehmood
et al. [40] developed a Siamese Convolutional Neural Network (SCNN) which is
based on VGG16 architecture to classify various stages of AD in better manner, that
is, between no dementia and AD. In another study, attributes of the multi-task CNN
and DenseNet have been utilized to automate hippocampal segmentation and classify
the AD using structured MRI data [41]. A six-layer CNN model has been utilized by
Wada et al. [42] based on MRI feature analysis to classify AD and distinguish it from
dementia with Lewy body with an accuracy of 73.00%. Using a deep multi-task multi-
channel learning (DM2L) scheme, Liu et al. [43] identified anatomical landmarks from
MRI scans with the aid of data-driven algorithm and established a combined clas-
sification and regression framework for diagnosis of AD. Similar studies have been
conducted by Fulton et al. [44], Wang et al. [45], and Qiao et al. [46] that extract data
from MRI scans and utilize DL architecture for classification of AD.
TABLE 10.3
DL Models in Diagnosis of Dementia of the Alzheimer Type
Imaging/ Number of
Method Dataset Technique Subjects (n) Outcome Reference
CNN ADNI MRI 750 Accuracy of 90.50% [54]
WSL-based DL KACD and MRI 12800 Sensitivity of 99.69% [55]
(ADGNET) ROAD
FCN ADNI fMRI 170 Accuracy of 85.20% [56]
ANN and Data from Raman 38 Sensitivity and specificity [83]
SVM-DA individuals Spectroscopy more than 80%
(CSF)
DMFNet ADNI MRI and PET 500 Accuracy of 95.21% [72]
CNN ADNI MRI and DTI 406 Accuracy of 93.50% [57]
TL (ResNet) ADNI and MRI 2179 Accuracy of 86.05% [63]
OASIS
3DAN ADNI MRI 1832 Accuracy of 72.00% [60]
CNN ADNI MRI 1075 Accuracy of 97.01% [61]
DNN ADNI MRI 801 Accuracy of 71.18% [58]
RSBN ADNI fMRI 170 Accuracy more than 55% [66]
for various methods
3D CNN ADNI MRI 315 Accuracy of 98.06% [59]
DL (multimodal ADNI MRI 72 Accuracy of 83.30% [60]
fusion)
SVM ADNI FDG-PET 466 Accuracy of 91.50% [70]
CNN architecture ADNI 18F-FDG PET 1002 specificity of 82% and [71]
Inception sensitivity of 100%
V3
DL, RF, EMIF-AD CSF metabolites 883 AUC greater than 0.85 [84]
XGBoost for all
DNN ADNI MRI, PET and SNP 805 Accuracy of 90.00% [74]
LUPI ADNI MRI and PET 103 Accuracy more than 85% [75]
for various algorithm
3D CNN ADNI MRI and 731 Accuracy of 87.46% [78]
FDG-PET
DNN ADNI MRI and 1242 Accuracy of 86.40% [79]
FDG-PET
MM-SDPN ADNI MRI and PET 202 Accuracy more than [76]
95.00% for various
classifiers
DNN ADNI MRI, PET and 805 Accuracy of 65.00% [74]
SNP
AAL (DL) ADNI MRI, PET and 818 Accuracy of 90.00% [77]
CSF and blood
markers
SVMs, MLPs, Data from Kinematic 108 Accuracy more than [85]
RBNs, and DBNs individuals acquisition and 70.00%
assessment
Deep Learning Frameworks for Alzheimer’s Disease 193
image processing and analysis [90]. Incapacitating these shortcomings will neces-
sitate the hard work of interdisciplinary squads of physicians, engineers, computer
scientists, ethicists, and legal experts. It is one of the ways by which humans could
actually understand the prospective uses of DL in medicine to improve the profi-
ciency of frontline workers and improve the degree of care to patients.
10.6 CONCLUSION
AD is a growing neurodegenerative ailment manifested by decay in brain functions
with no proper management. Since it is crucial to avert the progression of disease,
efficient, self-learning techniques are required to predict and diagnose it. The nature
of ample dimensions of neural data, extracted mainly from neuroimaging with
computer-aided algorithms, brings out the conception of accurate computer-aided
diagnostic systems. DL, a high-tech ML tactic, has outperformed contemporary ML
techniques in recognizing entangled structures in multi-dimensional data which is
complex in nature. Structural alterations in the brain could be detected by DL tech-
niques which could be utilized to detect features with information related to AD.
Automated classification could also be applied for early detection during prodromic
stages of the disease.
In this chapter, based on close scrutiny of the existing literature, a wide-ranging
investigation of recent automated approaches to classify AD, algorithm-based neuro-
imaging procedures for dementia diagnosis, and systematic explanation of the most
recent DL schemes for early prediction of AD have been presented. Also, progres-
sion monitoring of dementia of the Alzheimer type using medical image analysis and
DL algorithms have been discussed. The focus was mainly on DL techniques and
collective methods, along with some other ML techniques. The research inference,
hurdles, and the future instructions pertaining to the study have also been empha-
sized. Significant research issues in the integration of DL tools in the clinical scenery
and the problems for embarking on overcoming current challenges are also noted.
DL holds the capability to deeply modify the practice of medicine. Researchers
have gathered data from different sources like hospitals, research laboratories and
online repositories (AIBL, BIOCARD, EMIF-AD, FHS, KACD, NACC, NIFD,
MIRIAD and ROAD). It is perceived that MRI and PET information and CNN could
be effectively employed in order to predict and diagnose AD. In a partial accessible
neuroimaging data set, fusion methods have exhibited accuracies of up to 96%, 99%
and 99.5% for AD prediction, classification and diagnosis respectively.
DL approaches, which are the deeper version of neural networks, continue to
advance in performance and seem to hold promise for diagnostic classification of
AD using multimodal neuroimaging data. This is true because DL aims to under-
stand the data representations in better ways, which can be built into any type of
ML techniques. Research on AD that uses DL is still budding, refining performance
by fusing additional data, increasing transparency that increases understanding of
specific disease-associated attributes. In the future, DL methods in the hybridizing
of nature-inspired systems should be considered for research in view of more effi-
cacious presentation in the prediction, classification, and diagnosis of neurological
disorders.
Deep Learning Frameworks for Alzheimer’s Disease 197
REFERENCES
[1] Pascoal, Tharick A, Sulantha Mathotaarachchi, Monica Shin, et al. “Synergistic inter-
action between amyloid and tau predicts the progression to dementia.” Alzheimer’s
Dementia 13 (2017): 644–53.
[2] Rajput, Mithun Singh, Nilesh Prakash Nirmal, Devashish Rathore, and Rashmi Dahima.
“Dimethyl fumarate exerts neuroprotection by modulating calcineurin/NFAT1 and
NFκ B dependent BACE1 activity in Aβ1-42 treated neuroblastoma SH-SY5Y cells.”
Brain Research Bulletin 165 (2020): 97–107.
[3] Salehi, Ahmad Waleed, Preety Baglat, and Gaurav Gupta. “Alzheimer’s disease diag-
nosis using deep learning techniques.” Int. J. Eng. Adv. Technol 9, no. 3 (2020): 874–80.
[4] Veitch, Dallas P., Michael W. Weiner, Paul S. Aisen, Laurel A. Beckett, Nigel J. Cairns,
Robert C. Green, and Danielle Harvey et al. “Understanding disease progression and
improving Alzheimer’s disease clinical trials: Recent highlights from the Alzheimer’s
Disease Neuroimaging Initiative.” Alzheimer’s & Dementia 15, no. 1 (2019): 106–52.
[5] Jindal, Mansi, Gupta Jatin, and Bhushan Bharat. “Machine learning methods for
IoT and their future applications”. In 2019 International Conference on Computing,
Communication and Intelligent Systems (ICCCIS), pp. 430–34. IEEE, 2019.
[6] Samper-González, Jorge, Ninon Burgos, Simona Bottani, Sabrina Fontanella, Pascal
Lu, Arnaud Marcoux, Alexandre Routier et al. “Reproducible evaluation of classifi-
cation methods in Alzheimer’s disease: Framework and application to MRI and PET
data.” NeuroImage 183 (2018): 504–21.
[7] LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. “Deep learning.” Nature 521, no.
7553 (2015): 436–44.
[8] Sharma, Nikhil, Ila Kaushik, Bharat Bhushan, and Siddharth Gautam. “Applicability of
WSN and biometric models in the field of healthcare”. In 2020 Deep Learning Strategies for
Security Enhancement in Wireless Sensor Networks, pp. 304–29, IGI Global, India, 2020.
[9] Jo, Taeho, Kwangsik Nho, and Andrew J. Saykin. “Deep learning in Alzheimer’s dis-
ease: diagnostic classification and prognostic prediction using neuroimaging data.”
Frontiers in Aging Neuroscience 11 (2019): 220.
[10] Khamparia, Aditya, Prakash Kumar Singh, Poonam Rani, Debabrata Samanta, Ashish
Khanna, and Bharat Bhushan. “An internet of health things-driven deep learning
framework for detection and classification of skin cancer using transfer learning.”
Transactions on Emerging Telecommunication Technologies Sp. Issue (2020): e3963.
[11] Zhang, Xiaobo, Yan Yang, Tianrui Li, Yiling Zhang, Hao Wang, and Hamido Fujita.
“CMC: A consensus multi-view clustering model for predicting Alzheimer’s disease
progression.” Computer Methods and Programs in Biomedicine 199 (2021): 105895.
[12] Kim, Sung-Hyun, Sumin Yang, Key-Hwan Lim, Euiseng Ko, Hyun-Jun Jang, Mingon
Kang, Pann-Ghill Suh, and Jae-Yeol Joo. “Prediction of Alzheimer’s disease-specific
phospholipase c gamma-1 SNV by deep learning-based approach for high-throughput
screening.” Proceedings of the National Academy of Sciences 118, no. 3 (2021).
[13] Liu, Yuanyuan, Zhouxuan Li, Qiyang Ge, Nan Lin, and Momiao Xiong. “Deep Feature
Selection and Causal Analysis of Alzheimer’s Disease.” Frontiers in Neuroscience 13
(2019): 1198.
[14] Abrol, Anees, Zening Fu, Yuhui Du, and Vince D. Calhoun. “Multimodal data fusion
of deep learning and dynamic functional connectivity features to predict Alzheimer’s
disease progression.” In 2019 41st Annual International Conference of the IEEE
Engineering in Medicine and Biology Society (EMBC), pp. 4409–13. IEEE, 2019.
[15] Kim, Donghyeon, and Kiseon Kim. “Detection of early stage Alzheimer’s disease using
EEG relative power with deep neural network.” In 2018 40th Annual International
Conference of the IEEE Engineering in Medicine and Biology Society (EMBC),
pp. 352–55. IEEE, 2018.
198 Machine Learning and Deep Learning in Medical Data Analytics
[16] Yang, Mengya, Peng Yang, Ahmed Elazab, Wen Hou, Xia Li, Tianfu Wang, Wenbin
Zou, and Baiying Lei. “Join and Deep Ensemble Regression of Clinical Scores for
Alzheimer’s Disease Using Longitudinal and Incomplete Data.” In 2018 40th Annual
International Conference of the IEEE Engineering in Medicine and Biology Society
(EMBC), pp. 1254–57. IEEE, 2018.
[17] Wojciechowski, Jakub W., and Małgorzata Kotulska. “pAtH-prediction of
Amyloidogenicity by threading and Machine Learning.” Scientific Reports 10, no. 1
(2020): 1–9.
[18] Vizcarra, Juan C, Marla Gearing, Michael J Keiser, et al. “Validation of machine learn-
ing models to detect amyloid pathologies across institutions.” Acta Neuropathology
Communications 8, (2020): 59.
[19] Pan, Dan, An Zeng, Longfei Jia, Yin Huang, Tory Frizzell, and Xiaowei Song. “Early
detection of Alzheimer’s disease using magnetic resonance imaging: A novel approach
combining convolutional neural networks and ensemble learning.” Frontiers in
Neuroscience 14 (2020): 259.
[20] Alegro, Maryana, Yuheng Chen, Dulce Ovando, Helmut Heinser, Rana Eser, Daniela
Ushizima, Duygu Tosun, and Lea T. Grinberg. “Deep learning for Alzheimer’s disease:
Mapping large-scale histological tau protein for neuroimaging biomarker validation.”
BioRxiv (2020): 698902.
[21] Spasov, Simeon, Luca Passamonti, Andrea Duggento, Pietro Liò, Nicola Toschi, and
Alzheimer’s Disease Neuroimaging Initiative. “A parameter-efficient deep learning
approach to predict conversion from mild cognitive impairment to Alzheimer’s dis-
ease.” Neuroimage 189 (2019): 276–87.
[22] Shen, Ting, Jiehui Jiang, Jiaying Lu, Min Wang, Chuantao Zuo, Zhihua Yu, and
Zhuangzhi Yan. “Predicting Alzheimer Disease from mild cognitive impairment with
a deep belief network based on 18F-FDG-PET Images.” Molecular Imaging 18 (2019):
15–36.
[23] Liu, Chin-Fu, Shreyas Padhy, Sandhya Ramachandran, Victor X. Wang, Andrew
Efimov, Alonso Bernal, Linyuan Shi et al. “Using deep Siamese neural networks for
detection of brain asymmetries associated with Alzheimer’s disease and mild cognitive
impairment.” Magnetic Resonance Imaging 64 (2019): 190–99.
[24] Lee, Garam, Kwangsik Nho, Byungkon Kang, Kyung-Ah Sohn, and Dokyoon
Kim. “Predicting Alzheimer’s disease progression using multi-modal deep learning
approach.” Scientific Reports 9, no. 1 (2019): 1–12.
[25] Li, Hongming, Mohamad Habes, David A. Wolk, Yong Fan, and Alzheimer’s Disease
Neuroimaging Initiative. “A deep learning model for early prediction of Alzheimer’s
disease dementia based on hippocampal magnetic resonance imaging data.” Alzheimer’s
& Dementia 15, no. 8 (2019): 1059–70.
[26] Kim, Jongin, Boreom Lee. “Identification of Alzheimer’s disease and mild cognitive
impairment using multimodal sparse hierarchical extreme learning machine.” Human
Brain Mapping 39, no. 9 (2018): 3728–41.
[27] Al-Janabi, Shaimaa, André Huisman, Paul J. Van Diest. “Digital pathology: current
status and future perspectives.” Histopathology 61, no. 1 (2012): 1–9.
[28] Rajput, Mithun Singh, Nilesh Prakash Nirmal, Devashish Rathore, and Rashmi
Dahima. “Dimethyl Fumarate Mitigates Tauopathy in Aβ -Induced Neuroblastoma
SH-SY5Y Cells.” Neurochemical Research 45, no. 11 (2020): 2641–52.
[29] Kumar, Santosh, Bharat Bhusan, Debabrata Singh, Dilip kumar Choubey.
“Classification of Diabetes using Deep Learning.” In 2020 International Conference
on Communication and Signal Processing, pp. 651–55. IEEE, 2020.
[30] Maqsood, Muazzam, Faria Nazir, Umair Khan, Farhan Aadil, Habibullah Jamal, Irfan
Mehmood, and Oh-young Song. “Transfer learning assisted classification and detection
of Alzheimer’s disease stages using 3D MRI scans.” Sensors 19, no. 11 (2019): 2645.
Deep Learning Frameworks for Alzheimer’s Disease 199
[31] O’Bryant, Sid E., Stephen C. Waring, C. Munro Cullum, James Hall, Laura Lacritz,
Paul J. Massman, Philip J. Lupo, Joan S. Reisch, and Rachelle Doody. “Staging demen-
tia using Clinical Dementia Rating Scale Sum of Boxes scores: a Texas Alzheimer’s
research consortium study.” Archives of Neurology 65, no. 8 (2008): 1091–95.
[32] Sarraf, Saman, Ghassem Tofighi, and Alzheimer’s Disease Neuroimaging Initiative.
“DeepAD: Alzheimer’s disease classification via deep convolutional neural networks
using MRI and fMRI.” BioRxiv (2016): 070441.
[33] Hosseini-Asl, Ehsan, Robert Keynton, and Ayman El-Baz. “Alzheimer’s disease
diagnostics by adaptation of 3D convolutional network.” In 2016 IEEE International
Conference on Image Processing (ICIP), pp. 126–30. IEEE, 2016.
[34] Hosny, Ahmed, Chintan Parmar, John Quackenbush, Lawrence H. Schwartz, and Hugo
J.W.L. Aerts. “Artificial intelligence in radiology.” Nature Reviews Cancer 18, no. 8
(2018): 500–10.
[35] Khagi, Bijen, Goo-Rak Kwon, and Ramesh Lama. “Comparative analysis of Alzheimer’s
disease classification by CDR level using CNN, feature selection, and machine-learn-
ing techniques.” International Journal of Imaging Systems and Technology 29, no. 3
(2019): 297–310.
[36] Oh, Kanghan, Young-Chul Chung, Ko Woon Kim, Woo-Sung Kim, and Il-Seok Oh.
“Classification and visualization of Alzheimer’s disease using volumetric convolu-
tional neural network and transfer learning.” Scientific Reports 9, no. 1 (2019): 1–16.
[37] Hu, Jingjing, Zhao Qing, Renyuan Liu, Xin Zhang, Pin Lv, Maoxue Wang, Yang Wang,
Kelei He, Yang Gao, and Bing Zhang. “Deep learning-based classification and voxel-
based visualization of frontotemporal dementia and Alzheimer’s disease.” Frontiers in
Neuroscience 14 (2020): 1468.
[38] Suh, C.H., W.H. Shim, S.J. Kim, J.H. Roh, J-H. Lee, M-J. Kim, S. Park et al.
“Development and validation of a deep learning–based automatic brain segmentation
and classification algorithm for Alzheimer disease using 3D T1-weighted volumetric
images.” American Journal of Neuroradiology 41, no. 12 (2020): 2227–34.
[39] Qiu, Shangran, Prajakta S. Joshi, Matthew I. Miller, Chonghua Xue, Xiao Zhou, Cody
Karjadi, Gary H. Chang et al. “Development and validation of an interpretable deep learn-
ing framework for Alzheimer’s disease classification.” Brain 143, no. 6 (2020): 1920–33.
[40] Mehmood, Atif, Muazzam Maqsood, Muzaffar Bashir, and Yang Shuyuan. “A deep
Siamese convolution neural network for multi-class classification of Alzheimer dis-
ease.” Brain Sciences 10, no. 2 (2020): 84.
[41] Liu, Manhua, Fan Li, Hao Yan, Kundong Wang, Yixin Ma, Li Shen, Mingqing Xu, and
Alzheimer’s Disease Neuroimaging Initiative. “A multi-model deep convolutional neu-
ral network for automatic hippocampus segmentation and classification in Alzheimer’s
disease.” NeuroImage 208 (2020): 116459.
[42] Wada, Akihiko, Kohei Tsuruta, Ryusuke Irie, Koji Kamagata, Tomoko Maekawa,
Shohei Fujita, Saori Koshino et al. “Differentiating Alzheimer’s disease from dementia
with Lewy bodies using a deep learning technique based on structural brain connectiv-
ity.” Magnetic Resonance in Medical Sciences 18, no. 3 (2019): 219.
[43] Liu, Mingxia, Jun Zhang, Ehsan Adeli, and Dinggang Shen. “Joint classification and
regression via deep multi-task multi-channel learning for Alzheimer’s disease diagno-
sis.” IEEE Transactions on Biomedical Engineering 66, no. 5 (2019): 1195–206.
[44] Fulton, Lawrence V., Diane Dolezel, Jordan Harrop, Yan Yan, and Christopher P.
Fulton. “Classification of Alzheimer’s disease with and without imagery using gradient
boosted machines and ResNet-50.” Brain Sciences 9, no. 9 (2019): 212.
[45] Wang, Shui-Hua, Preetha Phillips, Yuxiu Sui, Bin Liu, Ming Yang, and Hong Cheng.
“Classification of Alzheimer’s disease based on eight-layer convolutional neural net-
work with leaky rectified linear unit and max pooling.” Journal of MedicalSsystems
42, no. 5 (2018): 1–11.
200 Machine Learning and Deep Learning in Medical Data Analytics
[46] Qiao, Jianping, Yingru Lv, Chongfeng Cao, Zhishun Wang, and Anning Li. “Multivariate
deep learning classification of Alzheimer’s disease based on hierarchical partner match-
ing independent component analysis.” Frontiers in Aging Neuroscience 10 (2018): 417.
[47] Jo, Taeho, Kwangsik Nho, Shannon L. Risacher, and Andrew J. Saykin. “Deep learn-
ing detection of informative features in tau PET for Alzheimer’s disease classification.”
BMC Bioinformatics 21, no. 21 (2020): 1–13.
[48] Son, Hye Joo, Jungsu S. Oh, Minyoung Oh, Soo Jong Kim, Jae-Hong Lee, Jee Hoon
Roh, and Jae Seung Kim. “The clinical feasibility of deep learning-based classification
of amyloid PET images in visually equivocal cases.” European Journal of Nuclear
Medicine and Molecular Imaging 47, no. 2 (2020): 332–41.
[49] Punjabi, Arjun, Adam Martersteck, Yanran Wang, Todd B. Parrish, Aggelos K.
Katsaggelos, and Alzheimer’s Disease Neuroimaging Initiative. “Neuroimaging
modality fusion in Alzheimer’s classification using convolutional neural networks.”
PloS one 14, no. 12 (2019): e0225759.
[50] Iizuka, Tomomichi, Makoto Fukasawa, and Masashi Kameyama. “Deep-learning-
based imaging-classification identified cingulate island sign in dementia with Lewy
bodies.” Scientific Reports 9 (2019): 8944.
[51] Liu, Manhua, Danni Cheng, Weiwu Yan, and Alzheimer’s Disease Neuroimaging Initiative.
“Classification of Alzheimer’s disease by combination of convolutional and recurrent neu-
ral networks using FDG-PET images.” Frontiers in Neuroinformatics 12 (2018): 35.
[52] Li, Feng, Loc Tran, Kim-Han Thung, Shuiwang Ji, Dinggang Shen, and Jiang Li. “A
robust deep model for improved classification of AD/MCI patients.” IEEE Journal of
Biomedical and Health Informatics 19, no. 5 (2015): 1610–16.
[53] Lazli, Lilia, Mounir Boukadoum, and Otmane Ait Mohamed. “A survey on computer-
aided diagnosis of brain disorders through MRI based on machine learning and data
mining methodologies with an emphasis on Alzheimer disease diagnosis and the con-
tribution of the multimodal fusion.” Applied Sciences 10, no. 5 (2020): 1894.
[54] Herzog, Nitsa J., and George D. Magoulas. “Brain asymmetry detection and machine
learning classification for diagnosis of early Dementia.” Sensors 21, no. 3 (2021): 778.
[55] Liang, Shuang, and Gu Yu. “Computer-aided diagnosis of Alzheimer’s disease through
weak supervision deep learning framework with attention mechanism.” Sensors 21, no. 1
(2021): 220.
[56] Lei, Baiying, Shuangzhi Yu, Xin Zhao, Alejandro F. Frangi, Ee-Leng Tan, Ahmed Elazab,
Tianfu Wang, and Shuqiang Wang. “Diagnosis of early Alzheimer’s disease based on
dynamic high order networks.” Brain Imaging and Behavior 15, no. 1 (2021): 276–87.
[57] Marzban, Eman N., Ayman M. Eldeib, Inas A. Yassine, Yasser M. Kadah, and
Alzheimer’s Disease Neurodegenerative Initiative. “Alzheimer’s disease diagnosis
from diffusion tensor images using convolutional neural networks.” PloS One 15, no. 3
(2020): e0230409.
[58] Lee, Eunho, Jun-Sik Choi, Minjeong Kim, Heung-Il Suk, and Alzheimer’s Disease
Neuroimaging Initiative. “Toward an interpretable Alzheimer’s disease diagnostic
model with regional abnormality representation via deep learning.” NeuroImage 202
(2019): 116113.
[59] Goceri, Evgin. “Diagnosis of Alzheimer’s disease with Sobolev gradient-based opti-
mization and 3D convolutional neural network.” International Journal for Numerical
Methods in Biomedical Engineering 35, no. 7 (2019): e3225.
[60] Bi, Xia-an, Ruipeng Cai, Yang Wang, and Yingchao Liu. “Effective diagnosis of
Alzheimer’s disease via multimodal fusion analysis framework.” Frontiers in Genetics
10 (2019): 976.
[61] Bi, Xiuli, Shutong Li, Bin Xiao, Yu Li, Guoyin Wang, and Xu Ma. “Computer
aided Alzheimer’s disease diagnosis by an unsupervised deep learning technology.”
Neurocomputing 392 (2020): 296–304.
Deep Learning Frameworks for Alzheimer’s Disease 201
[62] Jin, Dan, Bo Zhou, Ying Han, Jiaji Ren, Tong Han, Bing Liu, Jie Lu et al. “Generalizable,
reproducible, and neuroscientifically interpretable imaging biomarkers for Alzheimer’s
disease.” Advanced Science 7, no. 14 (2020): 2000675.
[63] Puente-Castro, Alejandro, Enrique Fernandez-Blanco, Alejandro Pazos, and Cristian
R. Munteanu. “Automatic assessment of Alzheimer’s disease diagnosis based on deep
learning techniques.” Computers in Biology and Medicine 120 (2020): 103764.
[64] Basheera, Shaik, and M. Satya Sai Ram. “Convolution neural network–based
Alzheimer’s disease classification using hybrid enhanced independent component
analysis based segmented gray matter of T2 weighted magnetic resonance imaging
with clinical valuation.” Alzheimer’s & Dementia: Translational Research & Clinical
Interventions 5 (2019): 974–86.
[65] Cui, Ruoxuan, and Manhua Liu. “Hippocampus Analysis by Combination of 3-D
DenseNet and Shapes for Alzheimer’s Disease Diagnosis.” IEEE Journal of Biomedical
and Health Informatics 23, no. 5 (2018): 2099–107.
[66] Ju, Ronghui, Chenhui Hu, and Quanzheng Li. “Early diagnosis of Alzheimer’s disease
based on resting-state brain networks and deep learning.” IEEE/ACM Transactions on
Computational Biology and Bioinformatics 16, no. 1 (2017): 244–57.
[67] Esmaeilzadeh, Soheil, Dimitrios Ioannis Belivanis, Kilian M. Pohl, and Ehsan Adeli.
“End-to-end Alzheimer’s disease diagnosis and biomarker identification.” In International
Workshop on ML in Medical Imaging, pp. 337–345. Springer, Cham, 2018.
[68] Kam, Tae-Eui, Han Zhang, and Dinggang Shen. “A novel deep learning framework
on brain functional networks for early MCI diagnosis.” In International Conference
on Medical Image Computing and Computer-Assisted Intervention, pp. 293–301.
Springer, Cham, 2018.
[69] Li, Fan, Manhua Liu, and Alzheimer’s Disease Neuroimaging Initiative. “Alzheimer’s
disease diagnosis based on multiple cluster dense convolutional networks.”
Computerized Medical Imaging and Graphics 70 (2018): 101–110.
[70] Li, Yupeng, Jiehui Jiang, Jiaying Lu, Juanjuan Jiang, Huiwei Zhang, and Chuantao Zuo.
“Radiomics: a novel feature extraction method for brain neuron degeneration disease
using 18F-FDG PET imaging and its implementation for Alzheimer’s disease and mild
cognitive impairment.” Therapeutic Advances in Neurological Disorders 12 (2019):
1756286419838682.
[71] Ding, Yiming, Jae Ho Sohn, Michael G. Kawczynski, Hari Trivedi, Roy Harnish,
Nathaniel W. Jenkins, Dmytro Lituiev et al. “A deep learning model to predict a diag-
nosis of Alzheimer disease by using 18F-FDG PET of the brain.” Radiology 290, no. 2
(2019): 456–64.
[72] Zhang, Tao, and Mingyang Shi. “Multi-modal neuroimaging feature fusion for diagno-
sis of Alzheimer’s disease.” Journal of Neuroscience Methods 341 (2020): 108795.
[73] Zhou, Tao, Kim-Han Thung, Xiaofeng Zhu, and Dinggang Shen. “Effective feature
learning and fusion of multimodality data using stage wise deep neural network for
dementia diagnosis.” Human Brain Mapping 40, no. 3 (2019): 1001–16.
[74] Zhou, Tao, Kim-Han Thung, Xiaofeng Zhu, and Dinggang Shen. “Feature learning
and fusion of multimodality neuroimaging and genetic data for multi-status dementia
diagnosis.” In International Workshop on Machine Learning in Medical Imaging, pp.
132–40. Springer, Cham, 2017.
[75] Li, Yan, Fanqing Meng, and Jun Shi. “Learning using privileged information improves
neuroimaging-based CAD of Alzheimer’s disease: a comparative study.” Medical &
Biological Engineering & Computing 57, no. 7 (2019): 1605–16.
[76] Shi, Jun, Xiao Zheng, Yan Li, Qi Zhang, and Shihui Ying. “Multimodal neuroimaging
feature learning with multimodal stacked deep polynomial networks for diagnosis of
Alzheimer’s disease.” IEEE Journal of Biomedical and Health Informatics 22, no. 1
(2018): 173–83.
202 Machine Learning and Deep Learning in Medical Data Analytics
[77] Ortiz, Andres, Jorge Munilla, Juan M. Gorriz, and Javier Ramirez. “Ensembles of deep
learning architectures for the early diagnosis of the Alzheimer’s disease.” International
Journal of Neural Systems 26, no. 07 (2016): 1650025.
[78] Huang, Yechong, Jiahang Xu, Yuncheng Zhou, Tong Tong, Xiahai Zhuang, and Alzheimer’s
Disease Neuroimaging Initiative. “Diagnosis of Alzheimer’s disease via multi-modality 3D
convolutional neural network.” Frontiers in Neuroscience 13 (2019): 509.
[79] Lu, Donghuan, Karteek Popuri, Gavin Weiguang Ding, Rakesh Balachandar, and
Mirza Faisal Beg. “Multimodal and multiscale deep neural networks for the early diag-
nosis of Alzheimer’s disease using structural MR and FDG-PET images.” Scientific
Reports 8, no. 1 (2018): 1–13.
[80] Thung, Kim-Han, Pew-Thian Yap, and Dinggang Shen. “Multi-stage diagnosis of
Alzheimer’s disease with incomplete multimodal data via multi-task deep learning.”
In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical
Decision Support, pp. 160–68. Springer, Cham, 2017.
[81] Suk, Heung-Il, Seong-Whan Lee, and Dinggang Shen. “Deep sparse multi-task learning
for feature selection in Alzheimer’s disease diagnosis.” Brain Structure and Function
221, no. 5 (2016): 2569–87.
[82] Suk, Heung-Il, Seong-Whan Lee, Dinggang Shen, and Alzheimer’s Disease
Neuroimaging Initiative. “Hierarchical feature representation and multimodal fusion
with deep learning for AD/MCI diagnosis.” NeuroImage 101 (2014): 569–82.
[83] Ryzhikova, Elena, Nicole M. Ralbovsky, Vitali Sikirzhytski, Oleksandr Kazakov,
Lenka Halamkova, Joseph Quinn, Earl A. Zimmerman, and Igor K. Lednev. “Raman
spectroscopy and machine learning for biomedical applications: Alzheimer’s disease
diagnosis based on the analysis of cerebrospinal fluid.” Spectrochimica Acta Part A:
Molecular and Biomolecular Spectroscopy 248 (2021): 119188.
[84] Stamate, Daniel, Min Kim, Petroula Proitsi, Sarah Westwood, Alison Baird, Alejo
Nevado Holgado, Abdul Hye et al. “A metabolite-based machine learning approach
to diagnose Alzheimer-type dementia in blood: Results from the European Medical
Information Framework for Alzheimer disease biomarker discovery cohort.” Alzheimer’s
& Dementia: Translational Research & Clinical Interventions 5, no. 1 (2019): 933–38.
[85] Costa, Luís, Miguel F. Gago, Darya Yelshyna, Jaime Ferreira, Hélder David Silva,
Luís Rocha, Nuno Sousa, and Estela Bicho. “Application of machine learning in pos-
tural control kinematics for the diagnosis of Alzheimer’s disease.” Computational
Intelligence and Neuroscience 2016 (2016), 1186–200.
[86] Pattnayak, Parthasarathi, and Om Prakash Jena. “Innovation on machine learn-
ing in healthcare services–an introduction.” 2021 Machine Learning for Healthcare
Applications 1 (2021): 1–15.
[87] Goyal, Sukriti, Nikhil Sharma, Bharat Bhushan, Achyut Shankar, and Martin Sagayam.
“IoT enabled technology in secured healthcare: applications, challenges and future
directions.” 2021 Cognitive Internet of Medical Things for Smart Healthcare. Studies
in Systems, Decision and Control 311 (2021): 25–48.
[88] Panigrahi, Niranjan, Ishan Ayus, and Om Prakash Jena. “An expert system-based clini-
cal decision support system for hepatitis-B prediction & diagnosis.” 2021 Machine
Learning for Healthcare Applications 4 (2021): 57–75.
[89] Patra, Sudhanshu Shekhar, Om Prakash Jena, Gaurav Kumar, Sreyashi Pramanik,
Chinmaya Misra, and Kamakhya Narain Singh. “Random forest algorithm in imbal-
ance genomics classification.” 2021 Data Analytics in Bioinformatics 7 (2021): 173–90.
[90] Paramesha, K., H.L. Gururaj, and Om Prakash Jena. “Applications of machine learn-
ing in biomedical text processing and food industry.” 2021 Machine Learning for
Healthcare Applications 10 (2021): 151–67.
11 Machine Learning
Algorithms and
COVID-19
A Step for Predicting
Future Pandemics with
a Systematic Overview
Madhumita Pal, Ruchi Tiwari, Kuldeep
Dhama, Smita Parija, Om Prakash Jena,
and Ranjan K. Mohapatra
CONTENTS
11.1 Introduction.................................................................................................204
11.2 Different ML Techniques............................................................................205
11.2.1 Supervised Learning......................................................................205
11.2.1.1 Artificial Neural Networks...........................................205
11.2.1.2 Logistic Regression.......................................................207
11.2.1.3 K-Nearest Neighbor......................................................207
11.2.1.4 Decision Tree................................................................207
11.2.1.5 Naïve Bayes...................................................................207
11.2.1.6 Support Vector Machine...............................................208
11.2.1.7 Random Forest..............................................................208
11.2.2 Unsupervised Learning..................................................................208
11.2.2.1 K-Mean Clustering........................................................208
11.2.2.2 C-Mean Clustering........................................................209
11.2.2.3 Hierarchical Clustering.................................................209
11.2.2.4 Gaussian Mixture Algorithm........................................209
11.2.3 Reinforcement Learning................................................................209
11.2.4 Deep Learning Techniques............................................................ 210
11.2.4.1 Auto Encoder................................................................ 210
11.2.4.2 Sparse Auto Encoders................................................... 210
11.2.4.3 Stacked Sparse Auto Encoder....................................... 210
11.2.5 Convolutional Neural Network...................................................... 210
11.2.5.1 Recurrent Neural Network............................................ 210
DOI: 10.1201/9781003226147-11 203
204 Machine Learning and Deep Learning in Medical Data Analytics
11.1 INTRODUCTION
After the birth of SARS-CoV-2 from the Huanan Seafood wholesale market of
Wuhan in China in December 2019, the deadly virus has spread very quickly around
the whole world and become the major cause of serious health concern worldwide
(Mohapatra et al. 2020a, 2020b; Mohapatra and Rahman 2021). This deadly pathogen
is responsible for the ongoing and never-ending COVID-19 pandemic and on March
11, 2020, the World Health Organization (WHO) has declared a global pandemic due
to this (Dhama et al. 2020a). This pathogenic viral infection is believed to have origi-
nated from animals such as pangolins and bats, however, bats are suspected to be the
major source (Mohapatra et al. 2021b). The widespread propagation of this disease has
caused nearly three million deaths worldwide to date due to rapid human-to-human
transmission. It is also responsible for more than 20.5 million years of life loss globally
and has radically changed the common practice of our normal life (Arolas et al. 2021).
SARS-CoV-2 is mainly transmitted through the respiratory aerosols/droplets
and fecal-oral route (Chan et al. 2020). Although several other means of transmis-
sion have been identified, human-to-human transmission mainly occurs by direct
and indirect contacts (i.e. contaminated objects/surfaces/fomite) (Mohapatra et al.
2021a). Moreover, the airborne transmission and hospital-associated transmissions
were also reported as a predominant mode of virus spread (Morawska and Cao 2020;
Wang et al. 2020a; Zhang et al. 2020a). The most common COVID-19 symptoms are
respiratory infections such as cough, shortness of breath, fever, followed by pneu-
monia. Apart from this, SARS-CoV-2 also affects kidneys, heart, nervous system,
and finally progresses to multiple organ damage (Dhama et al. 2020b). It may cause
severe complications among immunocompromised persons having diabetes and car-
diovascular disorders (Arumugam et al. 2020).
Moreover, this pandemic has devastated the stock and financial markets, and the
global economy dramatically (Lenzen et al. 2020; Nicola et al. 2020). The efficient rate
of human-to-human transmission makes it challenging to prevent community trans-
mission and to formulate the evidence-based proper infection control strategies to save
health workers, children, and old-aged individuals (Kucharski et al. 2020). Moreover,
no approved drugs are available to combat SARS-CoV-2 and hence, scientists, doctors,
and researchers are trying days and nights to find a solution to combat SARS-CoV-2 and
its emerging variants (Mohapatra et al. 2020c, 2021c; Sah et al. 2021). Hence, the accu-
rate prediction of the disease will help in providing high-quality healthcare services and
may reduce the disease severity and mortality. Timely actions are needed for the accu-
rate prediction of the disease to provide high-quality healthcare service management.
Machine Learning Algorithms and COVID-19 205
Artificial neural networks (ANNs), Internet of Things (IoT), and ML techniques may
provide valuable suggestions in numerous fields, such as agriculture, environmental
science, food industry (Paramesha et al. 2021), and classification of diabetes (Kumar
et al. 2020), chronic diseases (Reddy and Imler 2017), skin cancer (Khamparia et al.
2020), epidemiology, public health, and smart healthcare systems (Uddin et al. 2019;
Hassanien et al. 2021; Panigrahi et al. 2021; Patra et al. 2021; Pattnayak and Jena 2021).
Furthermore, some researchers have explained the challenges related to the use of vari-
ous ML techniques in order to extract the results with good efficiency (Jindal et al. 2019;
Rana et al. 2019). Kumar et al. (2020) have investigated deep learning (DL) models for
the classification of diabetes and the results were compared with Naïve Bayes (NB)
and Random Forest (RF) algorithms. In this overview, we have discussed different ML
techniques for the automotive detection, prediction, and diagnosis of COVID-19 out-
break which may help to increase the survival rate of patients.
This chapter is organized into five sections as follows: The introductory section
describes the current understanding of the disease and outlines how ANN, IoT, and
different ML techniques are helpful to provide valuable suggestions in numerous
fields. Section 11.2 explains the theoretical concept of several ML techniques with
the performance measurement parameters. The detailed methodology of the study
is illustrated in Section 11.3. Further, Section 11.4 discusses how ML techniques are
currently used for the prediction and accurate diagnosis of COVID-19 for the better
healthcare services. Finally, Section 11.5 summarizes and offers a conclusion with
suggestions for further improvements in this direction.
• Supervised learning
• Unsupervised learning
• Semi supervised learning
• DL
layer, and output layer (Uzun and Tezel 2012; Singhal and Pareek 2018). Several
types of activation functions are employed in hidden layer for nonlinear mapping.
There are two perceptron types of ANNs. One is simple ANN used for binary
classification; another one is multilayer perceptron used for complex regression
Machine Learning Algorithms and COVID-19 207
Output = ai + ∑w y (11.1)
k =1
ik i
Wij = weight propagated from i/p to o/p. aj bias value, bi input value.
The activation function which are commonly used in the ANN are:
1
sigmoid = (11.2)
1 + e− x
e x− e − x
Tanh = (11.3)
e x + e− x
RELU = {0,for y ≤ 0 y, for y > 0 (11.4)
ex
Y (x) = (11.5)
1 + ex
X is the input variable.
as y= {C1, C2, C3, …………. Cz}. Suppose there are k classes represented as D1, D2, D3.
For the tuple y, the classifier forecast class belongingness of y with highest posterior
probability conditioned on y. This classifier predicts that y belongs to Dj class if,
( )
p D j y > p ( D k y ) for 1 ≤ k ≤ n, j ≠ k. (11.7)
( ) ( )( )
p D j y = p y D j p D j /p ( y ) (11.8)
( ) ( )
P y Di = ∏ p y D j (11.9)
11.2.2 Unsupervised Learning
It operates on unlabeled dataset. It is mainly used for clustering operation and for
pattern recognition of newly dataset. For training the model it neither requires clas-
sified data nor labeled data. K-mean, C-mean, Hierarchical clustering algorithm
comes under unsupervised learning.
11.2.2.1.1 Advantages
• It is simple, easy to implement and to understand.
• It is efficient where the time taken to cluster k-means rises linearly with the
number of data points.
• No other clustering algorithms perform better than k-means.
11.2.2.1.2 Disadvantages
• The initial value of k needs to be specified.
• The process of clusters finding may not converge.
• It may not be applicable for discovering clusters which are not hyper spheres
or hyper ellipsoids.
11.2.3 Reinforcement Learning
In this type of learning, the machine learns from the environment. It doesn’t require
any labeled data. In the absence of training dataset, the machine improves its per-
formance by learning from experience. Reinforcement learning is a type where
an agent learns to behave in an environment by performing actions and seeing the
results. There is no expected output as in supervised learning. It doesn’t require any
labeled data. Reinforcement learning system comprises two main components: agent
and environment. The agent decides what action is to be taken to perform a task.
Reinforcement learning is all about an agent who is put about an unknown environ-
ment, and he is going to take a hit and trial method in order to figure out the environ-
ment and then come up with an outcome. Q-Learning algorithm is an example of
reinforcement learning.
210 Machine Learning and Deep Learning in Medical Data Analytics
11.3 METHODOLOGY
A systematic literature review was performed to collect data from different data-
bases such as Science Direct, IEEE, PubMed, Scopus, and Google Scholar resources.
For critical covering the most relevant literature contents, the key terms searched
included ML tools in COVID-19, ML models predicting COVID-19, COVID-19
diagnosis model by using ML techniques, predicting COVID-19 fatality rates, pre-
dicting COVID-19 casualties, and diagnosis of COVID-19 from CT images with
ML models. The closely matched data were carefully examined and considered for
Machine Learning Algorithms and COVID-19 211
11.4.2 Prediction of Outcomes
Jimenez-Solem et al. (2021) have predicted the risk of death as 0.906 at diagnosis,
0.721 at ICU admission, and 0.818 at hospital admission by ML models. The United
Kingdom Biobank SARS-CoV-2 positive cases dataset was used for external valida-
tion. In this study, common risk factors are body mass index, age, and hyperten-
sion, with top risk features in ICU patients (shock, organ dysfunction). The authors
suggested that the ML models may be used for accurate prediction of outcomes in
COVID-19 (disease progression and death) at different stages of management.
212 Machine Learning and Deep Learning in Medical Data Analytics
data) of DL in COVID-19 with the availability of big data. They have discussed
several aspects such as public sentiment analysis, medical image analysis, ambient
intelligence, protein structure prediction, precision diagnostics, and also for drug
repurposing. DL technique is also used in forecasting for epidemiology. Hence, the
study will be helpful to adopt DL technique for COVID-19 and/or related research.
As we all know, early stage detection/diagnosis is essential to control/prevent the
spreading of COVID-19. With this in mind, Silva et al. (2020) have proposed a novel
efficient DL technique to screen COVID-19 patients with voting-based approach.
The model was analyzed with a cross-dataset study to evaluate robustness. However,
the accuracy drops (87.68% → 56.16%) on the best evaluation scenario and suggest
improving the model significantly for consideration clinically. Moreover, Shibly et al.
(2020) have suggested that the use of DNN techniques coupled with radiological imag-
ing may be useful to identify COVID-19 disease accurately. They have introduced a
VGG-16 network-based faster R-CNN framework to detect/screen the patients from
chest x-ray images with accuracy of 97.36%, sensitivity 97.65%, and precision 99.28%.
So, the proposed model may be helpful to assess COVID-19 initially. The workflow
representation with working procedure of the proposed model is shown in Figure 11.4.
11.5 CONCLUSION
Infectious diseases, in particular, are continuously causing seasonal epidemics and
pandemics. Hence, the accurate prediction of such diseases is attracting continued
interest due to global importance. The uncertainty regarding the growth rate of the
recent COVID-19 infection makes it difficult for the healthcare system to adapt
according to the increasing requirements. Also, health professionals, certain ages,
gender, and races are mostly affected. Some studies have also been reported for
the modeling of the recent COVID-19 pandemic in several countries using AI and
ML algorithms. The detection and diagnosis of this disease in the early stage is a
challenging task. The accurate prediction of the disease will help in providing high-
quality healthcare services and may reduce the disease severity and mortality. In this
overview, we have discussed different ML techniques for the automotive detection,
Machine Learning Algorithms and COVID-19 215
prediction, and diagnosis of COVID-19 which may help to increase the patient sur-
vival rate. Therefore, we hope the study may provide useful information for monitor-
ing such a pandemic as the COVID-19 outbreak in the future.
ACKNOWLEDGMENT
All authors acknowledge their respective institute and university for providing the
necessary facilities and support.
REFERENCES
Ahmad, F.K., and N. Yusoff. 2013. Classifying breast cancer types based on fine needle aspira-
tion biopsy data using random forest classifier, in Proceedings of the 13th International
Conference on Intelligent Systems Design and Applications. 121–25.
Ardabili, S.F., A. Mosavi, P. Ghamisi, et al. 2020. COVID-19 outbreak prediction with
machine learning. Algorithms. 13:249. https://doi.org/10.3390/a13100249
Arolas, H.P., E. Acosta, G. López-Casasnovas, et al. 2021. Years of life lost to COVID-19 in
81 countries. Scientific Reports. 11:3504.
Arumugam, V.A., S. Thangavelu, Z. Fathah, et al. 2020. COVID-19 and the world with
co-morbidities of heart disease, hypertension and diabetes. Journal of Pure and Applied
Microbiology. 14(3):1623–38. https://doi.org/10.22207/JPAM.14.3.01
Bezdek, J.C., R. Ehrlich, and W. Full. 1984. FCM: The fuzzy c-means clustering algorithm.
Computers & Geosciences. 10(2–3):191–203.
Breiman, L. 2001. Random forests. Machine Learning. 45(1):5–32.
Chan, J.F., S. Yuan, K.H. Kok, et al. 2020. A familial cluster of pneumonia associated with
the 2019 novel Coronavirus indicating person-to-person transmission: a study of a fam-
ily cluster. Lancet. https://doi.org/S0140-6736(20)30154-9
Dhama, K., S. Khan, R. Tiwari, et al. 2020a. Coronavirus disease 2019-COVID-19. Clinical
Microbiology Reviews. 33(4):e00028–20.
Dhama, K., S.K. Patel, M. Pathak, et al. 2020b. An update on SARS-CoV-2/COVID-19 with
particular reference to its clinical pathology, pathogenesis, immunopathology and miti-
gation strategies. Travel Medicine and Infectious Disease. 37:101755.
Evgeniou, T., and M. Pontil. 2005. Support vector machines: Theory and applications, in
Advanced Course on Artificial Intelligence. Berlin, Germany: Springer, 249–257.
Fakoor, R., F. Ladhak, A. Nazi, et al. 2013. Using deep learning to enhance cancer diag-
nosis and classification, in Proceedings of the International Conference on Machine
Learning., New York, NY, 28, 1–7.
Fatima, N., L. Liu, S. Hong, et al. 2020.Prediction of breast cancer, comparative review of
machine learning techniques, and their analysis. Journals & Magazines, IEEE Access.
8:150360–76.
Ghisolfi, S., I. Almås, J.C. Sandefur, et al. 2020. Predicted COVID-19 fatality rates based on
age, sex, comorbidities and health system capacity. BMJ Global Health. 5:e003094.
https://doi.org/10.1136/bmjgh-2020-003094
Hamed, G., M.A.E.-R. Marey, S.E.-S. Amin, et al. 2020. Deep learning in breast cancer detec-
tion and classification, in Proceeding of Joint Euro-US Workshop Appl. Invariance
Computing. Vis. Cham, Switzerland: Springer, 322–33.
Haque, K.F., and A. Abdelgawad. 2020. A deep learning approach to detect COVID-19
patients from chest x-ray images. AI. 1:418–35. https://doi.org/10.3390/ai1030027
Hasan, N. 2020. A methodological approach for predicting COVID-19 epidemic using
EEMD-ANN hybrid model. Internet of Things. 11:100228.
216 Machine Learning and Deep Learning in Medical Data Analytics
Hassanien, A.E., A. Khamparia, D. Gupta, et al. 2021. Cognitive Internet of Medical Things
for Smart Healthcare. https://doi.org/10.1007/978-3-030-55833-8
Ibrahim, A.A., A.I. Hashad, and N.E.M. Shawky. 2017. A comparison of open source data
mining tools for breast cancer classification, in Handbook of Research on Machine
Learning Innovations and Trends. Hershey, PA: IGI Global, 636–51.
Imandoust, S.B., and M. Bolandraftar, 2013. Application of k-nearest neighbor (KNN)
approach for predicting economic events: Theoretical background. International
Journal of Engineering Research and Applications. 3:605–10.
Iwendi, C., A.K. Bashir, A. Peshkar, et al. 2020. COVID-19 patient health prediction using
boosted Random Forest algorithm. Frontiers in Public Health. 8:357. https://doi.
org/10.3389/fpubh.2020.00357
Jain, N., S. Jhunthra, H. Garg, et al. 2021. Prediction modelling of COVID using machine
learning methods from B-cell dataset. Results in Physics. 21:103813.
Jimenez-Solem, E., T.S. Petersen, C. Hansen, et al. 2021. Developing and validating COVID-
19 adverse outcome risk prediction models from a bi-national European cohort of 5594
patients. Scientific Reports. 11:3246. https://doi.org/10.1038/s41598-021-81844-x
Jindal, M., J. Gupta, and B. Bhushan, 2019. Machine learning methods for IoT and
their Future Applications, 2019 International Conference on Computing,
Communication, and Intelligent Systems (ICCCIS), in IEEE Xplore. 430–434.
ISBN: 978-1-7281-4826-7.
Khakharia, A., V. Shah, S. Jain, et al. 2020. Outbreak prediction of COVID‑19 for dense
and populated countries using machine learning. Annals of Data Science. https://doi.
org/10.1007/s40745-020-00314-9
Khamparia, A., P.K. Singh, P. Rani, et al. 2020. An internet of health things-driven deep
learning framework for detection and classification of skin cancer using transfer learn-
ing. Transactions on Emerging Telecommunications Technologies. e3963. https://doi.
org/10.1002/ett.3963
Kucharski, A.J., T.W. Russell, C. Diamond, et al. 2020. Early dynamics of transmission and
control of COVID-19: a mathematical modeling study. Lancet Infectious Diseases.
https://doi.org/10.1016/S1473-3099(20)30144-4
Kumar, S., B. Bhusan, D. Singh, et al. 2020. Classification of Diabetes using Deep Learning,
International Conference on Communication and Signal Processing, in IEEE Xplore.
651–55.
LeCun, Y., Y. Bengio, and G. Hinton. 2015. Deep learning, Nature, 521(7553):436–44.
Lenzen, M., M. Li, A. Malik, et al. 2020.Global socio-economic losses and environmental
gains from the Coronavirus pandemic. PLoS One. 15(7):e0235654.
Li, W.T., J. Ma, N. Shende, et al. 2020.Using machine learning of clinical data to diagnose
COVID-19: a systematic review and meta-analysis. BMC Medical Informatics and
Decision Making. 20:247. https://doi.org/10.1186/s12911-020-01266-z
Li, Y., and H. Wu, 2012. A clustering method based on K-means algorithm. Physics Procedia.
25:1104–109.
Mohapatra, R.K., P.K. Das, and V. Kandi. 2020a. Challenges in controlling COVID-19 in
migrants in Odisha, India. Diabetes & Metabolic Syndrome: Clinical Research &
Reviews. 14:1593–94.
Mohapatra, R.K., L. Pintilie, V. Kandi, et al. 2020b. The recent challenges of highly conta-
gious COVID-19; causing respiratory infections: symptoms, diagnosis, transmission,
possible vaccines, animal models and immunotherapy. Chemical Biology & Drug
Design. 96:1187–208. https://doi.org/10.1111/cbdd.13761
Mohapatra, R.K., V.P. Saikishore, M. Azam, et al. 2020c. Synthesis and physicochemical
studies of a series of mixed ligand transition metal complexes and their molecular dock-
ing investigations against Coronavirus main protease. Open Chemistry. 18:1495–506.
Machine Learning Algorithms and COVID-19 217
Mohapatra, R.K., P.K. Das, L. Pintilie, et al. 2021a. Infection capability of SARS-CoV-2 on
different surfaces. Egyptian Journal of Basic and Applied Science. 8(1):75–80.
Mohapatra, R.K., S. Mishra, M. Azam, et al. 2021b. COVID-19, WHO guidelines, pedagogy,
and respite. Open Medicine. 16:491–93.
Mohapatra, R.K., L. Perekhoda, M. Azam, et al. 2021c. Computational investigations of three
main drugs and their comparison with synthesized compounds as potent inhibitors of
SARS-CoV-2 main protease (Mpro): DFT, QSAR, molecular docking, and in silico
toxicity analysis. Journal of King Saud University – Science. 33:101315.
Mohapatra, R.K., and M. Rahman. 2021. Is it possible to control the outbreak of COVID-19
in Dharavi, Asia’s largest slum situated in Mumbai? Anti-Infective Agents. 19(4): 1–2.
https://doi.org/10.2174/2211352518999200831142851
Morawska, L., and J. Cao. 2020. Airborne transmission of SARS-CoV-2: The world should
face the reality. Environment International. 139:105730.
Muhammad, L.J., E.A. Algehyne, S.S. Usman, et al. 2021. Supervised machine learning
models for prediction of COVID‑19 infection using epidemiology dataset.SN Computer
Science. 2:11. https://doi.org/10.1007/s42979-020-00394-7
Munir, K., H. Elahi, A. Ayub, et al. 2019. Cancer diagnosis using deep learning: A biblio-
graphic review. Cancers. 11(9):1235.
Nicola, M., Z. Alsafi, C. Sohrabi, et al. 2020.The socio-economic implications of the Coronavirus
pandemic (COVID-19): A review. International Journal of Surgery. 78:185–93.
Panigrahi, N., I. Ayus, and O.P. Jena, 2021. An expert system-based clinical decision
support system for hepatitis-b prediction & diagnosis, Chapter 4. https://doi.org/
10.1002/9781119792611.ch4
Paramesha, K., H.L. Gururaj, and O.P. Jena, 2021. Applications of machine learn-
ing in biomedical text processing and food industry, Chapter 10. https://doi.org/
10.1002/9781119792611.ch10
Patel, S., S. Sihmar, and A. Jatain. 2015. A study of hierarchical clustering algorithms, in
Proceeding of Second International. Conference on Computing for Sustainable Global
Development (INDIA.com). 537–41.
Patra, S.S., O.P. Jena, G. Kumar, et al. 2021. Random Forest algorithm in imbalance genomics
classification, Chapter 7. https://doi.org/10.1002/9781119785620.ch7
Pattnayak, P., and O.P. Jena, 2021. Innovation on machine learning in healthcare services–an
introduction, Chapter 1, https://doi.org/10.1002/9781119792611.ch1
Rana, A.K., A.O. Salau, S. Gupta, et al. 2019. Machine learning methods for IoT and
their Future Applications, in 2019 International Conference on Computing,
Communication, and Intelligent Systems (ICCCIS), IEEE Xplore. https://doi.
org/10.1109/icccis48478.2019.8974551
Reddy, R., and T.D. Imler. 2017. Artificial neural networks are highly predictive for hepato-
cellular carcinoma in patients with cirrhosis. Gastroenterology. 152:S1193.
Roberts, M., D. Driggs, M. Thorpe, et al. 2021. Common pitfalls and recommendations for
using machine learning to detect and prognosticate for COVID-19 using chest radio-
graphs and CT scans. Nature Machine Intelligence. 3:199–217. https://doi.org/10.1038/
s42256-021-00307-0
Sah, R., A.P. Khatiwada, S. Shrestha, et al. 2021. The COVID-19 vaccination campaign in
Nepal, emerging UK variant and futuristic vaccination strategies to combat the ongoing
pandemic, Travel Medicine and Infectious Disease. 41:102037.
Selvathi, D., and A. A. Poornila. 2018. Deep learning techniques for breast cancer detection
using medical image analysis, in Biologically Rationalized Computing Techniques for
Image Processing Applications. Cham, Switzerland: Springer, 159–86.
Sharma, H., and S. Kumar, 2016. A survey on decision tree algorithms of classification in data
mining. International Journal of Science and Research,. 5(4):2094–97.
218 Machine Learning and Deep Learning in Medical Data Analytics
Shibly, K.H., S.K. Dey, T.-U. Islam, et al. 2020. COVID faster R–CNN: A novel framework
to diagnose novel Coronavirus disease (COVID-19) in X-ray images. Informatics in
Medicine Unlocked. 20:100405.
Shorten, C., T.M. Khoshgoftaar, and B. Furht. 2021. Deep learning applications for
COVID-19. Journal of Big Data. 8:18. https://doi.org/10.1186/s40537-020-00392-9
Silva, P., E. Luz, G. Silva, et al. 2020. COVID-19 detection in CT images with deep learning:
A voting-based scheme and cross-datasets analysis. Informatics in Medicine Unlocked.
20:100427.
Singhal, P., and S. Pareek, 2018. Artificial neural network for prediction of breast cancer, in
Proc. 2nd Int. Conf. I-SMAC (IoT Social, Mobile, Anal. Cloud) (I-SMAC). 464–68.
Tiwari, M., R. Bharuka, P. Shah, et al. 2020. Breast cancer prediction using deep learning
and machine learning techniques.
Togacar, M., and B. Ergen. 2018. Deep learning approach for classification of breast cancer,
in Proc. Int. Conf. Artif. Intell. Data Process. (IDAP). 1–5.
Tutsoy, O., U. Çolak, A. Polat, et al. 2020. A novel parametric model for the predic-
tion and analysis of the COVID-19 casualties. IEEE Access. https://doi.org/10.1109/
ACCESS.2020.3033146
Uddin, S., A. Khan, E. Hossain, et al. 2019. Comparing different supervised machine learn-
ing algorithms for disease prediction. BMC Medical Informatics and Decision Making.
19:281. https://doi.org/10.1186/s12911-019-1004-8
Uzun, Y., and G. Tezel, 2012. Rule learning with machine learning algorithms and artificial
neural networks. Journal of Selcuk University Natural and Applied Science 1(2):1–11.
Wang, D., B. Hu, C. Hu, et al. 2020a. Clinical characteristics of 138 hospitalized patients
with 2019 novel Coronavirus-infected pneumonia in Wuhan, China. JAMA. https://doi.
org/10.1001/jama.2020.1585
Wang, J., H. Yu, Q. Hua, et al. 2020b. A descriptive study of random forest algorithm for pre-
dicting COVID-19 patients outcome. Peer J. 8:e9945. https://doi.org/10.7717/peerj.9945
Wu, W., S. Nagarajan, and Z. Chen, 2015. Bayesian machine learning: EEGMEG signal pro-
cessing measurements. IEEE Signal Process. Mag. 33(1):14–36.
Xu, X., X. Jiang, C. Ma, et al. 2020. A deep learning system to screen novel Coronavirus
disease 2019 pneumonia. Engineering. 6:1122–29.
Zhang, J., X. Hong, S.-U. Guan, et al. 2016. Maximum Gaussian mixture model for classifi-
cation, in 8th International Conference on Information Technology in Medicine and
Education, (ITME). 587–91.
Zhang, R., Y. Li, A. L. Zhang, et al. 2020a. Identifying airborne transmission as the dominant
route for the spread of COVID-19. Proceedings of the National Academy of Sciences
USA. 117(26):14857–63.
Zhang, X., H. Saleh, E.M.G. Younis, et al. 2020b. Predicting Coronavirus pandemic in real-
time using machine learning and big data streaming system. Complexity. 2020, Article
ID 6688912, https://doi.org/10.1155/2020/6688912
12 Transferred Learning-based
TRNetCoV
CONTENTS
12.1 Introduction............................................................................................... 220
12.2 Role of Chest X-Ray Images in Covid-19 Detection................................. 220
12.3 Motivation.................................................................................................. 221
12.4 Challenges in Chest X-Ray Modality........................................................ 221
12.5 Contributions............................................................................................. 221
12.6 Literature Review...................................................................................... 222
12.7 Deep Learning Models.............................................................................. 222
12.7.1 Artificial Neural Networks........................................................ 222
12.7.2 Deep Convolution Neural Networks.......................................... 223
12.7.2.1 Feature Extraction.................................................... 223
12.7.2.2 Classification............................................................224
12.7.3 Related Work Against COVID-19 Detection.............................224
12.8 Methodologies........................................................................................... 225
12.8.1 Data Preprocessing Technique.................................................. 225
12.8.2 Data Augmentation Technique.................................................. 226
12.8.3 Proposed Deep Transfer-Based Learning Model...................... 226
12.8.3.1 Basic Blocks............................................................. 227
12.9 Experimental Setup and Result Analysis.................................................. 228
12.9.1 Collection of Dataset and Resources......................................... 228
12.9.2 Experiment and Results............................................................. 228
12.10 Performance Analysis................................................................................ 229
12.10.1 Metrics Used for Performance Measurement............................ 229
12.10.2 Comparison of the Proposed Technique
with Literature Methods............................................................ 231
12.11 Conclusion and Future Work..................................................................... 231
References�������������������������������������������������������������������������������������������������������������� 232
DOI: 10.1201/9781003226147-12 219
220 Machine Learning and Deep Learning in Medical Data Analytics
12.1 INTRODUCTION
The new coronavirus disease pandemic, named COVID-19, was identified in Wuhan,
China in the December of 2019. The total number of COVID-19 confirmed cases and
deaths has been rapidly increasing globally. Following these serious consequences,
the World Health Organization (WHO) declared the COVID-19 as a pandemic on
March 11, 2020. Since the COVID-19 disease shows a major impact on the respira-
tory system, chest X-rays (CXRs) and CT scans can help in identifying them easily.
Thus, a fast and immediate diagnosis can be done using medical imaging such as
CXRs and CT scans, which are used widely. However, CXR image processing has
already been proven to be an essential imaging modality while identifying most
cases of other respiratory issues. Hence the researchers are widely using this modal-
ity for COVID-19 estimation. There are many deep convolutional neural network
(DCNN) models proposed during the COVID-19 pandemic. The main scope of deep
learning (DL) is widely used in various aspects in terms of medical image inspec-
tion, new drug discovery [1], disease detection [2] and diagnosis, and other case
study problems of hepatitis virus [3], and COVID-19 prediction.
FIGURE 12.1 Overview of the proposed workflow of modern X-ray system for classifica-
tion of COVID-19.
TL-based ResNet Model for COVID-19 detection 221
12.3 MOTIVATION
COVID-19 is a kind of Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-
CoV-2), according to Wang et al. [5], and it could lead to acute respiratory distress
syndrome and other additional symptoms (i.e. fever, dry cough, body pains, nasal
congestion, sore throat, organ failures, etc.). On the other hand, due to the rapid
spread of COVID-19 and its long incubation periods and lesser healthcare systems,
many countries were unable to manage the infected patients during detection and
diagnosis of COVID-19. Ai et al. [6], along with many scientific research institutions
and most of the world’s governments, have declared that a necessary standard detec-
tion system in real-time could be done using the test named: Reverse Transcriptase
Polymerase Chain Reaction (RT-PCR). Most of the countries are following this
RT-PCR test as an immediate and faster detection kit. However, Kong et al. [7]
found that RT-PCR tests often face a high false-negative rate. Moreover, in case the
COVID positive cases suddenly increase, these kits are not readily available at all the
clinical systems, which could be a reason for the virus spreading. Medical inspection
is another clinical complementary method for COVID-19 detection, includes radi-
ography chest images, that is, CXR and CT scan. CT and X-ray images can help in
predicting the severity of COVID-19 by identifying the infected regions. Therefore,
many AI-aided methods have been proposed and used in the detection of COVID-19
recently. Thus, the main objective of this chapter is to implement a TRNetCoV net-
work model to detect COVID-19 disease with CXR images.
12.5 CONTRIBUTIONS
Our major contributions for this chapter include:
TABLE 12.1
Description of AI Techniques Used in COVID-19 Research
f ( x ) = max ( 0, x ) (12.1)
Output : ( n − f + 1) X ( n − f + 1) (12.4)
With padding:
Input : mXm (12.5)
Paddingpissettobe 1 (12.6)
224 Machine Learning and Deep Learning in Medical Data Analytics
12.7.2.2 Classification
This block consists of a set of fully connected layers that takes the group of input
features into a one-dimensional vector. It looks at the observed values that are closed
to target values by applying some distribution functions.
Softmax Layer: It is a kind of multi-label classifier to distribute the probability
of each label. Furthermore, the output of each class label is to be normalizing in the
range of (0.1). The standard equation of softmax is defined as follows:
z
σ ( Z ) = ei (12.9)
∑
K
e zj
j =1
TABLE 12.2
AI-assisted CXR Image Detection Methods for COVID-19
Dataset Size
Authors of X-ray Images COVID-19 Cases AI-based Deep Models
Wang 13,800 183 COVID-Net
Zhang 1531 100 New DL
Ioannis 1472 224 VGG, MobileNet, Inception
Castiglioni 610 324 ResNet
Loey 306 69 AlexNet, GoogleNet, ResNet
Maghdid 170 60 ALexNet
TL-based ResNet Model for COVID-19 detection 225
TABLE 12.3
Various Deep Learning Models Against COVID-19
In this way, few models were constructed as image feature extractors based on
VGG, AlexNet, GoogleNet, DenseNet, and InceptionResNet. Szegedy et al. [12] and
Krizhevsky et al. [13] had also used similar feature extraction methods.
Simonyan et al. [14] had integrated DCNN models built on CXR images, such
as the VGG19 model given by Huang et al. [15], DenseNet201, ResNEtV2, and
InceptionV3 from Chebet et al. [16].
Later, some of these typical DCNN models –VGG, ResNet50, and InceptionV3 –
were improved and recommended for medical image classification on small-
scale CXR images for COVID-19. Their results displayed that ResNet50 had
achieved 98% accuracy, which was the superior performance among other AI
methods they used. Table 12.3 gives more detailed information about various DL
models against COVID-19.
12.8 METHODOLOGIES
12.8.1 Data Preprocessing Technique
Initially, the collected groups of COVID-19 CXR images could be relatively
massive in noise information and this data could sometimes be fake. In the era
of social media, filtering fake information is a crucial step in medical imaging.
However, AI-based models can be used to identify fake news from online media
platforms. As a consequence, this issue can limit the performance of DCNN
models in the epidemic prediction. The CXR images are resized to the size
of 224x224x3. Then on each image of the dataset, a normalization method is
applied, which responds to the model and could benefit in overall performance.
Figure 12.2 displays the complete flowchart of data processing stages proposed
in this chapter.
FIGURE 12.2 Steps involved in the preprocessing stage of the proposed work.
226 Machine Learning and Deep Learning in Medical Data Analytics
FIGURE 12.3 Basic architecture of residual learning Model. (Modified from He et al. [17].)
TL-based ResNet Model for COVID-19 detection 227
• Hinge loss function: To minimize the loss error from the learning prob-
lem, selection of a loss function that will help to reduce the risk. In
this multi-label classification, the loss will be controlled and defined as
follows:
where Wt and Wy are the control parameters for the training model.
• Adam optimizer: This was adapted to combine the advantages of both
RMSprop and AdaGrad, improves the performance over gradient problem.
This strategy helps in fixing the updated weight for old weights. All the
updating weights will be regularizing by the following rule:
m∧
wt = wt − 1 − η + δ wt − 1 (12.11)
vt + ∈
TABLE 12.4
Statistics of the Dataset Distribution
TABLE 12.5
New Hyperparameters Controls in the Proposed Model
FIGURE 12.5 Training and validation progress of the proposed TRNetCoV model.
Furthermore, Figures 12.5 and 12.6 present the resultant graphs of the pro-
posed model. We computed training and validation accuracy for every epoch
which extracts the best accuracy during the forward pass. It also compares the
validation loss and training loss at every epoch. The proposed model gave us a
better accuracy of 98.33% and validation loss at around 0.12 which can be con-
firmed by Figures 12.6 and 12.7.
FIGURE 12.6 (a) Loss history of the proposed TRNetCoV model and (b) Accuracy of the
proposed TRNetCoV model.
230 Machine Learning and Deep Learning in Medical Data Analytics
∑ TPi + TNi
l
∑ TPi (12.13)
l
i =1
Precisionm =
∑ TPi + FPi
l
i =1
∑ TPi (12.14)
l
i =1
Recallm =
∑ TPi + FNi
l
i =1
∑ TNi (12.15)
l
i =1
Specificity =
∑ TNi + FPi
l
i =1
F1 − Score =
( 2 * Precisionm * Recallm ) (12.16)
Precisionm + Recallm
where,
TPi = truepositivie, FPi = falsepositive, TNi = truenegative, and FNi = falsenegative
These are referred to the COVID-19 class and other subclasses of non-COVID-19
(viral and normal) and also 'l' represents the set of class labels. Table 12.6 shows the
precision and recall values estimated for the proposed model.
TL-based ResNet Model for COVID-19 detection 231
TABLE 12.6
Precision and Recall for the Proposed Work
TABLE 12.7
Comparative Analysis of Individual Evaluation Performance
A confusion matrix was obtained for the proposed TRNetCoV model and is pre-
sented in Figure 12.7. It is observed that the proposed model predicts true values for
most of the classes based on the validation set. Due to the fact of overfitting behavior
of DL models, few labels were misclassified. To classify the COVID-19, the pro-
posed model has performed with better accuracy of 98%.
The proposed model has been compared with the existing popular DL models.
Table 12.7 lists the most recent and most popular relevant studies on the diagnosis of
COVID-19 using X-ray image classification methods. From the table, it can be found
that the proposed model has gained importance due to its higher accuracy rate and
lower validation loss while considering the COVID-19 cases.
The proposed method in this chapter had provided better performance in terms of
higher accuracy and lower validation loss. Thus, it is encouraging to researchers
and doctors in detecting and classifying COVID-19 for respiratory issues in less
time with higher efficiency.
In the future, this work can be extended to deal with large datasets of COVID-19
CXR images by combining multiple classifiers to achieve a more reliable estimation
of disease with wide challenges in the dataset such as image noise, fine and minute
classifications between the normal respiratory issues and covid cases, and better pre-
processing approaches to avoid illumination changes in the dataset images.
REFERENCES
[1] Paramesha K, Gururaj H L, & Jena O P. (2021). Applications of Machine Learning
in Biomedical Text Processing and Food Industry. Machine Learning for Healthcare
Applications, 151–67, doi: 10.1002/9781119792611.ch10
[2] Gnter K, Unterthiner T, Mayr A, & Hochreiter S. (2017). Self-normalizing Neural
Networks. Advances in Neural Information Processing Systems, 5, 971–80.
[3] Panigrahi N, Ayus I, & Jena, O P. (2021). An Expert System-Based Clinical Decision
Support System for Hepatitis-B Prediction & Diagnosis. Machine Learning for
Healthcare Applications, 57–75, doi: 10.1002/9781119792611.ch4
[4] Apostolopoulos I D, & Bessiana T. (2003). Covid-19: Automatic Detection from X-Ray
Images Utilizing Transfer Learning with Convolutional Neural Networks. Report
Number; arXiv, 11617, doi: 10.1007/s13246-020-00865-4
[5] Wang Y, Wang Y, Chen Y, & Qin, Q J. (2020). Unique Epidemiological and Clinical
Features of the Emerging Novel Coronavirus Pneumonia (COVID-19) Implicate
Special Control Measure. Medical Virology, 92(6), 568–76.
[6] Ai T, Yang Z, Hou H, Zhan C et al. (2020). Correlation of Chest CT and RT-PCR
Testing in Coronavirus Disease 2019 (COVID-19) in China. Radiology; Report num-
ber: 200642, 296(2), E32–40.
[7] Kon W, Li Y, Peng M, & Kong D. (2020). SARS-CoV-2 Detection in Patients
with Influenza-like Illness. Nature Microbiology, 5(5), 675678, doi: 10.1038/
s41564-020-0713-1
[8] Patra S S, Jena O P, Kumar G, Pramanik S, Misra C, & Singh, K N. (2021). Random Forest
Algorithm in Imbalance Genomics Classification. Data Analytics in Bioinformatics: A
Machine Learning Perspective, 173–90. doi:10.1002/9781119785620.ch7
[9] Hemdan E, Shouman M, & Karar M. (2020). COVIDX-Net: A Framework of Deep
Learning Classifiers to Diagnose COVID-19 in X-ray Images, Electrical Engineering
and Systems Science. 1, 11055.
[10] Pattnayak P, & Jena O P. (2021). Innovation on Machine Learning in Healthcare
Services–An Introduction. Machine Learning for Healthcare Applications, 1–15, doi:
10.1002/9781119792611.ch1
[11] Kumar S, Bhusan B, Singh D, & Choubey DK. (2020). Classification of Diabetes using
Deep Learning. International Conference on Communication and Signal Processing
(ICCSP), 0651–5, doi: 10.1109/ICCSP48568.2020.9182293
[12] Szegedy C, Ioffe S, Vanhoucke V, & Alemin A. (2017). Inception-v4, Inception-ResNet
and the Impact of Residual Connections on Learning. AAAI Conference on Artificial
Intelligence, 31(1). 4278–4284.
[13] Krizhevsky A, Sutskever I, & Hinton G. (2017). Imagenet Classification with Deep
Convolutional Neural Networks. Advances in Neural Information Processing Systems,
60, 84–90, doi: 10.1145/3065386
TL-based ResNet Model for COVID-19 detection 233
[14] Simonyan K. & Zisserman A. (2014). Very Deep Convolutional Networks for Large-
scale Image Recognition. Computer Vision and Pattern Recognition, 6, 1409–556.
[15] Huang G, Liu Z, & Weinberger K. (2017). Densely Connected Convolutional Networks.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4700–08, doi:
10.1109/CVPR.2017.243
[16] Chebet T, Li Y, & Sam Liu Y. (2019). A comparative Study of Fine-tuning Deep
Learning Models for Plant Disease Identification. Computers and Electronics in
Agriculture, 161, 272–79.
[17] He K, Zhang X, Ren S, & Sun J. (2016). Deep Residual Learning for Image Recognition.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–78.
[18] Shin H.-C. et al. (2016). Deep Convolutional Neural Networks for Computer-aided
Detection: CNN Architectures, Dataset Characteristics and Transfer Learning. IEEE
Transactions on Medical Imaging, 35, 1285–98.
[19] A, Singh PK, Rani P, Samanta D, Khanna A, & Bhushan B. (2020). An Internet of
Health Things-driven Deep Learning Framework for Detection and Classification of
Skin Cancer Using Transfer Learning. Transactions on Emerging Telecommunications
Technologies, doi: 10.1002/ett.3963
[20] COVID-19 Radiography Database [Online] Available from: https://www.kaggle.com/
tawsifurrahman/covid19-radiography-database [Accessed February 5, 2021].
13 The Influence of
COVID-19 on Air Pollution
and Human Health
L. Bouhachlaf, J. Mabrouki, and S. El Hajjaji
CONTENTS
13.1 Introduction................................................................................................. 235
13.2 Air Quality and COVID-19 in Morocco During
the State of Health Emergency.................................................................... 236
13.2.1 Effect of COVID-19 and Air Pollution on Human Health............ 237
13.3 Materials and Methods................................................................................ 237
13.3.1 Study Area..................................................................................... 237
13.3.2 Methodology.................................................................................. 238
13.4 Results and Discussion................................................................................ 239
13.4.1 Nitrogen Dioxide Measurement..................................................... 239
13.4.2 Spatiotemporal Evolution of Nitrogen Dioxide............................. 239
13.5 Conclusion................................................................................................... 241
References............................................................................................................... 242
13.1 INTRODUCTION
Air pollution can have various health effects either in the short or long term. The
danger of acute and chronic breathing and cardiovascular diseases increases with air
pollution (Manisalidis et al. 2020). In 2016, the World Health Organization (WHO)
stated that 92% of the total population globally breathes substandard air (WHO
2016). This results in significant rates of illness and death especially among the
elderly, those with respiratory problems, and young children living in polluted cities
(Lelieveld et al. 2015). Air pollution is caused by the occurrence of harmful elements
in the air, mostly generated by the actions of humans. These poisonous substances
produce a number of events and effects on the various systems and living beings that
populate our world; they affect everyone and all sectors (World Bank 2016). It has a
high effect on the evolution of plants by inhibiting photosynthesis in the majority of
situations, with significant impact on the purification of the air we breathe (Biswal
et al. 2021). The accumulation of these substances in the air creates environmental
issues whose impact is well noted, such as the destruction of the ozone layer, global
climate change, and the greenhouse phenomenon (Menut et al. 2020). The level of
these pollutants in the environment is growing at an annual rate of 1%. It is due to
the characteristics of some gases like nitrogen dioxide (NO2), chlorofluorocarbons,
carbon dioxide, ozone, and methane, which retain the temperature of the sun in the
DOI: 10.1201/9781003226147-13 235
236 Machine Learning and Deep Learning in Medical Data Analytics
air, keeping it from getting back into the atmosphere once it has been returned by the
ground (Pénard-Morand and Annesi-Maesano 2004; Khomsi et al. 2020).
The main gases that can pollute the air by their high concentration are sulfur
dioxide, carbon dioxide, NO2, and ozone, along with fine dust particles (Sekmoudi
et al. 2021). This study focuses on the monitoring of air pollutants that can pollute
the air by their high concentration. In this chapter we will focus on monitoring the
pollutant NO2, to analyze the effect of COVID-19 and this air pollutant on human
health, as well as examining the level of NO2 pollutants present in the air, by moni-
toring the spatial and temporal variability of NO2 pollution in Morocco from January
2020 to July 2021 using data from MERRA-2 and sentinel 5-P satellites, during the
global epidemic and the state of health emergency in Morocco.
Most Moroccan industrial units are located at the level of great Casablanca,
followed by Fes-Boulemane, Rabat-Sale-Zemmours-Zaers, Tanger-Tetouan, Sous-
Massa-Daraa, and Marrakech-Tensift-Haouz.
This assessment is based on monitoring station data analysis both before and after
the state of health emergency. This assessment will allow a more detailed analysis of
the air pollution situation, including the baseline situation, to draw lessons and make
recommendations for limiting post-COVID air pollution (Macd onald et al. 2021).
13.3.2 Methodology
The ozone-measuring instrument is a Finnish-Dutch image spectrum meter for
ozone monitoring (Lokhandwala and Gautam 2020). The instrument is designed
to distinguish ozone from other atmospheric species (Ghosh and Ghosh 2020). The
Influence of COVID-19 on Air Pollution and Human Health 239
high spatial and spectral response of the instrument is important for the analysis of
air pollution at the city level (Gelaro et al. 2017). The acquisition of measurements
of the stratospheric and tropospheric stages of the earth’s atmosphere is the principal
aim of the instrument. MERRA-2 is the acronym for Modern Era Retrospective
analysis for Research and Applications version 2 (Boersma et al. 2011). MERRA
focuses on analyses of the past climate for a variety of weather and climatic periods
and situates the NASA Earth Observing System (EOS) suite of images in a climate
context (Xu et al. 2008).
OMNO2d stands for OMI and Aura NO2 Total and Tropospheric Column Filtered
(Ghosh and Ghosh 2020). The OMNO2d data is a Level 3 gridded data element in
which good quality pixel-level data are averaged and combined into global grids
of 0.25-degree x 0.25-degree resolution. This dataset provides the total column
of ground-level NO2 for all atmospheric and cloud fraction conditions below 30%
(Duncan et al. 2016; Zambrano-Monserrate et al. 2020).
FIGURE 13.2 Spatiotemporal evolution of NO2 concentration in Morocco (a) before lockdown
(January to March) and (b) in lockdown (March to May). (From NASA Earth Observatory.)
the NO2 had decreased. This reduction in air pollution has had beneficial effects on
people’s health and has decreased deaths and saving lives, mostly due to cardiovas-
cular diseases.
In the same way for the months of June and July, it remains low in comparison
with the first months (Figure 13.3).
This number represents the number of NO2 molecules in an atmosphere column
from the planet’s surface to the upper atmosphere, measured upwards one square
centimeter of the surface (NASA Earth Observatory, 2021).
The decrease in the concentration of NO2 from 5.25 to 4.25 (1015 molec.cm−2)
from March to May during the containment (Figure 13.4), which is certainly due
to the decrease of anthropogenic emissions. Human activities, due to the strict
restrictions imposed by the countries during the lockdown, after the reduction of
restrictions in some areas, a gradual increase in the level of pollutants (values of
9.056 *1015 molec.cm−2) was observed on (Figure 13.4).
Influence of COVID-19 on Air Pollution and Human Health 241
FIGURE 13.4 Time series, area averaged of NO2 tropospheric column from January to June
2020.
13.5 CONCLUSION
This research presents an evaluation of air monitoring in Morocco during the
COVID-19 pandemic from March to June 2020. Satellite data comparing levels of
NO2 concentrations after the shutdown show significant reductions. The satellite
data show that, in the confined areas, the average NO2 levels between March and
May 2020 were lower than the June 2020 concentrations after the shutdown. We
therefore conclude that the traffic restrictions applied during the quarantine for the
COVID-19 pandemic in Morocco were remarkably effective in reducing NOx emis-
sions. This reduction in ozone precursors reduces ozone.
242 Machine Learning and Deep Learning in Medical Data Analytics
REFERENCES
Biswal, Akash, Vikas Singh, Shweta Singh, Amit P Kesarkar, Ravindra Khaiwal, et al. 2021.
“COVID-19 lockdown-induced changes in NO2 levels across India observed by multi-
satellite and surface observations.” Atmospheric Chemistry and Physics 21 (6): 5235–
51. https://doi.org/10.5194/acp-21-5235-2021
Boersma, K. F., H. J. Eskes, R. J. Dirksen, R. J. Van Der A, J. P. Veefkind, P. Stammes, V.
Huijnen, et al. 2011. “An improved tropospheric no2 column retrieval algorithm for the
ozone monitoring instrument.” Atmospheric Measurement Techniques 4 (9): 1905–28.
https://doi.org/10.5194/amt-4-1905
Dirksen, R. J., K. F. Boersma, H. J. Eskes, D. V. Ionov, E. J. Bucsela, P. F. Levelt, and H. M.
Kelder. 2011. “Evaluation of stratospheric NO2 retrieved from the Ozone Monitoring
Instrument: Intercomparison, diurnal cycle, and trending.” Journal of Geophysical
Research 116: D08305, https://doi.org/10.1029/2010JD014943
Duncan, Bryan N.; Geigert, Michael; Lamsal, Lok.2016. “A brief tutorial on using the
ozone monitoring instrument (OMI) nitrogen dioxide (NO2) data product for SIPS
preparation.”NASA HAQAST Tiger Team: Supporting the Use of Satellite Data in State
Implementation Plans(SIPS). https://doi.org/10.7916/D80K3S3W. Draft Version 2: 1–21.
Gelaro, R., W. McCarty, M. J. Suárez, R. Todling, A. Molod, L. Takacs, C. A. Randles,
et al. 2017. “The modern-era retrospective analysis for research and applications,
version 2 (MERRA-2).” Journal of Climate 30: 5419–54. https://doi.org/10.1175/
JCLI-D-16-0758.1
Ghosh, Shilpi, and Shatabdi Ghosh. 2020. “Air quality during Covid-19 lockdown: Blessing
in disguise.” Indian Journal of Biochemistry and Biophysics 57 (4): 420–30.
Haddout, S., and K. L. Priya. 2020. “Unveiling the causes of reduction in troposphere no2
in two cities of Morocco during COVID-19 lockdown.” Environmental Forensics 21
(3–4): 237–40. https://doi.org/10.1080/15275922.2020.1784313
Khomsi Kenza, Houda Najmi, Hassan Amghar, Youssef Chelhaoui, and Zineb Souhaili.
2020. “COVID-19 national lockdown in Morocco: Impacts on air quality 1 and public
health.” MedRxiv 20146589. https://doi.org/10.1101/2020.07.05.20146589.
Khomsi, K., H. Najmi, H. Amghar, Y. Chelhaoui, and Z. Souhaili. 2021. “COVID-19 national
lockdown in Morocco: Impacts on air quality and public health.” One Health 11: 1–13.
https://doi.org/10.1016/j.onehlt.2020.100200
Lelieveld, J., J. S. Evans, M. Fnais, D. Giannadaki, and A. Pozzer. 2015. “The contribution
of outdoor air pollution sources to premature mortality on a global scale.” Nature 525
(7569): 367–71. https://doi.org/10.1038/nature15371
Lokhandwala S. and P. Gautam. 2020. “Indirect impact of COVID-19 on environment: A
brief study in Indian context,” Environmental Research. 188:109807.
Macdonald, E., N. Otero Felipe, and T. M. Butler. 2021. “A comparison of long-term trends
in observations and emission inventories of NOx.” Atmospheric Chemistry and Physics
21(5): 4007–23. https://doi.org/10.5194/acp-21-4007-2021
Manisalidis, Ioannis, Elisavet Stavropoulou, Agathangelos Stavropoulos, and Eugenia
Bezirtzoglou. 2020. “Environmental and health impacts of air pollution: A review.”
Frontiers in Public Health. https://doi.org/10.3389/fpubh.2020.00014
Marais, E. A., J. F. Roberts, R. G. Ryan, H. Eskes, K. F. Boersma, S. Choi, J. Joiner, N.
Abuhassan, et al. 2021 “New observations of NO2 in the upper troposphere from
TROPOMI.” Atmospheric Measurement Techniques. 14: 2389–2408, https://doi.
org/10.5194/amt-14-2389-2021
Menut, Laurent, Bertrand Bessagnet, Guillaume Siour, Sylvain Mailler, Romain Pennel, and
Arineh Cholakian. 2020. “Impact of lockdown measures to combat Covid-19 on Air
quality over western Europe.” Science of the Total Environment 741: 140426. https://
doi.org/10.1016/j.scitotenv.2020.140426
Influence of COVID-19 on Air Pollution and Human Health 243
Metya A., P. Dagupta, S. Halder, S. Chakraborty, and Y. K. Tiwari. 2020. “COVID-19 lock-
downs improve air quality in the South-East Asian regions, as seen by the remote sens-
ing satellites.” Aerosol and Air Quality Research. 20(8):1772–1782.
Minister of Energy, Mines and the Environment of the Kingdom of Morocco. 2020. Qualité
de l’air au Maroc pendant l’état d’urgence sanitaire liée au COVID 19. https://www.
environnement.gov.ma/fr/134-actualites/3338-qualite-de-l-air-aumaroc-pendant-l-
etat-d-urgence-sanitaire-liee-au-covid-19
NASA Earth Observatory. 2021. “Giovanni Parameter Definitions: Nitrogen Dioxide.”
Atmospheric Composition, Water & Energy Cycles and Climate Variability. https://disc.
gsfc.nasa.gov/information/glossary?title=Giovanni%20Parameter%20Definitions:%20
Nitrogen%20Dioxide
World Health Organization (WHO). 2016. WHO releases country estimates on air pollution
exposure and health impact. Accessed March 5, 2021. http://www.who.int/mediacentre/
news/releases/2016/air-pollution-estimates/en/
Otmani, A., A. Benchrif, M. Tahri, M. Bounakhla, E.M. Chakir, M. El Bouch, and M.
Krombi. 2020. “Impact of Covid-19 lockdown on PM10, SO2 and NO2 concentrations
in Salé City (Morocco).” Science of the Total Environment. 735: 139541. https://doi.
org/10.1016/j.scitotenv.2020.139541
Parrish David D., Richard G. Derwent, and Johannes Staehelin. 2021. “Version for publica-
tion Long-term changes in northern mid-latitude tropospheric ozone concentrations:
Synthesis of two recent analyses.” Atmospheric Environment, 118227. https://doi.
org/10.1016/j.atmosenv.2021.118227
Pénard-Morand, C., and I. Annesi-Maesano. 2004. “Air pollution: From sources of emissions
to health effects.” Breathe 1 (2): 108–19. https://doi.org/10.1183/18106838.0102.108
Sarfraz, Muddassar, Khurram Shehzad, and Syed Ghulam Meran Shah. 2020. “The
Impact of COVID-19 as a necessary evil on air pollution in India during the lock-
down.” Environmental Pollution 266: 115080. https://doi.org/10.1016/j.envpol.
2020.115080
Sekmoudi, I., K. Khomsi, S. Faieq, and L. Idrissi. 2021. “Assessment of global and regional
PM10 CAMSRA data: comparison to observed data in Morocco”. Environmental
Science and Pollution Research. https://doi.org/10.1007/s11356-021-12783-3
Suresh, Arjun, Diksha Chauhan, Amina Othmani, Neha Bhadauria, Aswin S, Jais Jose, and
Nezha Mejjad. 2020. “Diagnostic comparison of changes in air quality over China
before and during the COVID-19 pandemic.” Research Square 1:1–20. https://doi.
org/10.21203/rs.3.rs-30482/v1
Tong Daniel Q., Lok Lamsal, Li Pan, Charles Ding, Hyuncheol Kim, Pius Lee, Tianfeng
Chai, Kenneth E. Pickering, and Ivanka Stajner. 2015. “Long-term NOx trends over
large cities in the United States during the great recession: Comparison of satellite
retrievals, ground observations, and emission inventories.” Atmospheric Environment
107: 70–84. https://doi.org/10.1016/j.atmosenv.2015.01.035
van Geffen, J., K. F. Boersma, H. Eskes, M. Sneep, M. ter Linden, M. Zara, and J. P. Veefkind.
2020 “S5P TROPOMI NO2 slant column retrieval: Method, stability, uncertainties and
comparisons with OMI.” Atmospheric Measurement Techniques 13: 1315–35. https://
doi.org/10.5194/amt-13-1315-2020
Wargan, Krzysztof, Gordon Labow, Stacey Frith, Steven Pawson, Nathaniel Livesey, and
Gary Partyka. 2017. “Evaluation of the ozone fields in NASA’s MERRA-2 reanalysis.”
Journal of Climate 30 (8): 2961–88. https://doi.org/10.1175/JCLI-D-16-0699.1World
Bank and Institute for Health Metrics and Evaluation. 2016. The Cost of Air Pollution:
Strengthening the Economic Case for Action. Washington, DC: World Bank.
Xu Pengcheng, Zhixuan Cheng, Qingyi Pan, Jiaqiang Xu, Qun Xiang, Weijun Yu, and
Yuliang Chu, 2008. “High aspect ratio In2O3 nanowires: Synthesis, mechanism and
NO2 gas-sensing properties,” Sensors and Actuators B: Chemical 130(2): 802–08.
244 Machine Learning and Deep Learning in Medical Data Analytics
CONTENTS
14.1 Introduction.................................................................................................. 245
14.2 Materials and Methods................................................................................ 247
14.2.1 Geographic Information Systems and COVID-19
Containment/Deconfinement Operations...................................... 247
14.2.2 About Geo-Pandemic Data............................................................ 247
14.2.3 Geoprocessing Basic Operations................................................... 249
14.2.3.1 Operation 1: Continue to Contain Infected Areas........ 249
14.2.4 Viral Infection Risk Analysis Based on the Proximity
of Infected Areas............................................................................ 252
14.2.5 Viral Infection Risk Analysis Based on the Duration
of Exposure.................................................................................... 253
14.3 Spatial Network Voronoï Diagram Operations............................................ 253
14.3.1 Spatial Network Voronoï Diagrams............................................... 254
14.3.2 Spatial Voronoï Accessibility......................................................... 254
14.3.3 Spatial Relationships and Statistical Evaluation Offered
by Spatial Voronoï Diagrams......................................................... 255
14.3.4 The Shortest and Safest Paths Based on Network
Voronoï Diagrams.......................................................................... 255
14.4 Results and Discussion................................................................................. 256
14.4.1 Isolate Infected Areas.................................................................... 257
14.4.2 Ensure Safe Travel......................................................................... 257
14.4.3 Evaluate the Various Security Levels Compared
to Infected Areas............................................................................ 257
14.5 Conclusion.................................................................................................... 259
References............................................................................................................... 259
14.1 INTRODUCTION
The pandemic looks like a war: people are dying, medical personnel are on the
front lines, and authorities are working overtime to tighten control over compliance
with protective measures and mandatory health procedures. According to the World
DOI: 10.1201/9781003226147-14 245
246 Machine Learning and Deep Learning in Medical Data Analytics
Health Organization (WHO) (World Health Organization 2005), the term pandemic
applies when talking about the global spread of a new disease. It occurs when a new
virus appears and spreads around the world, affecting a large geographic extent,
with no immunity in the vast majority of the population. In the past, pandemics were
caused by influenza viruses in animals. Recently, a heretofore unknown agent, the
Severe Acute Respiratory Syndrome Coronavirus (SARS) broke out. SARS is
the first serious and communicable disease to emerge in the 21st century. The epi-
demic, which began in China at the end of 2002, erupted around the world in 2003,
infecting more than 8000 people and killing nearly 800 people. The SARS virus
is spread primarily by droplets through person-to-person contact, although aerosol
transmission may also play a role. Nowadays, public health issues have taken on
great importance in our society, especially the viral spread in populated areas. This
coronavirus, also called COVID-19 (Franch-Pardo et al. 2020; Sun et al. 2020), is
a new family of viruses that infects humans. By navigating and moving, infected
humans infect other individuals and consequently a transmission caused epidemic
outbreaks with almost exponential growth and generated chaos on the planet, where
more than a third of the world’s population called for containment. This has caused
the saturation of health systems, and social and economic disruption. In terms of
accountability, the competent national authorities are responsible for managing the
national risk of pandemic influenza. Each country is required to develop or update a
national preparedness plan for influenza as recommended by the WHO. This man-
agement is the solution to reduce the risk of the influenza pandemic spread. For
example, during a pandemic alert period, the priority goals of containing the new
virus in limited foci or delaying its spread to save time, and then doing everything to
contain its spread (Atluri et al. 2018).
On the other hand, this pandemic mobilized scientific research and accelerated
the production of knowledge on this virus as well as the means of curing and pre-
venting it. In addition, technologies play a decisive role in ensuring the proper func-
tioning of society in times of containment and quarantine. These technologies can
have a lasting impact even after the pandemic (Zhou et al. 2020), (Cherradi et al.
2017; Das and Ghosh 2020). The year 2020, which marks the start of an exciting
decade in medicine and science, has allowed the development of many digital tech-
nologies. This is artificial intelligence (AI) which uses deep learning; the Internet of
Things (IoT) (Pamučar1996; D’silva et al. 2017; Maguerra et al. 2020) with the new
5G; big data analysis; and blockchain technology (Maguerra et al. 2020).
In this work, we propose a set of processes to effectively conduct urgent viral
pandemic management operations (Duan et al. 2018). These processes concern the
management of confinement/deconfinement at the local or global scale by using
spatio-temporal networks of the tessellation/partitioning buffer type (Erwig 2000;
Okabe et al. 2008) or of the Voronoï type (buffer, graphs, and polygons) and the
structural patterns of complex graphs (Zheng et al. 2011).
This chapter is organized as follows. It starts with an overview of a set of materi-
als and then it presents methods, which are given in Section 14.2. Section 14.3 dis-
cusses the different operations of Voronoï spatial diagrams. Section 14.4 shows the
results obtained as well as the analysis of these results. Using the interactive smart
maps that we developed in this work, the infected areas of the city are visualized
Smart COVID-19 GeoStrategies using Spatial Network Voronoï Diagrams 247
and spatial decision support tools are offered to authorities to monitor the pandemic
spreading. Finally, in Section 14.5, we draw some conclusions from this study and
highlight the future directions of this work.
TABLE 14.1
Cell Table Attributes
Name Name of the Location
cell id Cell identifier (code)
S Number susceptible individuals
R Number recovered individuals
I Number infected individuals
Geom. Point
TABLE 14.2
Network Table Attributes
Name Name of the Street
Type Road type (primary, secondary,
residential, motorway)
Geo Line
TABLE 14.3
Houses Table Attributes
Name Name of the Street
Type Road type (primary, secondary,
residential, motorway)
Geo Polygon
Covid_Outbreak makes up the cell layer of COVID-19 virus infection in the city.
Each cell is characterized by epidemiological information.
The buffer operation is a common operation in GIS and it is available in PostGIS.
ST_Buffer (geometry, distance) takes a buffer distance and a geometry type, and gen-
erates a polygon with a limit to the buffer distance of the input geometry as follows:
The geospatial data concerning the pandemic are processed according to the
pipeline presented in Figure 14.1.
Smart COVID-19 GeoStrategies using Spatial Network Voronoï Diagrams 249
OSM − > GML − > shapefile − > Qgis− > shapefiles − > postGis − > tables
+ queries − > Qgis − > WEB || svg
This process performs the deconfinement structures. It defines the green network
(uncontaminated) as well as green housing (uncontaminated). The associated pro-
cesses are described below:
{( )
k
}
A p = Si , S j | Si , S j ∈ S , k = 1, 2, ···, m p ; (14.1)
In a path p which belongs to the set of available paths, the risk Rp is calculated by
using the following formula [2,3]:
Rp = ∑ P C (14.2)
( )
i , j ∈A p
v
ij
v
ij
With Pijv representing the probability of a danger occurrence and Cijv representing the
consequences on the vulnerable which may result from it given environment elements.
The analysis of the viral infection risk and of the human targets’ vulnerability
based on the systemic risk approach leads us to conclude that more people get closer
to infected people or areas; these targets are more and more in danger (more and
more vulnerable). Indeed, if the viral infection probability is higher, then the viral
infection risk is higher (Figure 14.6).
FIGURE 14.6 Viral infection risk analysis based on the proximity of infected areas.
Smart COVID-19 GeoStrategies using Spatial Network Voronoï Diagrams 253
FIGURE 14.7 Viral infection risk analysis based on the duration of exposure.
Let Ri,n and Ri+1,n and Ri,m be the risks assessed respectively on the pi,n and pi+1,n
and pi,m paths according to formula (equation 14.2), where Pi ,vn and Pi +1,
v
n and Pi ,m rep-
v
resent the probabilities of viral infection of people heading respectively from node Si
to nodes Sn and m. dij represents the travel distance between nodes Si and Sj.
On the other hand, when the individual changes the direction at node Si+1 toward
the zone m which is not infected, he moves away from the infected zone n. In fact,
the probability of Pi +1,
v
m infection decreases and consequently the Ri,m risk decreases
in parallel with the distance of the individual from the infected area.
In fact, the exposure time plays a very important role in the change in the risk of
viral infection rates. Let Te1 and Te2 respectively be the travel times between nodes i
and i+1 by the two individuals Ind1 and Ind2 within an infected zone (Figure 14.7).
Let Pe1 and Pe2 be the probabilities of viral infection of individuals Ind1 and Ind2
respectively. If the duration of exposure of individual Ind1 is greater than that of
individual Ind2, then the likelihood of viral infection of individual Ind1 is greater
than that of individual Ind2. Therefore, the risk of Re1 infection of individual Ind2 is
greater than the risk of Re2 infection of individual Ind2.
{ ( ) }
Vori = ∀x ∈ E | d ( x , pi ) ≤ d x , p j , j ≠ i, j = 1,…, n (14.4)
Okabe and al. (2008) showed that the Euclidean distance is significantly differ-
ent from the distance of the shortest path in the urbanized sector. So, in an urban
environment, this distance must be calculated on the spatial network. The network
Voronoï diagram (NVD) is then defined by the division of the network into Voronoï
sub-networks, each of which contains the points closest to each Voronoï generator by
traversing the shortest path between these components (Figure 14.8).
For a set of vertices (represent the Voronoï generators) G = {g1,…, gn} with G ⊆
S, the NVD divides a network N (S, E) of vertices S and arcs E, into n Voronoï sub-
networks Vor(1),…, Vor(n) with:
with P(v, w) represents the weight of the shortest path from v to w in the network N
considering v and w two vertices belonging to S.
points is a solution to associate, with each set of points surrounding each accessibil-
ity point, the Voronoï generators and the distances of the shortest paths to reach these
generators. Figure 14.9 shows the polygons of the PVD which are generated by the
Voronoï accessibility points. Each polygon contains information about the Voronoï
generator, and the shortest path distance from the accessibility point to the Voronoï
generator associated with that point.
Mabrouk and Boulmakoul (2017) proposed an approach for routing and planning
safe trips in an urban setting. The purpose of this approach is to find the shortest
routes for the safe movement of people (Figure 14.10). It helps to reduce risk and
256 Machine Learning and Deep Learning in Medical Data Analytics
ultimately keep people as far away from harm as possible. Indeed, these authors
propose an assessment process based on spatial network modeling, which consists in
finding short and safe routes by moving as far as possible from the Voronoï genera-
tors (vulnerable sites). These points, called division points, are generated using the
NVD (Figure 14.10). They are found at the same distance from every two or more
Voronoï generators.
implement a set of means and coordinated actions to achieve the objectives of our
geostrategy which aims to provide answers to a set of questions that allow citizens
to move in a secure urban space far from the flow of danger propagated by areas
contaminated by COVID-19. So, we used a set of software components (already
developed in the framework of a previous work) and also a set of geoprocessing and
geovisualization tools of a GIS system to develop a set of intelligent and interac-
tive maps that allow authorities to isolate infected areas, ensure safe movement and
assess different levels of security in relation to infected areas.
FIGURE 14.12 Interactive map used to find the safest paths (green lines).
(Figure 14.13). This map offers decision-makers the possibility of measuring the risk
and consequently of deciding the secure places for the necessary meetings (pick-up
points for students, etc.)
A mobile application manipulating the information provided by this spatial sup-
port will be able to guide people who move around the city and warn them in real
time of infected areas that are found in their proximity, while showing them the
distance separating their current geographical position and these infected areas
(Figure 14.14). Other services will be added based on the work given in Boulmakoul
and Bouziri (2012) and Cassini (1998).
FIGURE 14.13 Interactive map used to evaluate the various security levels compared to
infected areas.
Smart COVID-19 GeoStrategies using Spatial Network Voronoï Diagrams 259
14.5 CONCLUSION
In this chapter we have discussed a set of tools and methods capable of construct-
ing a safe spatial Voronoï graph, and spatially and temporally analyzing the risk of
COVID-19 infection. In addition, a set of geoprocessing, geovisualization and intel-
ligent spatial analysis operations will make it possible to choose and implement a set
of means and coordinated actions to achieve the objectives of a smart geostrategy
which aims to provide answers to a set of questions. Using these methods and tools,
we have developed a set of interactive maps that allow citizens to move around a
secure urban space far from the risk of COVID-19 contamination and make avail-
able to decision makers, during the period of partial containment, a spatial deci-
sion-making aid illustrating spatial information on the location and proximity of
the infected places, the distance and routes to the nearest infected places, and the
security levels of the various infected areas.
REFERENCES
Atluri G., Karpatne A., Kumar V. Spatio-temporal data mining: A survey of problems and
methods. ACM Computing Surveys, 2018, 51(4): 1–41. https://doi.org/10.1145/3161602
Boulmakoul A., Bouziri A.E. Mobile Object Framework and Fuzzy Graph Modelling to Boost
HazMat Telegeomonitoring. In: Garbolino E., Tkiouat M., Yankevich N., Lachtar D.
(eds), Transport of Dangerous Goods. NATO Science for Peace and Security Series C:
Environmental Security. Springer, Dordrecht, 2012.
Cassini P. Road transportation of dangerous goods: quantitative risk assessment and route
comparison. Journal of Hazardous Materials, 1998, 61(1–3). https://doi.org/10.1016/
S0304-3894(98)00117-4
Cherradi G., El Bouziri A., Boulmakoul A., Zeitouni K. Real-Time HazMat Environmental
Information System: A micro-service based architecture. Procedia Computer Science,
2017, 109: 982–87.
Das M., Ghosh S.K. Data-driven approaches for spatio-temporal analysis: a survey of the
state-of-the-arts. Journal of Computer Science and Technology, 2020, 35: 665–96.
https://doi.org/10.1007/s11390-020-9349-0
D’silva G.M., Khan A., Gaurav, J., Bari S. Real-time processing of IoT events with his-
toric data using Apache Kafka and Apache Spark with dashing framework, 2nd
IEEE International Conference on Recent Trends in Electronics, Information &
Communication Technology (RTEICT), Bangalore, 2017, 1804–09. https://doi.
org/10.1109/RTEICT.2017.8256910
260 Machine Learning and Deep Learning in Medical Data Analytics
Duan P, Mao G, Liang W, Zhang D. A unified spatiotemporal model for short-term traffic
flow prediction. IEEE Transactions on Intelligent Transportation Systems, 2018, 20(9):
3212–23.
Erwig M. The graph Voronoï diagram with applications. Networks, 2000, 36(3): 156–63.
Fabiano B., Curro F., Palazzi E., Pastorino R. A framework for risk assessment and decision-
making strategies in dangerous good transportation. Journal of Hazardous Materials,
2002, 93: 1–15. https://doi.org/10.1016/S0304-3894(02)00034-1
Franch-Pardo I., Napoletano B., Rosete-Verges F., Billa L. Spatial analysis and GIS in the
study of COVID-19: a review. Science of The Total Environment, 2020, 739: 140033.
https://doi.org/10.1016/j.scitotenv.2020.140033.
Karim L., Boulmakoul A., Mabrouk A., Lbath A. Deploying Real Time Big Data Analytics
in Cloud Ecosystem for Hazmat Stochastic Risk Trajectories. Procedia Computer
Science, 2017, 109: 180–87. https://doi.org/10.1016/j.procs.2017.05.322.
Mabrouk A., Boulmakoul A. Modèle spatial objet base sur les diagrammes spatiaux de voronoï
pour la geo-gouvernance des espaces urbains. Workshop International sur l’Innovation
et Nouvelles Tendances dans les Systèmes d’Information, INTIS, 2012, pp. 105–16.
Mabrouk A., Boulmakoul A. Nouvelle approche basée sur le calcul des itinéraires courts et
sûrs pour le transport des matières dangereuses favorisant l’accès rapide aux secours,
The sixth International Conference on Innovation and New Trends in Information
Systems, INTIS, 2017, ISBN : 978-9954-34-378-4, ISSN: 2351-9215.
Mabrouk A. Boulmakoul A., Karim L., Lbath A. Safest and shortest itineraries for transport-
ing hazardous materials using split points of Voronoï spatial diagrams based on spatial
modeling of vulnerable zones. Procedia Computer Science, 2017, 109: 156–63. https://
doi.org/10.1016/j.procs.2017.05.311
Maguerra S., Boulmakoul A., Karim L., et al. Towards a reactive system for managing big
trajectory data. J Ambient Intell Human Comput, Springer Journal., 2020, 11: 3895–906,
https://doi.org/10.1007/s12652-019-01625-3
OCG. 2002. Open-Geospatial-Consortium. Web map service (1.1.1) implementation specifi-
cation. http://portal.opengeospatial.org
Okabe A., Satoh T., Furuta T., Suzuki A., Okano K. Generalized network Voronoï diagrams:
Concepts, computational methods, and applications, International Journal of Geographical
Information Science, 2008: 965–94, https://doi.org/10.1080/13658810701587891
Pamučar Dragan, Ljubojević Srđan, Kostadinović Dragan, Đorović Boban. Cost and risk
aggregation in multi-objective route planning for hazardous materials transportation—
A neuro-fuzzy and artificial bee colony approach. Expert Systems with Applications,
1996, 65: 1–15.
PostGis-Project. 2013. Spatial support for postgresql. http://postgis.refractions.net/
QGIS Development Team. 2009. QGIS Geographic Information System. Open-Source
Geospatial Foundation. http://qgis.org.
Snow J. The cholera near Golden-square, and at Deptford. Med Times Gazette, 1854, 9:
321–22.
Sun Feinuo, Matthews Stephen A., Yang Tse-Chuan, Hu Ming-Hsiao. 2020. A spatial analysis
of COVID-19 period prevalence in US counties through June 28, 2020: Where geogra-
phy matters? Annals of Epidemiology, https://doi.org/10.1016/j.annepidem.2020.07.014
World Health Organization. 2005. WHO global influenza preparedness plan: the role of WHO
and recommendations for national measures before and during pandemics. https://apps.
who.int/iris/handle/10665/68998
Zheng Y., Xiaofang Zhou (Eds). Computing with Spatial Trajectories. Springer, 2011, 308.
https://doi.org/10.1080/13658816.2012.741688
Zhou Chenghu, Su Fenzhen, Pei Tao, Zhang An et al. COVID-19: Challenges to GIS with
Big Data, Geography and Sustainability, 2020, 1(1): 77–87, https://doi.org/10.1016/j.
geosus.2020.03.005
15 Healthcare Providers
Recommender System
Based on Collaborative
Filtering Techniques
Abdelaaziz Hessane, Ahmed El Youssefi,
Yousef Farhaoui, Badraddine Aghoutane,
Noureddine Ait Ali, and Ayasha Malik
CONTENTS
15.1 Introduction................................................................................................. 261
15.2 Research Background.................................................................................. 263
15.3 Healthcare Provider Recommender Systems.............................................. 263
15.3.1 HPRS Based on Collaborative Filtering........................................ 263
15.3.2 HPRS Based on Content Filtering.................................................264
15.3.3 Hybrid HPRSs............................................................................... 265
15.4 Methodology............................................................................................... 265
15.4.1 Data Description............................................................................ 265
15.4.2 Data Preprocessing........................................................................ 265
15.4.3 Proposed Solution..........................................................................266
15.5 Machine Learning Models.......................................................................... 267
15.5.1 Neighbors-Based Models............................................................... 267
15.5.2 Latent Factors Models................................................................... 268
15.5.3 Slope-One-Based Model................................................................ 268
15.5.4 Co-Clustering Based Model.......................................................... 269
15.5.5 Simple Baseline Model.................................................................. 269
15.6 Evaluation Metric........................................................................................ 270
15.7 Results......................................................................................................... 270
15.8 Conclusion and Perspectives....................................................................... 272
References............................................................................................................... 272
15.1 INTRODUCTION
Health information systems (HIS) are becoming an increasingly valuable medium
for providing healthcare service [1]. Today, a large amount of health data is spread
around various websites on the internet. Overchoice and information reliability
are some of the challenges a patient may encounter when looking for suitable
DOI: 10.1201/9781003226147-15 261
262 Machine Learning and Deep Learning in Medical Data Analytics
The first step in CF is to identify users who have similar rating patterns to the new
active user, and the second step is to use the ratings of these similar users to make
a recommendation for that new user [15]. The filtering process may be one of two
types. The first one is user-based CF, in which we want to identify users that are close
to a new active user, find their ratings, and then aggregate those ratings to create sug-
gestions for the new user. The second one is item-based CF, in which the similarity
of two items is determined based on user reviews of those items. Figure 15.1 shows
the working process of CF techniques.
15.4 METHODOLOGY
In this section, the work planning with the internal working process in a proper
sequence is discussed, including which kind of data is required by describing them,
then it is stated how data get preprocessed and finally proposed solutions are explained.
15.4.1 Data Description
Several review-based HPRSs have been created in recent years to integrate the useful
information found in user-generated textual or numerical scale rating feedback into the
user modeling and recommendation process [19]. The amount of clientele information
in Yelp’s database has grown dramatically over the years, and we aim to test the predic-
tive efficiency of various CF techniques on this dataset. There are five json files in this
dataset named business, review, user, check-in, and tips. Only three of them were used
to construct the dataset, namely: Business.json, which contains business data including
location data, attributes, and categories. Categories.json can be used to retrieve category
lists and individual category information respectively; this file is used in the filtering
process as described in Figure 15.3. The third file is Review.json; it contains full review
text data including the ID of the user who wrote the review, the ID of the business the
review is written for, the star rating, the review date, and other information.
15.4.2 Data Preprocessing
Since we are interested in healthcare providers’ businesses only, the first step of data
preprocessing was necessary [20]. Figure 15.3 resumes the preprocessing steps to
prepare the data in the right format for the ML-based recommendation algorithms.
TABLE 15.1
Statistics on Final Dataset
Number of unique healthcare providers 1086
Number of unique users 8344
Number of ratings +28,000
The first preprocessing step was to extract a list of healthcare providers’ business
aliases and titles by filtering the categories.json. We aimed in the second step to
apply filters on the business.json file to retain only the open businesses and to remove
the unnecessary columns. We then build the ready-to-use dataset by combining the
files resulting from the two previous steps with the review.json file.
In this study, we limited ourselves to data from the city of Las Vegas only. In
order to boost the high sparsity presented by this data, we only keep users who have
given many reviews greater than the average number of reviews per user, which is
1.76. (users with only one review are removed). Despite this, the constructed patient-
healthcare providers’ ratings matrix presents almost 99.7% of sparsity. The challenge
was to find the best ML model that can predict the missing ratings with high accu-
racy. Table 15.1 shows the statistics on final dataset.
15.4.3 Proposed Solution
For the establishment of a HPRS based on predictive patient-healthcare professionals
rating model, the following approach is proposed. First, data in the form of explicit
user ratings of healthcare professionals must be collected and preprocessed. Second,
by using the cross-validation technique, we have tested and evaluated the different ML
models on the preprocessed data, and we then selected the best performing model. The
third step was to tune the hyper-parameters of the chosen model to improve their perfor-
mance. Finally, we deployed the trained model to construct the intended recommenda-
tion system. Figure 15.4 summarizes the different stages of the proposed solution.
15.5.1 Neighbors-Based Models
Recommendations based on neighbors models are done in two main phases: one
is the neighborhood formation phase and another one is the recommendation
phase.
A similarity between users (user-based approach) or items (item-based approach)
is measured during the neighborhood creation process. According to Zhang et al.
[23] the two most popular types of similarity measures are Pearson’s correlation
(PC) coefficient (equation 15.1) and cosine-based similarity (equation 15.2).
∑
n
( ai − a′ )( bi − b′ )
PC ( a, b ) = i =1
(15.1)
∑ ∑
n n
i =1
( ai − a′ ) 2
i =1
( bi − b′ ) 2
where a and b are two n-pointed vectors. The mean value of vector a and vector b is
represented by a’ and b’, respectively. In the above equation, PC finds the correlation
between the two sets a and b.
∑ A ×B
n
i i
cos (θ ) = i =1
(15.2)
∑ A ∑
n n
i
2 Bi 2
i =1 i =1
where A and B are two sets of n data points or n feature values. The values of char-
acteristic i in sets A and B are described by Ai and Bi, respectively.
The next step is to predict a rating r̂ui that user u will probably give to item i that
he didn’t rate yet. One way is to use the calculated similarities and the correspond-
ing ratings. Many variations are possible by adding biases like Z-Score or the user’s/
item’s rating average. In our study, we have tested four variations: KNN baseline,
KNN with means, KNN with Z-Score, and the basic KNN
If the approach is user based, then the predicted rating will be calculated as shown
in the formula (equation 15.3):
rˆui =
∑ v ∈N ik ( u )
sim ( u, v ).rvi
(15.3)
∑ v ∈N ik (u )
sim ( u, v )
268 Machine Learning and Deep Learning in Medical Data Analytics
If the approach is item based, then the predicted rating will be calculated as
shown in the formula (equation 15.4):
rˆui =
∑ sim ( i, j ) .ruj
j ∈N uk ( i )
(15.4)
∑ sim ( i, j )
j ∈N uk ( i )
where,
• qi: Vector associated with the item i.
• pu: Vector associated with user u.
rˆui = µu +
1
Ri ( u ) ∑ dev (i, j ) (15.6)
j ∈Ri ( u )
where,
• Ri ( u ) is the set of relevant elements, the set of elements j rated by u which
also have at least one common user with i.
• dev ( i, j ) is defined as the mean difference between the ratings of item I and
those of item j calculated as (equation 15.7):
dev ( i, j ) =
1
Uij ∑r
u ∈Uij
ui − ruj (15.7)
Uij represents the set of all users who have evaluated the elements i and j.
bui = µ + bu + bi (15.9)
Where:
μ is the mean rating of all reviews in the system.
bu indicates the difference between the average rating of user u and μ.
bi indicates the difference between the average rating of business i and μ.
270 Machine Learning and Deep Learning in Medical Data Analytics
• In this model, we take into account the user’s mean rating and the mean
of the healthcare providers to eliminate any potential bias when forecast-
ing (a consumer has a propensity to give something a higher rating than it
deserves).
• Model based on maximum likelihood estimation, a statistical estimator is
used to infer the parameters of a normal distribution from a given sample
by looking for the values of the parameters maximizing the likelihood
function.
∑(rˆ − ru ,i )
2
u ,i
RMSE = (15.10)
n
where,
• rˆu ,i is the predicted rating from user u on item i.
• ru ,i is the actual rating.
• n is the size of the test set.
15.7 RESULTS
Table 15.2 summarizes the results. We have got when testing and evaluating different
methods issued from the main models described above.
As shown in Figure 15.6, the user-based KNN model allows to predict the missing
ratings with the smallest RMSE; this means that this model is the best among the
11 models tested. The baseline here was used to adjust the data because some users
tend to give higher ratings than others, and for some items to receive higher evalua-
tions than others. For this, an estimation (baseline) is calculated using the following
formula (equation 15.11):
bui = µ + bu + bi (15.11)
with bui is the estimation (baseline) for an unknown score r̂ui and μ the overall aver-
age score. The parameters bu and bi respectively indicate the observed deviations of
Healthcare Providers Recommender System 271
TABLE 15.2
Summary of Results
the user u and the element i from the mean. Other estimations were tested such as
Z-Score and means biases.
As described in Figure 15.4, the step of hyper-parameters tuning comes right
after the evaluation step. As shown in Table 15.2, the baseline user-based CF KNN
was the best model in terms of RMSE value. The main goal of this phase was to find
the optimal hyper-parameters of the model. After applying the hyper-parameters
TABLE 15.3
Some Results of the Hyper-parameters Tuning Phase
tuning, the RMSE decreased from 1.041687 to 0.984700. Table 15.3 summarizes
some results of this phase.
REFERENCES
[1] Haried, P., Claybaugh, C., & Dai, H. (2017). Evaluation of health information systems
research in information systems research: A meta-analysis. Health Informatics Journal,
25(1), 186–202. https://doi.org/10.1177/1460458217704259
Healthcare Providers Recommender System 273
[2] Goyal, S., Sharma, N., Bhushan, B., Shankar, A., & Sagayam, M. (2020). IoT enabled
technology in secured healthcare: Applications, challenges and future directions.
Cognitive Internet of Medical Things for Smart Healthcare, 25–48. https://doi.
org/10.1007/978-3-030-55833-8_2
[3] Calero,ValdezA.,Ziefle,M.,Verbert,K.,Felfernig,A.,&Holzinger,A.(2016).Recommender
systems for health informatics: State-of-the-art and future perspectives. Lecture Notes in
Computer Science, 391–414. https://doi.org/10.1007/978-3-319-50478-0_20
[4] Aggarwal, C. C. (2016). An introduction to recommender systems. Recommender
Systems, 1–28. https://doi.org/10.1007/978-3-319-29659-3_1
[5] Sharma, N., Kaushik, I., Bhushan, B., Gautam, S., & Khamparia, A. (2020). Applicability
of WSN and biometric models in the field of healthcare. Deep Learning Strategies for
Security Enhancement in Wireless Sensor Networks Advances in Information Security,
Privacy, and Ethics, 304–329. https://doi.org/10.4018/978-1-7998-5068-7.ch016
[6] Narducci, F., Lops, P., & Semeraro, G. (2017). Power to the patients: The HealthNetsocial
network. Information Systems, 71, 111–122. https://doi.org/10.1016/j.is.2017.07.005
[7] Archana, B. & Smita, L. (2015). Personalized recommendation system for medical
assistance using Hybrid Filtering. International Journal of Computer Applications,
128(9), 6–10. https://doi.org/10.5120/ijca2015906626
[8] Guo, L., Jin, B., Yao, C., Yang, H., Huang, D., & Wang, F. (2016). Which doctor to trust:
A Recommender system for identifying the right doctors. Journal of Medical Internet
Research, 18(7), e186. https://doi.org/10.2196/jmir.6015
[9] Waqar, M., Majeed, N., Dawood, H., Daud, A., & Aljohani, N. R. (2019). An adap-
tive doctor-recommender system. Behaviour & Information Technology, 38(9), 959–73.
https://doi.org/10.1080/0144929X.2019.1625441
[10] Han, Q., Ji, M., de Troya, I. M. de R., Gaur, M., & Zejnilovic, L. (2019). A Hybrid
Recommender System for Patient-Doctor Matchmaking in Primary Care. ArXiv:
1808.03265 [Cs, Stat]. http://arxiv.org/abs/1808.03265
[11] Zhang, Y., Chen, M., Huang, D., Wu, D., & Li, Y. (2017). iDoctor: Personalized and pro-
fessionalized medical recommendations based on hybrid matrix factorization. Future
Generation Computer Systems, 66, 30–35. https://doi.org/10.1016/j.future.2015.12.001
[12] Puri, D., & Bhushan, B. (2019). Enhancement of security and energy efficiency in
WSNs: Machine Learning to the rescue. 2019 International Conference on Computing,
Communication, and Intelligent Systems (ICCCIS). https://doi.org/10.1109/icccis48478.
2019.8974465
[13] Gulyani, V., Dhiman, T., & Bhushan, B. (2020). Introducing machine learning to wire-
less sensor networks. Deep Learning Strategies for Security Enhancement in Wireless
Sensor Networks Advances in Information Security, Privacy, and Ethics, 1–22. https://
doi.org/10.4018/978-1-7998-5068-7.ch001
[14] Ricci, F., Rokach, L., & Shapira, B. (2015). Recommender systems: Introduction and
challenges. Recommender Systems Handbook, 1–34. https://doi.org/10.1007/978-
1-4899-7637-6_1
[15] Bhushan, B., & Sahoo, G. (2019). ISFC-BLS (intelligent and secured fuzzy clustering
algorithm using balanced load sub-cluster formation) in WSN environment. Wireless
Personal Communications. https://doi.org/10.1007/s11277-019-06948-0
[16] Soni, S., & Bhushan, B. (2019). Use of Machine Learning algorithms for designing
efficient cyber security solutions. 2019 2nd International Conference on Intelligent
Computing, Instrumentation and Control Technologies (ICICICT). https://doi.org/
10.1109/icicict46008.2019.8993253
[17] Gaur, J., Goel, A. K., Rose, A., & Bhushan, B. (2019). Emerging Trends in
Machine Learning. 2019 2nd International Conference on Intelligent Computing,
Instrumentation and Control Technologies (ICICICT). https://doi.org/10.1109/
icicict46008.2019.8993192
274 Machine Learning and Deep Learning in Medical Data Analytics
[18] Erritali, M., Hssina, B., & Grota, A. (2021). Building recommendation systems using
the algorithms KNN and SVD. International Journal of Recent Contributions from
Engineering, Science & IT (IJES), 9(1), 71. https://doi.org/10.3991/ijes.v9i1.20569
[19] Chen, L., Chen, G., & Wang, F. (2015). Recommender systems based on user reviews:
The state of the art. User Modeling and User-Adapted Interaction, 25(2), 99–154.
https://doi.org/10.1007/s11257-015-9155-5
[20] Jindal, M., Gupta, J., & Bhushan, B. (2019). Machine learning methods for IoT and their
Future Applications. 2019 International Conference on Computing, Communication,
and Intelligent Systems (ICCCIS). https://doi.org/10.1109/icccis48478.2019.8974551
[21] George, T., & Merugu, S. (2005). A Scalable Collaborative Filtering Framework Based
on Co-Clustering. Fifth IEEE International Conference on Data Mining (ICDM’05),
625–28. https://doi.org/10.1109/ICDM.2005.14
[22] Sil, R., Roy, A., Bhushan, B., & Mazumdar, A. (2019). Artificial Intelligence and
Machine Learning based Legal Application: The State-of-the-Art and Future Research
Trends. 2019 International Conference on Computing, Communication, and Intelligent
Systems (ICCCIS). https://doi.org/10.1109/icccis48478.2019.8974479
[23] Zhang, S., Yao, L., Sun, A., & Tay, Y. (2019). Deep learning based recommender sys-
tem: A survey and new perspectives. ACM Computing Surveys, 52(1), 1–38. https://doi.
org/10.1145/3285029
[24] Montesinos-López, O. A., Montesinos-López, A., Crossa, J., Montesinos-López, J.
C., Mota-Sanchez, D., Estrada-González, F., Gillberg, J., Singh, R., Mondal, S., &
Juliana, P. (2018). Prediction of multiple-trait and multiple-environment genomic data
using recommender systems. G3 Genes|Genomes|Genetics, 8(1), 131–47. https://doi.
org/10.1534/g3.117.300309
[25] Goel, A., Bhushan, B., Tyagi, B., Garg, H., & Gautam, S. (2021). Blockchain and
machine learning: Background, integration challenges and application areas.
Emerging Technologies in Data Mining and Information Security, 295–304. https://
doi.org/10.1007/978-981-15-9774-9_29
[26] Lemire, D., & Maclachlan, A. (2008). Slope One Predictors for Online Rating-Based
Collaborative Filtering. ArXiv:Cs/0702144. http://arxiv.org/abs/cs/0702144
[27] Malik, A., Gautam, S., Abidin, S., & Bhushan, B. (2019). Blockchain Technology-
Future of IoT: Including Structure, Limitations and Various Possible Attacks. 2019
2nd International Conference on Intelligent Computing, Instrumentation and Control
Technologies (ICICICT). https://doi.org/10.1109/icicict46008.2019.8993144
[28] Koren, Y. (2008). Factorization meets the neighborhood. Proceeding of the 14th ACM
SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD
08. https://doi.org/10.1145/1401890.1401944
[29] Shani, G., & Gunawardana, A. (2011). Evaluating Recommendation Systems. In F.
Ricci, L. Rokach, B. Shapira, & P. B. Kantor (Eds), Recommender Systems Handbook
(pp. 257–297). Springer US. https://doi.org/10.1007/978-0-387-85820-3_8
Index
A CADx, 74
Cancer(s), 69, 77, 78
Abdominal, 152, 154, 160 Cancerous, 80
Abnormal, 75, 76, 78, 80, 82 Cardiac surgery, 53
Abnormalities, 75–78, 80, 81 Categorical, 165
Accuracy, 132, 134, 141, 145, 154, 156, 159, 161, CDI ID format, 11, 16–17
163, 166, 220–221, 225–226, 230–232 CDI subspace creation, 16
Activation function, 223, 228 Center for Biologics Evaluation and
Adam, 227–228 Research (CBER), 43
ADGNET, 192, 193 Center for Device and Radiological
AHP (analytic hierarchy process), 263 Health (CDRH), 43
AI-aided methods, 221 Center for Drug Evaluation and
Air pollution, 235–244 Research (CDER), 43
Algorithm, 67–68, 70, 74–77, 79 Cerebrospinal fluid (CSF), 184, 187–190, 192, 194
Algorithmic geometry, 247 CF (collaborative filtering), 262–265, 267–269, 271
Algorithms, 184, 187, 191, 196 Challenge(s), 66, 69, 74, 76, 81
Alzheimer’s disease, 185–186, 189, 191, 193, Challenging, 81–82
195, 197 Chaotic maps, 135
Alzheimer’s disease neuro-imaging initiative Chest X-ray, 220
(ADNI), 187, 189–192 Classification, 70, 74, 76, 77, 79–80, 132, 141,
Amyloid beta, 183, 188 154, 156, 158, 160–162, 165, 184–196,
Analysis, 154–156, 159–163, 165 220–227, 229, 231–233
Analytic hierarchy process, see AHP Classify, 77, 80
Anatomical, 77, 81 Clinic(s), 76, 78–79, 82
Ant Colony Optimization, 133 Clinical, 71, 76, 82
Anthologies, 71, 74 Clustering, 70–71, 78
Anthropogenic emissions, 239–240 CNN/CNNs, 75, 80–81
Application(s), 69–72, 74, 76–79, 82 Cobalt–chromium alloy, 50–51
Architecture, 154–156, 159, 160, 165, 226–227, 233 Co-clustering, 261–262, 267, 269, 271–272
Artificial neural network (ANN), 192, 194, 222 Common data interface, 3, 12
Augmentation, 226 Complications, 152–154
Auto-encoder, 159, 160 Compound tomography (CT), 184, 189
Automated anatomical labelling (AAL), 192, 194 Content filtering, 261–262, 264
Content-based recommendation system, 272
B Convergence ability, 135
Convolutional Neural Networks (CNN), 155, 184,
Batch size, 228 187–191, 232–233
BCI (Brain-computer interface), 23 Coronavirus 2 (SARS-CoV-2), 221
Bi-clustering, 269 Cosine, 267, 272
Binder jetting (BJ), 48 COVID-19, 77, 80, 220–233
Biogeography-Based Optimization, 136 Craniomaxillofacial surgery, 53–54
Biological system, 222 CRUD, 7
Brain-computer interface, see BCI
Brain-machine interface, see BCI
Brownian, 134, 136, 137, 138
D
Data science, 132
C Ddatabase(s), 68–69, 73–74
DCNN models, 220, 222, 225
C-section, 152 Decision(s), 76, 79, 82
CAD, 75–76, 78, 80 Decision-making process, 262–263
CADe, 74 Deep Belief Network (DBN), 160, 187, 188, 194
275
276 Index
Deep learning, 155, 156, 158–166, 184–196, 220, Fully convolutional network (FCN), 190,
222, 225, 232–233 192, 193
Dementia, 183, 185, 187, 189–194 Functional Magnetic Resonance Imaging,
DenseNet, 190, 191 see fMRI
Detection, 154–156, 160, 162, 165
Diabetes, 68, 78 G
Diabetic, 78
Diagnose, 74, 77 Gaussian distribution, 140
Diagnosis, 70, 74–75, 77, 82, 152–154, 163, 165, Geographic Information Systems, 247
184–196 buffer operation, 248
Diagnostic, 66, 78, 82 geo-pandemic data, 247
Differential evolution, 133, 148, 149 geoprocessing, 247, 249, 256–259
Digital imaging and communications in geospatial data, 247, 248
medicine (DICOM), 45 geospatial database, 247
Dimension, 70 geo-visualization, 1, 16, 19
Dimensionality, 79 interactive map, 257–259
Disease(s), 68, 70–71, 74–75, 77–79 spatial analysis, 247, 259
DNN, 186, 187, 192 spatial information, 247, 259
Doctors, 69, 71 spatial network, 252–256
Document(s), 70–71, 73–74 spatial query, 248–251
Dosage, 78 spatio-temporal networks, 246
Dropout, 224, 228 Geostrategy, 257, 259
Drug, 68–69, 77–78, 82 spatial decision-making, 257, 259
Generative Adversarial Network, 160
E Grey wolf optimization, 133
Gynecology, 153
Eddy formation, 136, 138
EEG (Electroencephalography), 25; see also H
Magnetoencephalography
Efficiency, 265, 268–269 Harmony search, 133, 147
EHR Standards, 8–10 Harris hawk optimization, 135
Electroencephalography, see EEG Health data, 261
Elite matrix, 137, 139 Health effects, 235–237, 243–244
Emotiv Epoc-EEG headset, 29 Health emergency, 236–238
Epidemic risk, 246, 247, 254, 257 Healthcare, 135
Epidemiological information, 248 Healthcare architecture, 11
Epochs, 228 Healthcare provider recommender system,
Evolutionary-based, 133 see HPRS
Exploration, 134, 136, 137, 138 High-dimensional, 132, 134
Extreme Learning Machine Auto Encoder, 188 HIS (health information systems), 261
HPRS (Healthcare provider recommender
F system), 261–266, 268, 270
Human health, 235–241
F1-score, 145, 146, 229 Human-based, 133
False-negative, 221 Hybrid approach, 134
Fast health interoperable resources (FHIR), 12 Hybrid filtering systems, 265
Feature(s), 67, 70, 74, 78–80, 82 Hyperparameters, 221, 228
Feature selection, 132, 135, 136, 146 Hyper-parameter tuning, 272
Federated system, 14 Hyper-parameters, 266, 271–272
Feed-forward, 68
Filter, 132, 223–225 I
Fine-tuning, 221, 231, 233
fMRI (Functional Magnetic Resonance Imaging), Image acquisition, 221
26; see also Magnetoencephalography Image recognition, 152, 155, 156, 159, 161,
Food industry, 135 164, 165
Fraud, 68 Imbalance, 82
Fruit fly optimization algorithms, 136 Information gain, 132
Index 277
Term(s), 70–74 U
Testing, 226, 228, 232
Texture mapping, 45–46 Urban data, 247
TF (term frequency), 73–74 pedestrian paths, 252
TF/IDF, 73–74 transport network, 247
Thoracic, 77 Uterus, 152–154
Three-dimensional printing Uterine surgery, 152
applications, 52–55
artificial intelligence (AI), 43 V
biometals, 48–51
history, 40 Validation, 228–229, 231–232
Internet of Things (IoT), 43 VGG, 224–225, 231
machine learning (ML), 43 Voronoï spatial diagrams, 246–255
technologies, 46–48 spatial relationships, 255
workflow, 43 spatial Voronoï accessibility, 253–256
Three-dimensional reconstruction brain tumor, 42
Time-consuming, 70, 74 W
Tissue(s), 74–82
Titanium alloy, 49 Water breaking, 152
TN (true negative), 75–76 Wavelet transform, 165
Tokenization, 72 Weakly supervised learning (WSL), 192, 193
Tomography, 74 Whale optimization algorithm, 132
Training, 75 WHO, 220
Training and test accuracy, 143 Womb, 152
Training data, 221, 224, 226
Transfer learning, 159, 160, 162 X
Transformations, 82
Tuberculosis, 77 XGBoost, 190, 192
TUC, 72
Tumor, 69, 75, 77 Z
Twin Pregnancy, 152
Typo(s), 73–74 Zinc, 51