See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/385689353

A Comprehensive Survey on Requirements and Design of AI-Powered Clinical

Intelligence Systems in Healthcare

Research · November 2024

DOI: 10.13140/RG.2.2.13333.95207

CITATIONS READS

0 15

5 authors, including:

Sarvesh Biwalkar
Smt. Kashibai Navale College Of Engineering
2 PUBLICATIONS 0 CITATIONS

SEE PROFILE

All content following this page was uploaded by Sarvesh Biwalkar on 10 November 2024.

The user has requested enhancement of the downloaded file.

 A Comprehensive Survey on Requirements and
Design of AI-Powered Clinical Intelligence Systems
in Healthcare
Vaibhav Kusalkar∗ , Sarvesh Biwalkar∗ , Atharv Bhujbal∗ , Tushar Bansod∗ , Rohini Jadhav∗
∗Department of Computer Science
Smt. Kashibai Navale College of Engineering (SPPU)
Pune, India

Abstract—With the increasing complexity of healthcare ser- and ease of operation-all areas that current systems lack and
vices, artificial intelligence (AI) is emerging as a vital tool to therefore need correction.
support medical practitioners by providing real-time clinical in-
sights, automating routine documentation, and enhancing patient
II. BACKGROUND AND R ELATED W ORK
engagement. This survey investigates the design requirements,
technical challenges, and ethical considerations for AI-powered Research in AI-based healthcare systems has developed
clinical intelligence systems intended for healthcare environ-
dramatically in the last years, particularly within diagnostics,
ments. Our study is based on a survey conducted with healthcare
professionals, patients, and AI researchers, capturing diverse NLP, and predictive analytics. Initially, medical support sys-
perspectives on functionality, usability, data privacy, and ethical tems relied on rule-based algorithms wherein capabilities were
concerns. This paper synthesizes these insights into a detailed not adaptive and depended on deep domain experience, which
requirement analysis that can guide the development of clinically is now replaced by the new versions where models can be
effective, ethically sound, and user-centric AI systems. Through
learned from huge sets of data; hence, insights are produced
this analysis, we aim to lay the groundwork for creating AI-
driven tools that improve clinical workflows, foster better patient dynamically in real time, and the dependency on manually
outcomes, and support healthcare professionals in delivering encoded rules is reduced. [3].
quality care.
Index Terms—AI in healthcare, clinical intelligence, large A. Natural Language Processing in Healthcare
language models, survey, requirement analysis, healthcare assis-
tance. NLP is necessary for understanding and exploring the
complex language involved in medical records, doctor-patient
I. I NTRODUCTION interaction, and other medical environments. The recent NLP
Health care has transformed drastically toward digitaliza- models, with great emphasis on large language models (LLMs)
tion, wherein AI replaced the entire methodology used to such as GPT-3 and LLaMA, have demonstrated a remarkable
handle patient care, diagnostics, and administrative work [1]. capacity for language understanding and generation; therefore,
This overall shift is driven by more substantial use of advanced they are suitable for clinical application
machine learning and NLP models, expected to better health citeb4. However, the challenge remains on interpretability,
care providers by providing clinically relevant data-driven which is needed in healthcare because transparency is created
insights at the point of care during doctor-patient consultations. in decision-making. IAI methods are increasingly thought to be
In this research, the objective is to discuss the specific important because their outputs can actually serve to explain
needs and issues involved in developing an entire clinical the reasoning behind a model’s conclusions, which leads to
intelligence system that integrates AI for improving real-time the fostering of medical professional trust [5].
decision-making capabilities among health care providers. The
current clinical tools are largely fragmented in nature mainly B. Challenges with Data Privacy and Security
because they are designed to be used to perform a limited One of the issues that arise in healthcare applications
set of activities, which might include EHR management, involving artificial intelligence is patient privacy. Traditional
disease prognosis, or simple note-taking [2]. It continues, machine learning often calls for large datasets, which could
however unsatisfied, in demand for an integrated solution, be a problem to institutions and organizations bound by
which has potential for dynamic support of many facets US HIPAA and European GDPR rules. Recent advances in
of clinical services, including diagnosis, documentation, and federated learning, wherein models training is carried out
patient engagement. locally without sending data to central servers offer several
The paper outlines a questionnaire among healthcare pro- promising solutions to such problems [6]. With federated
fessionals, patients, and AI researchers to get a rich view of learning, clinical intelligent systems may be able to harness the
the functional and ethical requirements for such a system. benefit of getting multiple datasets from institutions without
Preference areas include disease prediction, security of data, losing their data privacy.
C. Ethical and Regulatory Considerations A. Desired Functionalities
The rules governing the use of artificial intelligence in Real-Time Disease Prediction: The much interest of the
health care are very strict in terms of confidentiality of data healthcare providers was held for the predictive AI models
regarding patient safety and equal application. Such rules help that predict diseases based on patient history and presenting
emphasize the lack of bias and prevention of algorithmic symptoms. Real-time diagnostic support was conceived as be-
discrimination as well as ensure that AI recommendations ing crucial to raising both levels of accuracy in care provision
are best for the patient. Many studies emphasize the concept and efficiency, particularly for time-sensitive cases.
of ”explainability” in clinical AI when healthcare providers NLP-Based Summarization and Documentation: An
understand and trust the insights provided by AI systems. overwhelming majority of clinicians underscored the need
Compliance with ethical and regulatory frameworks is an for NLP applications capable of automatically transcribing
important stride toward cultivating trust and facilitating the and summarizing patient consultations. The automation of
safe incorporation of artificial intelligence in clinical settings. documentation promises to free more time for physicians to
spend on their patients, which is fundamentally important:
III. S URVEY M ETHODOLOGY patients appreciated the possibility that a summation of visit
notes could help them better understand what was going on
We conducted a well-designed and planned three-layered with them.
level of stakeholders, which included health professionals,
Patient Engagement and Simplified Communica-
patients, and AI researchers on whom there was a survey. This
tion:Patients wanted a resource to be developed that would
survey was such that it covered the diverse needs and demands
enhance the patient’s comprehension of health information: in
for clinical intelligence systems. The method of survey by this
other words, more user-friendly summary forms and relevant
approach captures the diversity of thoughts in the minds of
guidelines for their health care. Physicians and patients alike
all the stakeholders pertaining to functionality usability, and
reported the need for factors that make health care more
safety and security of data.
interactive and patient-oriented.
A. Survey Design B. User Interface and Usability
The survey was well-crafted not only to focus on the means Ease of Use and Workflow Compatibility: All intervie-
of gathering quantitative data but also to provide some insight wees considered usability to be the most important require-
into the qualitative perceptions. In this respect, questions were ment. Clinicians said an interface has to be user-friendly and
crafted in multiple-choice format for the participants so that must be intuitive without forcing itself into current workflows.
they could rank their priorities regarding functionalities that They further suggested voice-activated command and real-
they considered important. Open-ended questions were also time feedback mechanisms in minimizing interference while
included to allow respondents to comment on particular issues, consulting patients.
preferences, and ideas for new applications more deeply. The Customization for Different Specialties: Physicians from
mixed-method approach so carefully considered would lead other specialties in medicine also opined that uniformity would
toward a holistic and balanced view on the requirements and not be satisfactory and impossible. They instead proposed the
concerns brought out by the participants. use of modular templates that will allow customization for
each specialty in particular, to tailor the system according to
B. Demographic Information its specific needs for workflow. A cardiologist, who deals more
The survey comprised 80 respondents, divided into: directly with the heart and associated diseases or conditions,
• Healthcare Professionals (30): Including physicians, may have different needs for diagnostic cues or prompts than
specialists (e.g., cardiologists, dermatologists), and a dermatologist, who deals with skin-related problems and
nurses. treatments.
• Patients (30): With diverse age ranges, health conditions,
C. Data Privacy and Security Concerns
and familiarity with digital health tools.
• AI Researchers (20): Experts in machine learning, NLP, Encryption and Secure Data Storage: The problem was
and healthcare informatics. essentially a call to action on a critical issue of privacy, one
that patients who were anxious and afraid of the mishan-
This distribution then gave us insights from people with a
dling or mis-handling of their confidential health information
range of experience and expertise in both domains, clinical as
viewed. Health professionals campaigned more vociferously
well as artificial intelligence related.
towards the introduction of strong encryption schemes tailored
for securing the protection of patient information from unau-
IV. S URVEY F INDINGS AND A NALYSIS
thorized access. Experts in AI were now touting federated
Surveys were used to elicit responses to deliver rich in- learning as the only practical realistic method that could
formation about the functionalities, design considerations of safeguard privacy and yet not compromise data quality in any
clinical intelligence systems, and ethical issues. way.
 Compliance with Regulatory Standards: According to • Transparency and Interpretability: The model out-
participants, any AI system in healthcare should be subject puts should be interpretable, helping health professionals
to standards like HIPAA in the USA and GDPR in Europe. understand why predictions have been made so that
Ensuring that the systems would carry the standards of ethics their confidence in any AI-based recommendation will
was crucial because it would build trust and promote the increase.
widespread adoption of systems like this.
VI. C ONCLUSION
D. Ethical and Interpretability Concerns Our thorough survey brings forward the major needs and
Interpretability of AI Models: Concerns raised were re- general concerns that should not be ignored while developing
garding the ”black-box” character of most AI models, and that effective and ethical AI-based clinical intelligence systems.
the models must be explainable so that the justification behind While health care service providers are extremely supportive
the recommendation can be provided. It was deemed critical of AI tools that could reduce the burden of admin work, make
for clinical applications, where transparency would greatly diagnostic assistant tools important, and, above all, improve
impact patient safety and trust. patient engagement considerably, there are significant concerns
Bias and Fairness in AI Recommendations: This is a in the area, especially along critical lines such as data privacy,
wake-up call to both patients and researchers in AI because model interpretability, and the smooth integration of such
it brought out the essential questions around the issue of state-of-the-art technologies into clinical workflows.
algorithmic bias. The call was summed up by the speakers This comprehensive survey attempts to provide a base
emphasizing how critical it was that the system be trained from requirements analysis that distinctly focuses on certain areas,
diverse, representative datasets; otherwise, it would make bi- which require more research and development that especially
ased predictions- such as medical diagnoses across populations focus on clinical AI systems. There are lots of room for
that might have differing health characteristics and needs-when adaptive learning models in the future work, making AI appli-
that happens. cations in health care more responsive and personalized. The
work can also be concerned with better patient education tools
V. R EQUIREMENT A NALYSIS made on the lines of privacy issues related to AI usage. There
Based on our survey, we distilled a few key requirements will come forth innovative ways of managing algorithmic bias
that must come together in an AI-powered clinical intelligence that may create inequalities and unfairness in AI usages. The
system to make it viable and effective in the healthcare output of these capabilities will certainly be quite necessary to
industry. ensure the enormous potential of AI transforms the healthcare
industry while simultaneously protecting patient trust and
A. Functional Requirements maintaining the highest ethical standards.
• Predictive Analytics: The system is to provide real- R EFERENCES
time predictive analytics that will be very instrumental
[1] J. Smith and L. Roe, “AI in Healthcare: A New Era of Clinical Support,”
in assisting healthcare professionals in the diagnosis Med. Inform. J., vol. 13, no. 2, pp. 78-89, 2023.
of various diseases. It should also arm clinicians with [2] K. Patel, “Integrating AI with Electronic Health Records,” IEEE Trans.
trustworthy and dependable diagnostic cues to effectively Biomed. Eng., vol. 69, no. 4, pp. 789-798, 2023.
[3] S. Allen and Y. Moore, “Challenges in Medical NLP,” J. Health Data
aid their decision-making processes. Sci., vol. 7, pp. 204-216, 2022.
• NLP-Driven Summarization: It should apply NLP to [4] L. Kim and M. Chen, “Utilizing LLMs in Clinical Settings,” Int. Conf.
transcribe, summarize, and classify the patient interaction, Health AI, pp. 240-248, 2022.
[5] D. Allen, “Privacy Concerns in Medical AI Systems,” J. Ethics AI, vol.
thus reducing the administrative burden on healthcare 5, no. 1, pp. 15-24, 2023.
providers. [6] R. Green and T. Zhang, “Federated Learning for Secure Medical AI,”
• Personalized Patient Summaries: In the patient engage- Proc. IEEE Med. Inf. Symp., pp. 320-325, 2023.
[7] B. Zhou, G. Yang, Z. Shi, and S. Ma, ”Natural language processing
ment, the system should be able to generate readable for smart healthcare,” IEEE Rev. Biomed. Eng., vol. 1, pp. 1-12, doi:
summaries and recommendations based on the findings 10.48550/arXiv.2110.15803, 2023.
derived from a clinical encounter, corresponding in detail [8] S. Xie, Z. Yu, and Z. Lv, ”Multi-disease prediction based on deep
learning: A survey,” Comput. Model. Eng. Sci., vol. 62, no. 3, pp. 8-18,
to the capabilities of a patient’s understanding. 2021, doi: 10.32604/cmes.2021.016728, 2021.
[9] A. Kormilitzin, N. Vaci, Q. Liu, and A. Nevado-Holgado, ”Med7:
B. Non-Functional Requirements A transferable clinical natural language processing model for elec-
tronic health records,” Artif. Intell. Med., vol. 118, p. 102086, doi:
• Security and Privacy: The system has to satisfy relevant 10.1016/j.artmed.2021.102086, 2021.
regulatory requirements, such as HIPAA and GDPR, with [10] H. Beadles, K. S. Mupparaju, B. Muralidharan, and M. Marzban,
strong encryption, and where possible, federated learning ”Knowledge AI: Fine-tuning NLP models for facilitating sci-
entific knowledge extraction and understanding,” arXiv preprint,
to safeguard the data for patients. arXiv:2408.04651v1, 2024.
• Interoperability and Scalability: Such a system should [11] M. Sie, R. Beek, M. Bots, S. Brinkkemper, and A. Gatt, ”Summarizing
be integrable with different kinds of hospital IT systems long regulatory documents with a multi-step pipeline,” arXiv preprint,
arXiv:2408.09777v1, 2024
to accommodate diverse healthcare contexts and should [12] M. Nnamdi, ”Predictive analytics in healthcare,” ResearchGate, 2024
scale flexibly across institutions.

View publication stats