JMIR MEDICAL INFORMATICS Gray & Sockolow

Original Paper

Conceptual Models in Health Informatics Research: A Literature

Review and Suggestions for Development

Kathleen Gray1*, PhD; Paulina Sockolow2*, DrPH

1
Health and Biomedical Informatics Centre, University of Melbourne, Melbourne, Australia
2
Health Systems & Sciences Research, College of Nursing and Health Professions, Drexel University, Philadelphia, PA, United States
*
all authors contributed equally

Corresponding Author:
Kathleen Gray, PhD
Health and Biomedical Informatics Centre
University of Melbourne
Level 1
202 Berkeley Street
Melbourne, 3010
Australia
Phone: 61 3 8344 8936
Fax: 61 3 8344 8936
Email: [email protected]

Abstract
Background: Contributing to health informatics research means using conceptual models that are integrative and explain the
research in terms of the two broad domains of health science and information science. However, it can be hard for novice health
informatics researchers to find exemplars and guidelines in working with integrative conceptual models.
Objectives: The aim of this paper is to support the use of integrative conceptual models in research on information and
communication technologies in the health sector, and to encourage discussion of these conceptual models in scholarly forums.
Methods: A two-part method was used to summarize and structure ideas about how to work effectively with conceptual models
in health informatics research that included (1) a selective review and summary of the literature of conceptual models; and (2)
the construction of a step-by-step approach to developing a conceptual model.
Results: The seven-step methodology for developing conceptual models in health informatics research explained in this paper
involves (1) acknowledging the limitations of health science and information science conceptual models; (2) giving a rationale
for one’s choice of integrative conceptual model; (3) explicating a conceptual model verbally and graphically; (4) seeking feedback
about the conceptual model from stakeholders in both the health science and information science domains; (5) aligning a conceptual
model with an appropriate research plan; (6) adapting a conceptual model in response to new knowledge over time; and (7)
disseminating conceptual models in scholarly and scientific forums.
Conclusions: Making explicit the conceptual model that underpins a health informatics research project can contribute to
increasing the number of well-formed and strongly grounded health informatics research projects. This explication has distinct
benefits for researchers in training, research teams, and researchers and practitioners in information, health, and other disciplines.

(JMIR Med Inform 2016;4(1):e7) doi: 10.2196/medinform.5021

KEYWORDS
medical informatics; theoretical models; conceptual framework; conceptual model; design-based research; implementation
research; evaluation research; health informatics; research design; research training

http://medinform.jmir.org/2016/1/e7/ JMIR Med Inform 2016 | vol. 4 | iss. 1 | e7 | p. 1

(page number not for citation purposes)
XSL• FO
RenderX
 JMIR MEDICAL INFORMATICS Gray & Sockolow

a perceived clinical problem omitted any mention of consultation

Introduction with clinical experts [7]; and a description of a technical solution
Conceptualizing Research in Health Informatics to a clinical issue did not fit the clinical workflow [8]. If the
research does not capture the processes integral to both the
There is consensus that the discipline of health informatics is world of health science and the world of information science,
characterized by the integration of elements from many other valuable efforts are expended developing applications that do
fields of knowledge. The components of health informatics, not address the intended problem. Many innovations fail to
apart from the biomedical sciences, include computer science, achieve sustainability or other measures of success because “the
information science, decision science, statistics, cognitive current development of eHealth technology often disregards the
science, organizational theory, and others [1]. In essence, health interdependencies between technology, human characteristics,
informatics is “cross-training” between broadly defined and the socioeconomic environment, resulting in technology
information sciences and health sciences [2]. that has a low impact in health care practices” [9]. Evaluation
Ideally, research rests on “methodologies that capture the research too may fall short of offering key insights. For example,
processes integral to applications, the users and the world in researchers who evaluated an electronic health record
which the users function” [3]. However, a growing number of implementation using the Delone and McLean framework for
researchers who do not identify themselves as health evaluating generic information systems success recognized that
informaticians are now doing research into the design, they would have done better to develop a more health-specific
implementation, and evaluation of information and approach to evaluation [10], and indeed other researchers had
communication technologies in the health sector. This growth customized Delone and McLean for evaluating health-specific
is fuelled by new technologies that reduce the health information systems [11].
professionals’ barriers to application development, and also by Recognizing Conceptual Models in Health Informatics
the growing market for consumer technologies that are not
Research
subject to medical device regulations. For example, the current
emergence of apps for mobile phones and the increased ease of Researchers new to the field of health informatics need to learn
programming these apps is said to "enable busy clinicians to how to work with its conceptual models. However, this may
develop simple mobile Web-based apps for academic, not be taught formally, and exemplars and guidance in the
educational, and research purposes, without any prior knowledge literature are sporadic and scattered. Possibly due to publication
of programming" [4]. The ensuing research appears in the word limits, much of the published health informatics research
journal and conference literature of a variety of fields including conveys an absence of discussion, even a lack of awareness,
clinical specialties, health policy, information management, and about the importance of conceptual models. In addition, papers
software engineering, to name a few. This paper is aimed at may mention a conceptual model approach without specifying
researchers still in training, or practitioners new to the field who how it came into existence.
wish to align their work more strongly with the discipline of Some descriptions of the development of conceptual models in
health informatics. specific health informatics research studies are available. Gordon
Taking a disciplined approach to health informatics research et al described how models of clinical guideline knowledge had
means operating across the component domains of expertise by to be integrated with models of health care activities and
using integrative conceptual models. A working definition of processes in a conceptual model approach for automating
a conceptual model is that it is an explanation of the researcher’s distribution of clinical guidelines [12], and Ruland and Bakken
thinking about the key constituents of the research problem, enumerated the components of a conceptual model to support
and why the whole problem is greater than the sum of its parts inclusion of patient preferences in clinical decision making and
because of the way these interconnect and interact. In any field underscored the importance of incorporating knowledge from
of knowledge, using a conceptual model to describe something four domains [13]. In addition, Kaplan and Shaw compared a
about a subset or an aspect of the domain has value; that is, a variety of ways that evaluation researchers have conceptualized
conceptual model makes explicit the intended meaning of terms the complex social and institutional dynamics of health
and concepts used and avoids ambiguity and misinterpretation. information technology implementations [14], and Yusof et al
The terms conceptual framework and conceptual model are used [15] explained how theories from information science and
interchangeably in the literature [5], and we use the latter evidence from health science informed the structure of a human-,
throughout this paper. Health informatics conceptual models organization-, and technology-fit evaluation framework for
that connect the knowledge and thus explain the research in the health information systems. There is no single right or wrong
language of two broad domains, health science and information conceptual model that brings order to a set of ideas about a
science, can help to ensure that research is effective and has health informatics problem. The exercise of making the
impact. conceptual model explicit in a study, in words and/or figures,
is critical to clarify what is known and to identify what is in
Too often, research on information and communication question or not known, from the perspective of the researchers.
technologies in the health sector appears to miss either the health Through the process of debating the merits of alternative
problem or the information technology problem. Some examples conceptual models, their explanatory power, completeness, and
of missing the health problem are: a review of 55 heart failure other aspects about how well they represent the research
risk computational models noted that few had been implemented objectives, new theories are formed.
in clinical practice [6]; a description of a technical solution to

http://medinform.jmir.org/2016/1/e7/ JMIR Med Inform 2016 | vol. 4 | iss. 1 | e7 | p. 2

(page number not for citation purposes)
XSL• FO
RenderX
 JMIR MEDICAL INFORMATICS Gray & Sockolow

This paper aims to promote more explicit use of health of these peer-reviewed publications added external validity to
informatics conceptual models in research on information and our method.
communication technologies in the health sector, and encourage
discussion of these conceptual models in scholarly forums. Results
Learning how to develop and apply integrative conceptual
models is an educational issue for researchers in training, and The process of critical review and reflection led us to a
so too for those who train them and those who review their consensus that supported a seven-step approach to developing
work. Using conceptual models has research significance for a conceptual model for health informatics research. In this
building the discipline of health informatics and benefits for section we offer these steps and examples from the literature as
many stakeholders in health informatics research. a guide to the novice health informatics researcher. The
methodology for working with conceptual models in health
Methods informatics involves (1) acknowledging the limitations of health
science and information science conceptual models; (2) giving
A two-part method was used to structure and illustrate ideas of a rationale for one’s choice of an integrative conceptual model;
how to work effectively with conceptual models in health (3) explicating a conceptual model verbally and graphically;
informatics research. First, we conducted a search of the (4) seeking feedback about a conceptual model from
literature of conceptual models, and then we used a qualitative stakeholders in both the health science and information science
research process to formulate a step-by-step approach to domains; (5) aligning a conceptual model with an appropriate
developing a conceptual model. research plan; (6) adapting a conceptual model in response to
new knowledge over time; and (7) disseminating conceptual
We looked for papers published up to 2014 that described the
models in scholarly and scientific forums.
development of conceptual models in health information science
and technology research in PubMed, IEEE Xplore, ACM Digital Acknowledge the Conceptual Models of Contributing
Library, Scopus, and Web of Knowledge. First the Medical Domains
Subject Heading (MeSH) "models, theoretical" paired with
It is important to acknowledge that both health science and
"medical informatics" were used. Then search terms were
information science use a variety of conceptual models to
widened to include "research design" (especially where there
represent entities and relationships in their respective domains;
was discussion of why a design was chosen). An additional
however, these fields of knowledge are differentiated by their
search looked for possible pairings of "design science" or
approaches to conceptualizing problems. A defining
"design-based research", "implementation", and "evaluation"
characteristic of any given health informatics research problem
with health information systems and technology and with
is how it responds to the challenge to acknowledge the
electronic health (e-health). Selection of a cross-section of full
applicability and also the inadequacy of conceptual models from
papers that made substantial mention of conceptual models was
both health science and information science. That is, you should
based on reading abstracts and also on mining reference lists
be able to describe some parts of the research question using
from selected papers for further examples.
concepts that appropriately represent the problem to the separate
Critical reflection was used to formulate a step-by-step approach audiences or stakeholders from each domain. You must also go
to developing a conceptual model. We examined the beyond this, to frame the problem in a conceptual model that
assumptions embedded in our experiences, associated them with transcends and integrates these domains.
a range of different factors, re-evaluated them using external
In the information sciences, a conceptual model may also be
reference points, and re-worked our ideas and practices [16].
referred to as a domain model. Furthermore, conceptual
Specifically, we drew on our separate experiences working in
modeling should not be confused with other modeling
multidisciplinary health information technology research teams
disciplines, such as data modeling, logic modeling, or physical
internationally over five years. We reconsidered the bases of
modeling. Many different conceptual models may be used for
our expertise as researchers and also as reviewers, supervisors,
demonstration, optimization, construction, simulation and other
advisors, and examiners of research in health informatics. We
activities in the application domain [19]. In the field of
analyzed the literature we had retrieved, looking at ways authors
information science, recent examples in the literature can be
named, explicated, and sequenced key components in the
found that discuss specific conceptual models in detail [20-22].
development of a conceptual model. We agreed on specific
The complexity of health is a major reason why health
steps in development of a conceptual model and went back to
informatics is not just another application domain in information
the literature and to our experience repeatedly for examples.
science [23]. In the health sciences, an introduction to a range
Using this method, we produced a set of suggestions, which of conceptual models for defining and conceptualizing health
remains untested in terms of its technical validity and argues that simplistic definitions of health lead to equally
sufficiency. Nevertheless, after formulating these suggestions, simplistic measures of health, health outcomes, and quality of
we found that teaching novice researchers to use conceptual care [24]. In the field of health, recent examples which explore
models in a step-wise manner is recognized as effective by conceptual models can be found [25-28].
academics in other fields [17]. We also found that a similar idea
had appeared in the literature of a different field of health
research, namely health program evaluation [18]. The existence

http://medinform.jmir.org/2016/1/e7/ JMIR Med Inform 2016 | vol. 4 | iss. 1 | e7 | p. 3

(page number not for citation purposes)
XSL• FO
RenderX
 JMIR MEDICAL INFORMATICS Gray & Sockolow

Review Conceptual Models Already Used in Health who does not have access to the graphic. It should be clear why
Informatics you have chosen the visual representations you are using to
represent the key entities and relationships that your problem
The next step is to review the health informatics literature for
involves. For example, a study of unfulfilled and unrecognized
conceptual models that can be either applied or adapted to the
or hidden health information needs explained the research
research question. A clearly described search strategy for
framework using the graphic image and written analogy of an
reviewing this literature is an indicator of the rigor you need to
iceberg [38].
apply to the process of conceptualizing the problem [29,30].
The starting point is to compare various overarching conceptual Your first attempt to represent the entities and relationships in
models based on the power each may have to fully account for your research problem using a Venn diagram, matrix, or
all the elements of the problem as you have chosen to define it. flowchart may not suffice to give adequate detail or insight into
The introduction section of this paper offered examples from the problem space. The visualization of knowledge is a field of
the 1997-2008 decade, and more recent examples can be found study in its own right, and you may find it helpful to consult
in references [9,31-34]. It is essential to justify your choice of general works [39,40]. Although the visualization of data has
a pre-existing conceptual model as it relates to the key features become an active health informatics research area [41,42], the
of your research problem. Alternatively, you may conclude that visualization of concepts is a very different order of activity.
no pre-existing, cross-cutting conceptual models are adequate
to explore this problem. This judgment also requires Seek Critical Feedback on the Conceptual Model From
justification, and it opens the way for thought experiments about Multiple Perspectives
options for combining and elaborating the particular conceptual Your conceptual model may appear completely sensible from
models that you have previously acknowledged. In one example, your point of view. However, at this point in development, you
an investigation of a novel and under-researched technology in should be thinking of your conceptual model as a
health care was able to proceed by integrating concepts of communication tool. This tool should help you engage other
evidence-based treatment (from health science) and technology people who are direct and indirect stakeholders in your research.
affordances (from information science) into a new conceptual Thus, stakeholders with other perspectives need to test its
model of therapeutic affordances of social media [35]. communicative power to assure you that it is making sense of
the problem.
Schematize the Chosen Conceptual Model of the
Research Problem A conceptual model in health informatics research should pass
the "goodness of fit" test for domain experts in both information
Making a schematic representation of your chosen conceptual
science and health science. This is an informal but important
model captures and refines the thought processes behind your
step where you seek critique of your conceptual model from
choice. A visual artifact in the form of a diagram, motif, map
others who are at a distance from your research question. Their
or other type of figure (eg, a foil or straw man) can be used for
reactions allow you to refine and strengthen the supporting
reacting to and testing the thinking about a problem, to guide
arguments for your conceptual model as needed. You are
collaboration, and to assess research progress and outcomes.
looking for toughness; now is the time to establish whether your
An example of schematizing a health informatics conceptual
conceptual model will stand up to scrutiny and be persuasive
model to represent the relationships among health information
in a room full of either clinical specialists or computer scientists.
technology characteristics is the Health Information Technology
Reference-based Evaluation Framework [36]. A second example For a researcher in training, often the best way to ensure access
illustrates the temporal dimensions of five measures of health to this kind of feedback is to make sure that the supervisory or
information system adoption in the Clinical Adoption Meta advisory committee comprises people who bring information
Model [37]. science and health science perspectives. Alternatively, the
researcher needs to find suitable critical colleagues by tapping
The challenge for every health informatics researcher is to think
into networks of clinicians and researchers within the health
deeply about an apt way to visualize the specific problem space.
and biomedicine community, and into networks of industry
Part of the contribution that your research makes to the field is
experts and researchers within the information science and
determined by the originality you show in this step. Questions
technology community. Organizational mentoring programs,
to consider include: How does the visualization of your
professional associations, or scientific societies can provide
conceptual model position the information science and the health
access to networks appropriate to the study.
science elements of the problem (eg, side-by-side versus above
and below)? Does the level of detail match the intended level If the research involves human participants, another method of
of investigation (eg, evaluating the impact of a policy may need seeking feedback on a conceptual model is to engage actively
to represent issues at a macro level or assessing software with prospective participants including patients and consumers.
functionality may need a finer grained picture)? Does it leave For example, Belanger et al. advocated involving patients from
too much to be inferred (eg, not indicating the direction in which the inception of a research project centered on electronic health
a multi-part image should be "read") or use conventions in an records [43].
unconventional way (eg, using the colors red for "go" and green
for "stop")?
The schema also needs to be interpreted in words that explain
it to someone who is unfamiliar with the research problem or
http://medinform.jmir.org/2016/1/e7/ JMIR Med Inform 2016 | vol. 4 | iss. 1 | e7 | p. 4
(page number not for citation purposes)
XSL• FO
RenderX
 JMIR MEDICAL INFORMATICS Gray & Sockolow

Allow the Conceptual Model to Influence the Research are found in studies of clinical information system success [47],
Design and open access publishing use [48].
There must be a close connection between your conceptual Disseminate the Conceptual Model
model and research design. In a different field of health research, The final step is to disseminate your conceptual model through
the alignment was expressed as follows: formal presentation and publication in scholarly and scientific
The CF (conceptual framework) provided the basis conferences and journals. It is important to give over the time
for decisions about the development of a and space to include the conceptual model among the
mixed-method research design and data collection publications that come out of your research, so that it is captured
measures. For each construct of interest, we and available for others to refer to and build on. You can
determined the most suitable approach to collecting structure a paper or presentation about your conceptual model
information. We arranged the constructs into two using steps one through six in this paper; or examples cited
categories: those where validated quantitative throughout this paper offer many other successful models for
measures were available (...); and second, those that publishing a description of your conceptual model.
were best suited for exploratory/qualitative methods. When you describe your work on a conceptual model you make
[18] a contribution to the theory of health informatics, because your
This is a good example of allowing the conceptual model to conceptual model adds a new theoretical representation of the
influence the research design, including the selection of the entities and relationships in a problem space. In disseminating
research procedures and outcome measures (for quantitative your conceptual model you address an acknowledged need in
research) or themes (for qualitative research). Feasibility factors the field. The absence of theory in health informatics diminishes
also influence research design, for example, access to sites for its status as a field of knowledge [49]; whereas, enunciating
field studies, available funding and human resources, the your conceptual model advances the field [3].
exigencies of ethics approval, and time constraints. However,
if any of these factors undermine the conceptual model to any There are many diverse forums where it is possible to
great extent, then the research question and conceptual model disseminate your conceptual model. This is one advantage of
need to be reformulated. the diffusion of published research in health informatics. Apart
from those already mentioned in this paper, three further
Revisit the Conceptual Model in Light of the Research examples of particular health informatics conceptual models
Findings published in recent years illustrate that it may be appropriate to
Your conceptual model is what guides the approach you take place your work in conferences and journals in information
to explore a real-world research problem. The corollary is that science [50], in health science [51], and in mainstream health
through exploring this problem, inevitably you are testing your informatics [52].
conceptual model for its usefulness. While you are analyzing
and discussing findings from your research project, you need Discussion
to reflect on whether and how these findings support your initial
conceptual model, identify where the conceptual model may Benefits of Developing and Using Conceptual Models
need to be modified, and consider broader circumstances where This paper outlines current challenges in developing a
the conceptual model may prove useful. This is an iterative and conceptual model that integrates the information science aspects
experiential process that stretches your thinking as your and the health science aspects of a health sector information
conceptual model "evolves and develops until it becomes refined and communication technology problem. The effort to overcome
and burnished, to emerge as a robust outcome of the research" these challenges can yield important benefits that are not only
[44]. theoretical but also practical. For the individual researcher, the
conceptual model provides a persistent reminder of the defined
In health informatics research, the gap between health sciences entities and relationships that give shape and direction to their
and information sciences that must be bridged by this step is research plan and the interpretation of their findings. For teams
substantial. comprising various disciplines in a collaborative research
In more mature fields such as medicine, it is standard project, the conceptual model helps to share the related
practice, even mandatory, to conduct empirical vocabulary and reach agreement on the underlying constructs.
research to evaluate the efficacy of proposed new For diverse researchers with different questions who are
practices prior to advocating their use (...). However formulating their own approach or discussing their own findings,
in IS design research, it is often sufficient for the conceptual model facilitates comparisons and
researchers to argue on logical or theoretical grounds cross-pollination of ideas. Beyond the research community,
that their approach is effective. [45] there are benefits from giving closer attention to health
How your health informatics conceptual model is tested depends informatics conceptual models for three professional practice
on the research design appropriate to your study. In one communities : health informatics practitioners, other health
example, thematic analysis of interview data is the basis for practitioners, and other IT practitioners.
revising an initial conceptual model of health professionals’ The community of health informatics professionals can use
mobile health use [46]. Examples where structured survey conceptual models more overtly to improve practice. By eliciting
methods are the basis for revisions to initial conceptual models organizational input into conceptualizing implementation issues
http://medinform.jmir.org/2016/1/e7/ JMIR Med Inform 2016 | vol. 4 | iss. 1 | e7 | p. 5
(page number not for citation purposes)
XSL• FO
RenderX
 JMIR MEDICAL INFORMATICS Gray & Sockolow

[13], they may be able to communicate and surmount notable Information professionals who are not health scientists can use
problems of sustaining health information and communication integrative conceptual models to access important opportunities
technology applications [30]. For instance, making deliberate for innovation in the health sector, to achieve advances on health
use of a socio-technical conceptual model can help to anticipate informatics grand challenges, and to build prized expertise and
unintended consequences before these emerge during system strong partnerships. Because information technology in health
implementation [53]. Building health informatics conceptual is developing massively and is being adopted rapidly, the health
modeling skills can enhance training and professional sector offers numerous opportunities to apply emerging
development for roles such as chief information officers in solutions. When you aim to transfer into the health sector the
health organizations, research and development managers in conceptual models that underlie solutions from non-health
health technology companies, and health informatics experts in sectors, acknowledging and articulating these enables you to
large consulting firms. For example, health informaticians in reflect on how they can be expected to have positive impacts
such roles may benefit from working with conceptual models and be sustainable in health [57]. Coming to terms with the way
to explain and deliver the business value of information information science conceptual models relate to the key health
technology in healthcare [54]. domain conceptual models where you are applying solutions
can make you more effective in solving higher order problems
Health professionals who are not information scientists can use
[58]. If you are able to relate to and communicate within the
conceptual models to facilitate inter-professional practice, exert
health sector on this conceptual level [59,60], you will have
collegial influence, and advance professional ethics in their
expertise that is critical for successful collaboration with highly
work in an increasingly technological sector. For the individual
educated and committed health professionals to bring about
health professional, making explicit the conceptual models you
technological transformation.
use in your professional practice enables you to integrate these
more strongly into planning for new work practices during Conclusion
periods of technology change and adoption [55]. In working Our aim is to provide a representative selection of examples to
with colleagues in your profession, your ability to communicate accompany our suggestions for researchers who are new to the
conceptual models that frame health information and health informatics discipline. It is important for health
communication technology projects can position you as a leader informatics researchers to elucidate their hard-won conceptual
and facilitator in the design and oversight of such projects [36]. modeling experience, even while recognizing that no conceptual
In addition, the way each health profession expresses its ethical model can ever be static or definitive. Sharing and comparing
commitment to the safety and quality of care can be subtly these foundations of knowledge can support good practice in
different. When you clarify the conceptual models that you research training (and in the commissioning, management, and
apply in health information technology projects that involve review of research), and thereby can contribute to the evolution
your own patients and/or clients in the settings where you of better-formed and more strongly grounded health informatics
provide care for them [56], you express more deeply the way research. Making explicit the defined entities and relationships
you think about their needs and how technology interventions in a health informatics research project facilitates deep
might address these. In doing so, you contribute to broadening engagement with cross-cutting problems, offers a way for
the discourse within your profession about the ethical practice researchers to be more effective, and enables research to have
of clinical informatics. greater impact.

Acknowledgments
Ms Cecily Gilbert, Health and Biomedical Informatics Centre, University of Melbourne, did the initial literature review for this
paper. JMIR reviewers’ feedback helped us greatly to sharpen the focus of this paper (reviewer names are acknowledged at the
end of this paper).

Authors' Contributions
KG conceived of the idea for the paper. KG and PS equally contributed to the structure and content of the paper and analyzed
and added to the literature identified in the initial review. KG prepared the manuscript for submission and prepared responses to
reviewers. PS approved the final version for submission.

Conflicts of Interest
None declared.

References
1. Kulikowski CA, Shortliffe EH, Currie LM, Elkin PL, Hunter LE, Johnson TR, et al. AMIA Board white paper: definition
of biomedical informatics and specification of core competencies for graduate education in the discipline. J Am Med Inform
Assoc 2012;19(6):931-938 [FREE Full text] [doi: 10.1136/amiajnl-2012-001053] [Medline: 22683918]
2. Friedman CP. What informatics is and isn't. J Am Med Inform Assoc 2013;20(2):224-226 [FREE Full text] [doi:
10.1136/amiajnl-2012-001206] [Medline: 23059730]

http://medinform.jmir.org/2016/1/e7/ JMIR Med Inform 2016 | vol. 4 | iss. 1 | e7 | p. 6

(page number not for citation purposes)
XSL• FO
RenderX
 JMIR MEDICAL INFORMATICS Gray & Sockolow

3. Currie LM. Evaluation frameworks for nursing informatics. Int J Med Inform 2005 Dec;74(11-12):908-916. [doi:
10.1016/j.ijmedinf.2005.07.007] [Medline: 16099711]
4. Zhang MW, Tsang T, Cheow E, Ho CS, Yeong NB, Ho RC. Enabling psychiatrists to be mobile phone app developers:
insights into app development methodologies. JMIR Mhealth Uhealth 2014;2(4):e53 [FREE Full text] [doi:
10.2196/mhealth.3425] [Medline: 25486985]
5. Green HE. Use of theoretical and conceptual frameworks in qualitative research. Nurse Res 2014 Jul;21(6):34-38. [doi:
10.7748/nr.21.6.34.e1252] [Medline: 25059086]
6. de Vries JJG, Geleijnse G, Tesanovic A, van de Ven ART. Heart failure risk models and their readiness for clinical practice.
2013 Presented at: IEEE International Conference on Healthcare Informatics (ICHI ); 9-11 September 2013; Philadelphia,
PA p. 239-247. [doi: 10.1109/ICHI.2013.26]
7. Jiang CA, Leong TY, Poh KL. PGMC: a framework for probabilistic graphic model combination. AMIA Annu Symp Proc
2005:370-374 [FREE Full text] [Medline: 16779064]
8. Michel G, Marcy T, Shiffman RN. A wireless, handheld decision support system to promote smoking cessation in primary
care. AMIA Annu Symp Proc 2005:530-534 [FREE Full text] [Medline: 16779096]
9. van Gemert-Pijnen JE, Nijland N, van LM, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve
the uptake and impact of eHealth technologies. J Med Internet Res 2011;13(4):e111 [FREE Full text] [doi: 10.2196/jmir.1672]
[Medline: 22155738]
10. Bossen C, Jensen LG, Udsen FW. Evaluation of a comprehensive EHR based on the DeLone and McLean model for IS
success: approach, results, and success factors. Int J Med Inform 2013 Oct;82(10):940-953. [doi:
10.1016/j.ijmedinf.2013.05.010] [Medline: 23827768]
11. Yusof MM, Kuljis J, Papazafeiropoulou A, Stergioulas LK. An evaluation framework for Health Information Systems:
human, organization and technology-fit factors (HOT-fit). Int J Med Inform 2008 Jun;77(6):386-398. [doi:
10.1016/j.ijmedinf.2007.08.011] [Medline: 17964851]
12. Gordon C, Herbert I, Johnson P, Nicklin P, Pitty D, Reeves P. Telematics for clinical guidelines: a conceptual modelling
approach. Stud Health Technol Inform 1997;43 Pt A:314-318. [Medline: 10179562]
13. Ruland CM, Bakken S. Developing, implementing, and evaluating decision support systems for shared decision making
in patient care: a conceptual model and case illustration. J Biomed Inform 2002;35(5-6):313-321. [Medline: 12968780]
14. Kaplan B, Shaw N. Future directions in evaluation research: people, organizational, and social issues. Methods Inf Med
2004;43(3):215-231. [doi: 10.1267/METH04030215] [Medline: 15227551]
15. Yusof MM, Papazafeiropoulou A, Paul RJ, Stergioulas LK. Investigating evaluation frameworks for health information
systems. Int J Med Inform 2008 Jun;77(6):377-385. [doi: 10.1016/j.ijmedinf.2007.08.004] [Medline: 17904898]
16. Fook J, White S, Gardner F. Critical reflection: a review of contemporary literature and understandings. In: White S, Fook
J, Gardner F, editors. Critical Reflection in Health and Social Care. Maidenhead, Berkshire, England: Open University
Press; 2006.
17. Shields P, Rangarajan N. A Playbook for Research Methods: Integrating Conceptual Frameworks and Project Management.
Stillwater, OK: New Forums Press; 2013.
18. Helitzer DL, Sussman AL, Hoffman RM, Getrich CM, Warner TD, Rhyne RL. Along the way to developing a theory of
the program: a re-examination of the conceptual framework as an organizing strategy. Eval Program Plann 2014
Aug;45:157-163. [doi: 10.1016/j.evalprogplan.2014.04.005] [Medline: 24836998]
19. Thalheim B. The theory of conceptual models, the theory of conceptual modelling and foundations of conceptual modelling.
In: Embley DW, Thalheim B, editors. Handbook of Conceptual Modeling: Theory, Practice, and Research Challenges.
Berlin: Springer; 2011:547-580.
20. Poelmans J, Dedene G, Snoeck M, Viaene S. Using formal concept analysis for the verification of process - data matrices
in conceptual domain models. In: Proceedings of IASTED International Conference on Software Engineering. Calgary,
Canada: Acta Press; 2010 Presented at: IASTED International Conference on Software Engineering; 16-18 February 2010;
Innsbruck, Austria p. 79-86.
21. Tamzalit D, Schätz B, Pierantonio A, Deridder D. Introduction to the SoSyM theme issue on models and evolution. Softw
Syst Model 2013 Apr 14;13(2):621-623. [doi: 10.1007/s10270-013-0338-4]
22. Thomas O, vom Brocke J. A value-driven approach to the design of service-oriented information systems—making use of
conceptual models. Inf Syst E-Bus Manage 2009 Feb 27;8(1):67-97. [doi: 10.1007/s10257-009-0110-z]
23. Kim J. Complex interventions in healthcare and health informatics: a scoping review. Stud Health Technol Inform
2013;183:263-268. [doi: 10.3233/978-1-61499-203-5-263]
24. Larson JS. The conceptualization of health. Med Care Res Rev 1999 Jun;56(2):123-136. [Medline: 10373720]
25. Aluttis C, den Broucke SV, Chiotan C, Costongs C, Michelsen K, Brand H. Public health and health promotion capacity
at national and regional level: a review of conceptual frameworks. J Public Health Res 2014 Mar 26;3(1):199 [FREE Full
text] [doi: 10.4081/jphr.2014.199] [Medline: 25170508]
26. Banzi R, Moja L, Pistotti V, Facchini A, Liberati A. Conceptual frameworks and empirical approaches used to assess the
impact of health research: an overview of reviews. Health Res Policy Syst 2011;9:26 [FREE Full text] [doi:
10.1186/1478-4505-9-26] [Medline: 21702930]

http://medinform.jmir.org/2016/1/e7/ JMIR Med Inform 2016 | vol. 4 | iss. 1 | e7 | p. 7

(page number not for citation purposes)
XSL• FO
RenderX
 JMIR MEDICAL INFORMATICS Gray & Sockolow

27. Courtney LG, Caltabiano NJ, Caltabiano ML. Developing conceptual models of psychosocial wellbeing: pursuing better
health outcomes. In: Caltabiano ML, Ricciardelli L, editors. Applied topics in health psychology. Somerset, NJ: Wiley;
2013:112-133.
28. Marmor T, Wendt C. Conceptual frameworks for comparing healthcare politics and policy. Health Policy 2012
Sep;107(1):11-20 [FREE Full text] [doi: 10.1016/j.healthpol.2012.06.003] [Medline: 22763200]
29. Kruse CS, DeShazo J, Kim F, Fulton L. Factors associated with adoption of health information technology: a conceptual
model based on a systematic review. JMIR Med Inform 2014;2(1):e9 [FREE Full text] [doi: 10.2196/medinform.3106]
[Medline: 25599673]
30. Cresswell K, Sheikh A. Organizational issues in the implementation and adoption of health information technology
innovations: an interpretative review. Int J Med Inform 2013 May;82(5):e73-e86. [doi: 10.1016/j.ijmedinf.2012.10.007]
[Medline: 23146626]
31. De Marco M, Ricciardi F, Vom Brocke J. Strategies and solutions in eHealth: a literature review. In: Wickramasinghe N,
Bali R, Suomi R, Kim S, editors. Critical Issues for the Development of Sustainable e-Health Solutions. Healthcare Delivery
in the Information Age. New York: Springer; 2012:135-147.
32. Mohr DC, Schueller SM, Montague E, Burns MN, Rashidi P. The behavioral intervention technology model: an integrated
conceptual and technological framework for eHealth and mHealth interventions. J Med Internet Res 2014;16(6):e146 [FREE
Full text] [doi: 10.2196/jmir.3077] [Medline: 24905070]
33. Morrison LG, Yardley L, Powell J, Michie S. What design features are used in effective e-health interventions? A review
using techniques from Critical Interpretive Synthesis. Telemed J E Health 2012 Mar;18(2):137-144. [doi:
10.1089/tmj.2011.0062] [Medline: 22381060]
34. Rippen HE, Pan EC, Russell C, Byrne CM, Swift EK. Organizational framework for health information technology. Int J
Med Inform 2013 Apr;82(4):e1-13. [doi: 10.1016/j.ijmedinf.2012.01.012] [Medline: 22377094]
35. Merolli M, Gray K, Martin-Sanchez F. Developing a framework to generate evidence of health outcomes from social media
use in chronic disease management. Med 2.0 2013;2(2):e3 [FREE Full text] [doi: 10.2196/med20.2717] [Medline: 25075238]
36. Sockolow PS, Crawford PR, Lehmann HP. Health services research evaluation principles. Methods Inf Med 2012 Feb
07;51(2):122-130. [doi: 10.3414/me10-01-0066]
37. Price M, Lau F. The clinical adoption meta-model: a temporal meta-model describing the clinical adoption of health
information systems. BMC Med Inform Decis Mak 2014;14:43 [FREE Full text] [doi: 10.1186/1472-6947-14-43] [Medline:
24884588]
38. Alzougool B, Chang S, Gray K. Towards a comprehensive understanding of health information needs. Electronic Journal
of Health Informatics 2008;3(2):e15 [FREE Full text]
39. Eppler M. What is an effective knowledge visualization? Insights from a review of seminal concepts. In: Marchese FT,
Banissi E, editors. Knowledge Visualization Currents: From Text to Art to Culture. London: Springer-Verlag; 2013:3-12.
40. Snyder J. Visual representation of information as communicative practice. J Assn Inf Sci Tec 2014 Apr 21;65(11):2233-2247.
[doi: 10.1002/asi.23103]
41. West VL, Borland D, Hammond WE. Innovative information visualization of electronic health record data: a systematic
review. J Am Med Inform Assoc 2015 Mar;22(2):330-339 [FREE Full text] [doi: 10.1136/amiajnl-2014-002955] [Medline:
25336597]
42. Zhang Z, Wang B, Ahmed F, Ramakrishnan IV, Zhao R, Viccellio A, et al. The five Ws for information visualization with
application to healthcare informatics. IEEE Trans Vis Comput Graph 2013 Nov;19(11):1895-1910. [doi:
10.1109/TVCG.2013.89] [Medline: 24029909]
43. Bélanger E, Bartlett G, Dawes M, Rodríguez C, Hasson-Gidoni I. Examining the evidence of the impact of health information
technology in primary care: an argument for participatory research with health professionals and patients. Int J Med Inform
2012 Oct;81(10):654-661. [doi: 10.1016/j.ijmedinf.2012.07.008] [Medline: 22910233]
44. Sinclair M. A guide to understanding theoretical and conceptual frameworks. Evidence Based Midwifery 2007;5(2):39.
45. Moody DL. Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions.
Data & Knowledge Engineering 2005;55(3):243-276.
46. O'Connor Y, O'Reilly P, O'Donoghue J. M-health infusion by healthcare practitioners in the national health services (NHS).
Health Policy and Technology 2013;2(1):26-35.
47. Garcia-Smith D, Effken JA. Development and initial evaluation of the Clinical Information Systems Success Model (CISSM).
Int J Med Inform 2013 Jun;82(6):539-552. [doi: 10.1016/j.ijmedinf.2013.01.011] [Medline: 23497819]
48. Lwoga E, Questier F. A model for measuring open access adoption and usage behavior of health sciences faculty members.
In: Proceedings of 13th Mediterranean Conference on Medical and Biological Engineering and Computing. Cham,
Switzerland: Springer; 2014 Presented at: 13th Mediterranean Conference on Medical and Biological Engineering and
Computing; 25-28 September 2013; Seville, Spain p. 1298-1301.
49. Scott P, Briggs J, Wyatt J, Georgiou A. How important is theory in health informatics? A survey of UK academics. Stud
Health Technol Inform 2011;169:223-227. [Medline: 21893746]

http://medinform.jmir.org/2016/1/e7/ JMIR Med Inform 2016 | vol. 4 | iss. 1 | e7 | p. 8

(page number not for citation purposes)
XSL• FO
RenderX
 JMIR MEDICAL INFORMATICS Gray & Sockolow

50. Heath M, Appen R. Impact of information technology culture conflict on meaningful use of electronic health records system:
A conceptual framework. In: aisel.aisnet.org. Atlanta, GA, USA: Association for Information Systems; 2014 Presented at:
Twentieth Americas Conference on Information Systems; 7-9 August 2014; Savannah, GA, USA.
51. Paez K, Roper RA, Andrews RM. Health information technology and hospital patient safety: a conceptual model to guide
research. Jt Comm J Qual Patient Saf 2013 Sep;39(9):415-425. [Medline: 24147353]
52. Messeri P, Khan S, Millery M, Campbell A, Merrill J, Shih S, et al. An information systems model of the determinants of
electronic health record use. Appl Clin Inform 2013 Apr 17;4(2):185-200. [doi: 10.4338/aci-2013-01-ra-0005]
53. Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care--an interactive
sociotechnical analysis. J Am Med Inform Assoc 2007;14(5):542-549 [FREE Full text] [doi: 10.1197/jamia.M2384]
[Medline: 17600093]
54. Haddad P, Wickramasinghe N. Conceptualizing business value of IT in healthcare to design sustainable e-Health solutions.
Atlanta, GA, USA: Association for Information Systems; 2014 Presented at: Twentieth Americas Conference on Information
Systems; 7-9 August 2014; Savannah, GA.
55. Olla P, Butrico C, Pichette C. Role of mHealth in nursing: a conceptual framework. In: Adibi S, editor. mHealth
Multidisciplinary Verticals. Boca Raton, FL: CRC Press; 2015:279-298.
56. Mburu S, Franz E, Springer T. A conceptual framework for designing mHealth solutions for developing countries. In:
Proceedings of the 3rd ACM MobiHoc Workshop on Pervasive Wireless Healthcare. New York: ACM; 2013 Presented
at: 3rd ACM MobiHoc Workshop on Pervasive Wireless Healthcare; 29 July - 1 August 2013; Bangalore, India p. 31-36.
57. Williams P, Maeder A. A conceptual framework for secure mobile health. Journal of the International Society for
Telemedicine and eHealth 2013;1(1):44.
58. Ritz D. Measuring e-health impact: an e-health evaluation framework that leverages process control theory and big data
analytics. In: Marconi K, Lehmann H, editors. Big Data and Health Analytics. Hoboken, NJ: Auerbach Publications;
2014:309-332.
59. Unertl KM, Novak LL, Johnson KB, Lorenzi NM. Traversing the many paths of workflow research: developing a conceptual
framework of workflow terminology through a systematic literature review. J Am Med Inform Assoc 2010;17(3):265-273
[FREE Full text] [doi: 10.1136/jamia.2010.004333] [Medline: 20442143]
60. Sanderson P, Xiao T, Freeman C, Broxham W. Envisioning healthcare work: models for prospective evaluation of new
systems. In: Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium. New York: ACM; 2012
Presented at: 2nd ACM SIGHIT International Health Informatics Symposium; 28-30 January 2012; Miami, Florida, USA
p. 779-784.

Edited by G Eysenbach; submitted 11.08.15; peer-reviewed by P Schulz, L Moja, M Lorenzo; comments to author 17.11.15; revised
version received 10.12.15; accepted 03.01.16; published 24.02.16
Please cite as:
Gray K, Sockolow P
Conceptual Models in Health Informatics Research: A Literature Review and Suggestions for Development
JMIR Med Inform 2016;4(1):e7
URL: http://medinform.jmir.org/2016/1/e7/
doi: 10.2196/medinform.5021
PMID: 26912288

©Kathleen Gray, Paulina Sockolow. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 24.02.2016.
This is an open-access article distributed under the terms of the Creative Commons Attribution License
(http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information,
a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.

http://medinform.jmir.org/2016/1/e7/ JMIR Med Inform 2016 | vol. 4 | iss. 1 | e7 | p. 9

(page number not for citation purposes)
XSL• FO
RenderX