See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/318562026

Investigation of Physicians' Awareness and Use of mHealth Apps: A Mixed

Method Study

Article  in  Health Policy and Technology · July 2017

DOI: 10.1016/j.hlpt.2017.07.007

CITATIONS READS
8 567

3 authors, including:

Soner Yildirim
Middle East Technical University
96 PUBLICATIONS   1,390 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Nomophobia View project

The roadmap of transition to independence: A patient-centered research on teens with chronic conditions View project

All content following this page was uploaded by Emre Sezgin on 25 February 2020.

The user has requested enhancement of the downloaded file.

Author’s Accepted Manuscript

Investigation of Physicians' Awareness and Use of

mHealth Apps: A Mixed Method Study

Emre Sezgin, Sevgi Özkan-Yildirim, Soner

Yildirim

www.elsevier.com/locate/hlpt

PII: S2211-8837(17)30053-9
DOI: http://dx.doi.org/10.1016/j.hlpt.2017.07.007
Reference: HLPT247
To appear in: Health Policy and Technology
Cite this article as: Emre Sezgin, Sevgi Özkan-Yildirim and Soner Yildirim,
Investigation of Physicians' Awareness and Use of mHealth Apps: A Mixed
Method Study, Health Policy and Technology,
http://dx.doi.org/10.1016/j.hlpt.2017.07.007
This is a PDF file of an unedited manuscript that has been accepted for
publication. As a service to our customers we are providing this early version of
the manuscript. The manuscript will undergo copyediting, typesetting, and
review of the resulting galley proof before it is published in its final citable form.
Please note that during the production process errors may be discovered which
could affect the content, and all legal disclaimers that apply to the journal pertain.
Health Policy and Technology (2017) 6, 251–267

Available online at www.sciencedirect.com

journal homepage: www.elsevier.com/locate/hlpt

Investigation of physicians' awareness and use

of mHealth apps: A mixed method study
Emre Sezgina,b,n, Sevgi Özkan-Yildirimb, Soner Yildirimc

a
Nationwide Children's Hospital, The Research Institute, 43215 Columbus, OH, USA
b
Middle East Technical University, School of Informatics, 06800 Çankaya, Ankara, Turkey
c
Middle East Technical University, Department of Computer Education & Instructional Technology, 06800
Çankaya, Ankara, Turkey

Available online 19 July 2017

KEYWORDS Abstract
Information Objective: The study aims to understand physicians’ awareness of mobile health (mHealth)
technology; apps and their intentions to use these apps in medical practice.
Healthcare; Method: Mobile Health Technology Acceptance Model (M-TAM) was tested employing the sequential
Mobile health; explanatory mixed method. An online survey and focus group interviews were conducted for data
Technology
collection. Physicians were invited to participate in the survey. Structural Equation Modeling (SEM)
acceptance;
was used in quantitative data analysis. Qualitative data were analyzed using coding, memo, and
Physicians;
Application use contextual analyses.
Results: 151 physicians participated in the survey, representing a 15% response rate. The model was
able to explain physicians’ intention to use mHealth apps by explaining 59% of the total variance.
Performance Expectancy, Mobile Anxiety, Perceived Service Availability and Personal Innovativeness
were major influencing factors of Behavioral Intention. Qualitative codes outlined that information
gathering and communication purposes were the major enablers in mHealth app usage. In that
regard, Communication and Consulting, Clinical Decision Making, Reference and Information
Gathering, and Information Management are the most popular app categories. On the other hand,
lack of knowledge and lack of investment were seen as the major barriers to mHealth app usage.
Conclusions: User perception and intentions are important factors in technology use. Thus, the
preferences, expectations, and characteristics of physicians which were outlined in this research
could be significant inputs for researchers, app developers, managers and policymakers.
& 2017 Fellowship of Postgraduate Medicine. Published by Elsevier Ltd. All rights reserved.

Corresponding author at: Nationwide Children's Hospital, The Research Institute, 43215 Columbus, OH, USA.
n

E-mail addresses: [email protected] (E. Sezgin), [email protected] (S. Özkan-Yildirim),

[email protected] (S. Yildirim).

http://dx.doi.org/10.1016/j.hlpt.2017.07.007
2211-8837/& 2017 Fellowship of Postgraduate Medicine. Published by Elsevier Ltd. All rights reserved.
252 E. Sezgin et al.

Introduction healthcare professionals use smartphones in daily practice,

and that six out of ten are using tablet PCs [16]. The
The healthcare information technology market is expanding literature demonstrated that mHealth is located in a posi-
globally. Reports reveal trajectories support the prolifera- tion that provides a promising technology integration
tion of mobile technology use in delivering healthcare opportunity among healthcare services. It provides a plat-
services is becoming the norm [1]. McKinsey's report pre- form (-1-) to increase healthcare quality, productivity and
sented that the tablet PC and smartphones market will efficiency [17], (-2-) to improve accuracy and data conve-
expand by up to 30% by 2018 [2]. On the software side, the nience as well as decision-making processes [7,18], and (-3-)
number of mobile apps has reached more than one hundred to enhance access to healthcare services [19]. There have
thousand in approximately three years [3]. been several core fields for the use of mHealth which have
In parallel to this, studies investigating the effects of been identified and categorized in the literature (Table 1).
mobile technology usage and apps in healthcare services are In these studies, the most common categories were com-
also increasing. The studies underlined the significant global munication, decision support, education, tracking, and
impact of mHealth and in particular, the importance of monitoring. These categories also summarize the primary
mHealth technologies in developing countries for accessing purpose of using mHealth in practice.
healthcare and for patient monitoring [4,5]. However, Amongst them, Ventola [17] proposed an eight-group
common concerns for the technology remain with regard categorization for mHealth apps (Table 2). The categories
to sustainability, scalability, security, and privacy, which outline the common field of mHealth app usage by health-
may subsequently lead to problems in service quality [6]. On care providers, and helps to identify physicians’ preferences
the other hand, Healthcare Providers (HCP) report that of mHealth app use in this study.
locational constraints, needs for interoperability, profes- The literature reports a number of new opportunities and
sional control and effective management for chronic condi- challenges that have emerged with mHealth. As an oppor-
tions are their particular concerns with regard to using tunity, mHealth was emphasized as an effective tool for
mHealth in healthcare delivery [7]. healthcare delivery in developing countries [13], and
The actual use of mHealth requires in-depth investigation mHealth was also argued as an improvement to commu-
in order to understand the true impact on its end users. nication among doctors [20–22]. In this context, the use of
Significant escalation in smartphone usage might have auto-notification, tele-consultation, and digital X-ray image
increased app usage in healthcare services, yet, its utiliza- apps have been demonstrated in medical practice [23].
tion has not reached an acceptable level in practice [8]. Further, clinical image storing at clinics has enabled quicker
Studies have demonstrated that acceptance issues for mobile access and the ability to share high-quality images [24]. On
healthcare services need to be resolved to assure better the other side, the public use of mHealth apps would
healthcare delivery [9,10]. In that regard, the use of mHealth provide more insight into patient health history which aids
apps and the attitudes of healthcare providers remain the diagnostic process. Studies revealed that self-monitor-
questionable and therefore in need of investigation [11]. ing apps were the most used mHealth apps [25], including
In Turkey, the Ministry of Health reports over 135 apps for diabetes, asthma, and depression control [26],
thousand physicians actively practice medicine [12]. The fitness and training [25,27], and diet and physical activities
number of patients per physician is around 600, and each [28,29]. Furthermore, mobile intervention methods were
hospital physician sees more than 4600 patients per year. also widely adopted in healthcare services, and SMS remin-
Thus, there is an excessive workload for physicians. To der was one of the effective and most frequently used
maintain quality in healthcare delivery, assistive technolo- methods [6,30].
gies were predicted to help physicians, such as mHealth As the challenges, technological, psychological, and
apps [13]. However, there is a gap in practical knowledge. regulatory barriers in mHealth app use were considered
The current state of mHealth app use in most developing the major challenges, which could potentially lead to the
countries is barely known, with few studies providing insight underutilization of mobile technologies [18]. Collaboration
about mHealth use in developing, low and mid-income was seen as another challenge to overcome. Interventions
countries [14]. and collaborations among systems developers, behavioral
This current study investigates physicians’ mHealth app experts, healthcare professionals, and patients need more
usage in Turkey. A Mobile Health Acceptance Model (M-TAM) investment and development in order to promote the use of
was proposed and tested as a means to understanding the the mobile health systems [18,28]. However, sharing perso-
influencing factors in mHealth app usage. Furthermore, the nal health information with mobile apps raised concerns
study presents a social validation retrieving a collective about their privacy [6,31,32]. In the work environment,
meaning among individuals through focus group interviews perceptions about interruption from the use of technology
in order to understand expectations and characteristics of in routine tasks, the required extra efforts in multitasking,
mHealth app use. perceived increase in workload within limited work time
and perceptions about weakening relationships were dis-
couraging elements for the use mHealth technologies as
mHealth overview seen by healthcare providers [7,20,21,33,34]. In addition,
incompetent use of healthcare technologies, trusting the
Clinical use of mobile apps and devices is increasing. technology and a lack of awareness about their potential
Clinicians report that mHealth apps will become an impor- and their functions were some of the personal challenges
tant tool for health management in the near future [15]. A [33]. Developers also demonstrated lack of awareness about
Wolters Kluwer Health report stated that eight out of ten the needs in mobile health, and it was also considered a
Investigation of physicians' awareness and use of mHealth apps 253

Table 1 mHealth categorization.

Research Categories

Status and trends of mobile-health applications for iOS devices: A ! Drug or medical information database
developer's perspective [29] ! Medical information reference
! Decision support
! Educational tools
! Tracking tools
! Medical calculator
! Others
Mobile Health (mHealth) Approaches and Lessons for Increased Perfor- ! Education and awareness
mance and Retention of Community Health Workers in Low- and ! Data access
Middle-Income Countries: A Review [8] ! Monitoring and compliance
! Disease and emergency tracking
! Health information systems
! Diagnosis and consultation
Mobile devices and apps for health care professionals: uses and benefits ! Information management
[27] ! Time management
! Health record maintenance and access
! Communications and consulting
! Reference and information gathering
! Clinical decision making
! Patient monitoring
! Medical education and training
The smartphone in medicine: A review of current and potential use ! Patient care and monitoring
among physicians and students [30] ! Health apps for the layperson
! Communication, education and research
! Physician/Student reference apps
Mobile Technologies and Geographic Information Systems to Improve ! Treatment and disease management
Health Care Systems: A Literature Review [31] ! Data collection and disease surveillance
! Health support systems
! Health promotion and disease prevention
! Communication between patients and health care
providers or among providers
! Medical education

barrier to the development of useful apps [25]. In develop- proposed, named as the Mobile Health Technology Accep-
ing countries, cultural, social, and educational barriers also tance Model, or M-TAM. The model was developed based on
affect mHealth dissemination [7,34]. the findings of an in-depth literature review on technology
The literature showed that mHealth still needs develop- acceptance, and the consensus of experts. A group of
ment. From a technical perspective, multiplatform devel- scholars (Ph.D. level of knowledge in behavioral science or
opment for mHealth requires compatibility across mobile healthcare) were informed about the purpose of the study
apps and hospital systems [7,18,35]. Similarly, mobile and the literature findings. A consensus was reached about
infrastructure is required to reduce system-level barriers the theories and constructs to be employed in the study.
in accessibility [7,36], as well as increased security for Technology Acceptance Model (TAM) [37], Unified Theory of
health records, and new standards and regulations [6,35]. Acceptance and Use of Technology (UTAUT) [38], Theory of
Above all, there is a need for reliable and trustworthy apps Planned Behavior (TPB) [39], and Innovation Diffusion
in the market [14,18,22,32], and to overcome sustainability Theory (IDT) [40] were the theories selected for the model's
issues, such as short battery life and delays in mobile development. Table 4 presents the theories, constructs, and
processing [35]. Table 3 summarizes the challenges seen in definitions. Computer anxiety and computer self-efficacy
the use of mHealth. constructs were used as mobile anxiety and mobile self-
efficacy respectively in this study.
Figure 1 demonstrates the conceptual model. The rela-
Methodology tionships among the constructs (arrows) represent the
hypotheses. The hypotheses of the research were formu-
Research model lated addressing the research question: “What are the
factors influencing physicians’ intention to use mHealth
In order to assess physicians’ perceptions and attitudes apps?” In that regard, constructs’ influence on the beha-
toward mHealth apps, a technology acceptance model was vioral intention was hypothesized. Following that, another
254 E. Sezgin et al.

Table 2 Use of mHealth devices and apps by health- Table 3 Challenges in use of mHealth.
care professionals [17].
Challenges Reference
Information management Reference and information
gathering Need of collaboration and intervention [18,28,36]
Privacy [6,31,32]
■ Write notes ■ Medical textbooks Interruptions & multitasking requirements [7,34]
■ Dictate notes ■ Medical journals Increased workloads & Time constraints [20,21,33]
■ Record audio ■ Medical literature Cultural, social and educational barriers [7,34]
■ Take photographs ■ Literature search portals Lacks in skills and awareness [25,33]
■ Organize information ■ Drug reference guides Lack of trust in quality [28]
and images ■ Medical news Weakened interpersonal relationships [22]
■ Use e-book reader Multiplatform development [7,18,35]
■ Access cloud service time Security [6,35]
Time management Clinical decision-making Standards and regulations [14,18,22,32]
■ Schedule appointments ■ Clinical decision support Delays [35]
■ Schedule meetings systems Battery life [35]
■ Record call schedule ■ Clinical treatment Infrastructure [7,36]
guidelines
■ Disease diagnosis aids
■ Differential diagnosis aids
■ Medical calculators
■ Laboratory test ordering
Convenience sampling was used, and the data was collected
& interpretation using online survey application (www.qualtrics.com). The
■ Medical exams target sample was physicians using mobile health apps and
Health record maintenance Patient monitoring who were actively working in health institutions in Turkey. The
and access ethics board of the University approved the survey prior to
■ Access EHRs and EMRs ■ Monitor patient health application. The questionnaire was tested for item integrity
■ Access images and scans ■ Monitor patient location and understandability with a pre-test study applied to a small
■ Electronic prescribing ■ Monitor patient subset of the sample. The survey was announced via the
■ Coding and billing rehabilitation institution web page. Participants were invited via online posts
■ Collect clinical data on social network groups and via e-mails sent to mailing lists.
■ Monitor heart function In addition, a paper-based questionnaire was distributed to
Communications and Medical education and physicians in the province. The invitation for online participa
consulting training tion reached approximately one thousand physicians, and
■ Voice calling ■ Continuing medical paper-based questionnaires were delivered to 53 physicians.
■ Video calling education In the data analysis phase, the normality of the data and
■ Texting ■ Knowledge assessment internal consistency were tested using SPSS software. Linear
■ E-mail tests and casual models were then tested using Structural
■ Multimedia messaging ■ Board exam preparation Equation Modeling (SEM), which provided a multivariate
■ Video conferencing ■ Case studies approach to observe latent relationships among the con-
■ Social networking ■ E-learning and teaching structs [55]. Partial Least Squares (PLS) test was then
■ Surgical simulation completed with a component-based approach employing
■ Skill assessment tests SmartPLS software [56].

Focus group interview procedure and analysis

set of hypotheses were formulated to seek influence of
constructs in use over performance expectancy and effort
expectancy (see Appendix A for hypotheses). The focus group interviews were administered by two
researchers, with one acting as a moderator/researcher
and the other as an observer. The researchers informed
Quantitative data collection and analysis participants about the purpose of the study, the procedure,
confidentiality of the information, and voluntary nature of
A cross-sectional survey method was used in the data their participation. Verbal consent was sought before each
collection. A structured questionnaire was developed to focus group interview. Focus Group 1 (FG1), Focus Group 2
test the hypotheses. A five-point, Likert-type scale was used (FG2) and Focus Group 3 (FG3) consisted of three, four and
(“1: Strongly disagree”, “2: Disagree”, “3: Neutral”, “4: three physicians respectively. The participants were physi-
Agree” and “5: Strongly agree”) [54]. The survey consisted cians using mHealth apps and working in health institutions
of three parts. The first part included an introduction about in Turkey. Each focus group was from a different hospital.
mHealth apps, information about the purpose of study and a The informants were grouped considering demographic
consent form. The second part included demographic ques- characteristics (gender, age, specialty, and experience in
tions, and the third and final part was close-ended survey their profession) in order to create heterogeneity and
questions. diversity of responses within the group. Snowball approach
Investigation of physicians' awareness and use of mHealth apps 255

Table 4 Constructs, definitions and references.

Constructs Definition Theories References

Behavioral intention (BI) “the degree to which a person has formulated conscious UTAUT [38]
plans to perform or not perform some specified future
behavior”
Effort expectancy (EE) “the degree of ease associated with the use of the system.” [38,41]
Performance expectancy (PE) “the degree to which an individual believes that using the [38,41]
system will help him or her to attain gains in job
performance”
Habit (HB) “constitutes the level of routinization of behavior, i.e. the [42,43]
frequency of its occurrence” (UTAUT 2)
Technical support and training “the technical support and the amount of training provided [11,38,44]
(TT) by individuals of knowledge”
Perceived service availability “the degree to which an innovation is perceived as being [38,45]
(PS) able to support pervasive and timely usage”
Personal innovativeness in the “the willingness of an individual to try out any new IT, plays IDT [40,46–48]
domain of IT (PI) an important role in determining the outcomes of user
acceptance of technology”
Compatibility (CO) “the degree to which an innovation is perceived as being [11,40,49,50]
consistent with the existing practices, values, needs and
experiences of the health care professional”
Result demonstrability (RD) “the extent to which the tangible results of using an TAM [46,51]
innovation can be observable and communicable” (TAM2)
Computer Self- efficacy “the degree to which an individual beliefs that he or she has [44,49,52]
the ability to perform specific task/job using computer”
(TAM3)
Computer anxiety “the degree of an individual's apprehension, or even fear, [44,49,52]
when she/he is faced with the possibility of using computers”
(TAM3)
Social influence(SI) “the degree to which an individual perceives that important TPB (and [38,41,53]
others believe he or she should use the new system” UTAUT)

was used in recruitment. Details about each group are interviews, the questions were directed in a deductive
provided in Table 5. Each focus group interview lasted setting.
approximately one hour. The responses were recorded and
observational notes were taken during the interviews. Results
During the sessions, an interview protocol was followed.
The methods of memoing, coding [57,58] and contextual After the data collection process, 151 physicians completed the
analysis [59] were employed. In the process of analysis, the questionnaire, which represented a 15% response rate. Incomplete
raw data, audio recordings, and notes were transcribed as a responses were then removed, and 137 complete responses were
first step (QDA Miner software was used in the transcribing used in the testing of the hypotheses.
and coding). The data was then read through to ensure the
accuracy of the information. Then, the codes and themes Demographics
were created, and the meaning of these themes was
interpreted. The process of reading and the interpretation As can be seen in Table 6, the demographic data shows that the
continued until all relevant information was grouped with majority of participants were young adults (53%) and male (56%).
codes and themes. During the procedure of transcription, a Three out of four participants were specialist medical practitioners
colleague assisted in order to mitigate the risk of researcher (74%). Smartphones were the mostly used mobile device (98%), and
bias and potential misunderstandings. Additionally, peer participants had one to five years of experience in using mobile
debriefing [60] was utilized, in which the researcher was devices (69%). Competency in mobile device use was reported
assisted by a colleague impartial to the study. Recker's [58] mostly at the “good” level (63%). Most of the participants were
key elements (Dependability, Credibility, Confirmability, using mHealth apps more than once a week (54%), and mostly used
mHealth apps for one to two years (53%). Participants reportedly
Transferability) were considered in the process of qualita-
used mHealth apps voluntarily (98%). The majority of participants
tive data evaluation. were practicing healthcare services in public hospitals (43%) and
The questions of qualitative approach were developed training and research hospitals (33%). Specialist medical practi-
based on the quantitative findings of the study. This tioners (74%) were physicians from Pulmonology (15%), Cardiology
approach helped to identify significant and non-significant (8%), Pediatrics (7%), Surgery (7%), Primary Care (6%), Anesthesia
elements in the mHealth use of physicians. During the (6%), Internal Medicine (5%), Ophthalmology (5%), Dentistry (5%),
256 E. Sezgin et al.

Figure 1 The conceptual model: M-TAM.

Gynecology (5%), Psychiatry (5%), Otorhinolaryngology (4%), Oncol- and Google Hangout in their medical communications. In clinical
ogy (4%), Emergency Medical Services (4%), Urology (4%), Orthope- decision making, 144 apps were reported. Medical calculators (e.g.
dics (4%), Pathology (3%) and Neurology (3%). Medcalc, Das28) (44%) and diagnostic assistance tools (e.g. Prognosis,
Dxsaurus) (43%) were the most used apps. Drug referencing apps
(Cepilaç) are the most used out of 137 reference and information
Mobile app use gathering apps (62%), followed by referencing (14%), cases and
The participants of the survey reported the names of mobile apps guidelines (Nature, Uptodate) (14%) and dictionary (Eponyms) (10%)
they used in healthcare delivery or medical practice. Some of the apps. For information management, 117 apps were noted, and
reported apps were not designed specifically for mHealth, but due mostly default mobile apps were reported as being used in this
to their purpose of use, they were included in the study. The apps category. Google Notes, e-book reader, Evernote and Photo apps
were categorized by their field of use employing Ventola's [17] were used for reading and keeping notes. For medical education and
categorization. Results showed that most of the apps were used for training, 74 apps were reported. Physicians were highly interested in
communication and consulting. This was followed by clinical commonly used medical education and information apps (e.g.
decision making, reference and information gathering, information Medscape) (62%) and visual training apps (e.g. OrthoApp, Vcell)
management, medical education and training, time management, (27%). Public health training apps (11%) were also in focus. For time
and health records, maintenance and access. The least used apps management, out of 63 reported apps, Google Calendar was the most
were in the category of patient monitoring (Figure 2). In total, 764 used app (68%). It was followed by default mobile calendar tools
mHealth apps were reported, and were grouped under one of eight (16%) and an appointment app developed by the Health Ministry,
categories. Categorization of apps was completed with the assis- known as MHRS (16%). For health record maintenance and access,
tance of academic and medical experts. Figure 2 shows the ratio of Enlil, a national hospital management information system, was
apps in each category to the total number of apps. reported as the most used app (52%). It was followed by other
The results presented that text and multimedia messaging apps medical health recording systems as Meddata (16%), E-nabiz (16%),
(e.g. WhatsApp, Google Hangout) are the popular apps in the PACSapp (8%), and Acibadem (8%). For the patient monitoring
communications and consulting category. In total, 162 apps were category, 17 apps were reported. This category had the least number
reported for communication and consulting, and almost half of the of apps reported. Pedometer (24%), calorie tracker (24%), heart rate
participants in this category (50% and 43%) reported using WhatsApp and information tracker tools (cardiograph) (18%), Apple health
Investigation of physicians' awareness and use of mHealth apps 257

Table 5 Focus group characteristics.

Focus group 1 Focus group 2 Focus group 3

Identifier FG1.a FG1.b FG1.c FG2.a FG2.b FG2.c FG2.d FG3.a FG3.b FG3.c
Gender Male Male Female Female Female Female Male Male Male Female
Age 39 29 33 28 31 35 40 26 24 24
Specialty Cardiology Pulmonology Gynecology Anesthesiology Urology Practitioner
Experience in Smart- 10 7 3 5 4 8 2 5 6 4
mobile device use
(years)
Experience in job 15 4 7 4 8 12 18 3 1 1
(years)
Institution City State Hospital University Research and Application Hospital Private Hospital
Interview duration 57 min 52 min 1 h 3 min

(18%), Instant health rate (12%) and Fitwell (6%) were the reported identifying ‘misspecified’ models, and PLS-SEM counts on measures
apps. The tracking apps were assumed to have been used by patients about the predictive capabilities of the model in order to assess the
and shared with physicians. quality of the model [68].

Structural model
Descriptive results
Since the items of model constructs can be identified as inter-
changeable among the constructs as well as having high correla-
Descriptive results outlined the distribution of the data. As given in tions, reflective measurement scale was employed in the PLS
Table 7, the mean values provided the central tendency of modeling [64]. Literature suggests that if the PLS algorithm
responses. Except for the mobile anxiety (MA) and technical support converges before the maximum iteration limit (set to 300), it
and training (TT), the responses were clustered at the favorable ensures the stability of estimation [69], and convergence of the
side (above 3) with standard deviations between 0.38 and 0.95. The algorithm completed at nine iterations. Estimating the normality of
data presented negative Skewness and positive Kurtosis, and the the data, bootstrapping method was employed using 5000 resam-
data was acceptable (71.5) to proceed to structural equation pling. Table 10 outlines the approved and rejected hypotheses
modeling [55]. Shapiro-Wilk test was conducted to test the normal- providing path coefficients, t-statistics, and multicollinearity of the
ity of the data [61], and the data were not normally distributed data. For the approved hypotheses, path coefficient values were
(po0.05) (Table 7). During the analysis, list-wise deletion approach suggested to be above 1.0, and t-statistics values were suggested to
was used to handle missing data. Cronbach's Alpha test was be above 1.96 at the significance level of po0.05 [68]. Multi-
conducted to assess internal consistency of the model. Alpha values collinearity values of the data were non-problematic with the
of constructs and the overall reliability (0.796) were acceptable variance inflation factor (VIF) value below five for each hypothesis
[62]. [70].
Results suggested that nine of the hypotheses were approved for
the model. According to the test results., for mHealth app users,
Structural equation modeling
mobile anxiety (β=-0.160, po0.05), performance expectancy
(β=0.359, po0.001), personal innovativeness (β =0.139, po 0.05)
Measurement model and perceived service availability (β=0.120, po0.05) had a sig-
Convergent validity and discriminant validity were tested to ensure nificant influence on behavioral intention. In addition, compatibility
construct validity of the model. In the first phase, Fornell and (β =0.383, po0.001), personal innovativeness (β =0.284, po 0.001)
Larcker's [63] procedure was followed to test convergent validity and result demonstrability (β=0.196, po0.05) had significant
(Table 8). Item reliability test was then conducted by extracting influence on performance expectancy. Mobile self-efficacy and
square values of item loadings, and they were expected to be above perceived service availability had influence on effort expectancy
0.4 [64]. Following that, composite reliability was tested and (β=0.365, po0.001 and β=0.175, po0.05). However, compatibil-
resulted between 0.7 and 0.92, which was acceptable with above ity, effort expectancy, habit, mobile self-efficacy, social influence
0.60 [65]. At the final phase, convergent validity test was con- and technical support and training had no influence on behavioral
ducted. AVE values of each construct were expected to be above intention. The remaining hypotheses were not supported as well.
0.50 [66]. As given in Table 8, AVE values of the constructs met the The determinants of behavioral intention (MA, PE, PI, and PS)
requirement with the values between 0.53 and 0.85. The model accounted for 59% of the total variance explained for the intention
met the requirements for convergent validity, except for items SI3 to use mHealth apps. In addition, the determinants of effort
and TT2, which were removed from m-Health users due to low item expectancy explained 51% of the variance, and the determinants
loadings. of performance expectancy explained 51% of the variance
Following the convergent validity, discriminant validity was (Table 10, Figure 3).
tested in order to measure divergence within constructs [67].
Discriminant validity test was conducted by calculating the square
roots of AVE values and analyzing the correlation (Table 9). The Findings of focus group study
square root of AVE was expected to be greater than the constructs’
correlation values (the diagonal path of the discriminant validity) Three focus group interviews were finalized as the rich data was
[64]. Fit indices for PLS-SEM (Goodness-to-fit) were excluded from obtained, and the researchers agreed that saturation was achieved
the analysis. Literature suggests that the measure was not fit for [71]. After coding of the transcripts, themes were created as
258 E. Sezgin et al.

also included sub-codes, which were identified as items influencing

Table 6 Demographic characteristics.
the actual use of mHealth apps. Response counts and percentages
are given in Table 11.
Demographics Percentages
Observational notes showed that the physicians had to fulfill
1. Gender their duties within a tight schedule and limited time, and the
hospital management expect them to work at their maximum limits
Female 44%
while providing healthcare services. Hospitals were equipped with
Male 56% mostly non-mobile devices and equipment, such as desktop PCs,
2. Age 25–35 (53%) X-ray devices. There was limited mobile equipment, such as mobile
36–45 (36%) electrocardiograph (ECG) and ultrasound devices, but they were
46–66 (11%) made available for particular healthcare services. Physicians and
3. Education Level other healthcare providers have an open and continuous commu-
General Practitioners 26% nication in their routine.
Specialist medical practitioners 74% Interviews demonstrated that, within the limited time to deliver
Specialist medical practitioners with 8% healthcare services, the physicians should be able to reach the
PhD degree information quickly when needed. Existing information and com-
munication technologies (e.g. Hospital management systems, SMS,
4. Mobile device preferences Smart Phone 98%
and e-mail) have been used in medical services and communica-
Tablet PC 61% tions, but to a limited level. Personal smartphones grant faster
5. Experience in mobile device use access to information, especially for communication among peers.
None – Web browsers and search engines were also used for medical
Less than 1 year 2% information searches, sometimes outperforming the mHealth apps.
1–5 years 69% Observations and interviews showed that mHealth apps have not
6–10 years 19% yet reached maturity due to its low effectiveness and lack of
More than 10 years 10% extensive use in the hospital system, limited development and the
6. Perceived competency in Mobile deficiency of standards and regulations. The informal use of mobile
device use communication apps in medical practice, lack of interoperability,
security, and control over apps would also support the low maturity
Excellent 18%
in mHealth.
Good 63%
Moderate 19%
Bad – Discussion
7. What is your mobile health appli-
cation use frequency? The literature argued that healthcare providers are aware
None 1% of mHealth technologies, and they acknowledge the bene-
More than once in a Month 28% fits of mHealth in clinical communication and healthcare
More than once in a Week 54% delivery [20,72]. This study supports this argument, out-
Everyday 17% lining a variety of evidence regarding physicians’ intentions
8. Do you use the mHealth applica- toward using mHealth apps.
tions on voluntary basis?
Yes 98%
No 2% mHealth Apps
9. How long have you been using the
mobile health applications? The self-reported mHealth apps outlined the common
None 1% trajectory in app preferences. Communication and consult-
Less than one year 20% ing, clinical decision making, reference and information
1–2 years 53% gathering and information management apps constituted
3–4 years 22% the popular categories in mHealth app use. For instance, a
5 years and above 4% physician may have WhatsApp, Medcalc, Epocrates, Up-to-
10. How can you define the type of date and Evernote in their mobile device as essential apps
your health institution? being used during the delivery of healthcare services. Web-
Public hospital 43% based searches and default mobile apps were also used by
Training and research hospital 33% physicians, such as Evernote, Google Drive or voice recor-
Health research center 7% der. Thus, each physician group may have their own “app
Community clinic 5% ecosystem”. These ecosystems may help developers to
Private hospital 12% identify the needs of physicians in medical practice and to
On-site medical services – develop specific apps to meet the demand.
The least preferred categories of apps were health record
maintenance and access and patient monitoring. Unlike the
other categories, these require a certain level of involve-
“enablers” (elements that enable or influence physicians to use
mHealth apps), “barriers” (elements of limiting or disabling physi-
ment of patients and network access to patient health
cians to use mHealth apps) and “enablers and barriers” (elements records. Thus, the possible reason would be mainly the need
that may either affect to use or not to use mHealth apps). Codes for mobile hardware, insufficient apps, lacking infrastruc-
were grouped as “personal” and “organizational” in order to ture as well as the regulations. These have been already
categorize according to personal and external influences. The codes regarded as issues for developing countries in the literature
Investigation of physicians' awareness and use of mHealth apps 259

Figure 2 mHealth app categories with usage rates.

Table 7 Descriptive statistics and reliability test.

Mean Std. deviation Skewness Kurtosis Shapiro-Wilk Cronbach's alpha

BI 4.2600 0.51613 "0.605 0.910 0.000 0.769

EE 3.9416 0.45696 "0.026 0.467 0.000 0.767
PE 4.1191 0.40583 "0.068 0.695 0.000 0.768
MA 1.7591 0.59440 0.689 0.841 0.000 0.856
MS 3.9295 0.47900 "0.073 "0.045 0.000 0.759
PI 3.6349 0.59647 "0.073 "0.170 0.000 0.767
HB 3.1752 0.95408 0.034 "1.068 0.000 0.759
SI 3.3186 0.67873 "0.752 0.238 0.000 0.783
CO 3.7104 0.52783 "0.263 "0.472 0.000 0.760
TT 2.6884 0.66311 0.509 ".012 0.000 0.796
RD 3.9635 0.38651 "0.455 1.229 0.000 0.779
PS 3.9126 0.48688 "0.796 1.137 0.000 0.780

[7]. Thus, there is a need for interconnected mobile devices helpful in their job routines. The literature has already
and apps for physicians as well as patients to create a provided that PE is one of the important indicators of the
network for keeping health records, accessing and monitor- adoption of health information systems [20,73,74], thus,
ing patients in real time. In that regard, the authorities the findings supported the impact of PE. On the other hand,
should consider investing in service infrastructure, giving the influence of PE on BI was an expected outcome from the
priority to mobile healthcare services. developing countries’ perspective [20]. In that regard, one
of the primary concerns of physicians can be stated as the
practical benefit of the mHealth apps, especially while
Research model practicing within a tight schedule [75]. Focus group inter-
views supported this argument, as one participant stated
M-TAM was able to explain behavioral intention (BI) to use that, physicians have to be quick to fulfill their tasks in
mHealth apps with 51% of total variance. In that regard, seconds. Access to information in a timely manner is crucial
performance expectancy (PE) was one of the factors during the physicians’ routine [76]. PE was also influenced
significantly influencing BI. In the study, it was observed by compatibility (CO), personal innovativeness (PI) and
that PE was more effective in explaining BI than effort result demonstrability (RD), which means the consistency
expectancy (EE). The significant effect of PE indicated that and integrity of mHealth apps, the degree of willingness to
physicians had beliefs about mHealth apps that would be use mHealth and availability of demonstrable results also
260 E. Sezgin et al.

influenced the perception of physicians about their job the expectations of physicians from mHealth apps was the
performances [46,47,77]. ability for pervasive and timely use. This finding supported
Perceived service availability (PS) was found as another Venkatesh's [38] argument that facilitating conditions are
significant factor influencing intention to use. Thus, one of influential in explaining the use of technology. In that
regard, service availability for specialty-specific mHealth
Table 8 Convergent validity (Item reliability, Compo- apps was a challenge, yet regarding the user interface,
site reliability, and AVE). language support was seen as expected by the physicians.
The physicians reported that use of mHealth apps was not
Constructs Items Item Composite AVE vitally important at the current level, however, when they
reliability reliability need to use it, they expect to have language support for
better access.
BI BI1 0.552 0.858 0.670 Similarly, PI was another factor that had a significant
BI2 0.701 influence on BI. Physicians were found to have a certain
BI3 0.755 level of willingness to use new technologies which even-
CO CO1 0.601 0.823 0.609 tually positively affect their attitudes toward using mHealth
CO2 0.493 apps [47]. In the focus groups, the physicians were observed
CO3 0.733 to have no barriers to new technologies, especially to
EE EE1 0.696 0.811 0.590 smartphones and mobile apps. They also stated that all
EE2 0.546 physicians around them own a smartphone and use mobile
EE3 0.527 apps, which would facilitate the process of technology
HB HB1 0.862 0.919 0.850 adoption.
HB2 0.838 On the other side, Mobile anxiety (MA) was identified to
MA MA1 0.462 0.730 0.578 have a negative influence on BI. Perceived intimidation,
MA2 0.693 hesitation or apprehension would negatively affect physi-
MS MS1 0.654 0.810 0.587 cians’ intention to use. In that regard, lack of initiatives and
MS2 0.538 perceived ability may increase physicians’ anxiety as well as
MS3 0.571 reducing their intention to use mHealth apps [20,49]. In the
PE PE1 0.544 0.775 0.535 interviews, as a disabler, anxiety was observed to have less
PE2 0.458 impact on physicians. The reason can be connected to a
PE3 0.602 couple of factors, such as existing trust with informally used
PI PI1 0.643 0.860 0.673 apps and the low level of importance of mHealth in common
PI2 0.652 practice. It was observed that, at the initial phases, validity
PI3 0.724 and reliability issues may create apprehension in terms of
PS PS1 0.672 0.820 0.607 using medical apps, but they were overcome by routine use.
PS2 0.405 Physicians’ perceptions toward consistency of mHealth
PS3 0.743 apps with the tasks and practices (HP4: CO to BI) and
RD RD1 0.617 0.810 0.681 physicians’ perceived abilities in performing daily task with
RD2 0.746 mHealth apps (HP9: MS to BI) did not have a significant
SI SI1 0.852 0.903 0.824 relationship with BI. Even though the literature states
SI2 0.796 otherwise [50,78], a lacking in routine practices with
TT TT1 0.526 0.830 0.713 mHealth and job conditions (as in a developing country)
TT3 0.900 would be effective drivers for this result [20]. The inter-
views revealed a certain level of compatibility issued among

Table 9 Discriminant validity.

BI CO EE HB MA MS PE PI PS RD SI TT

BI 0.818
CO 0.447 0.780
EE 0.563 0.528 0.768
HB 0.543 0.621 0.535 0.922
MA " 0.538 " 0.256 " 0.349 "0.422 0.760
MS 0.566 0.600 0.653 0.546 "0.375 0.766
PE 0.667 0.622 0.577 0.602 "0.483 0.562 0.731
PI 0.529 0.478 0.504 0.596 "0.367 0.588 0.540 0.820
PS 0.441 0.349 0.459 0.339 "0.335 0.463 0.354 0.229 0.779
RD 0.450 0.453 0.405 0.333 "0.254 0.470 0.494 0.337 0.516 0.825
SI 0.221 0.227 0.205 0.248 "0.172 0.223 0.160 0.020 0.365 0.214 0.908
TT 0.027 0.166 0.129 0.207 0.260 0.231 0.093 0.167 0.126 0.054 0.277 0.844
Investigation of physicians' awareness and use of mHealth apps 261

Figure 3 Path analysis.

mHealth apps and hospital information systems. However, usefulness. In this case, technical support and training could
since there is no active implementation of mHealth or use of be perceived as insignificant to physicians.
mHealth apps within an organized official capacity, there Habit (HB) and social influence (SI) were not found
may not be a perception about compatibility in behavioral influential on BI, which is consistent with the literature
intention, simply due to lack of practice [79,80]. Similarly, [42,81]. Here, the argument was that mHealth app usage
mobile self-efficacy (MS) was observed to exist; however, had not reached a sufficient level of routine usage to be
the conceptualization of mHealth use in practice might be considered habitual. Considering the interview findings,
ambiguous due to limited apps for each specialty. The there is a certain level of use of mobile apps which can be
physicians reported the need for apps for their own speci- considered as a habit (e.g. Communication apps), yet it is
alty. For instance, Gynecologist informants stated that not applicable for the use of other mHealth apps. On the
there are many popular apps for pregnancy monitoring; other hand, the insignificant effect of social influence could
however, there are few apps for use by physicians. Thus, be the result of time constraints, lack of interest and
self-efficacy might not have been assessed due to lack of awareness of using mobile health apps [33]. In addition,
specialty-specific mHealth apps, which require further physicians reported that they prefer searching for apps
development [76]. themselves, and do not engage in conversations about
For similar reasons, the insignificant relationship of mHealth apps very often. In cases where they were unable
technical support and training (TT) and BI can be explained. to find a particular app, they may ask a colleague. One
Since there is no formal use or regulations for mHealth apps, participant reported that he always uses web sources for all
voluntary use resulted in low motivation toward the needs apps he downloads.
of technical support and training [43]. Interview findings Unexpectedly, effort expectancy (EE) had no influence on
supported this argument. The physicians reported having a BI. The influence of EE had been repeatedly justified in
busy schedule and low-level willingness to participate in any many studies [73,74,82]. However, the findings suggest that
training program or to have technical support. Instead, they perceived ease of using mHealth apps had not reached a
prefer to use the internet for support and training. One of certain level of maturity. Interview findings supported that
the physicians reported that she used web blogs to search the insignificant impact could have been a result of the
for new mHealth apps and also for ratings of their frequency of app use in daily practice or using substitute
262 E. Sezgin et al.

Table 10 Results of hypothesis testing.

Hypotheses Relationship Path coefficients T Statistics p-Values VIF Status

HP1 PE-BI 0.359nn 4.072 0.000 2.334 Supported

HP6 PS-BI 0.120n 1.997 0.045 1.531
HP8 MA-BI " 0.160n 2.001 0.046 1.872
HP10 PI-BI 0.139n 1.996 0.047 2.051
HP22 CO-PE 0.383nn 4.536 0.000 1.521
HP15 PI-PE 0.284nn 3.548 0.000 1.344
HP19 RD-PE 0.196n 2.457 0.014 1.585
HP12 MS-EE 0.365nn 3.348 0.001 2.301
HP17 PS-EE 0.175n 2.007 0.041 1.613
HP4 CO-BI " 0.105 1.267 0.205 2.253 Not supported
HP23 CO-EE 0.103 1.347 0.178 2.061
HP2 EE-BI 0.106 1.475 0.140 2.139
HP7 HB-BI 0.077 0.905 0.366 2.360
HP14 HB-EE 0.146 1.505 0.132 2.278
HP11 MA-EE " 0.011 0.202 0.840 1.678
HP9 MS-BI 0.118 1.411 0.159 2.547
HP13 PI-EE 0.112 1.481 0.139 1.912
HP16 PS-PE 0.059 1.076 0.282 1.412
HP18 RD-EE 0.009 0.176 0.860 1.611
HP3 SI-BI 0.063 1.432 0.153 1.357
HP5 TT-BI " 0.06 1.204 0.229 1.463
HP21 TT-EE " 0.041 0.643 0.520 1.340
HP20 TT-PE -0.036 0.741 0.459 1.050
n
p o 0.05.
nn
po 0.001.

Table 11 Themes and codes of focus group interviews.

Themes Codes Sub-codes Count % Codes

Enablers Personal enablers Information gathering 27 12%

Communication 25 12%
Urgency 16 7%
Accessibility 10 5%
Interest in new technologies 9 4%
Education 8 4%
Ease of use 7 3%
Expectations 7 3%
Social sharing 5 2%
Leisure time 5 2%
Organizational enablers Compatibility 16 7%
Performance 13 6%
Assistance 6 3%
Barriers Personal barriers Lack of knowledge and interest 11 5%
Software problems 7 3%
Anxiety 5 2%
Organizational barriers Lack of investment 17 8%
Lack of control 15 7%
Enablers and barriers Habits 8 4%

apps or technologies (e.g. using web applications and desk- significant effect on EE. Here, it can be argued that ease of
top PCs in the visits). These might have created ambiguity in use of mHealth apps would be influenced by the physicians’
perceiving the ease of use and intention to use. ability and the availability of technology [44,45]. On the
In addition, even though the mediating and direct effect other hand, effort expectancy of physicians was found to
of EE was not significant in the study, MS and PS had a have no influence by HB, CO, MA, PI, RD, and TT. Since the
Investigation of physicians' awareness and use of mHealth apps 263

influence of EE on BI was not significant, its mediating effect and privacy issues, and also to promote standardization in
for the aforementioned factors remained redundant. Simi- mHealth app development [18].
larly, the PS and TT had no influence on performance Policymakers and managers need to consider the afore-
expectancy. Unlike Wu et al.’s [45] suggestion, PS showed mentioned implications in procedure and policy develop-
no encouraging indication to use mHealth by physicians. In ment in order to improve healthcare services for society.
fact, its impact could have been disregarded due to the lack There is a need for alignment between mHealth apps and
of conceptualization of the construct in real life. In a similar healthcare services to create an interoperable and control-
manner, the perception of TT might not have been identi- lable working environment for physicians [3]. In that regard,
fied by physicians due to lack of practice in mHealth it is important to note that mHealth apps need to be
support. Thus, the direct effect of TT on performance checked for reliability and consistency to ensure the main-
expectancy remained insignificant [11]. tenance of healthcare quality [32]. In the process of
mHealth platform development, costs, increasing workloads
and unscheduled tasks [20], trust, security [28], privacy
[31], standards and regulations [18,32] should be recognized
Practical and managerial implications and considered as major challenges. On-the-job training and
operational support were commonly suggested for the use
Communication, decision making, and information gather- of mHealth apps in order to enhance physicians’ ability to
ing are the primary aims of physicians using mHealth apps in perform their daily tasks. However, the physicians’ interest
healthcare services. This finding was supported by Franko in new technologies could be a leverage for promoting
and Tirrell's study [83], which outlined that commonly used training programs and to reduce their reluctance to use
apps among physicians were drug guides and medical mHealth apps. For instance, mobile game-based training
calculators, and the requested apps were about reference could be effective since physicians reported not to prefer
materials, treatment guides, and general medical knowl- on-site training programs [85].
edge. Thus, information gathering and communication
needs were the primary app choices of the physicians. Limitations
Blending these empirical results with the factors influen-
cing behavioral intention to use the technology, the evi-
A self-reported and cross-sectional survey may have caused
dence demonstrated that practical benefits are the key
the study to suffer from self-reporting biases. In addition,
elements in the actual use of mHealth apps (significant
the quantitative design limited the collection of all relevant
effect of PE, PS, and CO). In that regard, developers of
data within a scale in the research. Since TAM might have
mHealth apps should primarily focus on the practicality of
presented low predictive power in the study [86], the
apps more than focusing on the details [19,25]. In addition,
authors developed an integrated model, yet it might be
availability for timely use is another important element.
unable to explain the latent facts and relationships. On the
Even though the apps were available on the mobile plat-
other hand, the sample size was another limitation. Even
form, time is an important constraint, especially for physi-
though the sample size met the requirements to conduct
cians working to a tight scheduled [33]. Thus, high
the analyses [87], the data was limited in representing the
responsiveness and effectiveness would be an important
population. The sample size also limited the study to
benefit. Here, the optimization of mHealth apps specific to
capture differences among different specialties of physi-
medical specialties would enhance the usability.
cians. The constructs were adopted from previous studies
In the broader perspective, these needs require infra-
which were designed to assess new technologies (e.g.
structure development, focusing on compatibility and inter-
computers, handheld devices), but they were not specific
operability among mHealth apps, mobile healthcare
to mobile apps. This may be a reason for the unexplained
devices, and hospital management systems. In that regard,
variance in the study. Thus, the unexplained physician
government incentives would be influential to increase
behaviors may require further research in order to reveal
investments in mHealth infrastructure and to increase the
other factors in mHealth app usage. In the focus group
number of specialty-specific mHealth apps. Promoting the
interviews, the risk of collecting information from acquain-
use of mHealth apps could also positively affect job
tance physicians (as a result of the snowball approach),
performance, which would increase efficiency as well as
reluctant behaviors of informants, and the researchers’
the quality of doctor-patient time [76].
experiences and approach may have limited the depth of
Physicians’ willingness to use new technology should be
information gathered.
nurtured in order to promote mHealth app use. The trust in
mobile apps and the interest in new technologies are
positive attitudes which reduce the resistance toward new Conclusion
apps, but increase the vulnerability to possible malicious
content. Physicians’ reports in this study have already This study contributed to the literature in terms of provid-
raised some concerns about the lack of mHealth apps in ing a new model to explain the acceptance of mHealth apps
the domain [13] and privacy issues about health records by healthcare providers. In addition to that, providing a
[84]. Thus, the authorities should consider promoting the dataset from a developing country depicted an alternative
formal use of mHealth apps by controlling the content and outlook to influencing factors in using mHealth apps.
distribution. Authorities should supply apps considering the Furthermore, this study extended prior research about the
preferences of physicians in each specialty. Providing stan- perceptions and preferences on mobile healthcare apps
dards and regulations would also help to address security [36,76,88]. The authors would suggest further studying of
264 E. Sezgin et al.

mHealth apps in order to increase the depth of the current [2] Atluri V, Rao S, Rajah T, Schneider J, Thibaut M, Varanasi S. ,
study. In that regard, expanding the sample size and target et al. Unlocking digital health: opportunities for the mobile
group including different specialties would be a promising value chain; 2015.
step. A longitudinal approach is also suggested to report [3] OECD. OECD Digital Economy Outlook 2015 [Internet]; 2015.
behavioral changes on mHealth app use over time. Available from: 〈http://ec.europa.eu/eurostat/documents/
42577/3222224/Digital+economy+outlook+2015/
dbdec3c6-ca38-432c-82f2-1e330d9d6a24〉.
Author contribution [4] Istepanian RSH, Laxminarayan S, Pattichis CS. M-Health: emerging
mobile health systems [Internet]. In: Istepanian OSH, Laxminarayan
This research study is out of a PhD thesis work of the first S, Pattichis CS, editors. MHealth. London: Springer-Verlag; 2010.
author Mr. Emre Sezgin, who has completed his thesis work. p. 133–270. http://dx.doi.org/10.1007/b137697.
Emre Sezgin will defend his thesis in the next couple of [5] Hampton T. Recent advances in mobile technology benefit
global health, research, and care. Am Med Assoc 2012;307
months. The second and third authors Associate professor
(19):2013–4.
Dr. Sevgi Ozkan-Yildirim and professor Soner Yildirim are [6] Källander K, Tibenderana JK, Akpogheneta OJ, Strachan DL,
supervisors of him and they have initiated, directed and Hill Z, ten Asbroek AH, et al. Mobile health (mHealth)
critically contributed to the research and the paper. approaches and lessons for increased performance and reten-
tion of community health workers in low- and middle-income
Author statements countries: a review. J Med Internet Res 2013;15(1):e17.
Available from: 〈http://www.jmir.org/2013/1/e17/〉.
[7] Varshney U. Mobile health: four emerging themes of research.
Funding Decis Support Syst 2014;66:20–35. Available from: 〈http://
linkinghub.elsevier.com/retrieve/pii/S0167923614001754〉.
There is no direct funding for his research. However the [8] Rai A, Chen L, Pye J, Baird A. Understanding determinants of
authors have been receiving grants from TUBITAK (The consumer mobile health usage intentions, assimilation, and channel
Turkish Scientific and Research Council) for doing this preferences. J Med Internet Res [Internet] 2013;15(8):e149. Avail-
research. able from: 〈http://www.pubmedcentral.nih.gov/articlerender.fcgi?
artid=3742412&tool=pmcentrez&rendertype=abstract〉.
[9] Hung M-C, Jen W-Y. The adoption of mobile health manage-
Conflict of interest ment services: an empirical study. [cited 2012 Mar 21]. J Med
Syst [Internet] 2010;36:1381–8. Available from: http://www.
Up to the authors' knowledge, there is no conflict of ncbi.nlm.nih.gov/pubmed/20878452.
interest. The survey conducted has been accomplished [10] Wu I, Li J, Fu C, Wu S. The acceptance of wireless healthcare
voluntarily by the participating physicians' willing and for individuals : an integrative view. Proc 12th Int Conf Enterp
Inf Syst 5; 2010. p.1–6.
consent. The survey participants were informed about the
[11] Wu J-H, Wang S-C, Lin L-M. Mobile computing acceptance
research they are involved and they accepted the terms and
factors in the healthcare industry: a structural equation
conditions on the interpretation of the results of the model. [cited 2012 Mar 21]. Int J Med Inform [Internet]
research. 2007;76(1):66–77. Available from: http://www.ncbi.nlm.nih.
gov/pubmed/16901749.
Ethical approval [12] Number of physicians, number of persons per physician and
number of patient hospital visits per physician- 2009–2014
[Internet]. Ankara; 2015. Available from: 〈www.tuik.gov.tr/
Not required.
PreIstatistikTablo.do?Istab_id=1612〉.
[13] Kahn J, Yang J, Kahn J. Mobile health needs and opportunities
Acknowledgements in developing countries. Health Aff [Internet] 2010;29
(2):252–8. Available from: 〈http://content.healthaffairs.org/
content/29/2/252.short〉.
This study was supported by The Scientific and Technological
[14] Chib A, van Velthoven MH, Car J. mHealth adoption in low-
Research Council of Turkey (TUBITAK) with 2211C doctoral resource environments: a review of the use of mobile health-
research scholarship program. care in developing countries. J Health Commun [Internet]
2015;20(1):4–34. Available from: 〈http://www.ncbi.nlm.nih.
Appendix A. Supporting information gov/pubmed/24673171〉.
[15] PwC Health Research Institute. Top health industry issues of
2015 [Internet]; 2014. Available from: 〈http://www.pwc.com/
Supplementary data associated with this article can be found en_US/us/health-industries/top-health-industry-issues/down
in the online version at http://dx.doi.org/10.1016/j.hlpt. load.jhtml〉.
2017.07.007. [16] Wolters Kluwer Health 2013 Physician Outlook Survey [Inter-
net]; 2013. Available from: 〈http://www.wolterskluwerhealth.
com/News/Pages/latestnews.aspx〉.
[17] Ventola CL. Mobile devices and apps for health care profes-
References sionals: uses and benefits. [cited 2015 Feb 10]. Pharm Ther
[Internet] 2014;39(5):356–64. Available from: http://www.
[1] World Health Organization. European health report- Targets pubmedcentral.nih.gov/articlerender.fcgi?artid=4029126&
beyond reaching new frontiers in evidence; 2015. tool =pmcentrez&rendertype=abstract.
Investigation of physicians' awareness and use of mHealth apps 265

[18] Becker S, Miron-Shatz T, Schumacher N, Krocza J, Diamantidis University. Libr Rev [Internet] 2004;53(3):150–6. [Available
C, Albrecht U-V. mHealth 2.0: experiences, possibilities, and from] 〈http://www.emeraldinsight.com/doi/abs/10.1108/00
perspectives. JMIR mHealth uHealth [Internet] 2014;2(2):e24. 242530410526556〉.
Available from: 〈http://mhealth.jmir.org/2014/2/e24/〉. [34] Solvoll T, Scholl J, Hartvigsen G. Physicians interrupted by
[19] Liu C, Zhu Q, Holroyd K a, Seng EK. Status and trends of mobile devices in hospitals: understanding the interaction
mobile-health applications for iOS devices: a developer's between devices, roles, and duties. J Med Internet Res
perspective. Available from:. J Syst Softw [Internet] 2011;84 2013;15(3):1–12.
(11):2022–33. http://dx.doi.org/10.1016/j.jss.2011.06.049. [35] Jersak LC, Costa AC, Callegari Da, Jersak LC, Adriana C,
[20] Gagnon M-P, Ngangue P, Payne-Gagnon J, Desmartis M. m-Health Callegari DA. A Systematic Review on Mobile Health Care;
adoption by healthcare professionals: a systematic review. J Am 2013.
Med Inform Assoc [Internet] 2015:1–10. Available from: 〈http:// [36] Praveen D, Patel A, Raghu A, Clifford GD, Maulik PK, Moham-
jamia.oxfordjournals.org/cgi/doi/10.1093/jamia/ocv052〉. mad Abdul A, et al. SMARTHealth India: development and field
[21] Steven R, Steinhubl M. Can mobile health technologies transform evaluation of a mobile clinical decision support system for
health care? JAMA [Internet] 2013;310(22):2395–6. Available cardiovascular diseases in rural India. JMIR mHealth uHealth
from: 〈http://list.umassmed.edu/read/attachment/136612/1/ [Internet] 2014;2(4):e54. Available from: 〈http://www.ncbi.
Steinhubl_JAMA_MHTTransformHealthCare_2013.pdf〉. nlm.nih.gov/pubmed/25487047〉.
[22] Ozdalga E, Ozdalga A, Ahuja N. The smartphone in medicine: a [37] Davis FD. Perceived usefulness, perceived ease of use, and
review of current and potential use among physicians and user acceptance of information technology. Manag Inf Syst
students. J Med Internet Res 2012;14(5):1–14. 1989;13(3):319–40.
[23] Nhavoto JA, Grönlund Å. Mobile technologies and geographic [38] Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance
information systems to improve health care systems: a litera- of information technology: toward a unified view. MIS Q
ture review. JMIR mHealth uHealth [Internet] 2014;2(2):e21. 2003;27(3):425–78.
Available from: 〈http://mhealth.jmir.org/2014/2/e21/〉. [39] Fishbein M, Ajzen I. In: Park DC, Liu LL, editors. Belief, attitude,
[24] Zwart CM, He M, Wu T, Demaerschalk BM, Mitchell JR, Hara AK. intention and behaviour: An introduction to theory and research
Selection and pilot implementation of a mobile image viewer: a [Internet]. Reading MA AddisonWesley. MA: Addison-Wesley; 1975.
case study. JMIR mHealth uHealth [Internet] 2015;3(2):e45. Avail- p. 480. Available from: 〈http://www.people.umass.edu/aizen/
able from: 〈http://www.pubmedcentral.nih.gov/articlerender. f&a1975.html〉.
fcgi?artid=4463772&tool=pmcentrez&rendertype=abstract〉. [40] Rogers EM. Diffusion of Innovations [Internet]. In: Salwen MB,
[25] Sama PR, Eapen ZJ, Weinfurt KP, Shah BR, Schulman K a. An Stacks DW, editors. An integrated approach to communication
Evaluation of Mobile Health Application Tools. JMIR mHealth theory and research. NY: Free Press; 1995. p. 519. (Health
uHealth [Internet]. 2014;2(2):e19. Available from: 〈http:// Behavior and Health Education: Theory, Research, and Practice;
mhealth.jmir.org/2014/2/e19/〉. vol. 65) Available from: 〈http://books.google.com/books?
[26] Fiordelli M, Diviani N, Schulz PJ. Mapping mHealth research: a hl=en&lr=&id=v1ii4QsB7jIC&pgis=1〉.
decade of evolution. J Med Internet Res [Internet] 2013;15(5): [41] Kijsanayotin B, Pannarunothai S, Speedie SM. Factors influen-
e95. Available from: 〈http://www.jmir.org/2013/5/e95/〉. cing health information technology adoption in Thailand's
[27] Xu W, Liu Y. mHealthApps: a repository and database of mobile community health centers: applying the UTAUT model. Int J
health apps. JMIR Mhealth Uhealth [Internet] 2015;3(1):e28. Med Inform 2009;78:404–16.
Available from: 〈http://mhealth.jmir.org/2015/1/e28/〉. [42] Gagnon M-P, Godin G, Gagné C, Fortin J-P, Lamothe L,
[28] Hale K, Capra S, Bauer J. A framework to assist health Reinharz D. An adaptation of the theory of interpersonal
professionals in recommending high-quality apps for support- behaviour to the study of telemedicine adoption by physicians.
ing chronic disease self-management: illustrative assessment [cited 2012 Mar 26]. International J Med Inform [Internet]
of type 2 diabetes apps. JMIR mHealth uHealth [Internet] 2003;71(2-3). Available from: http://linkinghub.elsevier.com/
2015;3(3):e87. Available from: 〈http://www.ncbi.nlm.nih. retrieve/pii/S1386505603000947.
gov/pubmed/26369346〉. [43] Venkatesh V, Thong JYL, Xu X. Consumer acceptance and use
[29] Burke LE, Ma J, Azar KMJ, Bennett GG, Peterson ED, Zheng Y, of information technology: extending the unified theory of
et al. Current science on consumer use of mobile health for acceptance and use of technology. MIS Q [Internet] 2012;36
cardiovascular disease prevention: a scientific statement from (1):157–78. Available from: 〈http://papers.ssrn.com/sol3/
the American heart association. Circulation. 2015;132. papers.cfm?abstract_id=2002388〉.
[30] Free C, Phillips G, Watson L, Galli L, Felix L, Edwards P, et al. [44] Aggelidis VP, Chatzoglou PD. Using a modified technology
The effectiveness of mobile-health technologies to improve acceptance model in hospitals. Int J Med Inform, 78; 115–26.
health care service delivery processes: a systematic review [45] Wu I-L, Li J-Y, Fu C-Y. The adoption of mobile healthcare by
and meta-analysis. PLoS Med [Internet] 2013;10(1):e1001363. hospital’s professionals: An integrative perspective. [cited
Available from: 〈http://www.scopus.com/inward/record.url? 2012 Mar 1]. Decis Support Syst [Internet] 2011;51(3):587–96.
eid=2-s2.0-84873845389&partnerID=40&md5=7caa0d7bae3b Available from: http://linkinghub.elsevier.com/retrieve/pii/
9ede6a8d2a11803011ce〉. S0167923611000911.
[31] Kumar S, Nilsen WJ, Abernethy A, Atienza A, Patrick K, Pavel [46] Yi MY, Jackson JD, Park JS, Probst JC. Understanding informa-
M, et al. Mobile health technology evaluation: the mHealth tion technology acceptance by individual professionals:
evidence workshop. Am J Prev Med [Internet] 2013;45 Toward an integrative view. [cited 2012 Mar 9]. Inf Manag
(2):228–36. Available from: 〈http://www.ncbi.nlm.nih.gov/ [Internet] 2006;43:350–63. Available from: http://linkinghub.
pubmed/23867031〉. elsevier.com/retrieve/pii/S0378720605000716.
[32] Barton AJ. The regulation of mobile health applications. BMC [47] Hung S-Y, Ku Y-C, Chien J-C. Understanding physicians’
Med [Internet] 2012;10(1):46. Available from 〈http://www. acceptance of the Medline system for practicing evidence-
biomedcentral.com/1741-7015/10/46〉. based medicine: a decomposed TPB model. [cited 2012 Mar
[33] [ur] Rehman S, Ramzy V. Awareness and use of electronic 28]. Int J Med Inform [Internet] 2012;81(2):130–42. Available
information resources at the health sciences center of Kuwait from: http://www.ncbi.nlm.nih.gov/pubmed/22047627.
266 E. Sezgin et al.

[48] Agarwal R, Prasad J. A conceptual and operational definition of [67] Tabachnick BG, Fidell LS. Using multivariate statistics. Boston:
personal innovativeness in the domain of information technol- Pearson Education; 983.
ogy. Inf Syst Res 1998;9(2):204–15. [68] F. Hair Jr J, Sarstedt M, Hopkins L, G. Kuppelwieser V. Partial
[49] Schaper LK, Pervan GP. ICT and OTs: a model of information least squares structural equation modeling (PLS-SEM). Eur Bus
and communication technology acceptance and utilisation by Rev [Internet] 2014;26(2):106–21. [Available from]〈http://www.
occupational therapists. Int J Med Inform [Internet] 2007;76 emeraldinsight.com/doi/abs/10.1108/EBR-10-2013-0128〉.
(1):212–21. Available from: http://www.ncbi.nlm.nih.gov/ [69] Wong KK. Partial least squares structural equation modeling
pubmed/16828335. (PLS-SEM) techniques using smartPLS. Mark Bull [Internet]
[50] Chen J, Park Y, Putzer GJ. An examination of the components 2013;24:1–32. Available from: 〈http://marketing-bulletin.mas
that increase acceptance of smartphones among healthcare sey.ac.nz/v24/mb_v24_t1_wong.pdf〉.
professionals. Electron J Heal Inform 2010;5(2):1–12. [70] Grewal R, Cote JA, Baumgartner H. Multicollinearity and
[51] Venkatesh V, Davis FD. A theoretical extension of the technol- measurement error in structural equation models: implications
ogy acceptance model: four longitudinal field studies. Manag for theory testing. Available from:. Mark Sci [Internet] 2004;23
Sci 2000;46(2):186–204. (4):519–29. http://dx.doi.org/10.1287/mksc.1040.0070.
[52] Venkatesh V, Bala H. Technology acceptance model 3 and a http://pubsonline.informs.org/doi/.
research agenda on interventions. Decis Sci [Internet]., 39; [71] Fusch PI, Ness LR. Are we there yet? Data saturation in
273–315. http://dx.doi.org/10.1111/j.1540-5915.2008.00192.x. qualitative research Qual Rep [Internet] 2015;20(9):1408–16.
[53] Ajzen I. The theory of planned behavior. Organ Behav Hum Available from: http://tqr.nova.edu/wp-content/uploads/
Decis Process 1991;50:179–211. 2015/09/fusch1.pdf.
[54] Allen I, Seaman C. Likert scales and data analyses. Qual Prog [72] Vital Wave Consulting. mHealth for development: the oppor-
2007;7:64–5. tunity of mobile technology for healthcare in the developing
[55] Kline RB. Principles and practice of structural equation world. Washington, D.C. and Berkshire, UK; 2009.
modeling. Struct Equ Model 2010;156:427. [73] Sezgin E, Özkan-Yildirim S. A literature review on attitudes of
[56] Ringle C, Wende S, Will A. Smart-PLS Version 2.0 M3 [Internet]. health professionals towards health information systems: from
University of Hamburg; 2005. [cited 2015 Oct 2][. Available e-Health to m-Health. [cited 2015 Jun 2]. Procedia Technol
from]〈www.smartpls.de〉. [Internet] 2014;16:1317–26. Available from: http://www.scien
[57] Glaser BG, Strauss AL The discovery of grounded theory: cedirect.com/science/article/pii/S2212017314003752.
strategies for qualitative research [Internet]. Vol. 3, Sociology [74] Holden RJ, Karsh B-T. The technology acceptance model: its past
The Journal of The British Sociological Association. Aldine;
and its future in health care. [cited 2012 Mar 4]. J Biomed Inform
1967. 269-270 p. Available from: http://dx.doi.org/http://
[Internet] 2010;43(1):159–72. Available from: http://www.pub
soc.sagepub.com/cgi/doi/10.1177/003803856900300233.
medcentral.nih.gov/articlerender.fcgi?artid=2814963&tool=
[58] Recker J Scientific research in information systems: a beginner’s
pmcentrez&rendertype=abstract.
guide [Internet]. 2012 [cited 2015 Jan 15]. Available from: 〈http://
[75] Chau PYKP, Hu PJ. Investigating healthcare professionals’
books.google.com/books?Hl=en&lr=&id=LzGbLKxjKHAC&oi=
decisions to accept telemedicine technology: an empirical
fnd&pg=PR3&dq=Scientific+Research+in+Information+Systems
test of competing theories. [cited 2012 Jun 11]. Inf Manag
+A+Beginner%25E2%2580%2599s+Guide&ots=1fj_sF8jGf&sig=
[Internet] 2002;39:297–311. Available from: http://www.scien
VHAw-vGPiubS1OXmvln4FnNRrz0〉.
cedirect.com/science/article/pii/S0378720601000982.
[59] Mishler EG. Research Interviewing: context and Narrative
[76] Duhm J, Fleischmann R, Schmidt S, Hupperts H, Brandt SA.
[Internet]. Harvard University Press; 1991. [cited 2012 Jun 13]
Mobile electronic medical records promote workflow: physi-
[. 206 p. Available from]〈http://www.amazon.com/Research-In
cians’ perspective from a survey. [cited 2016 Jul 5]. JMIR
terviewing-Narrative-Elliot-Mishler/dp/0674764617〉.
[60] Spall S. Peer debriefing in qualitative research: emerging mHealth uHealth [Internet] 2016;4(2):e70. Available from:
operational models. [Available from: 〈http://qix.sagepub. http://mhealth.jmir.org/2016/2/e70/.
com/cgi/doi/〉. Qual Inq [Internet] 1998;4(2):280–92. http: [77] Ducey AJ, Coovert MD. Predicting tablet computer use: an
//dx.doi.org/10.1177/107780049800400208. extended technology acceptance model. Heal Policy Technol
[61] Ghasemi A, Zahediasl S. Normality tests for statistical analysis: [Internet] 2016;5(3):268–84. http://dx.doi.org/10.1016/j.
a guide for non-statisticians. [cited 2014 Jul 13]. Int J hlpt.2016.03.010.
Endocrinol Metab [Internet] 2012;10(2):486–9. Available from: [78] Wu J, Wang S, Lin L. What drives mobile health care ? An
http://www.pubmedcentral.nih.gov/articlerender.fcgi? empirical evaluation of technology acceptance. In: Proc 38th
artid =3693611&tool=pmcentrez&rendertype=abstract. Hawaii Int Conf onf Syst Sci; 2005. p. 1–9.
[62] Steel RGD, Torrie JH, Dickey DA. Principles and procedures of [79] Embi PJ, Yackel TR, Logan JR, Bowen JL, Cooney TG, Gorman
statistics: a biometrical approach. New York: McGraw-Hill; PN. Impacts of computerized physician documentation in a
352–401. teaching hospital: perceptions of faculty and resident physi-
[63] Fornell C, Larcker D. Evaluating structural equation models cians. J Am Med Inform Assoc 2004;11(4):300–9.
with unobservable variables and measurement error. [cited [80] Georgiou A, Ampt A, Creswick N, Westbrook JI, Braithwaite J.
2015 Sep 30]. J Mark Res [Internet] 1981. Available from: Computerized Provider Order Entry–what are health profes-
http://www.jstor.org/stable/3151312. sionals concerned about? A qualitative study in an Australian
[64] Hair J, Black W, Babib B, Anderson R, Tatham R. Multivariate hospital [cited 2011 Aug 1]. Int J Med Inform [Internet]
data analysis [Internet]. 7th ed. Analysis. NJ: Prentice Hall; 2009;78(1):60–70. Available from: http://www.ncbi.nlm.nih.
2009. p. 816. Available from: 〈http://www.lavoisier.fr/livre/ gov/pubmed/19010728.
notice.asp?ouvrage=1086590〉. [81] Yu P, Li H, Gagnon MP. Health IT acceptance factors in long-
[65] Nunnally J, Bernstein I. Psychometric theory. 3rd ed. New term care facilities: a cross-sectional survey. [cited 2012 Mar
York. NY: McGraw-Hill; 701. 28]. Int J Med Inform [Internet] 2009;78(4):219–29. Available
[66] Segars AH. Assessing the unidimensionality of measurement: a from: http://www.ncbi.nlm.nih.gov/pubmed/18768345.
paradigm and illustration within the context of information [82] Hsiao C-H, Tang K-Y, Liu JS. Citation-based analysis of
systems research. [cited 2015 Sep 30]. Omega [Internet] literature: a case study of technology acceptance research.
1997;25(1):107–21. Available from: http://www.sciencedir Scientometrics [Internet], 105; 1091–110. Available from:
ect.com/science/article/pii/S0305048396000515. 〈http://link.springer.com/10.1007/s11192-015-1749-5〉.
 Investigation of physicians' awareness and use of mHealth apps 267

[83] Franko OI, Tirrell TF. Smartphone app use among medical [86] Ward R. The application of technology acceptance and diffu-
providers in ACGME training programs. J Med Syst 2012;36 sion of innovation models in healthcare informatics. [cited
(5):3135–9. 2013 Aug 29]. Heal Policy Technol [Internet] 2013;2(4):222–8.
[84] Kessel KA, Vogel MM, Schmidt-Graf F, Combs SE. Mobile apps in Available from: http://linkinghub.elsevier.com/retrieve/pii/
oncology: a survey on health care professionals' attitude S2211883713000543.
toward telemedicine, mhealth, and oncological apps. J Med [87] Goodhue DL, Lewis W, Thompson R, Does PLS. have advantage
Internet Res [Internet] 2016;18(11):e312. Available from: for small sample size or non-normal data? MIS Q 2012;36
〈http://www.jmir.org/2016/11/e312/〉. (3):1–16.
[85] Edwards EA, Lumsden J, Rivas C, Steed L, Edwards LA, [88] Hao H, Padman R, Telang R Physician’s usage of mobile clinical
Thiyagarajan A, et al. Gamification for health promotion: applications in a community hospital: a longitudinal analysis of
systematic review of behaviour change techniques in smart- adoption behavior [Internet]. UK Academy for Information
phone apps. BMJ Open [Internet] 2016;6(10):e012447. [Avail- Systems Conference Proceedings; 2013. [cited 2016 Apr 8].
able from]〈http://www.ncbi.nlm.nih.gov/pubmed/27707829〉. Available from: 〈http://aisel.aisnet.org/ukais2013/11〉.

View publication stats