What patients want: relevant health

information technology for diabetes self-

management

Diane K. King, Deborah J. Toobert,

Jennifer Dickman Portz, Lisa
A. Strycker, Alyssa Doty, Carmen
Martin, Jennifer M. Boggs, et al.
Health and Technology

ISSN 2190-7188
Volume 2
Number 3

Health Technol. (2012) 2:147-157

DOI 10.1007/s12553-012-0022-7

1 23
Your article is protected by copyright and all
rights are held exclusively by IUPESM and
Springer-Verlag. This e-offprint is for personal
use only and shall not be self-archived in
electronic repositories. If you wish to self-
archive your work, please use the accepted
author’s version for posting to your own
website or your institution’s repository. You
may further deposit the accepted author’s
version on a funder’s repository at a funder’s
request, provided it is not made publicly
available until 12 months after publication.

1 23
 Author's personal copy
Health Technol. (2012) 2:147–157
DOI 10.1007/s12553-012-0022-7

ORIGINAL PAPER

What patients want: relevant health information technology

for diabetes self-management
Diane K. King & Deborah J. Toobert &
Jennifer Dickman Portz & Lisa A. Strycker &
Alyssa Doty & Carmen Martin & Jennifer M. Boggs &
Andrew J. Faber & Cristy R. Geno & Russell E. Glasgow

Received: 13 October 2011 / Accepted: 23 February 2012 / Published online: 5 March 2012
# IUPESM and Springer-Verlag 2012

Abstract Health information technology has great potential and to receive tailored information. Choice of technology,
to promote efficiency in patient care and increase patient- personalized instruction on how to use program features, and
provider communication, and patient engagement in their the ability to exchange information with their healthcare team
treatment. This paper explored qualitatively what patients were desired by all participants. Participants were divided on
with type 2 diabetes want from electronic resources that are whether virtual social support networks should be closed to
designed to support their diabetes self-management. Data friends and family, should include other program members
were collected via interviews and focus groups from man- (peers), or should be open to anyone with diabetes. Partici-
aged care patients who had completed participation in either pants aged 65 and older stressed the desire for technical
a web-based (MyPath) or in-person group-based (¡Viva support. What patients wanted from technology is real-time
Bien!) longitudinal diabetes self-management study. Con- assistance with daily behavioral decision-making, ability to
tent analysis identified common themes that highlighted share information with their healthcare team, connections with
participant interest in virtual and electronic programs to others for support, and choice.
support diabetes self-management goals, and their desired
content and features. Eighteen ¡Viva Bien! participants com- Keywords Technology . Diabetes . Self-management .
pleted telephone interviews and 30 MyPath participants Health behavior change
attended seven focus groups in 2010-2011. All participants
expressed a preference for face-to-face contact; however,
most participants were also interested in using technology 1 Introduction
as a tool to support daily diabetes self-management decisions
The Internet and availability of e-health interventions and
resources offer promise for assisting adults with diabetes to
D. K. King (*)
Center for Behavioral Health Research and Services, maintain health behaviors [1]. With the increase in the
University of Alaska Anchorage, number of people living with diabetes [2–4]; and with
Anchorage, AK, USA annual diabetes-related costs expected to increase to $192
e-mail: [email protected]
billion by 2020 [5], the demand for diabetes services far
D. J. Toobert : L. A. Strycker surpasses the time available in most primary care practices
Oregon Research Institute, [6, 7]. E-health technology could help relieve the strain [8].
Eugene, OR, USA Most diabetes management happens outside of clinics [9]
J. D. Portz : A. Doty : C. Martin : J. M. Boggs : A. J. Faber :
yet patients look to healthcare providers for counseling and
C. R. Geno support. Unfortunately, due to limitations in staffing and
Institute for Health Research, Kaiser Permanente Colorado, reimbursement, counseling patients on diet, exercise, and
Denver, CO, USA other important self-management behaviors is not accom-
plished as part of routine primary care [10, 11]. Research
R. E. Glasgow
National Cancer Institute, shows that in-person interventions can improve behavioral
Rockville, MD, USA and biologic outcomes [12–15], but questions remain about
 Author's personal copy
148 Health Technol. (2012) 2:147–157

whether their relatively high cost might be reduced using limitations reduce access to health-related programs and
technology without limiting their effectiveness. Well- services [62]. People older than age 60 with low incomes
designed, patient-centered [8, 16–18] e-health technologies or less education are less likely to be online [63], but
could enhance access to diabetes self-management programs Internet use is rising among these groups [23], in part due
and promote dissemination [19, 20]. to the widespread adoption of web-enabled mobile phones
[64]. To improve health outcomes for vulnerable popula-
1.1 Advantages of e-health technology tions [65, 66], Internet-delivered behavioral change pro-
grams must consider age [59, 67, 68], ethnicity [32, 63, 64,
Advantages include portability, timeliness, efficiency [21], 69–77], literacy levels [59, 78–80], and gender [68, 81].
scalability [22–24], and few barriers based on geography or Slower adoption of mobile technology by American Indians/
mobility [25]. E-health programs can be tailored [26], can Alaska Natives (AI/AN) has been reported—likely due to
deliver support as needed [21] and prevent relapse [27–31], rural connectivity issues [71]. A recent pilot web-based dia-
increase access to healthcare professionals, and may lower betes self-management study exploring ethnicity and mobile
healthcare costs [23, 32]. With the introduction of the technology adoption, compared AI/AN to non-AI/AN partic-
patient-centered medical home (PCMH) [16], opportunities ipants. The study found that the AI/AN who were more
exist for behavioral scientists to work with healthcare prac- frequent users were slightly younger, had less formal educa-
tices to design electronic technologies to achieve practice and tion, weighed more, were more depressed, and had more
patient goals [33]. hypoglycemic symptoms. AI/AN participants expressed a
strong preference for an “all-AI/AN” website [72, 73].
1.2 Use of electronic programs to improve health behaviors
1.5 Barriers to Internet use
Patients with diabetes are responsible for self-management
(e.g., eating a healthful diet, engaging in regular physical The healthcare sector has been slow to embrace information
activity, taking prescribed medications) to help them technology [82]. However, as adoption of electronic health
avoid complications [34] and risk of cardiovascular disease records spreads, along with increased incentives and regu-
[35–38]. With few exceptions [23] research has demonstrated lations to promote their meaningful use [21], patients are
improvements in diabetes outcomes and chronic illness self- experiencing increased opportunities to communicate with
management behaviors [32, 39–41] when behavioral support their doctor [83], view their health information, and receive
is available through a variety of technologies (e.g., cell phone health promotion information [84–87] electronically. Bar-
and automated text messaging [39, 42–45], hand-held devices riers to website use include fear of loss of privacy, intrusive-
[46] and e-communication with healthcare providers [47, 48]). ness, cost, and loss of interest over time [58]. Engagement
E-health interventions have helped improve dietary practices with health related programs could be enhanced by providing
[23, 49], physical activity [28, 50, 51], medication taking [32, patients with feedback that is frequent, tailored, interesting,
49, 52], clinical outcomes [32, 39, 40, 53, 54], and psychoso- encouraging, nonjudgmental, and unpredictable [58, 81, 88].
cial variables [55]. In addition, inclusion of self-monitoring features, such as
tracking minutes of exercise, can increase engagement [88]
1.3 Use of electronic programs to access support and enhance goal achievement [89].
and resources
1.6 Added value of human contact
Use of social networking services (SNS) [23] has nearly
doubled since 2008, with fastest growth reported among The value of using health information technology (HIT) to
Internet users ages 50–64 and females [56]. Studies suggest provide support for chronic disease self-management, may
that online support groups have positive effects on per- be limited if it does not include human contact from health-
ceived social support as well as on behavioral and clinical care providers or social support networks [81, 84, 90]. The
outcomes [57]. Recent studies of e-health technology [58, ideal combination of human contact and technology has not
59] found that patients desire real-time tailored support from been thoroughly explored [23, 61, 91, 92]. Reported attri-
other study participants and healthcare providers. tion rates vary widely in studies of e-health interventions
with and without human contact, For example, Rabin and
1.4 Reach of e-health interventions Glasgow, in their report on implementation studies of inter-
active health communication applications described attrition
E-health technology has the potential to help assure that rates as low as 3% for an interactive web-based intervention
interventions reach individuals in need of tailored support that included live telephone counseling, and attrition as high
[60, 61] or when geographic barriers and health services as 65% for a publically available web-based smoking
 Author's personal copy
Health Technol. (2012) 2:147–157 149

cessation intervention that lacked personalized human depth descriptions of the interventions and quantitative find-
follow-up [61]. E-health interventions with human contact ings are published elsewhere [14, 30, 89, 94, 95]. In brief,
may be more effective than technology-only interventions, MyPath tested a minimal human contact, 12-month web-
but more research is needed [8, 90]. based self-management intervention designed to provide
electronic support for adults with type 2 diabetes to improve
1.7 Study purpose their eating, physical activity, and medication-taking behav-
iors [30]. The program was largely stand-alone, and partic-
Surprisingly little is known about the preferences and ipant engagement declined over time [89]. ¡Viva Bien!
characteristics of patients for whom e-health applications tested a 24-month in-person, group-based intervention for
are intended. This paper analyzes qualitative data collect- Latinas with type 2 diabetes to improve diet, physical activ-
ed from two recent clinical trials with type 2 diabetes ity, stress management, smoking cessation, and social sup-
patients, to better understand patient likes and dislikes port [96]. The ¡Viva Bien! intervention did not use
with regard to using health information technology as a technology, and cost of implementation was identified as a
tool for diabetes self-management. Inclusion of patients barrier to widespread dissemination [97] . For this reason we
who had recently been exposed to an electronic or a non- were interested in exploring use of Interactive Voice Rec-
electronic diabetes self-management program provided ognition (IVR) telephone technology to deliver some
context for our qualitative focus groups and interviews, aspects of the program, to potentially increase the adoptabil-
allowing patients to think about how they would use ity of the intervention in primary care. To better understand
technology for a specific health-related purpose. Thus, how to improve cost-effectiveness and long-term engage-
while the content areas were established a priori, the ment with diabetes self-management interventions, qual-
themes that emerged are potentially generalizeable to a itative data were collected from a subset of participants
broader range of HIT applications. In particular we ex- that had completed the main study components to under-
plored features of telephone and Internet-based programs stand their satisfaction with specific program elements,
to understand what would be useful to supporting their what they thought about using different forms of tech-
health behavior goals, and what aspects of in-person nology to achieve their diabetes self-management goals,
encounters with clinical staff, within the context of dia- and their recommendations for changes and enhance-
betes self-management support, could be augmented or ments to future versions of the two interventions with
replaced by technology. We also explored how technol- regard to the integration of technology into their daily
ogy could be used to connect with peers, friends and lives. For the qualitative components of MyPath and
family, to get support for changing their health behavior. ¡Viva Bien! the sample composition and size were
The results provide answers to questions relevant to both planned to assure that participants in interviews or focus
researchers and practitioners seeking to design interven- groups represented the main sample of patients that par-
tions that are convenient, acceptable, and useful [93]. ticipated in the clinical trials.
Our specific research questions were: How comfortable
were patients in using technology as a tool to support 2.2 Measures
their diabetes self-management, particularly if it replaced
some encounters with human beings? How did patients The present investigation draws on qualitative data gen-
view using technology as a way to share diabetes self- erated from focus groups with selected MyPath partic-
management information with their healthcare providers ipants and telephone interviews with selected ¡Viva
and social support networks? What would improve the Bien! participants. In MyPath, four focus groups were
user-friendliness of health information technology? In conducted to understand how to improve the website to
what ways would patients use electronic resources or maximize its impact on sustaining behavior change, two
tools to help them manage their specific health behaviors focus groups were held with people older than 65 years
such as eating, physical activity, and medication taking? of age to explore relevance of various technologies as
well as potential digital divide or computer literacy
issues, and one group was conducted with people hav-
2 Methods ing poorer medication adherence to understand their
special needs. In ¡Viva Bien! telephone interviews were
2.1 Participants conducted with a diverse group of participants to collect their
thoughts on virtual vs. human support and to elicit their
Participants were managed-care patients in two different reactions to the idea of accessing components of the interven-
type 2 diabetes self-management intervention studies: ¡Viva tion via telephone. Focus group and interview topics are
Bien! and MyPath. Details of these studies, including in- detailed in Table 1.
 Author's personal copy
150 Health Technol. (2012) 2:147–157

Table 1 Semi-structured telephone interview and focus group topics and items

Topic Items

Telephone Interview Items Focus Group Items

Interest in technologies to help with stress, How comfortable would you feel getting pre- In general, what types of things would you
diet, exercise, medication taking, or recorded phone calls from ¡Viva Bien! staff look for in a web-based program?
smoking about helping with your stress, diet, exercise Is it important for a web-based program to
or smoking? target diabetes specifically?
Do you think that you would use a virtual
coach in your everyday life to help you with
diabetes self-management?
Interest in technologies to connect with What if, when this phone system called, it let Do you think that you would use an “ask an
program staff or healthcare team you check in with anyone you want from expert” feature in your everyday life to help
¡Viva Bien! staff? you with diabetes self-management?
Interest in technologies to connect with How much would you like it if the system let Would you like to invite people to cooperate
peers and others for social support you check in with a buddy/another in goals/activities with social media?
participant you know from the ¡Viva Bien! Do you think you would like to??? add people
program? to your support group through social media?
Would you like to publish your (diet, exercise,
medication taking) accomplishments in this
social media format?
Would you like to read about other people’s
achievements or success stories in this way?
Would you like your supporters to see how
you are doing with your diabetes though
social media?
Would you like to receive feedback from your
supporters through social media?
Indication to use technologies How much do you think you might use the Would you use this type of program?
check-in option?
How much do you think you might use the How often would you use this program?
buddy/peer support option?
Helpfulness of technologies Overall, how helpful do you think a phone What do you like about this program or
system like this would be to you? feature?
What do you dislike about this program or
feature?
Would a place to track your medications in
addition to your food and exercise be
helpful?
Do you think this tool would help you
remember to take your medications?
What do you like about this tool in helping
you remember to take you medications?
Are you more/less likely to use a refill
program in comparison to the reminder
program?
Do you think this tool would help you stay on
top of medication refills?
Would you prefer/not prefer to use a tool
aiding in medication taking or simply have
available information or resources for
medication?
Recommendations for technologies What, specifically, would make a phone How would you improve this program?
system helpful for you? If you wanted to access one of these program
features through a phone, what features
would you want included?
Design and features of technologies N/A What do you think about the design and
layout?
 Author's personal copy
Health Technol. (2012) 2:147–157 151

Table 1 (continued)

Topic Items

Telephone Interview Items Focus Group Items

What features would make you want to visit

this site daily?
What other features would encourage you to
visit this site daily?
Do you think you could easily find this tool?
Do you think you could use all the features of
the tool?
Does the tool look simple enough to use? Too
simple?
Do you think it would be difficult to manage
or use this program?
Thinking about the computer you use most
often, do you think your computer could
access this site?

2.3 Focus group and interview procedures with the specific programs and to solicit patient input
into use of various forms and features of technology to
The MyPath study was conducted from 2008–2011. Seven support their diabetes self-management goals, content
focus groups were conducted in 2011 at Kaiser Permanente analyses and coding were initially performed separately
Colorado clinical locations. The 90-minute sessions were for each dataset. The coded data were quantified [100] and
led by research staff in English. Participants were asked to condensed into major categories that emerged inductive-
discuss their experiences with the project website, then were ly from each dataset [98]. These categories were then
shown other online sites and asked to share their likes and merged to reorganize the content extracted from the two
dislikes of those sites as well as potential uses of site datasets in order to answer the more general research ques-
features and tools. Topics included diet, exercise, medica- tions posed in this paper [101]. Preliminary findings were
tion taking, social media, and different types of technology. debriefed with the research team to confirm findings [102,
The ¡Viva Bien! study was conducted from 2006–2010. 103].
Semi-structured telephone interviews by project staff were
completed in 2010 with 18 ¡Viva Bien! participants. Inter-
views lasted about 60 min; 16 interviews were in English
and two were in Spanish. 3 Results

3.1 Participant characteristics

2.4 Data analyses
Participants were invited purposefully to assure demo-
Notes and audiotapes from focus groups and interviews graphic representation of those participating in the larger
were transcribed and coded by research staff. First-cycle study, with regard to gender, age, race and ethnicity. All
coding employed descriptive coding [98] by two indi- participants were adults with a diagnosis of type 2
viduals to analyze content and identify preliminary diabetes and at least one other risk factor for heart disease
themes; coders then met to clarify and define data (i.e., were obese or had poorly controlled A1c, Lipids,
codes. Using the set definitions and inclusion/exclusion or blood pressure). Thirty MyPath intervention partici-
criteria, three coders completed second-cycle coding pants were selected to attend one to two focus groups
[98, 99] with 79.1% inter-coder agreement. Eighty per- (N 07 focus groups) with about six (range 05–12)
cent of material was double coded to assure reliability. attendees per group. For the ¡Viva Bien! interviews,
Codes and themes were discussed with research team nine women were selected from the control group and
members to hone definitions and assure consistency. As nine from the intervention group representing a range of
data were originally collected for purposes specific to adherence to intervention components. Participant character-
each study, i.e., to understand participant satisfaction istics are presented in Table 2.
 Author's personal copy
152 Health Technol. (2012) 2:147–157

Table 2 MyPath focus group

and ¡Viva Bien! semi-structured Focus Groups (N030) Phone Interviews (N018)
interview participant Mean (SD) or % Mean (SD) or %
characteristics
Gender (% female) 53.3% 100%
Age 58.9 (10.4) 59.5 (8.9)
Body mass index 36.1 (8.6) 35.3 (7.4)
Baseline smokers 20.0% 5.9%
Income
< $10,000 (¡Viva Bien! $0–$14,999) 0% 16.7%
$10,000–$29,999 (¡Viva Bien! $15,000–$29,999) 17.2% 27.8%
$30,000–$49,999 27.6% 27.8%
$50,000–$69,999 27.6% 5.6%
$70,000–$89,999 6.9% 5.6%
$90,000+ 20.7% 16.7%
Glucose-lowering medication
Oral 63.3% 66.7%
Insulin 10.0% 11.1%
Insulin and oral 20.0% 11.1%
None 6.7% 11.1%
Level of education
<High school graduate 3.3% 22.2%
High school graduate 16.7% 22.2%
Technical/some college 40.0% 27.8%
Completed college 40.0% 27.8%
Race
American Indian/Alaska Native 10.0%
Asian 0%
Native Hawaii/Pacific Islander 0%
Black or African American 10.0%
White 73.3%
Did not answer 6.7%
There were no significant differ-
ences between MyPath or ¡Viva Hispanic/Latino (% Latino) 13.8% 100%
Bien! focus group/exit interview Health literacy: 1 (low)—5 (high) 4.9 (0.3) 4.4 (0.8)
participants and main-study Numeracy: 1 (low)—6 (high) 4.3 (1.2) 3.8 (1.0)
participants.

3.2 Answers to research questions used the Internet to help with their diet, to find exercises, and
for information on stress management. Focus group partici-
3.2.1 Using technologies to support diabetes pants had more experience using the Internet than interview
self-management participants, but said they preferred websites with menus that
allowed them to access a wide variety of information related to
Most of the 18 phone interviewees (n013) said an interactive their diabetes self-care; and allowing them to find what they
voice response (IVR)-type program would help them maintain needed in one place. One commented, “I would like to choose
a healthful lifestyle. Twelve said they would be either “a little from a pull-down list, this would be very useful, particularly
comfortable” (n04) or “very comfortable” (n08) with such an with medications.” Another mentioned the desirability of
automated system; only one interviewee said she would be having a single place to track everything. One said, “I track
“very uncomfortable”. When asked about getting pre- so many things, it’s nice that it’s all in one place.” Others
recorded phone messages to support their self-management, stressed the need for information that was tailored to them,
respondents felt that these should be tailored reminders that and suggested “smart” programs that “ask the right questions”
were specific to their goals. One commented that it would be and then recommend what they need. They also liked the idea
helpful to have, “somebody to give me a little jab to get me of being able to customize their own page (e.g., iGoogle) so
going.” Four of the telephone interviewees also said they have that information and tools were relevant. One said, “Yeah,
 Author's personal copy
Health Technol. (2012) 2:147–157 153

2000 calories is standard, but it would be important to be able not necessarily solely from experts. One said, “I’m not so
to change it to what people need and want.” interested in following skinny people who don’t have dia-
betes, but you can get really great information from healthy
3.2.2 Using technologies to connect with members friends who have great tips.” Asked about an automated
of the healthcare team or program staff telephone system that could provide tips via voicemail or
text, some phone interviewees stated a strong preference for
Using technology to communicate with others was acceptable, human contact, saying, it is “really frustrating to use auto-
but patients were less interested in one-way communication mation,” and, “there is nothing like a human voice.”
with automated support messages via phone, email, or text. In
addition, all participants were supportive of technology that 3.2.4 Improving the user-friendliness of health information
would allow them to report progress and receive coaching technologies
from members of their healthcare team or diabetes program
staff. Focus group participants expressed a strong desire to “Navigability was the top issue for both focus group and
share progress with their providers either electronically or via telephone interviewees when it came to using technology.
a computer printout. One said, “I would like my tracking Phone interviewees were concerned about the technologic
journal to be hyperlinked to my medical chart.” Patients also aspects of accessing information or responding to automated
liked the idea of being able to ask anonymous questions over questions using their telephone keypad, such as navigating
the Internet and have continuous access to information through a push-button system. One noted, “I get very con-
through a website that allowed them to ask questions of a fused.” Focus group participants were also concerned about
qualified person, as long as responses were prompt. Phone navigating through complicated websites. Participants
interviewees said it would be important to not just report didn’t want to have to work to access information. One
progress using an IVR system, but also to receive feedback noted, “I do not like it when I have to scroll down the page
on their goals. They said they would welcome suggestions on to get the information. I would like information to be split
how they could improve or meet goals, and would like to have up across pages.” Older participants said they would like
their questions answered by experts, such as a dietitian. Some technical support whenever they needed it. They agreed that
participants also said they would attend virtual Internet work- a simple list of sequential steps to set up the main features of
shops with audio and visual features. When participants were the program could be useful, but they wanted instructions.
asked about substituting virtual support for live support, such One said, “I would want help to create the page, and then get
as a virtual expert or coach, participants were favorable so suggestions or ideas on what to add.” Readability was also
long as the virtual expert or coach knew something about emphasized by focus group participants. Most expressed a
them and provided tailored advice. Use of avatars or animated preference for interfaces that were clean, clear of advertise-
characters as buddies or coaches was generally favored as a ments, and in an easy-to-read typeface. For participants
way to make the program more fun and “social”; but most older than age 65, contrasting color including use of dark
respondents preferred using technology to increase their ac- print on light backgrounds, and the ability to choose colors,
cess to knowledgeable human beings. Of the nine phone font size, and other design elements were important features.
interviewees asked how much they would like to be able to Patients also wanted to choose their own passwords, if
talk to a ¡Viva Bien! program staff member or another partic- required, rather than having one assigned to them, so they
ipant after completing an automated IVR call that asked them could easily remember it, versus having to find where they
about their self-management goal progress, two said they wrote it down. Cost was also raised as a potential issue,
“might like” and six said they “would definitely like” that particularly with using cell phones. Participants agreed that
option. Interviewees commented that such a feature would diabetes self-management programs and tools should be
help provide social support that was sometimes lacking in easily accessed through multiple platforms (i.e., cellphones,
their lives. One said it is important to be accountable to a real smartphones and computers) but must also be free of charge.
person because “it’s easy to lie with pushing buttons.” One said, “Some of us have unlimited text plans and others
may pay per text, so it is important to set up reminders that I
3.2.3 Using technologies to connect with social support wouldn’t be charged for.”
networks
3.2.5 Using Internet-based resources to manage specific
Focus group participants universally preferred in-person health behaviors (focus groups only)
contact to social networking sites such as Facebook, saying
in-person contact was “more engaging” and “interactive.” Healthful eating Patients differed on the specific Internet
They also said they liked receiving advice from those food resources they preferred, but agreed that they desired
with diabetes or from those who live healthful lifestyles, choice, the ability to customize, and receive practical and
 Author's personal copy
154 Health Technol. (2012) 2:147–157

timely information. Some favored food diaries and the abil- for supporting their diabetes self-management goals. The
ity to track nutritional factors and some wanted guidance on study data were generally consistent with the literature
meal planning. Participants said they would like recipe [21], indicating patient agreement that technology would
searches and meal planning based on specific criteria (e.g., encourage them to attend to their health-behavior goals,
carbohydrates, sodium, calories; vegetarian, kosher, Indian; and would be useful as a tool to track progress [54] and
under 200 calories, under 15 carbohydrates). Patients also receive helpful information at any time [58].
were interested in mobile technology applications to track Results indicate that, while individual diabetes patients
their eating “on the go” and to find restaurants that meet differ on how they use technology for behavioral support,
their nutritional criteria and preferences. they agree that they want choice. Also, the features they
desire from an e-health application differ for different health
Exercise Patients wanted a program to help them set exer- behaviors. That is, patients favor a range of tracking tools
cise goals, track exercise, and receive tailored feedback. and “just-in-time” information to support healthful eating,
They said it would be helpful to include all lifestyle activity, but for physical activity they prefer feedback on their energy
not solely exercise. One said, “I garden and do housework, balance and for medications they mostly want an easy way
not fitness. I tried the treadmill, and it’s not my thing. to look up information and receive refills.
Gardening is what I like to do.” Patients also wanted to The study underscores the importance of two-way com-
review their weekly, monthly and annual progress. One said, munication and human contact. Participants clearly want e-
“It gives you some perspective as to where you are at, and health technology to help facilitate exchanges with their
you can reward yourself.” Some patients desired download- healthcare providers, program staff, and peers, and they
able graphs and charts to illustrate their progress. They also are less interested in virtual health coaches or public online
asked for feedback on caloric intake, calories burned, calo- social networks.
ries remaining after exercising, and glucose readings. They This study also suggests that Internet and mobile tech-
were less interested in general exercise information. nologies can improve access to diabetes self-management
programs. Most of the ethnically and socioeconomically
Medication Unlike exercise, patients most wanted to use e- diverse participants said they would use technology to sup-
technology to access information. Suggestions included port their diabetes self-care. To reduce barriers, though, such
pull-down menus with diabetes medications that included programs should be free or low in cost and easy to use, with
timing, dosage, and food consumption instructions, plus personalized instructions and technical support.
information on side effects. One said, “How does my dia- A limitation of this study is that focus group participants
betic medicine go with this cancer medication you gave me? and interviewees may have had preconceived ideas based on
That is pretty important.” Participants wanted the program their experiences with either the online MyPath program or
to allow them to request refills, and to receive reminders to the in-person ¡Viva Bien! program. In addition, since ¡Viva
take medications or to order refills. They also wanted the Bien! was a program for Latina women, both Latino men
program to be tailored and linked to their providers. One and non-Latinos were not interviewed, limiting generalize-
noted, “If it was on the (provider) website, you could email a ability. In addition, all participants received healthcare from
message to your doctor and set up the reminders.” a large HMO in an urban/suburban setting, so are not
representative of patients who are uninsured, or residing in
Social media Patients clearly appreciated the potential of rural areas. Strengths include a patient-centered approach
social media to help them interact with peers, receive moti- and samples that were representative of participants in the
vation and tips, and make friends. One said, “I’m very two large diabetes trials from which they were drawn; and
social. This would be a place where I can say, hey, I’m included a diverse mix of ages, genders, ethnicity, education
walking at so-and-so at this time. Want to join?” But they and income levels. Participants in this study provided a
were concerned about keeping personal information private. useful perspective and a surprising amount of agreement
One said, “I am a high school teacher. I worry about my on preferences for choice of technology and features, for
students having access to my profile.” programs that are timely and personalized, and for programs
that do not replace humans but rather facilitate accountabil-
ity to and communication with others.

4 Conclusions
Acknowledgments The ¡Viva Bien! study was supported by grant
HL077120 from the National Heart, Lung, and Blood Institute. The
This paper summarizes recent literature and presents quali- MyPath study was supported by grant DK35524 from the National
tative results from two studies to document patient prefer- Institute of Diabetes and Digestive and Kidney Diseases.
ences with regard to using web-based or phone-based tools No financial disclosures were reported by the authors of this paper.
 Author's personal copy
Health Technol. (2012) 2:147–157 155

References vulnerable patients with diabetes: a three-arm practical clinical

trial. Diabetes Care. 2009;32(4):559–66.
20. U.S.Census Bureau. Reported Internet usage for households by
1. Boucher JL. Technology and patient-provider interactions: im- selected householder characteristics. http://www.census gov/
proving quality of care, but is it improving communication and hhes/computer/publications/2009.html. Accessed 8-17-11
collaboration? Diabetes Spectrum. 2010;23(3):142–4. (October).
2. Danaei G, Finucane MM, Lu Y, Singh GM, Cowan MJ, Paciorek 21. Ahern DK, Woods SS, Lightowler MC, Finley SW, Houston TK.
CJ, et al. National, regional, and global trends in fasting plasma Promise of and potential for patient-facing technologies to enable
glucose and diabetes prevalence since 1980: systematic analysis meaningful use. Am J Prev Med. 2011;40(5 Suppl 2):S162–72.
of health examination surveys and epidemiological studies with 22. Bennett GG, Glasgow RE. The delivery of public health inter-
370 country-years and 2.7 million participants. Lancet. 2011;378 ventions via the Internet: actualizing their potential. Annu Rev
(9785):31–40. Public Health. 2009;30:273–92.
3. Centers for Disease Control and Prevention. National diabetes 23. Strecher V. Internet methods for delivering behavioral and health-
fact sheet: National estimates and general information on diabetes related interventions (eHealth). Annu Rev Clin Psychol. 2007;3:53–
and prediabetes in the United States, 2011. http://www.cdc.gov/ 76.
diabetes/pubs/pdf/ndfs_2011.pdf . Accessed 8-17-11 24. Bonander J, Gates S. Public health in an era of personal health
4. Roger VL, Go AS, Lloyd-Jones DM, Adams RJ, Berry JD, records: opportunities for innovation and new partnerships. J Med
Brown TM, et al. Heart disease and stroke statistics–2011 update: Internet Res. 2010;12(3):e33.
a report from the American Heart Association. Circulation. 25. Griffiths F, Lindenmeyer A, Powell J, Lowe P, Thorogood M.
2011;123(4):e18–e209. Why are health care interventions delivered over the Internet? A
5. Hogan P, Dall T, Nikolov P. Economic costs of diabetes in the US systematic review of the published literature. J Med Internet Res.
in 2002. Diabetes Care. 2003;26(3):917–32. 2006;8(2):e10.
6. Chin MH, Cook S, Jin L, Drum ML, Harrison JF, Koppert J, et al. 26. Chua HF, Polk T, Welsh R, Liberzon I, Strecher V. Neural
Barriers to providing diabetes care in community health centers. responses to elements of a web-based smoking cessation pro-
Diabetes Care. 2001;24(2):268–74. gram. Stud Health Technol Inform. 2009;144:174–8.
7. Powell MP, Glover SH, Probst JC, Laditka SB. Barriers associated 27. Oenema A, Brug J, Dijkstra A, de Weerdt I, De Vries H. Efficacy
with the delivery of Medicare-reimbursed diabetes self-management and use of an internet-delivered computer-tailored lifestyle inter-
education. Diabetes Educ. 2005;31(6):890–9. vention, targeting saturated fat intake, physical activity and smok-
8. Piette J. Interactive behavior change technology to support diabetes ing cessation: a randomized controlled trial. Ann Behav Med.
self-management. Where do we stand? Diabetes Care. 2007;30 2008;35(2):125–35.
(10):2425–32. 28. Vandelanotte C, DeBourdeaudhuij I, Brug J. Two-year follow-up
9. Wagner J. Behavioral interventions to promote diabetes self- of sequential and simultaneous interactive computer-tailored
management. Diabetes Spectrum. 2011;24:61–2. interventions for increasing physical activity and decreasing fat
10. Jansink R, Braspenning J, van der Weijden T, Elwyn G, Grol R. intake. Ann Behav Med. 2007;33(2):213–9.
Primary care nurses struggle with lifestyle counseling in diabetes 29. Prochaska JO, Velicer WF, Redding C, Rossi JS, Goldstein M,
care: a qualitative analysis. BMC Fam Pract. 2010;11:41. DePue J, et al. Stage-based expert systems to guide a population
11. Kolasa KM, Rickett K. Barriers to providing nutrition counseling of primary care patients to quit smoking, eat healthier, prevent
cited by physicians: a survey of primary care practitioners. Nutr skin cancer, and receive regular mammograms. Prev Med. 2005;41
Clin Pract. 2010;25(5):502–9. (2):406–16.
12. Silberman A, Banthia R, Estay IS, Kemp C, Studley J, Hareras D, 30. Glasgow RE, Kurz D, King DK, Dickman JM, Faber AJ, Halterman
et al. The effectiveness and efficacy of an intensive cardiac reha- E, et al. Outcomes of a minimal versus moderate support versions of
bilitation program in 24 sites. Am J Health Promot. 2010;24 an internet-based diabetes self-management support program. J Gen
(4):260–6. Intern Med. 2010;25(12):1315–22.
13. Lorig KR, Hurwicz ML, Sobel D, Hobbs M, Ritter PL. A national 31. Glasgow RE, Edwards LL, Whitesides H, Carroll N, Sanders TJ,
dissemination of an evidence-based self-management program: a McCray BL. Reach and effectiveness of DVD and in-person
process evaluation study. Patient Educ Couns. 2005;59(1):69–79. diabetes self-management education. Chronic Illn. 2009;5:243–9.
14. Toobert DJ, Strycker LA, Barrera Jr M, Osuna D, King DK, 32. Piette JD, Weinberger M, Kraemer FB, McPhee SJ. Impact of
Glasgow RE. Outcomes from a multiple-risk-factor diabetes automated calls with nurse follow-up on diabetes treatment out-
self-management trial for Latinas: ¡Viva Bien! Ann Behav Med. comes in a Department of Veterans Affairs Health Care System: a
2011;41(3):310–23. randomized controlled trial. Diabetes Care. 2001;24(2):202–8.
15. Toobert DJ, Glasgow R, Strycker LA, Barrera Jr M, Radcliffe JL, 33. National Committee for Quality Assurance (NCQA). Standards
Wander RC, et al. Biologic and quality-of-life outcomes from the for Patient-centered Medical Home (PCMH). Accessed 8-17-11.
Mediterranean Lifestyle Program: a randomized clinical trial. Washington, D.C.: 2011.
Diabetes Care. 2003;26(8):2288–93. 34. Okoro CA, Mokdad AH, Ford ES, Bowman BA, Vinicor F, Giles
16. Finkelstein J, Barr MS, Kothari PP, Nace DK, Quinn M. Patient- WH. Are persons with diabetes practicing healthier behaviors in
centered medical home cyberinfrastructure current and future the year 2001? Results from the Behavioral Risk Factor Surveil-
landscape. Am J Prev Med. 2011;40(5 Suppl 2):S225–33. lance System. Prev Med. 2004;38(2):203–8.
17. Reid RJ, Fishman PA, Yu O, Ross TR, Tufano JT, Soman MP, et 35. Gaede P, Vedel P, Larsen N, Jensen GV, Parving HH, Pedersen O.
al. Patient-centered medical home demonstration: a prospective, Multifactorial intervention and cardiovascular disease in patients
quasi-experimental, before and after evaluation. Am J Manag with type 2 diabetes. N Engl J Med. 2003;348(5):383–93.
Care. 2009;15(9):e71–87. 36. Lichtenstein AH, Appel LJ, Brands M, Carnethon M, Daniels S,
18. Seidman J, Eytan T. Helping patients plug in: lessons in the Franch HA, et al. Diet and lifestyle recommendations revision
adoptions of online consumer tools. Oakland: California Health 2006: a scientific statement from the American Heart Association
Care Foundation; 2008. Nutrition Committee. Circulation. 2006;114(1):82–96.
19. Schillinger D, Handley M, Wang F, Hammer H. Effects of self- 37. Chiuve SE, McCullough ML, Sacks FM, Rimm EB. Healthy
management support on structure, process, and outcomes among lifestyle factors in the primary prevention of coronary heart
 Author's personal copy
156 Health Technol. (2012) 2:147–157

disease among men: benefits among users and nonusers of lipid- civic and political involvement are connected to their use of
lowering and antihypertensive medications. Circulation. 2006;114 social networking sites and other technologies. http://www.
(2):160–7. pewinternet.org/~/media/Files/Reports/2011/PIP%20-%20Social
38. Richardson CR, Kriska AM, Lantz PM, Hayward RA. Physical %20networking%20sites%20and%20our%20lives.pdf. Accessed
activity and mortality across cardiovascular disease risk groups. 8-18-11
Med Sci Sports Exerc. 2004;36(11):1923–9. 57. Lorig KR, Ritter PL, Laurent DD, Plant K. Internet-based chronic
39. Krishna S, Boren SA. Diabetes self-management care via cell disease self-management: a randomized trial. Med Care. 2006;44
phone: a systematic review. J Diabetes Sci Technol. 2008;2(3):509– (11):964–71.
17. 58. Fukuoka Y, Kamitani E, Bonnet K, Lindgren T. Real-time social
40. Murray E, Burns J, See TS, Lai R, Nazareth I. Interactive health support through a mobile virtual community to improve healthy
communication applications for people with chronic disease. behavior in overweight and sedentary adults: a focus group
Cochrane Database Syst Rev 2005;(4):CD004274. analysis. J Med Internet Res. 2011;13(3):e49.
41. Gibbons M, Wilson R, Samal L, Lehmann CU, Dickersin K, 59. Chapman LS, Rowe D, Witte K. eHealth portals: who uses them
Lehmann HP, et al. Consumer health informatics: results of a and why? Am J Health Promot. 2010; 24(5):TAHP1-7, iii.
systematic evidence review and evidence based recommenda- 60. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health
tions. Transl Behav Med. 2011;1:72–82. impact of health promotion interventions: The RE-AIM frame-
42. Biem HJ, Turnell RW, D’Arcy C. Computer telephony: automat- work. Am J Public Health. 1999;89:1322–7.
ed calls for medical care. Clin Invest Med. 2003;26(5):259–68. 61. Rabin BA, Glasgow RE. Dissemination of interactive health
43. Piette JD. Satisfaction with automated telephone disease manage- communication programs. In: Noar SM, Harrington NG, editors.
ment calls and its relationship to their use. Diabetes Educ. Interactive health communication technologies: promising strate-
2000;26:1003–10. gies for health behavior change. First Edition ed. Routledge;
44. Balas EA, Jaffrey F, Kuperman GJ, Boren SA, Brown GD, Pinciroli 2012. p. In press.
F, et al. Electronic communication with patients: evaluation of 62. Carroll M, Cullen T, Ferguson S, Hogge N, Horton M, Kokesh J.
distance medicine technology. JAMA. 1997;278(2):152–9. Innovation in Indian healthcare: using health information tech-
45. McBride CM, Rimer BK. Using the telephone to improve health nology to achieve health equity for American Indian and Alaska
behavior and health service delivery. Patient Educ Couns. 1999;37:3– Native populations. Perspect Health Inf Manag. 2011;8:1d.
18. 63. Smith A. Mobile access 2010. http://www.pewinternet.org/
46. Morris ME, Kathawala Q, Leen TK, Gorenstein EE, Guilak F, Reports/2010/Mobile-Access-2010.aspx [2011 (Pew Internet &
Labhard M, et al. Mobile therapy: case study evaluations of a cell American Life Project, July 5, 2010). Accessed 8-18-11]
phone application for emotional self-awareness. J Med Internet 64. Smith A. Smartphone adoption and usage. http://www.pewinternet.
Res. 2010;12(2):e10. org/Reports/2011/Smartphones.aspx. 2011 (Pew Internet & Amer-
47. Jimison H, Gorman P, Woods S, Nygren P, Walker M, Norris S, et ican Life Project, July 11, 2011). Accessed 8-18-11.
al. Barriers and drivers of health information technology use for 65. Chou WY, Hunt YM, Beckjord EB, Moser RP, Hesse BW. Social
the elderly, chronically ill, and underserved. Evid Rep Technol media use in the United States: implications for health commu-
Assess (Full Rep). 2008;175:1–1422. nication. J Med Internet Res. 2009;11(4):e48.
48. Pare G, Moqadem K, Pineau G, St-Hilaire C. Clinical effects of 66. Crilly JF, Keefe RH, Volpe F. Use of electronic technologies to
home telemonitoring in the context of diabetes, asthma, heart promote community and personal health for individuals unconnect-
failure and hypertension: a systematic review. J Med Internet ed to health care systems. Am J Public Health. 2011;101(7):1163–7.
Res. 2010;12(2):e21. 67. Gracia E, Herrero J. Internet use and self-rated health among older
49. Kim HS, Kim NC, Ahn SH. Impact of a nurse short message people: a national survey. J Med Internet Res. 2009;11(4):e49.
service intervention for patients with diabetes. J Nurs Care Qual. 68. Van’t RJ, Crutzen R, de VH. Investigating predictors of visiting,
2006;21(3):266–71. using, and revisiting an online health-communication program: a
50. Hurling R, Catt M, Boni MD, Fairley BW, Hurst T, Murray P, et longitudinal study. J Med Internet Res. 2010;12(3):e37.
al. Using internet and mobile phone technology to deliver an 69. Albright CL, Pruitt L, Castro C, Gonzalez A, Woo S, King AC.
automated physical activity program: randomized controlled trial. Modifying physical activity in a multiethnic sample of low-
J Med Internet Res. 2007;9(2):e7. income women: one-year results from the IMPACT (Increasing
51. van den Berg MH, Schoones JW, Vliet Vlieland TP. Internet- Motivation for Physical ACTivity) project. Ann Behav Med.
based physical activity interventions: a systematic review of the 2005;30(3):191–200.
literature. J Med Internet Res. 2007;9(3):e26. 70. Rainie L. Internet, broadband, and cell phone statistics. http://
52. Durso SC, Wendel I, Letzt AM, Lefkowitz J, Kaseman DF, www.pewinternet.org/Reports/2010/Internet-broadband-and-cell-
Seifert RF. Older adults using cellular telephones for diabetes phone-statistics.aspx. 2010 (Pew Internet & American Life Proj-
management: a pilot study. Medsurg Nurs. 2003;12(5):313–7. ect, Jan 5, 2010). Accessed 8-18-11.
53. Walker EA, Shmukler C, Ullman R, Blanco E, Scollan- 71. Tirado M. Role of mobile health in the care of culturally and
Koliopoulus M, Cohen HW. Results of a successful telephonic linguistically diverse US populations. Perspect Health Inf Manag.
intervention to improve diabetes control in urban adults: a ran- 2011;8:1e.
domized trial. Diabetes Care. 2011;34(1):2–7. 72. Lorig K, Ritter PL, Laurent DD, Plant K, Green M, Jernigan VB,
54. Piette JD, Weinberger M, McPhee SJ, Mah CA, Kraemer FB, et al. Online diabetes self-management program: a randomized
Crapo LM. Do automated calls with nurse follow-up improve study. Diabetes Care. 2010;33(6):1275–81.
self-care and glycemic control among vulnerable patients with 73. Jernigan VBB, Lorig K. The Internet diabetes self-management
diabetes? Am J Med. 2000;108(1):20–7. workshop for American Indians and Alaska natives. Health Pro-
55. van Bastelaar KM, Pouwer F, Cuijpers P, Riper H, Snoek FJ. mot Pract. 2011;12(2):261–70.
Web-based depression treatment for type 1 and type 2 diabetic 74. Piette J. Enhancing support via interactive technologies. Curr
patients: a randomized, controlled trial. Diabetes Care. 2011;34 Diab Rep. 2002;2:160–5.
(2):320–5. 75. Piette JD. Patient education via automated calls: a study of
56. Hampton KN, Goulet LS, Rainie L, Purcell K. Social networking English and Spanish speakers with diabetes. Am J Prev Med.
sites and our lives: How people’s trust, personal relationships, and 1999;17(2):138–41.
 Author's personal copy
Health Technol. (2012) 2:147–157 157

76. Piette JD, Heisler M. The relationship between older adults’ internet-delivered healthy lifestyle promotion interventions? A sys-
knowledge of their drug coverage and medication cost problems. tematic review. J Med Internet Res. 2011;13(1):e2.
J Am Geriatr Soc. 2006;54(1):91–6. 89. Glasgow RE, Christiansen S, Kurz D, King D, Woolley T, Faber
77. Piette JD, Mendoza-Avelares MO, Ganser M, Mohamed M, A, et al. Engagement in a diabetes self-management website:
Marinec N, Krishnan S. A preliminary study of a cloud- Usage patterns and generalizability of program use. J Internet
computing model for chronic illness self-care support in an under- Med Res. 2011;13(1):e9.
developed country. Am J Prev Med. 2011;40(6):629–32. 90. Fisher L, Glasgow RE. A call for more effectively integrating
78. Atkinson NL, Saperstein SL, Desmond SM, Gold RS, Billing AS, behavioral and social science principles into comprehensive dia-
Tian J. Rural eHealth nutrition education for limited-income betes care. Diabetes Care. 2007;30(10):2746–9.
families: an iterative and user-centered design approach. J Med 91. Tate DF, Jackvony EH, Wing RR. Effects of Internet behavioral
Internet Res. 2009;11(2):e21. counseling on weight loss in adults at risk for type 2 diabetes: a
79. Jensen JD, King AJ, Davis LA, Guntzviller LM. Utilization of randomized trial. JAMA. 2003;289(14):1833–6.
internet technology by low-income adults: the role of health 92. Tate DF, Finkelstein EA, Khavjou O, Gustafson A. Cost-
literacy, health numeracy, and computer assistance. J Aging effectiveness of Internet interventions: review and recommenda-
Health. 2010;22(6):804–26. tions. Ann Behav Med. 2009;38(1):40–5.
80. Jensen JD, King AJ, Guntzviller LM, Davis LA. Patient-provider 93. Atkinson NL, Saperstein SL, Pleis J. Using the internet for
communication and low-income adults: age, race, literacy, and health-related activities: findings from a national probability sam-
optimism predict communication satisfaction. Patient Educ Couns. ple. J Med Internet Res. 2009;11(1):e4.
2010;79(1):30–5. 94. King DK, Glasgow RE, Toobert DJ, Strycker LA, Estabrooks PA,
81. Robroek SJ, Brouwer W, Lindeboom D, Oenema A, Burdorf A. Osuna D, et al. Self-efficacy, problem solving, and social-
Demographic, behavioral, and psychosocial correlates of using environmental support are associated with diabetes self-
the website component of a worksite physical activity and healthy management behaviors. Diabetes Care. 2010;33(4):751–3.
nutrition promotion program: a longitudinal study. J Med Internet 95. Toobert DJ, Strycker LA, Glasgow RE, Osuna D, Doty AT,
Res. 2010;12(3):e44. Barrera M, et al. Viva bien!: Overcoming recruitment challenges
82. Hawn C. Take two aspirin and tweet me in the morning: how in a multiple-risk-factor diabetes trial. Am J Health Behav.
Twitter, Facebook, and other social media are reshaping health 2010;34(4):432–41.
care. Health Aff (Millwood). 2009;28(2):361–8. 96. Toobert DJ, Strycker LA, King DK, Barrera M Jr., Osuna D,
83. Stevenson FA, Kerr C, Murray E, Nazareth I. Information from Glasgow RE. Long-term outcomes from a multiple-risk-factor dia-
the Internet and the doctor-patient relationship: the patient per- betes trial for Latinas: ¡Viva Bien! Transl Behav Med. 2011;Pub-
spective–a qualitative study. BMC Fam Pract. 2007;8:47. lished online January 12, 2011(doi:10.1007/s13142-010-0011-1).
84. Norman GJ, Zabinski MF, Adams MA, Rosenberg DE, Yaroch AL, 97. Ritzwoller DP, Sukhanova A, Gaglio B, Glasgow RE. Costing
Atienza AA. A review of eHealth interventions for physical activity behavioral interventions: a practical guide to enhance translation.
and dietary behavior change. Am J Prev Med. 2007;33(4):336–45. Ann Behav Med. 2009;37(2):218–27.
85. Glasgow RE, Nelson CC, Kearney KA, Reid R, Ritzwoller DP, 98. Saldana J. The coding manual for qualitative researchers. Thousand
Strecher VJ, et al. Reach, engagement, and retention in an Oaks: Sage Publications; 2009.
Internet-based weight loss program in a multi-site randomized 99. Coffey A, Atkinson P. Making sense of qualitative data: comple-
controlled trial. J Med Internet Res. 2007;9(2):e11. mentary research strategies. Thousand Oaks: Sage Publications;
86. Verheijden MW, Jans MP, Hildebrandt VH, Hopman-Rock M. 1996.
Rates and determinants of repeated participation in a web-based 100. Miles MB, Huberman AM. Qualitative data analysis. Newbury
behavior change program for healthy body weight and healthy Park: Sage Publications; 1994.
lifestyle. J Med Internet Res. 2007;9(1):e1. 101. Elo S, Kyngas H. The qualitative content analysis process. J Adv
87. Danaher BG, Seeley JR. Methodological issues in research on Nurs. 2008;62(1):107–15.
web-based behavioral interventions. Ann Behav Med. 2009;38 102. Creswell JW, Miller DL. Determining validity in qualitative inquiry.
(1):28–39. Theory into Practice. 2000;39(3):124–31.
88. Brouwer W, Kroeze W, Crutzen R, de NJ, de Vries NK, Brug J, et al. 103. Padgett DK. Qualitative methods in social work research. Thousand
Which intervention characteristics are related to more exposure to Oaks: Sage Publications; 2008.

View publication stats