This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3100885, IEEE Access

=Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
Digital Object Identifier 10.1109/ACCESS.2017.Doi Number

The Use of Chatbots in Digital Business

Transformation: A Systematic Literature Review
Andrej Miklosik1, Nina Evans2, Athar Mahmood Ahmed Qureshi3
1
Marketing Department, Faculty of Management, Comenius University in Bratislava, Bratislava, 83104 Slovakia
2
University of South Australia, UniSA STEM, Mawson Lakes Campus, Mawson Lakes Blvd, Mawson Lakes SA 5095, Australia
3
University of South Australia, UniSA Business, City West Campus, 61-68 North Terrace, Adelaide SA 5000, Australia

Corresponding author: Andrej Miklosik (e-mail: [email protected])

This paper is an output of RTIS seed funding grant from UniSA for a project on Digital Transformation of the Superannuation industry in Australia.

ABSTRACT The research on chatbots has gained momentum over the past few years. Academics and practitioners investigate
how these tools for communication with customers or internal team can be improved in terms of their performance, acceptance,
and deployment. Although there is a plethora of recent studies available, not all of them deal with the digital business
transformation implications of chatbots. The main aim of the research presented in this paper was to conduct a systematic
literature review of high-quality journal research papers in order to summarise the current state of research on chatbots, identify
their role in digital business transformation and suggest the areas warranting further attention. 74 papers were included in the
research. Topical (focus and applications), methodological (methods used, sample size, sample type, and countries studied)
and bibliometric (publication outlet, citations, and Altmetric Attention Score) aspects are evaluated and described. Scholars
and practitioners can use the results to identify topics, areas, and applications that are intensely discussed in the literature and
require further attention, select a methodology for their research that is well established in the field or is emerging, identify the
most influential publications not to be missed in their research or identify publication outlets for publishing their research on
chatbots.

INDEX TERMS AI, chatbots, conversational agents, digital business transformation, digital disruption, ML

I. INTRODUCTION chatbots), chatbots also have the ability to mimic human

We are currently witnessing an immense technological speech (voice-based chatbots) to improve user experience
development, resulting in the need for organisations to adopt and cultivate customer loyalty [15], [16].
new technologies and information systems (IS) [1]. The Chatbots can be found on websites, social media or instant
technologies disrupt organisations’ business processes, and messaging apps [17], [18]. They can be deployed within an
trigger the need for digital transformation (DT) [2]–[4]. The organisation to assist with various services and processes
need for DT has been reflected across all industries, such as internal support systems, IT Service Management
including manufacturing (industry 4.0), retail, logistics, and (ITSM), learning or human resources management (HRM)
services [5]–[8]. [15], [19]–[24].
One category of IS with increasing capabilities are For external communication, standalone chatbots can also
artificial intelligence (AI)-powered systems. Such systems represent an alternative to branded websites [25]. They have
have many potential applications in decision making been deployed to provide services in many areas such as
support, production automation, learning, communication, customer relationships management (CRM), customer
etc [9]. The communication between online users and service or sales and marketing [26]–[30]. Chatbots are used
organisations is shifting towards interactions with AI-driven to make product or service recommendations regarding
systems [10]. A chatbot is one example of technology that is shopping, financial or health related decisions [25], [31]–
used in computer-mediated communication where AI agents
[34]. Researchers are, amongst others, focusing on
increasingly occupy roles once served by humans [11]. Not
investigating how to build better social bots for interaction
all chatbots are developed as Machine Learning (ML) or AI
in business or commercial environments, how to improve
driven chatbots [12], but the advantage of deploying AI
services with chatbots, which factors affect user perceptions
driven chatbots is that they give the impression of being of chatbots or how to encourage repetitive use of chatbots
intelligent as they get smarter with increased data and user [35]–[42].
interactions [13].
Chatbots can be defined as a ‘software that accepts natural II. OBJECTIVES OF THE STUDY
language as input and generates natural language as output, Researchers have been examining the various uses of
engaging in a conversation’ [14]. Another definition chatbots, the factors affecting their acceptance by users, and
accentuates their attempted human-liked character: the creation of new algorithms and frameworks for chatbot
‘Chatbots are interactive virtual characters whose mission is deployment to increase their efficiency. The number of
to assist people in high-profile environments.’ [15]. Apart studies on chatbots has increased significantly over the past
from engaging in written conversations (text-based

VOLUME XXI, 2021

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3100885, IEEE Access

few years which can make it difficult for researchers to care, disaster management, forensics, or hospitality
navigate the space and identify areas that need further (restaurants); ii) COVID 19 and religion perspectives. The
attention. The aim of this paper is to fill in the gap and title and abstract screening resulted in 158 selected papers.
provide a comprehensive overview of academic studies on This step of the SLR was completed by 10 May 2021.
chatbots.
Although many papers have been written that focus purely C. DETAILED FULL TEXT ANALYSIS (ELIGIBILITY)
on the development of chatbots, our research recognises the For each of the 158 publications, full texts were obtained,
need for interdisciplinary research and therefore focuses on read, and analysed.
papers that identify clear business implications of chatbot The exclusion criteria applied at this stage included:
use and development, both inside an organisation (internal i) Paper type – although the filter in WoS was set to
environment), and targeted at various external stakeholders, show only journal papers, a few other than journal
mainly customers. papers were included by WoS, e.g. book chapters,
The paper provides an overview of relevant research in and therefore needed to be excluded at this stage;
high-quality journal research papers, in order to summarise ii) Content – papers with purely programming/
the current state of research on business implications of technical perspectives such as algorithm
chatbots and identify the research gap that requires further improvements; other too narrow implications such
attention. The paper aims to answer the following research as pedagogy, psychology, humour;
questions: iii) Language – papers written in languages other than
English;
RQ1: What are the focus areas and applications of the iv) Quality – papers with missing or insufficient
existing research on chatbots? methodology, literature review or other major
RQ2: Which methodologies have been used in the current deficiencies; and
research and what are the characteristics of the v) Full text unavailable – no full text can be obtained.
samples used? The decision to exclude papers written from a
RQ3: Which journals publish most of the research from this programming/technical perspective only, was based on the
field and which publications are the most influential? following assumptions: Although these papers could
RQ4: What are the potential directions for future research produce very interesting results, e.g. the ability to build
in this area? chatbots based on smaller data sets or making the chatbots
more human while not increasing the requirements for
III. METHODOLOGY resources drastically, they have limited application beyond
A systematic literature review (SLR) was selected as the best the IT/programming field.
method to achieve the defined objectives [43]. The process The decision on which papers to keep or exclude was
of identification and analysis of relevant papers for the made through consensus between the authors. Based on the
purpose of this SLR consisted of three steps: i) Initial consensus of the first two authors, records for exclusion
database search; ii) Title and abstract screening; iii) Detailed were identified in the screening and eligibility phases. The
full-text analysis. These steps are described below. third author performed quality control and served as a
mediator in case a dispute resolution was required.
A. INITIAL DATABASE SEARCH (IDENTIFICATION) The flow of information through the different phases of
The Web of Science database was selected as the source of this SLR is depicted in the PRISMA flow diagram (Figure
papers for this SLR. To list possibly matching papers, the I).
following search query was entered into the new (Beta)
interface of Web of Science:

chatbot* (Title) or chat bot* (Title) or chatterbot* (Title) or

chatter bot* (Title)

The results were refined to include only articles, by setting

the Document Types filter to ‘Articles’. In this step, 298
papers were identified. This step of the SLR was completed
on 27 April 2021.

B. TITLE AND ABSTRACT SCREENING (SCREENING)

In the second step, the appropriateness of the papers for this
SLR was determined by reviewing their title and abstract.
Only papers with direct business implications were
included. The following exclusion criteria were applied: i)
Specific application to an unrelated industry such as health

VOLUME XXI, 2021

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3100885, IEEE Access

To eliminate potential bias, key characteristics of the SLR

were determined by the authors prior to initiating phase 1.
This included the definition of research questions. Involving
the third author in the discussions regarding the research
framework also helped eliminate the risk of bias, as he has
not specialised in chatbot research before.
The research sample included 74 papers from 54 different
journals. Twelve of these journals published more than one
paper. The most popular journals were Computers in Human
Behavior (6 papers), International Journal of Advanced
Computer Science and Applications (4 papers), Electronic
Markets (3 papers), and Journal of Business Research (3
papers). Journals that published more than one paper are
shown in Table I.

FIGURE 1. PRISMA flow diagram for the systematic literature review

TABLE I
JOURNALS THAT PUBLISHED MORE THAN ONE RESEARCH PAPER

Journal Number of References Percentage

articles of total (%)
Computers in Human Behavior 6 van der Broeck, 2020; Go - Sundar, 2019; Araujo, 2018; 8.1%
Shumanov - Johnson, 2021; Hill et al., 2015; Youn - Yin, 2021
International Journal of Advanced Computer Science 4 Bakouan et al., 2018; Janati et al., 2020; Kar - Haldar, 2016; 5.4%
and Applications Nawaz - Gomes, 2019
Electronic Markets 3 Adam et al., 2020; Stoeckli et al., 2019; Li et al., 2021 4.1%
Journal of Business Research 3 Chung et al., 2020; Sheehan, et al., 2020; Roy - Naidoo, 2021; 4.1%
Communication Studies 2 Beattie et al., 2020; Westerman et al., 2019 2.7%
Interacting with Computers 2 Bertinussen Nordheim et al., 2019; McDonnell - Baxter, 2019 2.7%
International Journal of Human-Computer Interaction 2 Chaves - Gerosa, 2021; Lee et al., 2020 2.7%
Journal of Broadcasting & Electronic Media 2 Ischen et al., 2020; Cheng - Jiang, 2020 2.7%
Journal of Internet Commerce 2 Trivedi, 2019; Han, 2021 2.7%
Journal of Retailing and Consumer Services 2 Rese et al., 2020; Pantano - Pizzi, 2020 2.7%
Journal of Service Management 2 Sidaoui et al., 2020; Sands et al., 2021 2.7%
Knowledge-Based Systems 2 Kerly et al., 2007; Nuruzzaman - Hussain, 2020 2.7%

A protocol developed by Hao [43] was adopted for the source of citations, outperforming both traditional (Web of
purpose of this SLR and used to collect and evaluate data Science, SCOPUS) and new (Microsoft Academic,
about i) research focus and design (research type and Dimensions, and the OpenCitations Index of CrossRef open
terminology used); ii) methodology (data collection, sample DOI-to-DOI citations) sources of citations [46]. After
size, sample type, and countries studied); and iii) gathering the total number of citations for each publication
bibliometric aspects (publication outlet and citations). A on Google Scholar, the annual average number of citations
similar methodology was used in a paper written by was calculated for every publication.
Sepasgozar et al. [44] about the systems developed and The Altmetric Attention Score was used as a metric to
technologies used for smart homes, in which they i) complement the citation analysis, thereby providing
reviewed relevant papers published between 2010 and 2019, additional insights into the research impact and reach.
within databases such as Scopus, ii) analysed the papers in Altmetrics are metrics and qualitative data that include (but
terms of bibliography and content to identify more related are not limited to) peer reviews on Faculty of 1000, citations
systems, practices, and contributors, iii) used a systematic on Wikipedia and in public policy documents, discussions
review method to identify and select the relevant papers and on research blogs, mainstream media coverage, bookmarks
iv) reviewed these relevant papers for their content by means on reference managers like Mendeley, and mentions on
of coding. social networks such as Twitter [47]. The Altmetric
To assess the research impact, citations and Altmetrics Attention Score is an automatically calculated, weighted
were used. Citations are the traditional way of determining count of all the attention a research output has received,
the influence of academic work [45]. Google Scholar was based on three factors: volume, sources, and authors [48].
used to determine the total number of citations. The evidence Both the number of citations and the Altmetric Attention
shows that Google Scholar is still the most comprehensive Score data were collected on 25 May 2021.

VOLUME XXI, 2021

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3100885, IEEE Access

IV. FINDINGS A. RESEARCH FOCUS AND APPLICATIONS

The findings part of the paper is organised to provide data The focus was identified for every paper in the research
for answering the research questions. Firstly, we present the sample. If a paper covers more than one area, the most
analysis of focus areas and applications of the research, dominant area was selected. The highest number of studies
followed by details on methodologies used. Lastly, the most focuses on user perceptions of chatbots and their acceptance
influential publications are identified by means of citations by users (16 papers), followed by communication (8 papers),
analysis enhanced by Altmetric data. the use of chatbots for customer service (7 papers),
performance of and satisfaction with chatbots (7 papers),
and learning (6 papers). The focus, along with the
references, are shown in Table II.

TABLE II
MAIN RESEARCH FOCUS AREAS

Number of
Main focus area References
studies
McDonnell - Baxter, 2019; Rese et al., 2020; Sheehan, et al., 2020; Moriuchi, 2020; Roy -
Naidoo, 2021; Cheng - Jiang, 2020; Ciechanowski et al., 2019; Araujo, 2018; De Cicco et al.,
Perceptions/Acceptance 16
2020; Lee et al., 2020; Zarouali et al., 2018; Toader et al., 2020; Han, 2021; Schuetzler et al.,
2020; Yen - Chaing, 2020; Kasilingam, 2020;
Beattie et al., 2020; Gennaro et al., 2020; Chaves - Gerosa, 2021; Go - Sundar, 2019;
Communication 8
Westerman et al., 2019; Nuruzzaman - Hussain, 2020; Hill et al., 2015; Youn - Yin, 2021
Sidaoui et al., 2020; Adam et al., 2020; Chung et al., 2020; Oguntosin - Olomo, 2021; Trivedi,
Customer service 7
2019; Ashfaq et al., 2020; Doherty - Curren, 2019
Ischen et al., 2020; Nichifor et al., 2021; Eren, 2020; Przegalinska et al., 2019; Sands et al.,
Performance/Satisfaction 7
2021; Almansor et al., 2021; Li et al., 2021
Bakouan et al., 2018; Janati et al., 2020; Wu et al., 2020; Kerly et al., 2007; Bailey et al.,
Learning 6
2021; Pereira et al., 2019
Wolff et al., 2020; Stoeckli et al., 2019; Kondylakis et al., 2020; Schuetzler et al., 2021;
Development/Deployment 5
Telang et al., 2018
FAQs/Troubleshooting 3 Abushawar - Atwell, 2015; Massaro et al., 2018; Thorne, 2017
Recruiting 3 Majumder - Mondal, 2021; Nawaz - Gomes, 2019; Gigi - Gunaselli, 2020;
Relationships 2 Skjuve et al., 2021; Ho et al., 2018
Trust 2 Aoki, 2020; Bertinussen Nordheim et al., 2019
Advertising 1 van der Broeck et al., 2019
Health 1 Piao et al., 2020
Innovation 1 Johannsen et al., 2021
Interaction 1 Bhushan, 2020
IoT 1 Kar - Haldar, 2016
Labour organising 1 Flanagan - Walker, 2020
Linguistics 1 Dippold et al., 2020
News distribution 1 Arouali et al., 2021
Personalisation 1 Shumanov - Johnson, 2021
Purchasing 1 Luo et al., 2019
Research directions 1 Pantano - Pizzi, 2020
Security 1 Hamad - Yeferny, 2020
Stress management 1 Park et al., 2019
Surveys 1 Xiao et al., 2020
User classification 1 Rajaobelina - Ricard, 2021

In Table III, various applications of chatbots were identified. more general applications that were not restricted to a certain
The main application accentuated in the paper was used to area. One paper [19] identifies multiple applications of
map references to the applications in the table. 41 papers chatbots and therefore, this paper was also not added to one
could be mapped in total, the research in other papers had particular category.

TABLE III
OVERVIEW OF CHATBOT APPLICATIONS

Application References No. of

studies
Human resources Flanagan - Walker, 2020; Gennaro et al., 2020; Gigi - Gunaselli, 2020; Ho et al., 2018; 8
Majumder - Mondal, 2021; Nawaz - Gomes, 2019; Park et al., 2019; Piao et al., 2020

VOLUME XXI, 2021

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3100885, IEEE Access

E-commerce Han, 2021; Kasilingam, 2020; Moriuchi, 2020; Oguntosin - Olomo, 2021; Roy - Naidoo, 2021; 6
Yen - Chaing, 2020
Learning Management Bakouan et al., 2018; Hamad - Yeferny, 2020; Janati et al., 2020; Kerly et al., 2007; Pereira et 6
System (LMS) al., 2019; Wu et al., 2020
Customer service / Adam et al., 2020; Bertinussen Nordheim et al., 2019; Sands et al., 2021; Sheehan, et al., 2020; 5
experience Sidaoui et al., 2020; Xiao et al., 2020
Financial services and Doherty - Curren, 2019; Eren, 2020; Luo et al., 2019; Nuruzzaman - Hussain, 2020; 5
insurance Rajaobelina - Ricard, 2021; Trivedi, 2019
Sales Bhushan, 2020; Chung et al., 2020; Dippold et al., 2020; Nichifor et al., 2021; Rese et al., 2020 5
Marketing van der Broeck et al., 2019; Zarouali et al., 2021 2
CRM Youn - Yin, 2021 1
Internal support/ITSM Telang et al., 2018; Thorne, 2017 1
Innovation management Johannsen et al., 2021 1
Multiple touchpoints Kar - Haldar, 2016 1

B. METHODOLOGIES USED development/prototyping (10 papers). Details on the

The most frequently used research methods within the methods used, along with the references are listed in Table
sample of papers are experiment (26 papers) and IV.
questionnaire (16 papers), followed by

TABLE IV
OVERVIEW OF RESEARCH METHODS

Methodology Number of References Percentage

articles of total (%)
Experiment 26 Adam et al., 2020; Aoki, 2020; Araujo, 2018; Bailey et al., 2021; Beattie et al., 35.1
2020; De Cicco et al., 2020; Gennaro et al., 2020; Go - Sundar, 2019; Hill et al.,
2015; Ho et al., 2018; Ischen et al., 2020; Kondylakis et al., 2020; Lee et al., 2020;
Luo et al., 2019; McDonnell - Baxter, 2019; Pereira et al., 2019; Piao et al., 2020;
Rese et al., 2020; Sands et al., 2021; Schuetzler et al., 2020; Shumanov - Johnson,
2021; Sidaoui et al., 2020; van der Broeck et al., 2019; Westerman et al., 2019;
Youn - Yin, 2021; Arouali et al., 2021
Questionnaire 16 Ashfaq et al., 2020; Bertinussen Nordheim et al., 2019; Cheng - Jiang, 2020; 21.6
Chung et al., 2020; Eren, 2020; Gigi - Gunaselli, 2020; Han, 2021; Kasilingam,
2020; Li et al., 2021; Moriuchi, 2020; Rajaobelina - Ricard, 2021; Sheehan, et al.,
2020; Toader et al., 2020; Trivedi, 2019; Xiao et al., 2020; Zarouali et al., 2018

Development/Prototyping 10 Abushawar - Atwell, 2015; Almansor et al., 2021; Bakouan et al., 2018; Bhushan, 13.5
2020; Doherty - Curren, 2019; Hamad - Yeferny, 2020; Janati et al., 2020; Kar -
Haldar, 2016; Oguntosin - Olomo, 2021; Wu et al., 2020
Mixed method 6 Ciechanowski et al., 2019; Nuruzzaman - Hussain, 2020; Przegalinska et al., 2019; 8.1
Roy - Naidoo, 2021; Thorne, 2017; Yen - Chaing, 2020
Interviews 5 Johannsen et al., 2021; Skjuve et al., 2021; Stoeckli et al., 2019; Park et al., 2019; 6.8
Wolff et al., 2020
Case study 4 Dippold et al., 2020; Flanagan - Walker, 2020; Kerly et al., 2007; Massaro et al., 5.4
2018
Secondary research 4 Chaves - Gerosa, 2021; Majumder - Mondal, 2021; Nawaz - Gomes, 2019; 5.4
Schuetzler et al., 2021
Conceptual framework 1 Telang et al., 2018 1.4
Content analysis 1 Nichifor et al., 2021 1.4
Patent analysis 1 Pantano - Pizzi, 2020 1.4

In research studies where participants were involved, the TABLE V

COUNTRIES OF FOCUS
sample size ranged from 4 [49] to 6255 [50]. In most cases,
the structure of the research sample was diverse. Students Target References Frequency
were used as participants in case of 11 papers [11], [31], country
[32], [51] – [58]. For many research studies, the participants examined
resided in various countries, or the details of their residence USA Beattie et al., 2020; Sheehan, et al., 7
2020; Cheng - Jiang, 2020; Westerman
were not disclosed. For 26 studies, the country of focus was et al., 2019; Sands et al., 2021; Hill et
disclosed. Details are offered in Table V. al., 2015; Han, 2021
Germany Adam et al., 2020; Wolff et al., 2020; 3
Rese et al., 2020
South Bailey et al., 2021; Chung et al., 2020;
Korea Lee et al., 2020 3
Canada Rajaobelina - Ricard, 2021; Roy - 2
Naidoo, 2021

VOLUME XXI, 2021

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3100885, IEEE Access

India Trivedi, 2019; Kasilingam, 2020 2 van der Broeck et al., 2019 2019 13
Netherlands Ischen et al., 2020; Arouali et al., 2021 2 Skjuve et al., 2021 2021 12
China Li et al., 2021 1 Chaves - Gerosa, 2021 2021 11
Great Gennaro et al., 2020 1 Araujo, 2018 2018 10
Britain Zarouali et al., 2018 2018 7
Greece Kondylakis et al., 2020 1
Westerman et al., 2019 2019 7
Italy de Cicco et al., 2020 1
Pereira et al., 2019 2019 6
Japan Aoki, 2020 1
Hill et al., 2015 2015 5
Romania Nichifor et al., 2021 1 Adam et al., 2020 2020 5
Turkey Eren, 2020 1 Bhushan, 2020 2020 5

V. DISCUSSION
C. JOURNALS AND RESEARCH IMPACT In this section we indicate how the first three research
In Table I, the journals that published two or more papers questions are addressed.
from the SLR are displayed.
Table VI shows the 20 most cited papers, ranked in A. RESEARCH FOCUS AND APPLICATIONS
descending order by citations per year. The first research question (RQ1) refers to the identification
of research focus and applications of the research on
TABLE VI chatbots. Various focus areas have been identified (24 in
MOST CITED RESEARCH PAPERS total), as listed in Table II. The focus of the research studies
Authors Year Citation Citations mostly relates to perceptions and acceptance of chatbots (16
published count per year studies). Researchers investigated parameters and features
Chung et al., 2020 2020 136 68 that make chatbots more (or less) accepted by users and
Ciechanowski et al., 2019 2019 199 66 where their usage ultimately resulted in their higher (or
Hill et al., 2015 2015 460 66 lower) acceptance. Examples include studies about chatbot
Araujo, 2018 2018 258 65 gender perceptions [59], attitudes towards warm versus
Go - Sundar, 2019 2019 130 43 competent chatbots [32], discomfort when using chatbots
Luo et al., 2019 2019 108 36
and comparing reactions to a simple and animated avatar
Ho et al., 2018 2018 113 28
chatbot [60]. The studies that are centred around
Adam et al., 2020 2020 55 28
Zarouali et al., 2018 2018 76 19
communication (eight studies) focus on analysing chatbot
Abushawar - Atwell, 2015 2015 131 19 communication from various perspectives, comparing
van der Broeck et al., 2019 2019 53 18 various means of communication, the use of emojis [11] or
Przegalinska et al., 2019 2019 53 18 properties that make chatbots more human
Kerly et al., 2007 2007 258 17 (anthropomorphism) – [27], [52], [61]. Customer service
Trivedi, 2019 2019 43 14 was at the centre of seven studies. The researchers
Sheehan, et al., 2020 2020 27 14 investigated improving customer service via effective
Pantano - Pizzi, 2020 2020 22 11 chatbots [62], extracted feelings from chatbot data [63],
Kar - Haldar, 2016 2016 58 10 developed a chatbot with advanced learning skills [64] or
Ashfaq et al., 2020 2020 17 9
identified the factors affecting satisfaction with customer
McDonnell - Baxter, 2019 2019 25 8
Park et al., 2019 2019 25 8
service [65]. More popular topics included
performance/satisfaction (seven studies), learning (6
* Table is ordered by citations per year studies), and development/deployment (five studies).
In Table VII, papers with the highest Altmetric Attention Topics that were less frequently studied in the research
Scores are presented. relate to the use of chatbots for FAQs/troubleshooting [66]–
[68], recruiting [23], [24], [69], relationships with chatbots
TABLE VII [70], [71], trust [28], [72], advertising [30], health [73],
RESEARCH PAPERS WITH THE HIGHEST ALTMETRIC security [74], user classification [75], purchasing [50],
ATTENTION SCORE personalisation [76] or surveys [77].
Chatbots can play a role in digital transformation of many
Authors Year Altmetric
published Attention Score areas of the business. Identifying applications of chatbot
Luo et al., 2019 2019 168 deployment aims to determine in which processes and
Cheng - Jiang, 2020 2020 55 environment, whether it is internal or external, can and
Gennaro et al., 2020 2020 40 should chatbots be deployed. As per Table III., human
Aoki, 2020 2020 29 resources were the area which was featured in the highest
Arouali et al., 2021 2020 19 number of studies (eight studies), followed by e-commerce
Flanagan - Walker, 2020 2020 18 and Learning Management System, both in the focus of six
Go - Sundar, 2019 2019 17
studies. Studying the impact of social presence and
Kondylakis et al., 2020 2020 16
enjoyment of mobile messenger chatbots on consumers’
Ciechanowski et al., 2019 2019 14
purchase intentions [78], customer purchasing behaviour

VOLUME XXI, 2021

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3100885, IEEE Access

and trust in chatbots [79] or usefulness of chatbots for Identifying the most influential publications was the core
shopping [80] were some of the phenomena investigated in of RQ3. Here 20 papers with the most citations per year were
the e-commerce application area. In reference to LMS, listed in Table VI. There are five papers with more than 40
researchers looked for example at using voice messages in citations per annum [51], [53], [55], [60], [61]. The total
learning with chatbots [81] or suggesting he best e-learning number of citations varied significantly between studies.
content to the user including multimedia [82]. There were three studies with more than 200 citations
Researchers also investigated how customer service and identified in the Google Scholar database [22], [51], [53],
customer experience can benefit from chatbot deployment and further six papers with more than 100 citations [50],
(five studies). Next, the use of chatbots in financial services [55], [60], [61], [66], [71]. 24 papers were cited between 10
and insurance was investigated (five studies), for example and 100 times, and there were 13 papers without a citation.
which three factors affect customer satisfaction with Table VII also shows which papers are currently actively
chatbots in the banking industry [83] or the use of chatbots discussed in the online space. The papers with the highest
in insurance [84]. A chatbot that recognises user perceptions Altmetric Attention Score are listed, with 13 of them
via connected cameras, useful by conducting presentations featuring a score above 10.
[85] is an example from the sales application (five studies).
Two studies examined marketing applications of chatbots; VI. CONTRIBUTION OF THE STUDY
one study focused on one of the other four applications each: Identifying areas needing future research attention was in
CRM, internal support/ITSM, innovation management, and the centre of the fourth research question (RQ4) and
multiple touchpoints. represents one of the contributions of this study. We
identified topics, and applications that warrant further
B. METHODOLOGIES USED research. When thinking about a focus area for their
This section presents answers to the RQ2: ‘Which research, academic and practitioners could point their
methodologies prevail in the current research and what are attention towards some of the areas that are topical and
the characteristics of the samples used?’ actual but have not been investigated thoroughly. These
As Table IV reveals, experiment is the most frequently include, for example, the use of chatbots for innovation,
used method to examine chatbots and their business surveys, purchasing, stress management, news distribution
implications. 26 studies used experiment as their main or security. If researchers want to focus on a current
method, followed by questionnaire (16 papers). Some of the application of chatbots that is relevant and has not been the
least utilised methods include patent analysis [86], content focal point of many previous studies, they can investigate
analysis [29], conceptual framework creation [20], the use of chatbots in marketing, CRM, internal
secondary research [12], [23], [24], [87], and case study support/ITSM, innovation management or multiple
[22], [67], [88], [89]. touchpoints (how chatbots can help integrate or help serve
The research samples consisted of diverse types of customers using more touchpoints).
participants. Students were the most frequent participants in Another contribution of our study is a comprehensive
research with people involved. A significant number of overview of methods used in the field of chatbot research.
studies included participants from various backgrounds, and Researchers can now understand which methods dominate
they were often recruited via a crowdsourcing marketplace, the research field of chatbots and their business implications.
such as Amazon Mechanical Turk. The sample sizes with Experiment and questionnaire were found to be the most
participants ranged from 4 to 6,255. often used methodologies – either one of them was used in
For most studies (48 out of 74), the country of focus was more than half of the studies (56.7 percent). There are two
either not disclosed or the participants were from various possible perspectives of the implications of these findings.
regions and countries. For the 26 studies focusing on one If researchers want to use a method that is standardised and
country, USA was the most studied country (7 papers), widely accepted in the field, they can use one of the most
followed by Germany (3 papers) and South Korea (3 popular methodologies. Another option is to choose a
papers). Canada, India, and the Netherlands were all methodology that has not been used in a large number of
investigated in two studies. Other studies included studies, thereby enriching the field by not only providing
participants from China, Great Britain, Greece, Italy, Japan, results from a different sector, perspective or application,
Romania, and Turkey (one country each). but also by developing a methodological application that has
not received much attention. Patent analysis, content
C. JOURNALS AND RESEARCH IMPACT analysis, conceptualisation, secondary research, or case
To answer the third research question (RQ3), we analysed studies represent such opportunity.
which journals publish most of the research on chatbots and By identifying the publications that have published most
their business implications. There were 12 journals that of the research on chatbots and their business implications,
published more than one paper from this SLR. Computers in we helped the researchers to choose the publication outlet.
Human Behavior (six studies) and International Journal of The journals identified in Table I published more than one
Advanced Computer Science and Applications (four paper on this topic and therefore, if a quality paper is
studies) were most popular. prepared, the chance of being considered for publication will
be increased. Identifying the most cited studies also

VOLUME XXI, 2021

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3100885, IEEE Access

contributes to the current knowledge in the research field. [5] A. Grencikova, M. Kordos, and V. Berkovic, “The
Based on this overview, researchers can ensure that they Impact of Industry 4.0 on Jobs Creation within the Small
read the most impactful papers that have been published. and Medium-Sized Enterprises and Family Businesses in
There are nine studies with more than 100 citations, four of Slovakia,” Administrative Sci, vol. 10, no. 3, p. 71, 2020.
these are cited more than 200 times. We also created an [6] K. S, M. Kolgan, S. Suvorova, O. Martynenko, and
overview of citations per year as this takes the time factor IOP, “Digital technologies and transformation of modern
into account and helps reveal papers that have a very strong retail,” in International Scientific Conference Digital
impact over a shorter time. By including the Altmetric Transformation on Manufacturing, Infrastructure and
Attention Score in the impact analysis, we also enrich the Service, vol. 497, 2019, p. 012126.
theory and methodology of conducting systematic literature
reviews which mostly relied on traditional citations analysis. [7] M. Cichosz, C. M. Wallenburg, and A. M. Knemeyer,
It is mostly newer studies (2019 – 2021) with high Altmetric “Digital transformation at logistics service providers:
Attention Scores and these values do not necessary correlate barriers, success factors and leading practices,” Int J Logist
with the most cited studies. Thus, an overview from Table Management, vol. 31, no. 2, p. 209, 2020.
VII helps the researchers identify studies that are being [8] M. Jantti and S. Hyvarinen, “Exploring Digital
currently discussed and talked about. These 10 studies with Transformation and Digital Culture in Service
the highest Altmetric Attention Score [30], [50], [58], [60], Organizations,” in 15th International Conference on
[61], [72], [90]–[92] should not be omitted from reading if Service Systems and Service Management (ICSSSM), 2018.
conducting a study on chatbots which would also include
their business implications. [9] A. Miklosik and N. Evans, “Impact of Big Data and
Machine Learning on Digital Transformation in Marketing:
VII. LIMITATIONS A Literature Review,” Ieee Access, vol. 8, pp. 101284–
Our study has certain limitations. Firstly, although an effort 101292, 2020.
was made to search for relevant keywords in resources in [10] S. S. Sundar, “Rise of Machine Agency: A Framework
different databases, the search will never be entirely for Studying the Psychology of Human–AI Interaction
exhaustive, as some relevant studies may have been omitted (HAII),” J Comput-mediat Comm, vol. 25, no. 1, pp. 74–
due to the filtering process that was adopted. Only papers 88, 2020.
that specifically used the ‘chatbot’ terminology were
included, while papers that used a related terminology (e.g. [11] A. Beattie, A. P. Edwards, and C. Edwards, “A Bot
conversation agents) were not included in the research and a Smile: Interpersonal Impressions of Chatbots and
sample. Secondly, only journal papers were included in the Humans Using Emoji in Computer-mediated
research, while conference papers, books, book chapters, Communication,” Commun Stud, vol. 71, no. 3, pp. 1–19,
monographs, dissertations, and other potentially relevant 2020.
studies and reports were omitted. Thirdly, only studies [12] R. Schuetzler, G. Grimes, J. Giboney, and H. Rosser,
where the business implications of chatbots were clearly “Deciding Whether and How to Deploy Chatbots,” MIS
articulated or can be directly derived, were included. Quarterly Executive, vol. 20, no. 1, pp. 1–15, 2021.
Although the selection process was objectivised, some
[13] R. Kar and R. Haldar, “Applying Chatbots to the
papers describing implications for business (although it has
Internet of Things: Opportunities and Architectural
not been clearly articulated) might have been excluded.
Elements,” International Journal of Advanced Computer
REFERENCES Science and Applications, vol. 7, no. 11, pp. 147–154,
[1] A. Miklosik, M. Kuchta, N. Evans, and S. Zak, “Towards 2016.
the Adoption of Machine Learning-Based Analytical Tools [14] D. Griol, J. Carbó, and J. M. Molina, “An Automatic
in Digital Marketing,” Ieee Access, vol. 7, pp. 85705–85718, Dialog Simulation Technique to Develop and Evaluate
2019. Interactive Conversational Agents,” Appl Artif Intell, vol.
[2] D. Schwaferts and B. S, “Digital Transformation 27, no. 9, pp. 759–780, 2013.
Management and Digital Business Development,” in [15] M. Bakouan, T. Kone, B. Kamagate, S. Oumtanaga,
Business Information Systems and Technology 4.0: New and M. Babri, “A Chatbot for Automatic Processing of
Trends in the Age of Digital Change, vol. 141, 2018, p. Learner Concerns in an Online Learning Platform,”
147. International Journal of Advanced Computer Science and
[3] A. van der Meulen, P. Weill, and W. SL, “Managing Applications, vol. 9, no. 5, pp. 168–176, 2018.
Organizational Explosions During Digital Business [16] M.-H. Huang and R. T. Rust, “Artificial Intelligence
Transformations,” Mis Q Exec, vol. 19, no. 3, p. 165, 2020. in Service,” J Serv Res-us, vol. 21, no. 2, pp. 155–172,
[4] M. Kotarba, “Digital Transformation of Business 2018.
Models,” Found Management, vol. 10, no. 1, p. 123, 2018. [17] J. Hagberg, M. Sundstrom, and N. Egels-Zandén,
“The digitalization of retailing: an exploratory framework,”

VOLUME XXI, 2021

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3100885, IEEE Access

Int J Retail Distribution Management, vol. 44, no. 7, pp. [30] E. V. den Broeck, B. Zarouali, and K. Poels, “Chatbot
694–712, 2016. advertising effectiveness: When does the message get
[18] B. Zarouali, E. V. den Broeck, M. Walrave, and K. through?,” Comput Hum Behav, vol. 98, pp. 150–157,
Poels, “Predicting Consumer Responses to a Chatbot on 2019.
Facebook,” Cyberpsychology Behav Soc Netw, vol. 21, no. [31] E. Moriuchi, V. M. Landers, D. Colton, and N. Hair,
8, pp. 491–497, 2018. “Engagement with chatbots versus augmented reality
[19] R. von Wolff, S. Hobert, K. Masuch, and M. interactive technology in e-commerce,” J Strategic Mark,
Schumann, “Chatbots at Digital Workplaces - A Grounded- pp. 1–15, 2020.
Theory Approach for Surveying Application Areas and [32] R. Roy and V. Naidoo, “Enhancing chatbot
Objectives,” Pacific Asia Journal of The Association for effectiveness: The role of anthropomorphic conversational
Information Systems, vol. 12, no. 2, pp. 64–102, 2020. styles and time orientation,” J Bus Res, vol. 126, pp. 23–
[20] P. R. Telang, A. K. Kalia, M. Vukovic, R. Pandita, 34, 2021.
and M. P. Singh, “A Conceptual Framework for [33] J. Trivedi, “Examining the Customer Experience of
Engineering Chatbots,” Ieee Internet Comput, vol. 22, no. Using Banking Chatbots and Its Impact on Brand Love:
6, pp. 54–59, 2018. The Moderating Role of Perceived Risk,” J Internet
[21] E. Wu, C. Lin, Y. Ou, C. Liu, W. Wang, and C. Chao, Commer, vol. 18, no. 1, pp. 1–21, 2019.
“Advantages and Constraints of a Hybrid Model K-12 E- [34] D. Doherty and K. Curran, “Chatbots for online
Learning Assistant Chatbot,” Ieee Access, vol. 8, pp. banking services,” Web Intell, vol. 17, no. 4, pp. 327–342,
77788–77801, 2020. 2019.
[22] A. Kerly, P. Hall, and S. Bull, “Bringing chatbots into [35] A. Przegalinska, L. Ciechanowski, A. Stroz, P. Gloor,
education: Towards natural language negotiation of open and G. Mazurek, “In bot we trust: A new methodology of
learner models,” Knowl-based Syst, vol. 20, no. 2, pp. 177– chatbot performance measures,” Bus Horizons, vol. 62, no.
185, 2007. 6, pp. 785–797, 2019.
[23] S. Majumder and A. Mondal, “Are chatbots really [36] E. Almansor, F. Hussain, and O. Hussain, “Supervised
useful for human resource management?,” Int J Speech ensemble sentiment-based framework to measure chatbot
Technology, pp. 1–9, 2021. quality of services,” Computing, vol. 103, no. 3, pp. 491–
[24] N. Nawaz and A. Gomes, “Artificial Intelligence 507, 2021.
Chatbots are New Recruiters,” International Journal of [37] S. Sands, C. Ferraro, C. Campbell, and H. Tsao,
Advanced Computer Science and Applications, vol. 10, no. “Managing the human-chatbot divide: how service scripts
9, pp. 1–5, 2019. influence service experience,” J Serv Manage, vol. 32, no.
[25] C. Ischen, T. Araujo, G. van Noort, H. Voorveld, and 2, pp. 246–264, Jan. 9AD.
E. Smit, “‘I Am Here to Assist You Today’: The Role of [38] R. D. Cicco, S. C. Silva, and F. R. Alparone,
Entity, Interactivity and Experiential Perceptions in “Millennials’ attitude toward chatbots: an experimental
Chatbot Persuasion,” J Broadcast Electron, vol. 64, no. 4, study in a social relationship perspective,” Int J Retail
pp. 615–639, 2020. Distribution Management, vol. 48, no. 11, pp. 1213–1233,
[26] S. Youn and S. Jin, “‘In A.I. we trust?’ The effects of 2020.
parasocial interaction and technopian versus luddite [39] S. Lee, N. Lee, and Y. Sah, “Perceiving a Mind in a
ideological views on chatbot-based customer relationship Chatbot: Effect of Mind Perception and Social Cues on Co-
management in the emerging ‘feeling economy,’” Comput presence, Closeness, and Intention to Use,” Int J Hum
Hum Behav, vol. 119, 2021. Comput Interact, vol. 36, no. 10, pp. 930–940, 2020.
[27] B. Sheehan, H. S. Jin, and U. Gottlieb, “Customer [40] D. Toader, G. Boca, R. Toader, M. Macelaru, C.
service chatbots: Anthropomorphism and adoption,” J Bus Toader, D. Ighian, and A. Radulescu, “The Effect of Social
Res, vol. 115, pp. 14–24, 2020. Presence and Chatbot Errors on Trust,” Sustainability-
[28] C. B. Nordheim, A. Følstad, and C. A. Bjørkli, “An basel, vol. 12, no. 1, 2020.
Initial Model of Trust in Chatbots for Customer Service— [41] R. Schuetzler, G. Grimes, and J. Giboney, “The
Findings from a Questionnaire Study,” Interact Comput, impact of chatbot conversational skill on engagement and
vol. 31, no. 3, pp. 317–335, 2019. perceived humanness,” J Manage Inform Syst, vol. 37, no.
[29] E. Nichifor, A. Trifan, and E. Nechifor, “Artificial 3, pp. 875–900, Jan. 2AD.
Intelligence in Electronic Commerce: Basic Chatbots and [42] L. Li, K. Lee, E. Emokpae, and S. Yang, “What makes
Consumer Journey,” Www Amfiteatrueconomic Ro, vol. 23, you continuously use chatbot services? Evidence from
no. 56, pp. 88–103, 2021. chinese online travel agencies,” Electron Mark, n.d.

VOLUME XXI, 2021

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3100885, IEEE Access

[43] A. W. Hao, J. Paul, S. Trott, C. Guo, and H.-H. Wu, [55] M. Chung, E. Ko, H. Joung, and S. Kim, “Chatbot e-
“Two decades of research on nation branding: a review and service and customer satisfaction regarding luxury brands,”
future research agenda,” Int Market Rev, vol. 38, no. 1, pp. J Bus Res, vol. 117, pp. 587–595, 2020.
46–69, 2019. [56] A. Rese, L. Ganster, and D. Baier, “Chatbots in
[44] S. Sepasgozar, R. Karimi, L. Farahzadi, F. Moezzi, S. retailers’ customer communication: How to measure their
Shirowzhan, S. M. Ebrahimzadeh, F. Hui, and L. Aye, “A acceptance?,” J Retail Consumer Serv, vol. 56, p. 102176,
Systematic Content Review of Artificial Intelligence and 2020.
the Internet of Things Applications in Smart Home,” Appl [57] D. Bailey, A. Southam, and J. Costley, “Digital
Sci, vol. 10, no. 9, p. 3074, 2020. storytelling with chatbots: mapping L2 participation and
[45] I. R. R. Lu, L. A. Heslop, D. R. Thomas, and E. Kwan, perception patterns,” Interact Technology Smart Educ, vol.
“An examination of the status and evolution of country ahead-of-print, no. ahead-of-print, 2021.
image research,” Int Market Rev, vol. 33, no. 6, pp. 825– [58] H. Kondylakis, D. Tsirigotakis, G. Fragkiadakis, E.
850, 2016. Panteri, A. Papadakis, A. Fragkakis, E. Tzagkarakis, I.
[46] A. Martín-Martín, M. Thelwall, E. Orduna-Malea, and Rallis, Z. Saridakis, A. Trampas, G. Pirounakis, and N.
E. D. López-Cózar, “Google Scholar, Microsoft Academic, Papadaki, “R2D2: A Dbpedia Chatbot Using Triple-Pattern
Scopus, Dimensions, Web of Science, and OpenCitations’ Like Queries,” Lect Notes Comput Sc, vol. 13, no. 9, 2020.
COCI: a multidisciplinary comparison of coverage via [59] M. McDonnell and D. Baxter, “Chatbots and Gender
citations,” Scientometrics, vol. 126, no. 1, pp. 871–906, Stereotyping,” Interact Comput, vol. 31, no. 2, pp. 116–
2021. 121, 2019.
[47] Altmetric, “What are Altmetrics?,” 11-May-2021. [60] L. Ciechanowski, A. Przegalinska, M. Magnuski, and
[Online]. Available: https://www.altmetric.com/about- P. Gloor, “In the shades of the uncanny valley: An
altmetrics/what-are-altmetrics/. [Accessed: 11-May-2021]. experimental study of human-chatbot interaction,” Future
[48] Altmetric, “The donut and Altmetric Attention Score – Gener Comp Sy, vol. 92, pp. 539–548, 2019.
Altmetric,” 11-May-2021. [Online]. Available: [61] E. Go and S. S. Sundar, “Humanizing chatbots: The
https://www.altmetric.com/about-our-data/the-donut-and- effects of visual, identity and conversational cues on
score/. [Accessed: 11-May-2021]. humanness perceptions,” Comput Hum Behav, vol. 97, pp.
[49] F. Johannsen, D. Schaller, and M. F. Klus, “Value 304–316, 2019.
propositions of chatbots to support innovation management [62] M. Adam, M. Wessel, and A. Benlian, “AI-based
processes,” Inf Syst E-bus Manag, vol. 19, no. 1, pp. 205– chatbots in customer service and their effects on user
246, 2021. compliance,” Electron Mark, n.d.
[50] X. Luo, S. Tong, Z. Fang, and Z. Qu, “Frontiers: [63] K. Sidaoui, M. Jaakkola, and J. Burton, “AI feel you:
Machines vs. Humans: The Impact of Artificial customer experience assessment via chatbot interviews,” J
Intelligence Chatbot Disclosure on Customer Purchases,” Serv Manage, vol. 31, no. 4, pp. 745–766, Jan. 2AD.
Market Sci, vol. 38, no. 6, pp. 937–947, 2019.
[64] V. Oguntosin and A. Olomo, “Development of an E-
[51] T. Araujo, “Living up to the chatbot hype: The Commerce Chatbot for a University Shopping Mall,” Appl
influence of anthropomorphic design cues and Comput Intell Soft Comput, vol. 2021, 2021.
communicative agency framing on conversational agent
and company perceptions,” Comput Hum Behav, vol. 85, [65] M. Ashfaq, J. Yun, S. Yu, and S. Loureiro, “I,
pp. 183–189, 2018. Chatbot: Modeling the determinants of users’ satisfaction
and continuance intention of AI-powered service agents,”
[52] D. Westerman, A. C. Cross, and P. G. Lindmark, “I Telemat Inform, vol. 54, 2020.
Believe in a Thing Called Bot: Perceptions of the
Humanness of ‘Chatbots,’” Commun Stud, vol. 70, no. 3, [66] B. AbuShawar and E. Atwell, “ALICE Chatbot: Trials
pp. 1–18, 2019. and Outputs,” Computación y Sistemas, vol. 19, no. 4, pp.
625–632, 2015.
[53] J. Hill, W. Ford, and I. Farreras, “Real conversations
with artificial intelligence: A comparison between human- [67] A. Massaro, V. Maritati, and A. Galiano, “Automated
human online conversations and human-chatbot Self-Learning Chatbot Initially Built as a FAQs Database
conversations,” Comput Hum Behav, vol. 49, pp. 245–250, Information Retrieval System: Multi-level and Intelligent
2015. Universal Virtual Front-Office Implementing Neural
Network,” Informatica-Journal of Computing and
[54] S. Park, J. Choi, S. Lee, C. Oh, C. Kim, S. La, J. Lee, Informatics, vol. 42, no. 4, pp. 515–525, 2018.
and B. Suh, “Designing a Chatbot for a Brief Motivational
Interview on Stress Management: Qualitative Case Study,” [68] C. Thorne, “Chatbots for troubleshooting: A survey,”
J Med Internet Res, vol. 21, no. 4, 2019. Lang Linguistics Compass, vol. 11, no. 10, p. e12253,
2017.

VOLUME XXI, 2021

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3100885, IEEE Access

[69] G. Gigi and S. Gunaseeli, “HR Recruitment Through Indexing,” International Journal of Advanced Computer
Chatbot- An Innovative Approach,” J Contemp Issues Bus Science and Applications, vol. 11, no. 12, pp. 299–308,
Gov, vol. 26, no. 2, pp. 564–570, Jan. 2020. 2020.
[70] M. Skjuve, A. Folstad, K. Fostervold, and P. [83] B. Eren, “Determinants of customer satisfaction in
Brandtzaeg, “My Chatbot Companion - a Study of Human- chatbot use: evidence from a banking application in
Chatbot Relationships,” Int J Hum-comput St, vol. 149, Turkey,” Int J Bank Mark, vol. 39, no. 2, pp. 294–311,
2021. 2021.
[71] A. Ho, J. Hancock, and A. Miner, “Psychological, [84] M. Nuruzzaman and O. Hussain, “IntelliBot: A
Relational, and Emotional Effects of Self-Disclosure After Dialogue-based chatbot for the insurance industry,” Knowl-
Conversations with a Chatbot,” J Commun, vol. 68, no. 4, based Syst, vol. 196, 2020.
pp. 712–733, 2018. [85] R. Bhushan, K. Kulkarni, V. Pandey, C. Rawls, B.
[72] N. Aoki, “An experimental study of public trust in AI Mechtley, S. Jayasuriya, and C. Ziegler, “ODO: Design of
chatbots in the public sector,” Gov Inform Q, vol. 37, no. 4, Multimodal Chatbot for an Experiential Media System,”
p. 101490, 2020. Multimodal Technologies Interact, vol. 4, no. 4, 2020.
[73] M. Piao, H. Ryu, H. Lee, and J. Kim, “Use of the [86] E. Pantano and G. Pizzi, “Forecasting artificial
Healthy Lifestyle Coaching Chatbot App to Promote Stair- intelligence on online customer assistance: Evidence from
Climbing Habits Among Office Workers: Exploratory chatbot patents analysis,” J Retail Consumer Serv, vol. 55,
Randomized Controlled Trial,” Jmir Mhealth Uhealth, vol. 2020.
8, no. 5, Jan. 2019. [87] A. Chaves and M. Gerosa, “How Should My Chatbot
[74] S. Hamad and T. Yeferny, “A Chatbot for Information Interact? A Survey on Social Characteristics in Human-
Security,” International Journal of Computer Science and Chatbot Interaction Design,” Int J Hum Comput Interact,
Network Security, vol. 20, no. 4, pp. 287–291, Jan. 2030. vol. 37, no. 8, pp. 729–758, 2021.
[75] L. Rajaobelina and L. Ricard, “Classifying potential [88] D. Dippold, J. Lynden, R. Shrubsall, and R. Ingram,
users of live chat services and chatbots,” J Financial Serv “A turn to language: How interactional sociolinguistics
Mark, pp. 1–14, 2021. informs the redesign of prompt:response chatbot turns,”
[76] M. Shumanov and L. Johnson, “Making conversations Discourse Context Media, vol. 37, 2020.
with chatbots more personalized,” Comput Hum Behav, [89] F. Flanagan and M. Walker, “How can unions use
vol. 117, p. 106627, 2021. Artificial Intelligence to build power? The use of AI
[77] Z. Xiao, M. Zhou, Q. Liao, G. Mark, C. Chi, W. Chen, chatbots for labour organising in the US and Australia,”
and H. Yang, “Tell Me About Yourself: Using an AI- New Technology Work Employ, 2020.
Powered Chatbot to Conduct Conversational Surveys with [90] Y. Cheng and H. Jiang, “How Do AI-driven Chatbots
Open-ended Questions,” Acm T Comput-hum Int, vol. 27, Impact User Experience? Examining Gratifications,
no. 3, 2020. Perceived Privacy Risk, Satisfaction, Loyalty, and
[78] M. Han, “The Impact of Anthropomorphism on Continued Use,” J Broadcast Electron, vol. 64, no. 4, pp.
Consumers’ Purchase Decision in Chatbot Commerce,” J 1–23, Jan. 1AD.
Internet Commer, vol. 20, no. 1, pp. 46–65, Jan. 2AD. [91] M. de Gennaro, E. Krumhuber, and G. Lucas,
[79] C. Yen and M.-C. Chiang, “Trust me, if you can: a “Effectiveness of an Empathic Chatbot in Combating
study on the factors that influence consumers’ purchase Adverse Effects of Social Exclusion on Mood,” Front
intention triggered by chatbots based on brain image Psychol, vol. 10, 2020.
evidence and self-reported assessments,” Behav Inform [92] B. Zarouali, M. Makhortykh, M. Bastian, and T.
Technol, pp. 1–18, 2020. Araujo, “Overcoming polarization with chatbot news?
[80] D. L. Kasilingam, “Understanding the attitude and Investigating the impact of news content containing
intention to use smartphone chatbots for shopping,” opposing views on agreement and credibility,” Eur J
Technol Soc, vol. 62, p. 101280, 2020. Commun, vol. 36, no. 1, pp. 53–68, 2021.

[81] J. Pereira, M. Fernández-Raga, S. Osuna-Acedo, M.

Roura-Redondo, O. Almazán-López, and A. Buldón-
Olalla, “Promoting Learners’ Voice Productions Using
Chatbots as a Tool for Improving the Learning Process in a
MOOC,” Technology Knowl Learn, vol. 24, no. 4, pp. 545–
565, 2019.
[82] S. E. Janati, A. Maach, and D. E. Ghanami, “Adaptive
e-Learning AI-Powered Chatbot based on Multimedia

VOLUME XXI, 2021

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/