See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/333931397

Survey on Intelligent Chatbots: State-of-the-Art and Future Research

Directions

Chapter · January 2020

DOI: 10.1007/978-3-030-22354-0_47

CITATIONS READS

12 9,268

2 authors:

Ebtesam Almansor Farookh Khadeer Hussain

University of Technology Sydney University of Technology Sydney
9 PUBLICATIONS   20 CITATIONS    380 PUBLICATIONS   4,588 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Personalized Multi-dimensional process framework View project

Personalized web search using ontology View project

All content following this page was uploaded by Ebtesam Almansor on 04 August 2019.

The user has requested enhancement of the downloaded file.

 Survey on Intelligent Chatbots:
State-of-the-Art and Future Research
Directions

Ebtesam H. Almansor1,2(B) and Farookh Khadeer Hussain1

1
Faculty of Engineering and Information Technology,
University of Technology Sydney, Sydney, Australia
[email protected], [email protected]
2
Community College, Najran University, Najran, Saudi Arabia

Abstract. Human-computer interaction (HCI) is an area of interest

which plays a major role in understanding the interaction between
humans and machines. Dialogue systems or conversational systems
including chatbots, voice control interfaces and personal assistants are
examples of HCI application that have been developed to interact with
users using natural language. Chatbots can help customers find useful
information for their needs. Thus, numerous organizations are using chat-
bots to automate their customer service. Thus, the needs for using arti-
ficial intelligence has been increasing due to the needs of automated
services. However, devolving smart bots that can respond at the human
level is challenging. In this paper, we survey the state-of-art chatbot
approaches from based on the ability to generate appropriate responses
perspective. After summarizing the review from this aspect, we iden-
tify the research issues and challenges in chatbots. The findings of this
research will highlight directions for future work.

Keywords: Conversation system · Chatbot ·

Responses generating approach

1 Introduction
Human-computer interaction (HCI) is a technology that allows communication
between users and computers using natural language [1]. An automated conver-
sation system (chatbot) is one human-machine conversation approach that has
been designed to convince humans they are conversing with a human instead of a
machine. Chatbots have been widely used in several domains, such as customer
service, website help and education. Recent studies predict that 80% of busi-
nesses plan to implement chatbots by 2020 [2]. The main benefits of using chat-
bots for companies is that their customer service processes are automated as the
chatbot can answer customers’ questions about products or services. However,
building a smart chatbot is challenging as requires contextual understanding,
text entailment and language-understanding technology [3]. Therefore, various
c Springer Nature Switzerland AG 2020
L. Barolli et al. (Eds.): CISIS 2019, AISC 993, pp. 534–543, 2020.
https://doi.org/10.1007/978-3-030-22354-0_47
 Survey on Intelligent Chatbots 535

forms of artificial intelligence (AI) and natural language processing (NLP) are
required.
AI aims to make communication between humans and computers easier by
using natural language [4]. However, the complexity of human language has
resulted in the need for AI scientists to provide models that can understand
human language using the NLP approach. NLP is the area of research that
explores the capability of computers to understand the human language [5].
The main focus of this literature review is to survey the existing literature
and find the challenges and issues related to a chatbot. Therefore, in this paper,
we adopt a comprehensive survey of the conversation system.
This paper is organized as follows: the first section presents an overview of
chatbots; the second section presents a detailed background about chatbots; the
third section details the classification framework for different types of chatbots;
the fourth section presents the conclusion and future work.

2 Background of Chatbots

The idea of a chatbot comes from the “imitation” game or the Turing test which
was created by Alan Turing (1950) [1]. This game aimed to determine whether
a computer could imitate human behavior. The first chatbot, called ELIZA,
was developed in 1966 [11]. This system used keyword matching and minimal
context identification; however, this bot is a primitive system that lacked the
ability to maintain a conversation between humans and bots. In the 1980s, the
ALICE (Artificial Linguistic Internet Computer Entity) chatbot was created.
This bot was considered to be significant due to the use of the Artificial Intelli-
gent Markup Language (AIML) [12]. The idea behind AIML was to declare the
pattern-matching rules which connect user-submitted words and phrases. The
Jabberwack chatbot was built to simulate natural human language to learn from
previous conversations and then the contextual patterns were used to select the
most relevant response [13]. Additionally, commercial chatbots called Lingubots
were developed to customize the template to analyze the word structure and
grammar of the user’s input [14].

2.1 Chatbot Applications in Public Sectors

Recently, the importance of chatbot in the public sector has taken place for
example, chatbot was used for political purposes to inspire public opinion and
intervene any discussion in social media about politics [15]. Another chatbot has
been proposed as a digital channel of communication between citizens and the
government [16]. In the education sector, a chatbot has been used to enhance
critical thinking and support learners in learning a new language as the user
can learn from the chatbot through their conversations [17]. An educational
bot combining an intelligent tutoring system and learner modeling was designed
to support learners [18]. Another chatbot was proposed for medical students for
536 E. H. Almansor and F. K. Hussain

educational purposes [19]. In the health care sector, Your.MD chatbot was devel-
oped to provide relevant health information for patients [5]. Shawar and Atwell
developed an algorithm for retraining a chatbot in a specific domain about a
specific topic in any language [20]. Their algorithm was applied on two differ-
ent languages, Arabic and Afrikaans, using the different corpus, the Qu’ran to
compute frequently asked questions and the corpus of Spoken Afrikaans, respec-
tively.
In the past few years, chatbots have been increasingly used by several orga-
nizations to increase the response time to customers in answering their questions
and also reduce operational costs. Chatbot applications have been used in both
the private sector, including the virtual assistants that are powered by voice (e.g.
Siri, Alexa, Google now, Cortana) and public sector gaming agencies, telecom-
munications, banking (implementing transactions), tourism (booking hotels or
tickets), media (news provision), retail, stock market and insurance companies
[16]. Additionally, governments have used chatbots on social media platforms
such as Twitter as a new form of political communication [21].

3 Proposed Classification for Chatbot Approaches

With the growth of AI technology, NLP researchers aim to build automatic

conversation agents that respond to humans in a suitable time. There are two
categories of conversation agents, namely task-oriented and non-task-oriented
chatbots. Figure 1 shows our proposed classification of the chatbot categories.

Fig. 1. Proposed classification for chatbot

 Survey on Intelligent Chatbots 537

3.1 Task-Oriented Dialogue System

A task-oriented system is one which is designed based on hand-crafted rules to
help users achieve their goal or complete a specific task in a specific domain
such as making a booking, traveling, shopping or ordering food [3,22]. This
system has been widely used in both industry (Apple Siri, Microsoft Cortana,
Facebook Messenger and Google Now) and academia [23]. They are focusing
on developing natural language understanding (NLU) methods which parse the
utterance from the user into predefined semantic slots. In the next paragraph, we
discuss some of the task-oriented systems which have been implemented. There
are two approaches, the supervised approach and the non-supervised approach.

3.1.1 Supervised Approach

The supervised approach depends on handcrafted feature extraction and anno-
tated datasets [24]. As an example of this approach, Roy et al. [25] proposed a
model that parsed the user’s input into a semantic representation; then these
representations are exploited by the dialogue manager to decide on the appro-
priate response. The dialogue manager tracks the conversation to determine who
has spoken last, whether the information is private or shared; what plan should
be followed and to what degree does the system understand the user’s input.
Then the dialogue system will respond to the user’s input using NLP. There are
also several approaches that can be used by dialogue systems to respond to the
user, such as the slot fill approach [6]. Sequence-to-sequence (seq2seq) was used
in a task-oriented dialogue system in [26]. However, this model cannot map the
correct entities to the generated input. Thus, a copy-augmented Seq2Seq model
was used to copy the relevant information from the knowledge base (KB) in
[27]. The supervised approach has several limitations, such as the data annota-
tion process and handcrafted feature extraction incur high costs and has poor
scalability [24]. Thus, the unsupervised approach has been investigated.

3.1.2 Unsupervised Approach

The unsupervised approach learns features automatically from unlabeled
datasets [24]. One of the most effective approaches is a deep learning approach
which trained on an end-end neural network [28]. A convolution network model
has been developed to capture the interaction between message and response
[29]. An end-to-end-oriented dialogue system has been developed that uses the
pipe-lined Wizard-of-Oz framework to collect the dialogue datasets. The main
advantage of this model is the conversation developed without any need to use
handcraft features to make assumptions [28]. Wang et al. [30] developed a system
that leverages syntax features to measure text similarity.

3.2 Non-task-Oriented Dialogue System

A non-task-oriented dialogue system provides users with the means to participate
in different domains such as a game, chitchat or entertainment, without providing
538 E. H. Almansor and F. K. Hussain

the user with any help to complete any task in a specific job [31]. An example
of this system is the chatbot which chats with the user in a similar way to a
human and provides reasonable and relevant responses [3,31,32]. ELIZA is a
chatbot that uses a combination of rules and patterns. In 1981, another chatbot
was developed using simple text parsing rules to construct the dialogue system,
PARRY [33]. Both systems did not use data for learning purposes. Therefore,
a data-driven approach has been proposed to overcome some of the limitations
of the non-task-oriented dialogue system. This approach enables chatbots to
learn from the massive amount of available conversations on social media or
the Web2.0, which enhances the communication between humans and chatbots.
The developed methods are either retrieval-based or generation-based. Retrieval-
based models can obtain response candidates from a pre-built index, rank the
candidates then chose the response from the top-ranked ones [3,32].On the other
hand, generation-based methods use natural language generation (NLG) to select
the response [34].

3.2.1 Retrieval-Based Chatbot

A retrieval-based chatbot uses a selection of responses from ranking data rather
than generating a new response. A dialogue system with a knowledge base that
contains a considerable number of question-answer pairs was developed based
on a statistical language model to select suitable responses [14,35]. The informal
response interactive system (IRIS) is a chatbot that based on a vector space
model. This bot searches for candidate responses to users’ input using cosine
similarity metrics [31]. Another example of a retrieval-based chatbot is the chat-
bot that proposed in [32] which can obtain response candidates with a research
engine by utilizing the text similarity between the message and the response to
select a proper response. This approach combines several text similarity features
including topic similarity, cosine similarity and translation score. These features
are used to rank the candidate responses [32]. Another approach is based on cap-
turing the interaction between message and response using a convolution network
[29]. Measuring text similarity using leverage syntax features was proposed in
[30]. Considering the Using conversation history to support multi-turn conver-
sations was studied in [22,36]. The authors used a neural learning architecture
to select the best response and the model was tested on the Ubuntu Dialogue
Corpus. Another retrieval model which considers conversation history and topic
information to select a response in a single turn was proposed in [3], where the
authors used an attention mechanism to enhance message-response matching.
This chatbot can solve this problem by providing immediate responses. Thomas
[37] built a chatbot that supports e-business and gives an immediate response
to customers based on the frequently asked questions (FAQs). The author used
a combination of artificial intelligent markup language (AIML) to answer gen-
eral questions and latent semantic analysis (LSA) to answer questions related to
services [37]. Another chatbot that focuses on healthcare was built to support
people who have busy lives. This bot uses deep learning to address the prob-
lem of predefined rules as it focuses on the quality of the interaction between
 Survey on Intelligent Chatbots 539

humans and computers. This is considered to be a self-adapting approach as it

uses previous interactions with the chatbot [38].

3.2.2 Generation-Based Chatbot

A generation-based chatbot generates responses instead of selecting them from
the underlying model. Knight et al. stated that using data-driven generation
which depends on the utterance might be useful in a knowledge system [39]. A
hybrid generation algorithm that combines finite state machine (FSM) grammars
and a corpus-based language model has been developed in [40]. The FSM gram-
mar is controlled by the concept of n-gram and words that take terminal and
non-terminal co-occurrences into account. Due to the limited number of deriva-
tions, this approach achieved faster performance than real-time performance.
Statistical machine translation has been investigated in translating an internal
dialogue state into natural language [9]. Employing a statistical machine transla-
tion approach as a generic approach was proposed in [8,41]. Also, a sequence to
sequence (S2S) framework was developed to generate responses to users’ input
[34,42–44]. Shang et al. [45] proposed a sequence to sequence framework which
encodes a message with a neural network and generates responses using another
recurrent neural network with an attention mechanism. A conventional S2S gen-
eration model was utilized in [46]. The authors modified the loss function to
promote the diversity of responses.

4 Conclusion and Future Work

In this paper, we briefly introduced the history of chatbots and its use in public
sectors. Then the proposed classification of the existing approaches and the types
of chatbots was explained. In this research, some findings have been founded and
need to be investigated. As most of the existing research focuses on improving
the responses in chatbot other linguistic features need to be investigated such
as using emotional or sentiment analysis. Also, the importance of user interface
as an inter- active aspect of chatbots need more attention. There is a need for
standard a framework to measure the quality of chatbot which is lacking in the
existing literature. Also, using artificial intelligence to improve current chatbots
and services could be another research direction. Deep learning is a promising
approach that could be applied to develop more effective chatbots.

References
1. Shawar, B.A., Atwell, E.: Different measurements metrics to evaluate a chatbot
system. In: Proceedings of the Workshop on Bridging the Gap: Academic and
Industrial Research in Dialog Technologies, Association for Computational Lin-
guistics, pp. 89–96 (2007)
2. Sandbank, T., Shmueli-Scheuer, M., Herzig, J., Konopnicki, D., Richards, J.,
Piorkowski, D.: Detecting egregious conversations between customers and virtual
agents. arXiv preprint arXiv:1711.05780 (2017)
540 E. H. Almansor and F. K. Hussain

3. Wu, Y., Li, Z., Wu, W., Zhou, M.: Response selection with topic clues for retrieval-
based chatbots. Neurocomputing 316, 251–261 (2018)
4. Radziwill, N.M., Benton, M.C.: Evaluating quality of chatbots and intelligent con-
versational agents. arXiv preprint arXiv:1704.04579 (2017)
5. de Haan, H., Snijder, J., van Nimwegen, C., Beun, R.J.: Chatbot personality and
customer satisfaction (2018)
6. Mesnil, G., Dauphin, Y., Yao, K., Bengio, Y., Deng, L., Hakkani-Tur, D., He, X.,
Heck, L., Tur, G., Yu, D.: Using recurrent neural networks for slot filling in spoken
language understanding. IEEE/ACM Trans. Audio Speech Lang. Process. 23(3),
530–539 (2015)
7. Li, J., Monroe, W., Ritter, A., Galley, M., Gao, J., Jurafsky, D.: Deep reinforcement
learning for dialogue generation. arXiv preprint arXiv:1606.01541 (2016)
8. Ritter, A., Cherry, C., Dolan, B.: Unsupervised modeling of Twitter conversations.
In: Human Language Technologies: The 2010 Annual Conference of the North
American Chapter of the Association for Computational Linguistics, pp. 172–180.
Association for Computational Linguistics (2010)
9. Langner, B., Vogel, S., Black, A.W.: Evaluating a dialog language generation
system: comparing the mountain system to other NLG approaches. In: Eleventh
Annual Conference of the International Speech Communication Association (2010)
10. Venkatesh, A., Khatri, C., Ram, A., Guo, F., Gabriel, R., Nagar, A., Prasad, R.,
Cheng, M., Hedayatnia, B., Metallinou, A., et al.: On evaluating and comparing
conversational agents. arXiv preprint arXiv:1801.03625 4, 60–68 (2018)
11. Weizenbaum, J.: ELIZA—a computer program for the study of natural language
communication between man and machine. Commun. ACM 9(1), 36–45 (1966)
12. Wallace, R.: The elements of AIML style. Alice AI Foundation (2003)
13. Carpenter, R.: Jabberwocky. Jabberwocky.com (1997–2006). www.jabberwocky.
com. Accessed 16 May 2006
14. Leuski, A., Traum, D.: NPCeditor: creating virtual human dialogue using informa-
tion retrieval techniques. Ai Mag. 32(2), 42–56 (2011)
15. Suárez-Serrato, P., Roberts, M.E., Davis, C., Menczer, F.: On the influence of
social bots in online protests. In: International Conference on Social Informatics,
pp. 269–278. Springer (2016)
16. Androutsopoulou, A., Karacapilidis, N., Loukis, E., Charalabidis, Y.: Transforming
the communication between citizens and government through AI-guided chatbots.
Gov. Inf. Quart. 36, 358–367 (2018)
17. Goda, Y., Yamada, M., Matsukawa, H., Hata, K., Yasunami, S.: Conversation
with a chatbot before an online EFL group discussion and the effects on critical
thinking. J. Inf. Syst. Educ. 13(1), 1–7 (2014)
18. Kerly, A., Hall, P., Bull, S.: Bringing chatbots into education: towards natural
language negotiation of open learner models. Knowl. Based Syst. 20(2), 177–185
(2007)
19. Kerfoot, B.P., Baker, H., Jackson, T.L., Hulbert, W.C., Federman, D.D., Oates,
R.D., DeWolf, W.C.: A multi-institutional randomized controlled trial of adjuvant
Web-based teaching to medical students. Acad. Med. 81(3), 224–230 (2006)
20. Shawar, B.A., Atwell, E.: Fostering language learner autonomy through adaptive
conversation tutors. In: Proceedings of the The fourth Corpus Linguistics Confer-
ence
21. Forelle, M., Howard, P., Monroy-Hernández, A., Savage, S.: Political bots and
the manipulation of public opinion in Venezuela. arXiv preprint arXiv:1507.07109
(2015)
 Survey on Intelligent Chatbots 541

22. Yan, R., Song, Y., Wu, H.: Learning to respond with deep neural networks for
retrieval-based human-computer conversation system. In: Proceedings of the 39th
International ACM SIGIR Conference on Research and Development in Informa-
tion Retrieval, pp. 55–64. ACM (2016)
23. Henderson, M., Thomson, B., Young, S.: Deep neural network approach for the
dialog state tracking challenge. In: Proceedings of the SIGDIAL 2013 Conference,
pp. 467–471 (2013)
24. Chen, Y.N.: Unsupervised learning and modeling of knowledge and intent for spo-
ken dialogue systems. In: Proceedings of the ACL-IJCNLP 2015 Student Research
Workshop, pp. 1–7 (2015)
25. Roy, N., Pineau, J., Thrun, S.: Spoken dialogue management using probabilistic
reasoning. In: Proceedings of the 38th Annual Meeting on Association for Compu-
tational Linguistics, pp. 93–100. Association for Computational Linguistics (2000)
26. Zhao, T., Lu, A., Lee, K., Eskenazi, M.: Generative encoder-decoder models
for task-oriented spoken dialog systems with chatting capability. arXiv preprint
arXiv:1706.08476 (2017)
27. Eric, M., Manning, C.D.: A copy-augmented sequence-to-sequence architecture
gives good performance on task-oriented dialogue. arXiv preprint arXiv:1701.04024
(2017)
28. Wen, T.H., Vandyke, D., Mrksic, N., Gasic, M., Rojas-Barahona, L.M., Su, P.H.,
Ultes, S., Young, S.: A network-based end-to-end trainable task-oriented dialogue
system. arXiv preprint arXiv:1604.04562 (2016)
29. Hu, B., Lu, Z., Li, H., Chen, Q.: Convolutional neural network architectures for
matching natural language sentences. In: Advances in Neural Information Process-
ing Systems, pp. 2042–2050 (2014)
30. Wang, M., Lu, Z., Li, H., Liu, Q.: Syntax-based deep matching of short texts. arXiv
preprint arXiv:1503.02427 (2015)
31. Banchs, R.E., Li, H.: IRIS: a chat-oriented dialogue system based on the vector
space model. In: Proceedings of the ACL 2012 System Demonstrations, pp. 37–42.
Association for Computational Linguistics (2012)
32. Ji, Z., Lu, Z., Li, H.: An information retrieval approach to short text conversation.
arXiv preprint arXiv:1408.6988 (2014)
33. Colby, K.M.: Modeling a paranoid mind. Behav. Brain Sci. 4(4), 515–534 (1981)
34. Shang, L., Lu, Z., Li, H.: Neural responding machine for short-text conversation.
arXiv preprint arXiv:1503.02364 (2015)
35. Leuski, A., Patel, R., Traum, D., Kennedy, B.: Building effective question answer-
ing characters. In: Proceedings of the 7th SIGdial Workshop on Discourse and
Dialogue, pp. 18–27. Association for Computational Linguistics (2009)
36. Lowe, R., Pow, N., Serban, I., Pineau, J.: The Ubuntu dialogue corpus: a large
dataset for research in unstructured multi-turn dialogue systems. arXiv preprint
arXiv:1506.08909 (2015)
37. Thomas, N.: An e-business chatbot using AIML and LSA. In: 2016 Interna-
tional Conference on Advances in Computing, Communications and Informatics
(ICACCI), pp. 2740–2742. IEEE (2016)
38. Rai, S., Raut, A., Savaliya, A., Shankarmani, R.: Darwin: convolutional neural net-
work based intelligent health assistant. In: 2018 Second International Conference
on Electronics, Communication and Aerospace Technology (ICECA), pp. 1367–
1371. IEEE (2018)
39. Knight, K., Hatzivassiloglou, V.: Two-level, many-paths generation. In: Proceed-
ings of the 33rd Annual Meeting on Association for Computational Linguistics, pp.
252–260. Association for Computational Linguistics (1995)
542 E. H. Almansor and F. K. Hussain

40. Galley, M., Fosler-Lussier, E., Potamianos, A.: Hybrid natural language generation
for spoken dialogue systems. In: Seventh European Conference on Speech Commu-
nication and Technology (2001)
41. Ritter, A., Cherry, C., Dolan, W.B.: Data-driven response generation in social
media. In: Proceedings of the Conference on Empirical Methods in Natural Lan-
guage Processing, pp. 583–593. Association for Computational Linguistics (2011)
42. Serban, I.V., Sordoni, A., Bengio, Y., Courville, A., Pineau, J.: Building end-to-
end dialogue systems using generative hierarchical neural network models. In: 30th
AAAI Conference on Artificial Intelligence, AAAI 2016, pp. 3776–3783 (2016)
43. Vinyals, O., Le, Q.: A neural conversational model. arXiv preprint arXiv:1506.
05869 (2015)
44. Zhang, W.N., Zhu, Q., Wang, Y., Zhao, Y., Liu, T.: Neural personalized response
generation as domain adaptation. World Wide Web, pp. 1–20 (2017)
45. Shang, L., Sakai, T., Lu, Z., Li, H., Higashinaka, R., Miyao, Y.: Overview of the
NTCIR-12 short text conversation task. In: NTCIR (2016)
46. Li, J., Galley, M., Brockett, C., Gao, J., Dolan, B.: A diversity-promoting objective
function for neural conversation models. arXiv preprint arXiv:1510.03055 (2015)
47. Engelbrech, K.P., Gödde, F., Hartard, F., Ketabdar, H., Möller, S.: Modeling user
satisfaction with hidden markov model. In: Proceedings of the SIGDIAL 2009
Conference: the 10th Annual Meeting of the Special Interest Group on Discourse
and Dialogue, pp. 170–177. Association for Computational Linguistics (2009)
48. Ultes, S., Schmitt, A., Minker, W.: On quality ratings for spoken dialogue systems-
experts vs. users. In: Proceedings of the 2013 Conference of the North American
Chapter of the Association for Computational Linguistics: Human Language Tech-
nologies, pp. 569–578 (2013)
49. Hone, K.S., Graham, R.: Towards a tool for the subjective assessment of speech
system interfaces (SASSI). Nat. Lang. Eng. 6(3–4), 287–303 (2000)
50. Möller, S.: Subjective quality evaluation of telephone services based on spoken
dialogue systems. ITU-T Recommendation, p. 851 (2003)
51. Higashinaka, R., Minami, Y., Dohsaka, K., Meguro, T.: Issues in predicting user
satisfaction transitions in dialogues: individual differences, evaluation criteria, and
prediction models, pp. 48–60. Springer (2010a)
52. Higashinaka, R., Minami, Y., Dohsaka, K., Meguro, T.: Modeling user satisfaction
transitions in dialogues from overall ratings. In: Proceedings of the 11th Annual
Meeting of the Special Interest Group on Discourse and Dialogue, pp. 18–27. Asso-
ciation for Computational Linguistics (2010b)
53. Schmitt, A., Schatz, B., Minker, W.: Modeling and predicting quality in spoken
human-computer interaction. In: Proceedings of the SIGDIAL 2011 Conference,
pp. 173–184. Association for Computational Linguistics (2011)
54. Hara, S., Kitaoka, N., Takeda, K.: Estimation method of user satisfaction using
N-gram-based dialog history model for spoken dialog system. In: LREC (2010)
55. Schmitt, A., Schatz, B., Minker, W.: A statistical approach for estimating user
satisfaction in spoken human-machine interaction. In: 2011 IEEE Jordan Confer-
ence on Applied Electrical Engineering and Computing Technologies (AEECT),
pp. 1–6. IEEE (2011b)
56. Martinez, F.F., Blázquez, J., Ferreiros, J., Barra, R., Macias-Guarasa, J., Lucas-
Cuesta, J.M.: Evaluation of a spoken dialogue system for controlling a HiFi audio
system. In: Spoken Language Technology Workshop 2008, SLT 2008 IEEE, pp.
137–140. IEEE (2008)
57. Machinery, C.: Computing machinery and intelligence-AM turing. Mind 59(236),
433 (1950)
 Survey on Intelligent Chatbots 543

58. Liu, C.W., Lowe, R., Serban, I.V., Noseworthy, M., Charlin, L., Pineau, J.: How not
to evaluate your dialogue system: an empirical study of unsupervised evaluation
metrics for dialogue response generation. arXiv preprint arXiv:1603.08023 (2016)
59. Yuwono, S.K., Biao, W., D’Haro, L.F.: Automated scoring of chatbot responses in
conversational dialogue
60. Lowe, R., Noseworthy, M., Serban, I.V., Angelard-Gontier, N., Bengio, Y., Pineau,
J.: Towards an automatic turing test: learning to evaluate dialogue responses. arXiv
preprint arXiv:1708.07149 (2017)
61. DeVault, D., Leuski, A., Sagae, K.: Toward learning and evaluation of dialogue
policies with text examples. In: Proceedings of the SIGDIAL 2011 Conference, pp.
39–48. Association for Computational Linguistics (2011)
62. Gandhe, S., Traum, D.: A semi-automated evaluation metric for dialogue model
coherence, pp. 217–225. Springer (2016)
63. Guo, F., Metallinou, A., Khatri, C., Raju, A., Venkatesh, A., Ram, A.: Topic-based
evaluation for conversational bots. arXiv preprint arXiv:1801.03622 (2018)
64. Yu, Z., Xu, Z., Black, A.W., Rudnicky, A.: Strategy and policy learning for non-
task-oriented conversational systems. In: Proceedings of the 17th Annual Meeting
of the Special Interest Group on Discourse and Dialogue, pp. 404–412 (2016)
65. Bowman, S.R., Vilnis, L., Vinyals, O., Dai, A.M., Jozefowicz, R., Bengio, S.: Gen-
erating sentences from a continuous space. arXiv preprint arXiv:1511.06349 (2015)
66. Li, J., Monroe, W., Shi, T., Jean, S., Ritter, A., Jurafsky, D.: Adversarial learning
for neural dialogue generation. arXiv preprint arXiv:1701.06547 (2017)
67. Chakrabarti, C., Luger, G.F.: Artificial conversations for customer service chatter
bots: architecture, algorithms, and evaluation metrics. Expert Syst. Appl. 42(20),
6878–6897 (2015)

View publication stats