Development of Gamified Cognitive Agents for Primary

Schools - A Case Study

Markus Schatten Marinela Schatten
Artificial Intelligence Laboratory Construction, science and mining school, Varaždin
University of Zagreb Hallerova aleja 3, 42000 Varaždin, Croatia
Faculty of Organization and Informatics [email protected]
Pavlinska 2, 42000 Varaždin, Croatia
[email protected]

Ljiljana Inkret-Martinčević
2nd Primary School, Varaždin
Augusta Cesarca 10, 42000 Varaždin, Croatia
[email protected]

Abstract. In this paper we provide a first case study health therapy (Suganuma et al., 2018), cognitive ra-
on the development and implementation of cognitive dio (Mitola, 2002) and many more. These are systems
agents for primary school students’ supports using a that can interact with humans using a variety of tech-
design science approach. Cognitive agents are artifi- nologies, including but not limited to Text-to-Speech
cial intelligence systems that are able to interact with (TTS), Speech-to-Text (STT), Natural Language Pro-
humans using various technologies including but not cessing (NLP), Belief-Desire-Intention (BDI) architec-
limited to text to speech (TTS), speech to text (STT), ture, knowledge base (KB), as well as motion capture
natural language processing (NLP), belief-desire- (MoCap), lip sync and animation, providing an immer-
intention (BDI), as well as motion capture (MoCap), sive User Experience (UX) (Lee, 2010).
lip sync and animation in order to provide for an In our previous work, we have developed a cog-
immersive user experience (UX). We have developed nitive agent architecture called Beautiful ARtificial
four cognitive agents for a Croatian school especially Intelligence Cognitive Agent (B.A.R.I.C.A.) that is
for second, third and seventh grade pupils in col- based on a number of AI techniques and technolo-
laboration with their teachers. The cognitive agents gies (Schatten, Okreša Ðurić, and Peharda, 2021). We
development process has additionally been gamified in have also proposed a framework that combines cogni-
sense that students have participated in it through a tive agents, gamification practices, and telemedicine
competition to create the face of the agent as well as (Schatten, Okreša Ðurić, and Protrka, 2021). Fur-
to provide a name for it. thermore, we have developed the B.A.R.I.C.A. infras-
tructure, which allows for the implementation of open
Keywords. gamification, cognitive agents, primary source cognitive agents that can communicate using the
school, artificial intelligence, croatian language, design Croatian language (Schatten, Ðurić, et al., 2022).
science However, to the best of our knowledge, there has
been no study that introduces gamified cognitive agents
to primary school students. Therefore, this paper
1 Introduction presents a first case study based on a design science
approach on the development and implementation of
The advent of artificial intelligence (AI) has opened cognitive agents for primary school students’ supports,
up new horizons in various domains, including educa- specifically for second, third, and seventh grade pupils
tion. Cognitive agents have gained major popularity in a Croatian school. The cognitive agents develop-
through various systems by major software develop- ment process has additionally been gamified in the
ment vendors like Google (Assistant), Microsoft (Cor- sense that students have participated in it through a
tana), Apple (Siri) and Amazon (Alexa) as well as most competition to create the face of the agent as well as
recently various chatbots based on generative AI like to provide a name for it.
ChatGPT (OpenAI) and Bard (Google). their appli- The development of gamified cognitive agents is
cation domains include Internet of things (IoT) and fog part of the project anon. The project aims to estab-
computing (Foukalas, 2020), education (Baylor, 1999), lish the first STEAM (where STEAM stands for Sci-
home service robots (Van Dang et al., 2017), mental ence, Technology, Engineering, Arts, and Mathemat-
ics) primary school in anon. During the project, the learning materials provided in the Croatian language.
anon-primary-school currently undergoes a digi- This paper aims to fill this gap by presenting a case
tal transformation through the STEAM concept, in- study on the development and implementation of gam-
cluding equipping four classrooms (including an astro- ified cognitive agents for primary school students in
nomic observatory), comprehensive teacher education, Croatia.
and the development of curriculum and activities for
students. The project is led by the anon-school, with
partners from the anon-faculty and anon-company 3 Development Process
from the Kingdom of Norway and features the intro-
duction of advanced technologies into the school’s cur- To implement gamified cognitive agents for primary
ricula including but not limited to augmented reality school students we have adopted a design science ap-
systems, robotics, asteroid tracing systems, game de- proach. Design science is a research methodology in
velopment, 3D printing as well as cognitive agents.1 the field of information systems and technology that
The rest of the paper is organized as follows: Sec- aims to create novel artifacts in the form of models,
tion 2 provides a review of the related work. Section methods, and systems. These artifacts are designed to
3 describes the methodology used in the development support people in developing, using, and maintaining
and implementation of the cognitive agents. Section 4 IT solutions Johannesson and Perjons, 2014. The work
discusses our finding in four experimental usage ses- of design science is applied to various fields of human
sions conducted in the 2nd Primary School in Varaždin. practice, not limited to information systems and tech-
Finally, Section 5 concludes the paper and suggests fu- nology.
ture work directions.

Problem Identification & Motivation

2 Related Work
The use of cognitive agents in education has been ex- Problem
Define the Objectives for a Solution centered
plored in several studies. For instance, in previous initiation

work (Schatten, Okreša Ðurić, and Peharda, 2021) we

have developed a cognitive agent for university stu-
Objective
dents that is able to communicate using spoken Croat- Design & Development centered
solution
ian language and provides personalized support infor-
mation including but not limited to schedule, office
hours, campus information and similar. The agent does Design &
development
however not include learning materials. Another study Demonstration
centered
initiation
by Sikström et al. (Sikström et al., 2022) found that
cognitive agents can significantly enhance the learning
Client/
experience in a virtual learning environment by provid- Evaluation context
initiation
ing personalized feedback and guidance.
Chatbots have also been utilized in the educational
sector. A study by Kuhail et al. (Kuhail et al., 2023)
Communication
found that chatbots can provide instant responses to
students’ queries, enhancing their learning experience
in higher education. In the context of primary educa-
tion, several studies have explored the use of chatbots. Figure 1: Design science methodology (Azasoo and
For example, a study by Putjorn (Putjorn, 2022) found Boateng, 2015) adapted from (Peffers et al., 2007)
that chatbots can enhance the English language skills
of primary school students. Similarly, a study by Ghaz- Design science methodology (see figure 1) typically
ali et al. (Ghazali and Saad, 2022) found that chatbots involves a number of main steps which are not strictly
can significantly enhance the mathematics learning ex- sequential and may be iterated over multiple times in a
perience of primary school students. Lastly, a study design science research project. Bellow is a description
by Othman (Othman, 2023) discussed the use of AI of these steps and how we have applied them in the
mobile application chatbots for teaching English as a development of our gamified cognitive agents:
Foreign Language (EFL) to primary school students in
Saudi Arabia. 1. Problem Identification and Motivation: This
Despite these advancements, none of the above stud- step involves identifying a problem that needs
ies focus on the combination of gamification and cog- to be solved and establishing its relevance. It is
nitive agents. Moreover, none of these studies focus on crucial to understand the problem thoroughly and
to articulate the motivation for solving it.
1 More details abot the project are available here: https://steam.hr
 In our case the problem to be addressed is de- given the opportunity to design the cogni-
fined in the anon-project objectives to which tive agents’ faces and name them, fostering
the development of cognitive agents contribute. a sense of ownership and engagement in the
It involves enhancing the STEAM competencies learning process.
of students across all primary educational lev-
els. This shall be achieved through the inte- These objectives guide the design and develop-
gration of advanced technology and innovative ment process of the gamified cognitive agents, en-
teaching methods, including the organization of suring that the solution is tailored to meet the spe-
gamified workshops. These workshops will uti- cific needs of the primary school students, partic-
lize specially developed cognitive agents and an ularly those with special educational needs.
augmented reality systems tailored for various
3. Design and Development: In this step, an artifact
STEM-related subjects. Additionally, the project
is designed and developed to meet the defined ob-
aims to empower students with special educa-
jectives. The artifact could be a model, a method,
tional needs by developing and piloting four new
or an instantiation.
and improved extracurricular educational pro-
grams. These programs, equipped with specially The design of the cognitive agents has been iter-
designed learning materials for potentially gifted ative in the sense that each agent prototype was
students or those with special educational needs, developed in full and then tested and evaluated
will be developed by teachers under the mentor- in a special class session with students and their
ship of partner experts in relevant fields. By the teachers which were both involved in the devel-
end of the project, these programs are expected to opment of the cognitive agent. The feedback
become an integral part of the school curriculum. from each session was then used for the develop-
ment of the next cognitive agent prototype (with
2. Define the Objectives for a Solution: Based on retrofitting all useful functionality into the previ-
the problem identified, the objectives of the so- ous instances).
lution are defined. These objectives should be The development process had the following
achievable and should directly address the prob- phases:
lem.
The primary objectives for the development of our (a) Preparation phase Firstly all teachers that
gamified cognitive agents are outlined as follows: were going to participate in the development
process participated in a two day workshop
(a) Ease of Educational Processes: The fore- on the use of AI in education with a special
most objective is to simplify the educational accent on generative AI, chatbots and cogni-
processes for primary school students, with tive agents.
a particular emphasis on students with spe- (b) Teacher and student participation After-
cial educational needs. The gamified cogni- wards teachers were asked to prepare learn-
tive agents are designed to facilitate learn- ing materials in form of questions the cogni-
ing and make it more engaging for these stu- tive agents should be able to answer. Ad-
dents. ditionally, they were asked to organize a
student competition and group activity with
(b) Ease of Use: The cognitive agents are devel-
their students to draw a face picture for the
oped with a focus on user-friendliness. The
cognitive agent which will be animated by
interface and functionalities are designed to
the development team as well as to choose a
be intuitive and easy to navigate, reducing
name for the agent. The drawn face pictures
the learning curve for students and teachers
and chosen names are shown on figure 2.
alike.
(c) Agent implementation All agents’ backends
(c) Cross-Platform Compatibility: To ensure
were developed in Python using the chatter-
accessibility and ease of use from home, the
bot module for an implementation of a re-
cognitive agents should be implemented as
gression based natural language processing
web applications and are compatible with
(NLP) model and chatbot similar to the im-
both mobile and desktop platforms. This
plementation provided in (Schatten, Okreša
flexibility allows students to engage with the
Ðurić, and Peharda, 2021; Šokec, 2019)
learning material at their convenience, re-
as well as Flask for the implementation
gardless of the device they are using.
of a web application. The frontends were
(d) Immersiveness and Student Involvement: To developed using JavaScript especially us-
enhance the immersive learning experience, ing the SpeechRecognition application pro-
students are involved in the development gramming interface (API) that allowed us to
process of the cognitive agents. They are capture speech and convert it to text. The
 5. Evaluation: The artifact is evaluated to determine
how well it solves the problem. This could in-
volve observing the use of the artifact, conducting
experiments, or performing analysis.
In our case, the provided feedback was thoroughly
analyzed and various comments and suggestions
were included into the implementation. Some
example suggestions include: (1) the possibil-
ity to write down or select the text for a given
question since some (especially younger) students
with pronunciation problems weren’t recognized
well by the speech recognition system, (2) include
a button for starting the speech recognition since,
due to background noise, sometimes the question
was not recognized in full (i.e. cut in part) or
connected to other parts of speech, (3) for top-
ics that include mathematical formula or geomet-
ric shapes (especially mathematics and physics) to
include visualization of the formula or shape be-
ing talked about).

6. Communication: The results are then communi-

cated to the relevant audiences. This could involve
Figure 2: Cognitive agent’s faces and chosen names publishing research papers, presenting at confer-
(created by students of the anon-school) ences, or producing technical reports.
Apart from this paper, all the developed gami-
face images were animated using CrazyTalk fied cognitive agents have been published and are
and the speech was generated using various available for free use and testing on line2 . Addi-
online speech-to-text services. The agents tionally, results have been communicated to rele-
were additionally tested by a teacher before vant media outlets.
demonstration, and feedback was included
into the implementation.
4 Discussion
4. Demonstration: The artifact is then demon-
strated to solve one or more instances of the prob- The experimental sessions held during the demonstra-
lem. This could involve using the artifact in an tion phase of the study provided valuable insights into
experiment, a simulation, a case study, or other the effectiveness and potential of the gamified cogni-
appropriate activity. tive agents. Each cognitive agent was tested in a special
class session, organized in the form of a role-playing
For each cognitive agent a special class session
game, where students assumed the role of the teacher
was organized in which the students could test its
and interacted with the agent. The feedback collected
abilities. The sessions were held by the teacher
from these sessions, both oral and written, was instru-
who organized the competition (usually the head-
mental in identifying areas of improvement and gaug-
room teacher of the group), the teacher which
ing the overall success of the initiative.
tested it as well as one or two members from the
development team. The session was organized in The most significant finding from these sessions was
form of a role-playing game (RPG) in which each the increased level of student engagement. Teachers re-
student assumed the role of the teacher and could ported that student participation was much higher than
ask the agent questions about the agreed topic and usual, particularly among lower-grade students. This
provide feedback about how well the answer was. suggests that the gamified cognitive agents were suc-
After each session students were invited to pro- cessful in making the learning process more engag-
vide feedback to the development form (in oral ing and interactive. Furthermore, students expressed
form for 2nd and 3rd grade students, in written a high level of satisfaction and enthusiasm about being
form for 7th grade students). Additionally, after included in the development process and were eager to
the session a discussion involving the teachers and provide feedback.
the development team members was held in or- 2 The agents are available on the following URLs https://
der to clarify additional shortcomings and poten- dragon.foi.hr/micko/ https://dragon.foi.hr/zamrljan/ https://dragon.
tial improvements. foi.hr/iva/ https://dragon.foi.hr/drslavica/
 In terms of students with special educational needs, tation. We will continue to involve students in the
the sessions had varying impacts. Students with learn- development process, fostering a sense of ownership
ing disabilities were notably more engaged than usual, and engagement in the learning process. Furthermore,
indicating that the gamified cognitive agents could po- we will explore additional functionalities and improve-
tentially serve as an effective tool for this group. How- ments based on the feedback received from students
ever, there was no significant increase in engagement and teachers. This includes the possibility of inte-
among gifted students, although they were keen on grating visual aids for topics that include mathemati-
providing feedback and suggestions. This suggests cal formulas or geometric shapes, and improving the
that while the cognitive agents may not necessarily en- speech recognition system to better accommodate stu-
hance engagement among gifted students, they could dents with pronunciation difficulties. Through these
still serve as a platform for these students to express efforts, we aim to further enhance the learning experi-
their ideas and contribute to the learning process. ence for primary school students and contribute to the
An interesting side effect was observed among stu- digital transformation of education in Croatia.
dents with pronunciation difficulties. These students
made considerable efforts to pronounce each word cor-
rectly to ensure that the speech-to-text system could Acknowledgement
recognize it. This aligns with the findings of (Mitra
et al., 2003), who reported that automated instructional This work has been fully supported by the Full STEAM
approaches could improve English pronunciation. This Ahead! project which was funded under the call
suggests that the gamified cognitive agents could po- "Strengthening STEM skills in primary schools and
tentially serve as a tool for improving pronunciation development of regional science centers for primary
and language skills among students. education in the STEM field" (cro. Jačanje STEM
vještina u osnovnim školama i razvoj regionalnih
znanstvenih centara za osnovnoškolski odgoj i obra-
5 Conclusion and Future Research zovanje u STEM području) from the EEA Financial
Mechanism and the Norwegian Financial Mechanism
This paper presented a case study on the develop- for the period from 2014 to 2021.
ment and implementation of gamified cognitive agents We are very grateful to all students from the 2nd Pri-
for primary school students in Croatia. The initiative mary School Varaždin, their teachers (Natalija Lončar,
was part of the "Full STEAM Ahead!" project, which prof., Martina Dugonjić, prof., Goran Kreminski, prof.
aims to enhance the STEAM competencies of students and Željka Kopić, prof.) which participated in the cre-
across all primary educational levels through the inte- ation of the cognitive agents. Also, we would like to
gration of advanced technology and innovative teach- thank the members of the Artificial Intelligence Labo-
ing methods. The cognitive agents were developed us- ratory from the Faculty of Organization and Informat-
ing a design science approach and were tested in spe- ics (Bogdan Okreša Ðurić, PhD and Tomislav Peharda,
cial class sessions where students interacted with the mag.inf.) which helped in the organization of the spe-
agents and provided feedback. cial sessions.
The most significant finding from these sessions was
the increased level of student engagement, particularly
among lower-grade students and those with learning References
disabilities. This suggests that the gamified cognitive
agents were successful in making the learning process Azasoo, J., & Boateng, K. (2015). A retrofit design sci-
more engaging and interactive. Moreover, the students ence methodology for smart metering design
demonstrated a strong sense of satisfaction and excite- in developing countries. https : / / doi . org / 10 .
ment due to their involvement in the development pro- 1109/ICCSA.2015.23
cess, and they were proactive in offering their feed- Baylor, A. (1999). Intelligent agents as cognitive tools
back. An intriguing observation was made with stu- for education. Educational technology, 36–
dents who had difficulties with pronunciation. These 40.
students put in substantial effort to articulate each word Foukalas, F. (2020). Cognitive iot platform for fog
accurately so that the speech-to-text system could cor- computing industrial applications. Computers
rectly interpret it. This implies that the gamified cog- & Electrical Engineering, 87, 106770.
nitive agents could be used as an effective tool for en- Ghazali, N. D. S. M., & Saad, A. F. (2022). Online stu-
hancing pronunciation and language skills among stu- dent performance system integrating multidi-
dents. mensional data visualization and chatbot for
Looking ahead, our future research is aimed towards primary school. International Journal of Arti-
creating the next five iterations of gamified cognitive ficial Intelligence, 9(2), 63–73.
agents as part of the "Full STEAM Ahead!" project. Johannesson, P., & Perjons, E. (2014). An introduction
These agents will be developed based on the feed- to design science (Vol. 10). Springer.
back and insights gained from the initial implemen-
Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, smart mobility. Transportation Research Pro-
K. (2023). Interacting with educational chat- cedia, 64, 199–204.
bots: A systematic review. Education and In- Schatten, M., Okreša Ðurić, B., & Peharda, T.
formation Technologies, 28(1), 973–1018. (2021). A cognitive agent for university stu-
Lee, I. (2010). Encyclopedia of e-business development dent support. 2021 IEEE Technology & En-
and management in the global economy. IGI gineering Management Conference-Europe
Global. (TEMSCON-EUR), 1–6.
Mitola, J. (2002). Cognitive radio. an integrated Schatten, M., Okreša Ðurić, B., & Protrka, R. (2021).
agent architecture for software defined radio. Conceptual architecture of a cognitive agent
(Doctoral dissertation). Kungliga Tekniska for telemedicine based on gamification. Cen-
Hogskolan (Sweden). tral European Conference on Information and
Mitra, S., Tooley, J., Inamdar, P., & Dixon, P. (2003). Intelligent Systems, 3–10.
Improving english pronunciation: An au- Sikström, P., Valentini, C., Sivunen, A., & Kärkkäinen,
tomated instructional approach. Information T. (2022). How pedagogical agents communi-
Technologies & International Development, cate with students: A two-phase systematic re-
1(1), pp–75. view. Computers & Education, 188, 104564.
Othman, K. (2023). Towards implementing ai mobile Šokec, T. (2019). Modeliranje kontekstualno svjesnog
application chatbots for efl learners at primary agenta za razgovor na hrvatskom jeziku uz
schools in saudi arabia. Journal of Namib- pomoć konačnog automata i strojnog učenja
ian Studies: History Politics Culture, 33, 271– [Mentor: Markus Schatten].
287. Suganuma, S., Sakamoto, D., & Shimoyama, H.
Peffers, K., Tuunanen, T., Rothenberger, M. A., & (2018). An embodied conversational agent for
Chatterjee, S. (2007). A design science re- unguided internet-based cognitive behavior
search methodology for information systems therapy in preventative mental health: Feasi-
research. Journal of management information bility and acceptability pilot trial. JMIR men-
systems, 24(3), 45–77. tal health, 5(3), e10454.
Putjorn, P. (2022). Designing augmented learning plat- Van Dang, C., Tran, T. T., Gil, K.-J., Shin, Y.-B.,
form with iot and chatbot-based learning tech- Choi, J.-W., Park, G.-S., & Kim, J.-W. (2017).
nology for primary school students in north- Application of soar cognitive agent based
ern thailand. 2022 6th International Con- on utilitarian ethics theory for home service
ference on Information Technology (InCIT), robots. 2017 14th International Conference
275–279. on Ubiquitous Robots and Ambient Intelli-
Schatten, M., Ðurić, B. O., Peharda, T., & Tomičić, I. gence (URAI), 155–158.
(2022). A cognitive agent’s infrastructure for