Journal of

Clinical Medicine

Review
Current Cognition Tests, Potential Virtual Reality
Applications, and Serious Games in Cognitive
Assessment and Non-Pharmacological Therapy for
Neurocognitive Disorders
Roger Jin † , Alexander Pilozzi † and Xudong Huang *
Department of Psychiatry, Massachusetts General Hospital and Harvard Medical School,
Charlestown, MA 02129, USA; [email protected] (R.J.); [email protected] (A.P.)
* Correspondence: [email protected]; Tel.: +1-617-724-9778; Fax: +1-617-726-4078
† These authors contributed equally to this work.




Received: 13 August 2020; Accepted: 10 October 2020; Published: 13 October 2020


Abstract: As the global population ages, the incidence of major neurocognitive disorders (major
NCDs), such as the most common geriatric major NCD, Alzheimer’s disease (AD), has grown. Thus,
the need for more definitive cognitive assessment or even effective non-pharmacological intervention
for age-related NCDs is becoming more and more pressing given that no definitive diagnostics or
efficacious therapeutics are currently unavailable for them. We evaluate the current state of the art
of cognitive assessment for major NCDs, and then briefly glance ahead at potential application of
virtual reality (VR) technologies in major NCD assessment and in cognition training of visuospatial
reasoning in a 3D environment, as well as in the alleviation of depression and other symptoms of
cognitive disorders. We believe that VR-based technologies have tremendous potentials in cognitive
assessment and non-pharmacological therapy for major NCDs.

Keywords: aging; major neurocognitive disorder; Alzheimer’s disease; cognitive assessment;

mild neurocognitive disorder; mini-mental state examination; virtual reality; augmented reality;
mixed reality

1. Introduction
The present review seeks to define and describe some common tests for screening mild
neurocognitive disorders (mild NCDs) and major NCDs. Tests were selected based on their use
and the areas of cognition they cover. Furthermore, the review surveys the potential for virtual
reality and related technologies in major NCD screening, as well as for the amelioration of cognitive
deficiencies in cases of cognitive impairment and in the general older population.

1.1. Normal Aging

In a review on diagnosis of major NCDs, it is natural to first consider the state of normalcy
for comparison. Cognitive decline that occurs with normal aging is well characterized. Age brings
defects in many areas of cognition. It appears that while “crystallized intelligence”—cognition
involving learned or practiced information or procedures—is resistant to decline with age, “fluid
intelligence”—cognition involving reasoning with unfamiliar ideas or information—tends to become
impaired with age. Briefly, we consider some facets of fluid intelligence that are affected. One
afflicted area of cognition is the encoding of knowledge into memory. A 2003 cross-sectional study
by Haaland et al. examining recall and recognition ability in the context of logical memory and

J. Clin. Med. 2020, 9, 3287; doi:10.3390/jcm9103287 www.mdpi.com/journal/jcm

J. Clin. Med. 2020, 9, 3287 2 of 20

visual reproduction suggested that the largest factor contributing to learning deficits in aging was
the encoding of information into memory, rather than retrieval [1].
Another significant facet of cognition is selective attention—the ability to both focus on a point
of interest as well as to filter out irrelevant information. Using the reading-with-distraction task as
a metric, in which subjects are instructed to read several passages while in the presence of external
disturbances, Darowski et al. found in a cross-sectional study on patients aged 18–87 years that it
appears that selective attention declines with age [2].
A third area of cognition is logical and spatial reasoning. A cross-sectional study on 120 males of
ages 20–79 conducted by Salthouse et al. found that both verbal and spatial reasoning performance
declines with age [3].
It is worth noting, however, that while cross-sectional studies of cognition across many ages can
obtain results quickly, the groups were raised and educated in different eras, an unfortunate reality
that may greatly influence test performance. Indeed, a Brazilian study published in 2015 found that,
among 14,594 persons aged 35–74 years, participants’ education level had a higher influence over
cognitive test scores than did their age [4]. Furthermore, when age-related changes are measured
through longitudinal and cross-sectional studies, results are often different; cross-sectional analyses are
impacted by differences between study cohorts, while longitudinal analyses introduce factors such as
experience with the testing protocol, thus confounding results [5].

1.2. Mild Neurocognitive Disorder

Although cognitive abilities decline with normal aging, slightly accelerated states of cognitive
decline exist and are given the classification of mild cognitive disorders (mild NCDs) as an intermediate
state between normal aging and major NCDs. A primary concern of the prognosis of a mild NCD
patient is progression into a major NCD, though the etiology of mild NCDs is highly heterogenous,
and not all individuals exhibiting mild NCDs will develop major NCDs; many of those with mild
NCDs will even revert to cognitive normalcy [6,7]. However, individuals with mild NCDs have
a significantly higher risk of developing major NCDs [8,9], particularly in cases of the amnestic
mild NCD subtypes [10]. In practice, the feature that distinguishes mild NCDs from major NCDs
is the interference of cognitive decline with the performance of daily activities, such as cooking or
self-hygiene, or social functions [11].

1.3. Alzheimer’s Disease

Alzheimer’s disease (AD), a degenerative neurological disorder characterized by deficits in
memory and spatial reasoning, is the most common form of major NCD and, in most cases, does
not have a well-established underlying cause or treatment. In addition to cognitive deficits, AD is
characterized by intrasynaptic β-amyloid plaques and intraneuronal τ-protein neurofibrillary tangles,
although it is unknown whether either of these patent markers are causal in the disease etiology [12].
AD is also characterized by degeneration of cholinergic neurons on the basal forebrain. AD most
commonly affects patients over 65 years of age, although familial AD can begin affecting patients well
below the age of 65. Around 10% of those age 65 and older in the US have a major NCD in the form of
AD, and the frequency increases with age; 43% of those over the age of 85 have the disorder [13]. A
single definite cause has not been identified, though there is association with certain genetic mutations
and other abnormalities, such as trisomy 21 [14–16].
Because AD has well-studied physical manifestations in the brain, risk for AD can be assessed
decades before the potential onset of the disease through advances in neuroimaging. This implies that
unlike normal aging or mild NCDs, which currently must be diagnosed through cognitive assessments,
the AD state has the potential of a reference-standard disease state at autopsy, when the brain can
be sectioned and examined for amyloid deposits and neurofibrillary tangles. However, because
longitudinal studies are expensive and slow, magnetic resonance imaging (MRI) and other imaging
data must usually suffice as a reference standard. This also implies that for patients who have already
J. Clin. Med. 2020, 9, 3287 3 of 20

been diagnosed with preclinical signs of AD, the conversion from mild NCD to AD is a bit contrived
because diagnostic criteria for AD can be discerned decades before cognitive symptoms appear [17].
Thus, in some cases, mild NCDs are an intermediate diagnosis as the patient progresses into AD or
major NCDs more generally.

2. Motivation for Cognitive Assessment

Uncertain boundaries between what is currently considered normal age-related decline and
noteworthy cognitive impairment, alongside a lack of reliable biomarkers for major-NCD-causing
diseases, make the process of diagnosis somewhat difficult. Though MRIs and other methods can
identify notable aberrations in brain health, they are neither easy nor foolproof to administer. Cognitive
assessments fill the gap, providing a relatively easy, non-invasive means of assessing one’s cognitive
status. In practice, we find that the current amount of diagnostic information garnered from MRI data
is not a substitute for that which can be learned from cognitive assessment. Lebedeva et al. found
that adding Mini-Mental State Examination (MMSE) input to a random-forest classifier improved test
accuracy, sensitivity, and specificity compared to using MRI data alone [18]. Major NCDs are also not
homogenous conditions. Though AD is the most common cause of major NCDs, it is not the only
cause [13,19]. Though AD accounts for approximately 60–80% of major NCD cases, other forms of
major NCDs, such as Vascular, Parkinson-related, and frontotemporal major NCDs are prevalent [19].
Assessments that target the cognitive symptoms of major NCDs are useful for screening patients
for further, more in-depth analysis due to the lack of a catch-all physiological marker for all major
and minor NCD variants. Indeed, the current Diagnostic and Statistical Manual of Mental Disorders
(DSM)-5 criteria define minor NCDs as the presence of a marked decrease in cognitive ability, as noted
by an informant or a cognitive test that does not yet impede the subject’s day-to-day life and that is not
explained by delirium or a mental disorder [20].

2.1. Definition of an Effective Cognitive Assessment

Before considering the particulars of any one cognitive assessment, it is useful to list characteristics
that are important to the development of cognitive assessments. We segregate these properties into
practical needs, which deal with the details of administration, and diagnostic needs, which deal with
the ability to discern the subject’s cognitive status.

2.1.1. Practical Needs

• The test should be brief. Patients with major NCDs often suffer from deficiencies in attention;
thus, a long test may not be suitable for such patients. A brief test is also more convenient for
the clinician to administer, and more manageable in terms of health-care capacity.
• The questions of the test should not lead the patient to experience emotional distress.
• The test should ideally avoid involvement of the patient’s family and collaterals.
• The test should be easy to administer, requiring minimal training to give and score.
• The test should be cheap and minimalistic in terms of materials required.

2.1.2. Diagnostic Needs

• The test should have high sensitivity and specificity with respect to distinguishing between
the normal and diseased states.
• High sensitivity/specificity with respect to change or progression of major NCDs: This implies
that the test should be repeatable. Patients should not be able to easily memorize the answers
to the test questions. This also implies that the maximum score on the test should be high, and
the mean score of those who test positive should be around 50% of the maximum possible to
facilitate separation of results.
• The test should cover various areas of cognitive function.
J. Clin. Med. 2020, 9, 3287 4 of 20

• The test results should be consistent and independent of the administrator and rater.
• The test should have minimal ceiling and floor effects. That is, the scores should not be clustered
around the minimum or maximum possible scores. In terms of test design, the questions on
the test should span a large range of difficulty.

3. Common Cognitive Assessments

3.1. Mini-Mental State Examination

3.1.1. Background
The Mini-Mental State Examination (MMSE) is currently the most popular cognitive assessment.
It consists of 11 questions totaling 30 points used to categorize the degree of a subject’s cognitive
impairment from cognitive normalcy to severe impairment. The questions assess the cognitive areas
of orientation, registration, attention, calculation, recall, and language [21]. The cutoffs for cognitive
normalcy are generally varied based on education level, as the standard cutoff score of around 23
must be raised to achieve similar sensitivity/specificity when testing highly educated subjects [22].
Many variants of this test have arisen, some adapted to patients with special needs. Some variants
include standardized MMSE, a more scripted version of the MMSE, the modified MMSE (3MS),
which broadens scoring to 100 points and adds a few questions to test more areas of cognition, and
hearing-/vision-impaired MMSE [23].

3.1.2. Advantages
Perhaps the largest advantage of the MMSE is its popularity. Because it is the most popular
cognitive assessment, data from a large patient pool are available for analysis for calibration of test
scoring and comparison between patients. Another advantage of the MMSE is its simplicity. The MMSE
requires minimal training to administer and score, and can be completed in under 10 min. Despite its
simplicity, the MMSE has relatively high accuracy and specificity with respect to identifying patients
with major NCDs. A review of data including MMSE scores from 1141 persons who had 16+ years
of education found that the MMSE had an accuracy of 89%, sensitivity of 66%, and specificity of
99% when used to distinguish between educated patients who were either normal or had relatively
severe major NCDs [22]. In the same vein, despite its simplicity, the MMSE targets a broad range of
cognitive functions, with each area encompassing at least three points of the test. This allows for both
a fairly broad assessment of a patient’s cognition as well as an initial screen for the areas of cognition
in which the patient suffers the greatest deficit. Most impressively, perhaps, scores on the MMSE were
found to be correlated with morphological attributes of the brain, especially of the hippocampus [24],
notably involved in episodic and spatial memory [25], as well as the amygdala, cingulate gyrus,
and parahippocampal gyrus [24], meaning that the MMSE yields similar information to much more
expensive tests.

3.1.3. Disadvantages
Currently, one disadvantage of the MMSE is its associated intellectual property issues. Although
the material cost of administration is, in theory, negligible because the MMSE is patented, clinicians must
pay royalties to MiniMental, the current patent holders, for every instance of MMSE administration,
at least for the official, up-to-date version. This inconvenience has served as an impetus for
the development of royalty-free cognitive assessments [21]. Another disadvantage of the MMSE is that
while it focuses on several different areas of cognition, it is limited in its assessment of other important
aspects, such as visuospatial reasoning, for which there is only a single question. This is a critical
shortcoming of the test because a deficit in visuospatial navigation is a classic symptom of AD, as
well as amnestic mild NCDs [26]. The test is also heavily language-based, which can confound results
if the subject’s language ability and level of literacy are low [27–30]. Furthermore, the assessment
J. Clin. Med. 2020, 9, 3287 5 of 20

has a relatively low sensitivity, necessitating secondary assessment for patients who are able to attain
relatively high scores on the MMSE and yet exhibit symptoms of major NCDs [22]. In addition
to the low sensitivity, the test is limited in its ability to predict future decline; however, analytical
tools, such as COmputational Model to Predict the development of Alzheimer’s diSease Spectrum
(COMPASS), are being developed to remedy this shortcoming [31].

3.2. Clock-Drawing Test

3.2.1. Background
The clock-drawing test is one of the more conceptually simple of the cognitive tests in common
use. Though there are a variety of scoring systems that have been applied to the test, they generally
assign points to different elements of the clock, such as its shape, and how the numbers and clock
hands are positioned [32]. The patient is assigned the task of drawing an analog clockface at a certain
time, usually 11:10. This particular time is chosen because it forces the use of both visual fields as well
as the inhibition of “frontal pull” of the number 10. A common mistake in major NCD patients is
to draw the minute-hand pointing to the number 10 on the clockface rather than the number 2 [33].
The resistance of this allure is a sign of mental flexibility. There are various rubrics for scoring the clock,
but the classic one scores on a 10-point scale and focuses on visuospatial ability [34].

3.2.2. Advantages
Perhaps the main advantage of the clock-drawing test is its simplicity. The test can be administered
with minimal training in under two minutes. Another significant advantage of the test is its ability to
test multiple areas of cognition, especially visuospatial reasoning. The test assesses praxis through
the act of drawing fine details. The test assesses calculation through the need to position the minute
hand at the correct time. The test assesses visuospatial reasoning in the relative positioning and sizing
of the numbers of the clock face.

3.2.3. Disadvantages
The main disadvantage of the clock-drawing test is inconsistency of scoring. Because the prompt is
so open-ended, the scoring may vary from rater to rater. In addition, because the test is so simple, the test
suffers from floor and ceiling effects; patients with major NCDs may be unable to draw even the circle
of the clock face, while some patients with only mild NCDs may be able to draw the clock perfectly.
These two groups of patients will require secondary assessment to separate. Next, the simplicity
of the clock-drawing test means it excludes some significant areas of cognition, particularly those
related to language. These three disadvantages suggest that the clock-drawing test may work best as
a component of a larger test, so as to minimize the effect of variation in its scoring on the total score as
well as to mitigate floor and ceiling effects and broaden the areas of cognition tested. Lastly, this test
may be biased against those who have not received formal education. For example, a Brazilian study
of confounding factors that may influence a subject’s performance on the clock-drawing test found
a correlation between education level and performance on the clock-drawing test [35].

3.3. Addenbrooke’s Cognitive Examinations

3.3.1. Background
Addenbrooke’s Cognitive Examination (ACE) was developed as an extension of the MMSE to
incorporate questions that tested areas of cognition not covered by the MMSE, such as visuospatial
reasoning. The MMSE is incorporated verbatim into the assessment so that a 30-point MMSE sub-score
might be generated. The ACE has been the foundational work for many different variations—including
the ACE-Revised (ACE-R), developed to have clearly defined subdomain scores, and the ACE-III,
developed to reinforce certain areas of the ACE-R, such as repetition and comprehension, and replaces
J. Clin. Med. 2020, 9, 3287 6 of 20

the MMSE portion of the ACE and ACE-R [36]. The typical cut-off for major NCDs is 82–88 out of 100
points of the composite score for the ACE-III [36].

3.3.2. Advantages
One of the advantages of the ACE is its ability to test visuospatial capability in both 2D and 3D,
incorporating such challenges as reassembling fragmented letters and spatial reasoning with a wire
Necker cube, while preserving the plethora of cognitive areas tested by the MMSE. Because it fills
several holes left by the MMSE, the ACE outperforms the MMSE in diagnostic ability. The creators
of a 30-point simplified version of ACE, known as the mini-ACE (M-ACE), compared their test to
the MMSE, and found that the M-ACE was both more sensitive and less prone to ceiling effects than
the MMSE. It was found that patients scoring below 21/30 points on the M-ACE almost certainly had
a major NCD [37]. The ACE is also less prone to floor effects. A 2000 study validating ACE’s ability to
differentiate mild or major NCDs, AD, and frontotemporal major NCDs found that the test was able
to distinguish between different stages of Alzheimer’s disease [38]. Another advantage of the ACE
is that administration is easy and requires minimal training. Lastly, it appears that the test remains
effective when translated into other languages. For example, a Japanese version of the ACE achieved
a sensitivity of 0.889 and a specificity of 0.987 using a threshold of 74/100 [39].

3.3.3. Disadvantages
Aside from the ACE-III, which is independent of the MMSE, the versions of the ACE that contain
questions from the MMSE are subject to the same patent issues. In addition, since the ACE was
developed in the United Kingdom (UK), some of the questions are based on knowledge of the culture
and politics of the UK, making it biased towards those with cultural knowledge of the UK or other
European countries with similar political systems. For example, some of the questions ask for the name
of the prime minister or leader of the opposing political party [40].

3.4. General Practitioner Assessment of Cognition

3.4.1. Background
The General Practitioner Assessment of Cognition (GPCOG) was developed for use by doctors
in a primary care setting. The test format consists of two interviews—one with the patient and one
with an informant, a close friend or relative of the patient. The patient interview, which consists
of nine questions, can alternatively be completed as a written or online form [41]. The patient
interview incorporates the clock-drawing test as well as recollection of an address and a recent event
in the news. The informant interview consists of six questions surrounding the patient’s ability to
perform day-to-day tasks, such as finding objects, speaking, and managing finances [41].

3.4.2. Advantages
In just nine questions directed to the patient, the GPCOG tests the areas of orientation, visual
spatial abilities, executive function, retrieval of recent information, and delayed recall [41]. Despite
being shorter than the MMSE, taking about four minutes for the patient interview and two minutes for
the informant interview, the test has comparable sensitivity and specificity [41]. The written and online
options for the patient assessment also free the physician to perform other duties during this time. In
addition, while the MMSE is patented, the GPCOG is free to use. Lastly, it appears that the GPCOG is
agnostic to cultural background and education level [42].

3.4.3. Disadvantages
One disadvantage of the test is that it is reliant on the presence of an informant. Additionally,
while the patient’s age was not a factor in the informant-based section of the GPCOG, the age was
J. Clin. Med. 2020, 9, 3287 7 of 20

found to be a significant (corr = −0.187, p < 0.01) predictor of their score on the cognitive assessment
portion [43].

3.5. Montreal Cognitive Assessment

3.5.1. Background
The Montreal cognitive assessment (MoCA) is a screening tool developed to be an alternative to
the MMSE that is better suited to the detection of mild NCDs, with the initial test achieving better
sensitivity for both mild NCDs and AD, albeit with lesser specificity [44]; with tuning of the cutoff
scores, however, the MOCA can achieve better sensitivity and specificity in detecting mild NCDs
than the MMSE [45]. It can be used in conjunction with the MMSE to assess persons who fall within
the normal cognitive range of the MMSE score, but display other signs of cognitive impairment, such
as difficulty completing activities of daily living, or used to test those whose disorder is mild enough
that their daily lives are unaffected. The test is paper-based and free to use. It incorporates questions
and other cognitive tests that address a wide variety of common domains. A modified trail-making
test assesses visuomotor, visuoperceptual, and task-switching ability, and the clock-drawing test is
included as well. Other assessments in the MoCA examine language ability, conceptual thinking,
recall/memory, and orientation [46].

3.5.2. Advantages
The test was designed to take around 10 min to administer, and it is short and easily administered
in a standard clinical setting, though it also assesses many important areas of cognition. The test is
freely distributed as well [46]. The MoCA has also been found to indicate one’s cognitive reserve with
greater sensitivity than the MMSE, which may explain, in part, the MoCA’s heightened sensitivity to
early-stage AD [47]; individuals with greater cognitive reserves have been found to be at lower risk of
developing major NCDs [48].

3.5.3. Disadvantages
A notable disadvantage of the MoCA relates to sensory impairment. In a study comparing
the MoCA scores of 301 older persons, half of whom had no sensory impairments, while 38%, 5%, and
7% were hearing, vision, or dual-sense impaired, respectively, it was found that, despite modifying
the scores of the sensory-impaired persons, those who had no impairments were more likely to pass [49].
The test also needs to be corrected for age and education level; although, as with other tests, the total
score is taken and compared to a cutoff, there were certain questions with substantially higher failure
rates among 2653 older persons with cardiovascular disease [50].

3.6. Summary of Advantages/Disadvantages

Some of the key advantages and disadvantages of the tests we have discussed are summarized in
the Table 1 below.
J. Clin. Med. 2020, 9, 3287 8 of 20

Table 1. Summary of advantages and disadvantages of cognitive tests.

Test Name Advantages Disadvantages

Popular, so there is a large pool of Intellectual property issues create
available testing results. financial barriers for clinicians.
Simple and relatively short. Limited testing of visuospatial
Mini-Mental State
High accuracy and specificity, with reasoning.
Examination
moderate sensitivity [22]. Heavily language-based, making
Assesses a large range of cognitive results dependent on the subject’s
functions. overall language ability [27–30].
Open-ended, making scoring
inconsistent.
Simple and easy to administer.
Can be inaccurate on its own.
Clock-Drawing Test Short time to administer.
Does not test language.
Assesses visuospatial reasoning
Results correlate with education
level [35].
Easy to administer.
As it contains portions of
Tests visuospatial capability.
the MMSE, there are issues related
Addenbrooke’s More sensitive than MMSE, with
to intellectual property.
Cognitive Examinations lower impact of ceiling and floor
Some questions assume patient
effects [37].
knowledge of certain subjects.
Effective even when translated [39].
Short, but tests many different areas of
cognition [41].
Requires input from an informant.
General Practitioner Can be taken in written and online
Apparent correlation between
Assessment of Cognition formats.
scores and age.
Agnostic to cultural background and
education level [42].
Short, but tests many different areas of
cognition.
Sensory impairments can impact
Includes multiple tests of visuospatial
Montreal Cognitive score [49].
capacity.
Assessment Failure rates of questions vary
Freely available and easy to
considerably [50].
administer.
Short time to administer

4. Applications of Virtual Reality in Cognitive Disorder Testing and Therapy

4.1. Virtual Reality Diagnostics

4.1.1. Background
As a concept, virtual reality (VR) involves immersing an individual in a simulated environment.
Users are able to interact with this environment through some set of physical sensors, and typically
perceive that environment through a headset that provides visuals and audio for the environment; in
some cases, tactile feedback can be transmitted to the user through other systems, such as controllers [51].
The unique features of VR systems have generated considerable interest in the medical/psychiatric
fields; within the scope of major NCDs, the allure from VR comes from its ability to assess and
improve spatial reasoning, along with other cognitive abilities, depending on the focus and design of
the program [51,52].
The deterioration of visuospatial reasoning is a marker of Alzheimer’s and even amnestic mild
NCDs [26]. One flaw in the current state-of-the-art diagnostic examinations for major NCDs is that, while
it is possible to test spatial cognition with simple visuospatial tests, including some paper-and-pencil
methods, these tests tend to correlate only partially, if at all, with spatial navigation in large-scale
environments [53]. One potential method for incorporating large-scale spatial reasoning tasks in 3D
is setting the test environment in virtual reality. The navigation of virtual reality environments has
J. Clin. Med. 2020, 9, 3287 9 of 20

been found to correlate with their real-world counterparts, with the ability to safely navigate them in
correlation with MMSE score [54].

4.1.2. Egocentric and Allocentric Navigation

There are two broad categories of navigation strategies—egocentric, in which the individual
navigates by considering objects relative to his or her location, and allocentric, in which the individual
navigates by considering the positioning of objects relative to each other. Weniger et al. characterized
deficits in egocentric navigation in Alzheimer’s and mild NCDs by asking patients to memorize a VR
park environment. The authors furthermore characterized deficits in allocentric navigation by asking
patients to learn a VR maze. The study failed to differentiate those with mild NCDs that would
progress into major NCDs from those whose mild NCDs did not convert, but mild NCD individuals
performed worse than controls on both egocentric and allocentric navigation tasks [55]. Serino et al.
similarly investigated the allocentric and egocentric spatial abilities by having amnestic mild NCD,
AD, and control participants enter a virtual room containing certain objects, including a target object.
In the second phase, one of two tests was conducted. In the first, patients were presented with
a top-down representation of the room without the target object, but with other objects present. In
the second, patients entered a second VR room that was empty, with only an arrow for directional
orientation; patients entered this room from a different side from the original. Significant differences
were found between AD patients and controls, though these differences were dependent on both
the task and the direction from which participants entered the virtual room [56]. Cushman et al. tested
the ability of Alzheimer’s, mild NCD, and normal older individuals to navigate virtual and real-world
environments, testing navigation and recall abilities. Both the real-world and virtual tests were effective
in differentiating between groups, and, notably, there was no significant difference between real-world
and VR performances [57].

4.1.3. Navigational Memory

Lee et al. created a non-immersive virtual version of the classic radial arm maze (RAM) to test
spatial working memory. The maze consists of six arms, one of which contains a goal, attached to
a central platform. In order to visit an arm after exiting another arm, one must first visit the central
platform. The primary challenge of the maze is to remember which arms have been visited and which
have not. The maze can be revisited, and the mazes are designed so that some of the arms will never
have treasure. The act of learning and recalling which arms will never hold treasure is termed “spatial
reference memory”. Lee et al. found that in AD patients, both spatial reference and working memory
were impaired, while in mild NCD patients, only spatial reference memory was impaired [58]. Plancher
et al. investigated the episodic memories of amnestic mild NCD patients, AD patients, and controls by
challenging them to both active and passive vehicular exploration of a virtual reality city environment.
Participants were tasked with recalling details about the route, including allocentric and egocentric
navigational recollection (e.g., remembering the position/sequence of elements). The cognitive status
of the participant was a significant indicator of impaired performance, though this effect was much
greater for AD patients than those with amnestic mild NCDs [59].

4.1.4. Activities of Daily Living (ADL)

One’s ability to complete tasks that are part of daily life is integral to the definition of major
NCDs, which are classified as cognitive impairments that make completing activities of daily living
difficult [60]. However, using this criterion in diagnosis is difficult; though a wide variety of surveys
and questionnaires have been developed, they rely on interviews with a patient and/or caregiver,
which can be subjective and influenced by the subject’s interpretation of the specific wording of
a question; additionally, reliable administration is largely limited to outpatient settings [61–63]. VR
notably allows for the recreation of tasks a participant might reasonably encounter, so that participants
can be scored on their ability to complete them. Tarnanas et al. used a non-traditional virtual reality
J. Clin. Med. 2020, 9, 3287 10 of 20

setup featuring a treadmill and curved screen to simulate a “day out” task to screen for early major
NCDs; participants were tasked with navigating fire-building scenarios of varying degrees of difficulty.
Individuals’ performance scores were found to be a better predictor of mild NCD to AD conversion
than the Bristol ADL and Blessed ADL tests [64]. Dulua et al. created a VR experience called “A Day to
Remember”, which featured daily-life activities as well as gamified cognitive tests. Most participants
found the VR game to be easy to use, and noted that the experience was less stressful compared to
the direct screening tests. However, while the test was functionally correlated to the paper MMSE,
its precision was somewhat lacking, and a participant’s difficulty controlling the new technology
was suggested to be a confounding factor [65]. Zygouris et al. created a low-immersion virtual
supermarket, which measured aspects of task completion, such as the time to complete a shopping
list and the accuracy of the items bought, money paid, and types of items bought. All participants
were able to complete the exercise and were able to self-administer the test at home via a provided
tablet [66,67]. An initial study used the aforementioned performance metrics in order to differentiate 34
mild NCD patients from a sample of 55. A decision tree was constructed based on the mistakes made
and certain age/performance thresholds. A sensitivity of 82% and a specificity of 95% was achieved.
In a secondary study using the same method but applying data on six healthy and six mild NCD
individuals to a Naïve-Bayes classifier using the data from 20 of the participants’ trials, the classifier
achieved a sensitivity of 94% and a specificity of 89% in identifying persons with mild NCD [67].

4.1.5. Advantages
The main advantage of VR-based cognitive assessments is the potential for testing visuospatial
reasoning in 3D, a key indicator of AD- and major-NCD-related cognitive decline that traditional
tests fail to fully account for [68]. In addition to this, using computer-based testing allows for
streamlined cloud-storage of data and greatly decreases variability in test administration; tests
can even be administered by the patients themselves if they have the right equipment [67]. As
the tests are easily administered, data from multiple trials can be gathered, improving the accuracy
and sensitivity/specificity of classifiers [66,67]. This also allows for more complex queries to be
readily addressed in a controlled environment that would not be possible or feasible to do in
a real environment, and the parameters of that environment can be readily changed as needed [69].
Additionally, the “gamification” of the testing task can counter the anxiety of standard testing, changing
the activity into a more relaxed, engaging experience [65].
Neuropsychological tests also have limited relation to an individual’s ability to successfully
complete activities of daily living, which is important for ascertaining the capabilities of a person
and whether they can live independently [70]. VR tools have been found to relate strongly to their
real-world counterparts, particularly in navigation [57], and can be used to test certain facets such as
episodic memory in a more realistic, utilitarian manner [59]. VR tests have superior ecological validity
compared to their pen-and-paper counterparts [64,65]. Such tests of instrumental activities of daily
living are invaluable in the diagnosis of pre-major NCD cases, and can help to predict the conversion
of mild NCDs into AD [64]. VR also aids in the development of dual task tests that assess both one’s
cognition and ability to maintain gait.

4.1.6. Disadvantages
One disadvantage of VR-based diagnosis is that there is a relatively higher fixed cost than other
tests for purchasing the required equipment. The practicality of VR-based tests and treatments
improves, however, as the cost of VR headsets and equipment decreases with time [71]. VR-based
tests also require more expertise to administer than current tests. In addition, one aspect of cognition
in which VR may be limited is the evaluation of social cues. Traditional cognitive tests involve
a conversation between an administrator and patient, an aspect that is lost in VR. Adaptations of
current cognitive assessment tools such as the MMSE to VR may yield results that are incompatible
with those of the original versions simply because of loss of direct verbal communication between
J. Clin. Med. 2020, 9, 3287 11 of 20

administrator and patient. In addition, some users of VR systems have problems with motion sickness,
which can hinder their performance and can make use of the VR system unpleasant [51]. Tests such as
the “Simulator Sickness Questionnaire” can allow for the identification of those who are susceptible to
experiencing discomfort from the VR environment in a way that may alter their results [72]; certain
features of the VR experience, such as many graphical frames per second and limitation of sharp
turns, can be tuned to mitigate discomfort [65]. General technophobia is also an issue that can impede
the use of VR and other computerized tests [64,65]. Some of these issues in accessibility, such as motion
sickness, appear to be alleviated in recent work, perhaps due to improvements in the technology [73].
As VR technology is made more advanced and becomes more commonplace, these disadvantages may
be mitigated further.

4.2. Virtual-Reality-Based Therapy

4.2.1. Background
There is currently no known pharmaceutical treatment or preventive intervention for major NCDs
or mild NCDs. Certain behaviors and lifestyle choices can reduce a person’s risk for developing
cognitive impairment, however, and there is evidence that AD is inversely correlated with mental
stimulation and education [74]. Thus, as the search for an effective drug treatment continues, it may be
worthwhile to invest in the development of mental exercises that may be able to slow the progression
of cognitive decline. The use of computer environments to benefit cognitive capacity has been found
to be useful, and has been employed in the context of mild NCDs. Drills for training certain aspects of
cognition, specially developed videogames, dubbed “serious games”, and virtual reality scenarios
fall under the umbrella of computerized cognitive training. Though not necessarily taking place in
a VR environment, such programs indicate that gamified exercises have considerable utility. Cognitive
benefits, alongside certain emotional and well-being improvements, have been observed following
computerized cognitive training sessions [75–77]. Both specially designed and commercially available
(e.g., Nintendo Wii) “exergames” have been found to improve balance in the older population [78,79].
Serious games such as “Kitchen and Cooking”, which involve users completing cognitively taxing
tasks or emulating real-life situations that are important to daily living, are generally well received by
the older population and have significant benefits to cognition and independence [80]. VR games are
especially promising in the training of 3D navigation. In addition, a major risk factor for development
of major NCDs is depression. One potential function of VR is the ability to immerse patients in alternate
worlds with the goal of improving mood.

4.2.2. Virtual Reality Exercise

Lee et al. designed a 12-week virtual reality exercise program for mild NCD patients. Patients
were brought to three gaming sessions per week, for a total of 36 sessions. During this time, balance,
depression, and quality of life were tracked in the patients, and patients demonstrated significant
improvement across all three metrics. A control group enrolled in traditional therapy sessions saw
no significant improvement [81]. Hwang et al. utilized a VR training program on 24 older persons
with mild NCD, finding that balance and cognitive test scores were improved relative to controls,
though details of the VR training program used are unclear [82]. Htut et al. found that a virtual reality
video-game-based exercise, which is less beneficial than standard physical activity in assessments
of physical ability but better than controls, can also simultaneously improve cognitive function [83].
Integration of virtual environments with existing exercise modalities, such as stationary cycling, has
been found to be appropriate for younger and older individuals alike, while being rated as enjoyable
by users, promoting long-term adherence to the exercise program [81,84,85].
J. Clin. Med. 2020, 9, 3287 12 of 20

4.2.3. Virtual Cognitive Exercises

As VR tests are effective in evaluating the navigational abilities of a patient, they may prove useful
for rehabilitating those abilities. Kober et al. applied a VR navigation simulation based on Graz, Austria.
Participants did not have mild NCDs or Alzheimer’s disease, but rather exhibited brain damage with
notable deficiencies in spatial orientation. Users’ general performance on spatial cognitive tests was
found to improve following the VR training program [86]. Regarding cognitive disorders, Man et al.
found that VR-based memory training exhibited greater improvements in memory compared to taking
part in therapist-led memory training [87]. However, Fasilis et al. found little significant improvement
in major NCD patients engaged in serious VR games, though in this study, the total time participants
spent with the game was limited: The training regimen was 10 h over a 4–5 week period [88].

4.2.4. Dual-Task Training

Dual-task training involves the performance of a cognitively demanding task while the subject is
walking. Such tasks are useful for the testing of gait control, the loss of which is typical of mild NCDs,
and especially in cases of Parkinson’s disease [89]. Liao et al. created a dual-task VR exercise program
incorporating both physical and cognitive training, and found that the program improved executive
function and gait performance comparable to a combined physical and cognitive training regimen, but
the dual-task performance was significantly improved in the VR group relative to the non-VR group.
The unique ability VR systems have in combining physical activity with an interactive environment
appears to be a significant boon to addressing deficiencies in dual-task performance [90].

4.2.5. Advantages
VR-based therapy allows patients to train cognitive function without supervision, and users can
potentially train in their own homes provided they have the right equipment [91]. In addition, they
give the potential for patients to practice daily tasks with minimal risk. Patient performance data on
tasks can be easily stored in the cloud for analysis. Furthermore, users of some VR training programs
describe them as “fun”, which can encourage adherence to a VR-based cognitive training program [82],
and VR exercise programs are more well received than standard physical activity regimens [83]. As
noted previously, virtual reality exercise allows for the easy integration of a cognitive component
into the exercise [92], which can provide tandem physical and cognitive benefits that perhaps take
advantage of the promotion of neuroplastic processes induced by exercise [92,93]. The feeling of
immersion that VR brings with it also seems to be an important factor in improving spatial abilities.
The effects of immersive (VR) and non-immersive versions of the game “Fruit Ninja” were studied
on 33 persons that averaged 62 years old over a four-week period. The VR exergaming participants
experienced improvements in inhibition and task switching that the non-VR group did not [94].

4.2.6. Disadvantages
Again, there is a high fixed cost for initial purchase of equipment. Although VR would potentially
enable patients to train their mental faculties independently of clinicians, in practice, perhaps the VR
equipment would be too expensive for most people to invest in at home; thus, the responsibility of
guiding the training still falls to the physician. In addition, perhaps VR is limited in its ability to train
social faculties. Furthermore, as mentioned previously, some individuals experience motion sickness
from virtual environments, and the discomfort may limit or prevent their use of therapeutic VR [51];
though the aforementioned studies note that participants largely tolerated and enjoyed VR systems,
sample sizes were relatively small; larger studies with more rigorously explained inclusion and
exclusion criteria are necessary for further analysis of the prevalence of tolerability issues. Furthermore,
serious games in general are considered to be most adapted to those with more mild NCDs, rather
than those with AD and major NCDs; those with major NCDs also require the assistance of a caregiver
more so than those with mild NCDs [95].
J. Clin. Med. 2020, 9, 3287 13 of 20

4.3. Augmented and Mixed Reality

Augmented reality (AR) differs from virtual reality in that it involves the insertion of virtual
elements into the real environment, rather than the creation of a fully virtual space. It typically
involves an indirect display of the real environment, with virtual information and features overlaid
onto this projection, thereby enhancing a user’s sense of reality. “True” AR is largely limited to
displays of information and non-interactive elements, with the addition of interactive, completely
virtual elements falling under the umbrella of mixed reality (MR). Mixed reality exists between VR
and AR on the spectrum of reality, with virtual, interactable elements introduced into a display of
the real world [96]. For both MR and AR applications, cameras and other tracking mechanisms are
necessary for recognizing what is in the user’s environment and how the user is interacting with it;
systems can be stationary or mobile, though the computing power of mobile systems can be limited.
AR applications in daily life are somewhat limited by the “fashion” and social acceptability of these
systems. The computing power necessary for complex interactions can necessitate a noticeably bulky
system, and interactions through speech and gestures can be somewhat disruptive or awkward to
perform in public [97]. The near ubiquity of mobile devices and the advent of cloud computing provide
new opportunities for readily available AR, and mobile augmented reality (MAR) systems have been
employed in a variety of applications, such as navigation, education, and entertainment [98].
AR systems have been shown to enhance learning, though studies showing this have largely
involved relatively young students rather than older and/or cognitively impaired individuals [99,100].
Mobile augmented reality systems using one’s phone have also been proposed for the gamification of
certain activities, such as maintaining hydration in the older population [101].

Augmented and Mixed Reality for AD and Major NCDs

There have been several AR applications developed with AD patients in mind. Rohrbach et al.
investigated an AR assistant for activities of daily living. The researchers utilized a Microsoft HoloLens
and their application, TherapyLens, and tested AD patients’ abilities to make a cup of tea. The program
featured holographic cues that indicated the steps required. Through voice commands, users could
advance through the steps. The ultimate utility of the program was unclear, as those with the AR
assistance did not have improved performance, and their time to complete for each task was actually
increased. Researchers stated that some participants had difficulty remembering how to command
the system, noting that the impaired ability to learn in patients of neurodegenerative disorders may
make such AR systems difficult to use. Though participants expressed interest in the AR system, some
noted that the system was uncomfortable, and others did not notice or remember the holograms [102].
Other systems, such as cARe, have been proposed, using AR projections to help instruct major NCD
patients on activities of daily living. cARe notably includes the ability to “time out”, which again
prompts the user and provides instruction for the current step, which can be helpful if the patient
becomes distracted or forgets their task. Praise-based incentives were also included to motivate
patients. Studies for cARe are still ongoing, though initial results suggest cARe is more helpful in
a cooking scenario than written instructions. However, as with TherapyLens, problems arose from
the voice communication, and the system did not account for patients altering the order of steps [103].
Aruanno et al. created a HoloLens-based MR cognitive trainer. Two prototypes of the trainer
were tested; one was gesture controlled, while the second, which was modified based on feedback
from the first prototype, was both voice and gesture controlled. Both featured various text-instructed
activities geared toward short-term and spatial memory training, involving the location of certain
objects in boxes projected throughout the user’s environment. Though controls were not included
in the study, participants notably found the trainer engaging, and the second prototype was easy
to use even for participants unfamiliar with technology. There was minimal discomfort reported;
the HoloLens itself is designed to reduce the motion-sickness and fatigue that are typical of VR
systems. Furthermore, by foregoing the full immersion of VR, MR allows the user to remain aware
of the real environment. Feedback was positive overall, and the study illuminates some concerns
J. Clin. Med. 2020, 9, 3287 14 of 20

that need to be addressed in the development of MR cognitive trainers, such as where and how to
place textual instructions, what feedback (sound, visual, etc.) to provide the user, and interaction
methods [104]. Park et al. observed the effects of an MR cognitive trainer using the Consortium
to Establish a Registry for Alzheimer’s Disease, ultimately finding the program to be a significant
improvement over conventional cognitive training in improving visuospatial working memory [105].

5. Conclusions
Although pharmaceutical treatment is not yet available for major NCDs, there is still utility in
the early diagnosis of the condition for both assisting patients in managing it, as well as for aiding in
further research that may produce an effective treatment. In addition, non-pharmacological therapy,
such as serious games and VR-based cognitive training, can serve as an effective means of improving
and preserving cognitive functions. Currently, early diagnosis can facilitate decision-making regarding
a patient’s current and future assistive care options. Thus, there is a need for an effective cognitive test
for major NCDs. Both the pragmatism and the scientific merit of a test must be taken into consideration.
The most convenient cognitive test is perhaps the clock-drawing test, though this is lacking in its
testing of language abilities. The test that currently has the greatest utility is perhaps the MMSE,
which, due to its popularity, has become the standard for use in cognitive studies. However, due to its
associated intellectual property issues, as well as its deficiencies in testing visuospatial capabilities,
perhaps a more extensive assessment such as the ACE should be considered to maximize accuracy,
sensitivity, and specificity. To resolve the intellectual property issues, perhaps a simple test with no
associated patent, such as the GPCOG, would be of use. However, to date, no test in common use is
able to assess visuospatial reasoning in 3D, a major deficit of amnestic mild NCDs and AD.
Thus, perhaps there is much potential for the incorporation of VR-based 3D environments into
the diagnosis of major NCDs, as well as in the training of mental faculties and alleviation of depression.
As the technology powering virtual, augmented, and mixed realities becomes more accessible, it
becomes more feasible to employ the technologies in both clinical and at-home settings. We propose
that VR spatial tests such as virtual mazes be developed and studied, as the importance of visuospatial
reasoning ability for distinguishing one’s major NCD state is apparent. Such VR programs could serve
both as tests and as cognitive trainers themselves. AR and MR approaches may be optimal for home
use, as they allow the user to maintain environmental awareness, while still allowing for a 3D spatial
component to games and activities. ADL tasks also present a test that is relatively easy to understand
and carry out and that can help determine the level of one’s cognitive impairment. The increased
engagement noted with AR and MR activities can incentivize mild and major NCD patients to perform
tasks that are vital to preserving their mental faculties.

Author Contributions: Conceptualization, X.H. and R.J.; formal analysis, R.J. and A.P.; investigation, R.J. and
A.P.; resources, X.H.; writing—original draft preparation, R.J. and A.P.; writing—review and editing, A.P. and
X.H.; supervision, X.H.; project administration, X.H.; funding acquisition, X.H. All authors have read and agreed
to the published version of the manuscript.
Funding: This work was partially supported by the NIH grant R01AG056614 (to X.H.).
Acknowledgments: The authors would like to thank the MIT externship program for allowing R.J. to be a student
intern at the Massachusetts General Hospital (MGH).
Conflicts of Interest: The authors declare that they have no competing interests.

References
1. Haaland, K.Y.; Price, L.; Larue, A. What does the WMS-III tell us about memory changes with normal aging?
J. Int. Neuropsychol. Soc. JINS 2003, 9, 89–96. [CrossRef] [PubMed]
2. Darowski, E.S.; Helder, E.; Zacks, R.T.; Hasher, L.; Hambrick, D.Z. Age-related differences in cognition:
The role of distraction control. Neuropsychology 2008, 22, 638–644. [CrossRef] [PubMed]
3. Salthouse, T.A.; Mitchell, D.R.; Skovronek, E.; Babcock, R.L. Effects of adult age and working memory on
reasoning and spatial abilities. J. Exp. Psychol. Learn. Mem. Cogn. 1989, 15, 507–516. [CrossRef] [PubMed]
J. Clin. Med. 2020, 9, 3287 15 of 20

4. de Azeredo Passos, V.M.; Giatti, L.; Bensenor, I.; Tiemeier, H.; Ikram, M.A.; de Figueiredo, R.C.; Chor, D.;
Schmidt, M.I.; Barreto, S.M. Education plays a greater role than age in cognitive test performance among
participants of the Brazilian Longitudinal Study of Adult Health (ELSA-Brasil). BMC Neurol. 2015, 15, 191.
[CrossRef] [PubMed]
5. Salthouse, T.A. Trajectories of normal cognitive aging. Psychol. Aging 2019, 34, 17–24. [CrossRef] [PubMed]
6. Larrieu, S.; Letenneur, L.; Orgogozo, J.M.; Fabrigoule, C.; Amieva, H.; Le Carret, N.; Barberger-Gateau, P.;
Dartigues, J.F. Incidence and outcome of mild cognitive impairment in a population-based prospective
cohort. Neurology 2002, 59, 1594–1599. [CrossRef] [PubMed]
7. Canevelli, M.; Grande, G.; Lacorte, E.; Quarchioni, E.; Cesari, M.; Mariani, C.; Bruno, G.; Vanacore, N.
Spontaneous reversion of mild cognitive impairment to normal cognition: A systematic review of literature
and meta-analysis. J. Am. Med. Dir. Assoc. 2016, 17, 943–948. [CrossRef] [PubMed]
8. Vega, J.N.; Newhouse, P.A. Mild cognitive impairment: Diagnosis, longitudinal course, and emerging
treatments. Curr. Psychiatry Rep. 2014, 16, 490. [CrossRef]
9. Mitchell, A.J.; Shiri-Feshki, M. Rate of progression of mild cognitive impairment to dementia–meta-analysis
of 41 robust inception cohort studies. Acta Psychiatr. Scand. 2009, 119, 252–265. [CrossRef]
10. Aerts, L.; Heffernan, M.; Kochan, N.A.; Crawford, J.D.; Draper, B.; Trollor, J.N.; Sachdev, P.S.; Brodaty, H.
Effects of MCI subtype and reversion on progression to dementia in a community sample. Neurology 2017,
88, 2225–2232. [CrossRef]
11. Hugo, J.; Ganguli, M. Dementia and cognitive impairment: Epidemiology, diagnosis, and treatment. Clin.
Geriatr. Med. 2014, 30, 421–442. [CrossRef] [PubMed]
12. Wirth, M.; Madison, C.M.; Rabinovici, G.D.; Oh, H.; Landau, S.M.; Jagust, W.J. Alzheimer’s Disease
Neurodegenerative Biomarkers Are Associated with Decreased Cognitive Function but Not-Amyloid in
Cognitively Normal Older Individuals. J. Neurosci. 2013, 33, 5553–5563. [CrossRef] [PubMed]
13. Alzheimer’s-Association. 2019 Alzheimer’s Disease facts and figures. Alzheimer’s Dement. 2019, 15, 321–387.
[CrossRef]
14. Karch, C.M.; Goate, A.M. Alzheimer’s disease risk genes and mechanisms of disease pathogenesis. Biol.
Psychiatry 2015, 77, 43–51. [CrossRef] [PubMed]
15. Reitz, C.; Mayeux, R. Alzheimer disease: Epidemiology, diagnostic criteria, risk factors and biomarkers.
Biochem. Pharmacol. 2014, 88, 640–651. [CrossRef]
16. Schupf, N.; Kapell, D.; Lee, J.H.; Ottman, R.; Mayeux, R. Increased risk of Alzheimer’s disease in mothers of
adults with Down’s syndrome. Lancet 1994, 344, 353–356. [CrossRef]
17. Sperling, R.A.; Aisen, P.S.; Beckett, L.A.; Bennett, D.A.; Craft, S.; Fagan, A.M.; Iwatsubo, T.; Jack, C.R.;
Kaye, J.; Montine, T.J.; et al. Toward defining the preclinical stages of Alzheimer’s disease: Recommendations
from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for
Alzheimer’s disease. Alzheimer’s Dement. J. Alzheimer’s Assoc. 2011, 7, 280–292. [CrossRef]
18. Lebedeva, A.K.; Westman, E.; Borza, T.; Beyer, M.K.; Engedal, K.; Aarsland, D.; Selbaek, G.; Haberg, A.K.
MRI-Based Classification Models in Prediction of Mild Cognitive Impairment and Dementia in Late-Life
Depression. Front. Aging Neurosci. 2017, 9, 13. [CrossRef]
19. Falk, N.; Cole, A.; Meredith, T.J. Evaluation of Suspected Dementia. Am. Fam. Physician 2018, 97, 398–405.
20. Association, A.P. Diagnostic and Statistical Manual of Mental Disorders (DSM-5® ); American Psychiatric Pub:
Washington, DC, USA, 2013.
21. Folstein, M.F.; Folstein, S.E.; McHugh, P.R. “Mini-mental state”: A practical method for grading the cognitive
state of patients for the clinician. J. Psychiatr. Res. 1975, 12, 189–198. [CrossRef]
22. O’Bryant, S.E.; Humphreys, J.D.; Smith, G.E.; Ivnik, R.J.; Graff-Radford, N.R.; Petersen, R.C.; Lucas, J.A.
Detecting dementia with the mini-mental state examination in highly educated individuals. Arch. Neurol.
2008, 65, 963–967. [CrossRef] [PubMed]
23. Teng, E.L.; Chui, H.C. The Modified Mini-Mental State (3MS) examination. J. Clin. Psychiatry 1987, 48,
314–318. [PubMed]
24. Dinomais, M.; Celle, S.; Duval, G.T.; Roche, F.; Henni, S.; Bartha, R.; Beauchet, O.; Annweiler, C. Anatomic
correlation of the mini-mental state examination: A voxel-based morphometric study in older adults. PLoS
ONE 2016, 11, e0162889. [CrossRef] [PubMed]
25. Eichenbaum, H.; Dudchenko, P.; Wood, E.; Shapiro, M.; Tanila, H. The hippocampus, memory, and place
cells: Is it spatial memory or a memory space? Neuron 1999, 23, 209–226. [CrossRef]
J. Clin. Med. 2020, 9, 3287 16 of 20

26. Hort, J.; Laczó, J.; Vyhnálek, M.; Bojar, M.; Bures, J.; Vlcek, K. Spatial navigation deficit in amnestic mild
cognitive impairment. Proc. Natl. Acad. Sci. USA 2007, 104, 4042–4047. [CrossRef]
27. Park, J.; Jeong, E.; Seomun, G. The clock drawing test: A systematic review and meta-analysis of diagnostic
accuracy. J. Adv. Nurs. 2018, 74, 2742–2754. [CrossRef]
28. Albert, S.M.; Teresi, J.A. Reading ability, education, and cognitive status assessment among older adults in
Harlem, New York City. Am. J. Public Health 1999, 89, 95–97. [CrossRef]
29. Weiss, B.D.; Reed, R.; Kligman, E.W.; Abyad, A. Literacy and performance on the mini-mental state
examination. J. Am. Geriatr. Soc. 1995, 43, 807–810. [CrossRef]
30. Mayeaux, J.E.; Davis, T.; Jackson, R.; Henry, D.; Patton, P.; Slay, L.; Sentell, T. Literacy and self-reported
educational levels in relation to Mini-Mental State Examination scores. Fam. Med. 1995, 27, 658–662.
31. Zhu, F.; Panwar, B.; Dodge, H.H.; Li, H.; Hampstead, B.M.; Albin, R.L.; Paulson, H.L.; Guan, Y. COMPASS: A
computational model to predict changes in MMSE scores 24-months after initial assessment of Alzheimer’s
disease. Sci. Rep. 2016, 6, 34567. [CrossRef]
32. Palsetia, D.; Rao, G.P.; Tiwari, S.C.; Lodha, P.; De Sousa, A. The Clock Drawing Test versus Mini-mental
Status Examination as a Screening Tool for Dementia: A Clinical Comparison. Indian J. Psychol. Med. 2018,
40, 1–10. [CrossRef] [PubMed]
33. Ryan, J.J.; Glass, L.A.; Bartels, J.M.; Paolo, A.M. Base Rates of “10 to 11” Clocks in Alzheimer’s and Parkinson’s
Disease. Int. J. Neurosci. 2009, 119, 1261–1266. [CrossRef] [PubMed]
34. Sunderland, T.; Hill, J.L.; Mellow, A.M.; Lawlor, B.A.; Gundersheimer, J.; Newhouse, P.A.; Grafman, J.H.
Clock drawing in Alzheimer’s disease. A novel measure of dementia severity. J. Am. Geriatr. Soc. 1989, 37,
725–729. [CrossRef] [PubMed]
35. de Paula, J.J.; de Miranda, D.M.; de Moraes, E.N.; Malloy-Diniz, L.F. Mapeando as engrenagens: O que o
Teste do Desenho do Relógio avalia no envelhecimento normal e patológico? Arq. Neuro-Psiquiatr. 2013, 71,
763–768. [CrossRef]
36. Noone, P. Addenbrooke’s Cognitive Examination-III. Occup. Med. 2015, 65, 418–420. [CrossRef] [PubMed]
37. Hsieh, S.; McGrory, S.; Leslie, F.; Dawson, K.; Ahmed, S.; Butler, C.R.; Rowe, J.B.; Mioshi, E.; Hodges, J.R.
The Mini-Addenbrooke’s Cognitive Examination: A new assessment tool for dementia. Dement. Geriatr.
Cogn. Disord. 2015, 39, 1–11. [CrossRef] [PubMed]
38. Mathuranath, P.; Nestor, P.; Berrios, G.; Rakowicz, W.; Hodges, J. A brief cognitive test battery to differentiate
Alzheimer’s disease and frontotemporal dementia. Neurology 2000, 55, 1613–1620. [CrossRef] [PubMed]
39. Yoshida, H.; Terada, S.; Honda, H.; Ata, T.; Takeda, N.; Kishimoto, Y.; Oshima, E.; Ishihara, T.; Kuroda, S.
Validation of Addenbrooke’s cognitive examination for detecting early dementia in a Japanese population.
Psychiatry Res. 2011, 185, 211–214. [CrossRef]
40. Callow, L.; Alpass, F.; Leathem, J.; Stephens, C. Normative data for older New Zealanders on
the Addenbrooke’s Cognitive Examination-Revised. N. Z. J. Psychol. 2015, 44, 29–41.
41. Brodaty, H.; Pond, D.; Kemp, N.M.; Luscombe, G.; Harding, L.; Berman, K.; Huppert, F.A. The GPCOG:
A new screening test for dementia designed for general practice. J. Am. Geriatr. Soc. 2002, 50, 530–534.
[CrossRef]
42. Basic, D.; Khoo, A.; Conforti, D.; Rowland, J.; Vrantsidis, F.; Logiudice, D.; Hill, K.; Harry, J.; Lucero, K.;
Prowse, R. Rowland Universal Dementia Assessment Scale, Mini- Mental State Examination and General
Practitioner Assessment of Cognition in a multicultural cohort of community-dwelling older persons with
early dementia. Aust. Psychol. 2009, 44, 40–53. [CrossRef]
43. Brodaty, H.; Kemp, N.M.; Low, L.F. Characteristics of the GPCOG, a screening tool for cognitive impairment.
Int. J. Geriatr. Psychiatry 2004, 19, 870–874. [CrossRef] [PubMed]
44. Nasreddine, Z.S.; Phillips, N.A.; Bédirian, V.; Charbonneau, S.; Whitehead, V.; Collin, I.; Cummings, J.L.;
Chertkow, H. The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive
impairment. J. Am. Geriatr. Soc. 2005, 53, 695–699. [CrossRef] [PubMed]
45. Ciesielska, N.; Sokołowski, R.; Mazur, E.; Podhorecka, M.; Polak-Szabela, A.; K˛edziora-Kornatowska, K.
Is the Montreal Cognitive Assessment (MoCA) test better suited than the Mini-Mental State Examination
(MMSE) in mild cognitive impairment (MCI) detection among people aged over 60? Meta-analysis. Psychiatr.
Pol. 2016, 50, 1039–1052. [CrossRef]
46. Julayanont, P.; Nasreddine, Z.S. Montreal Cognitive Assessment (MoCA): Concept and clinical review. In
Cognitive Screening Instruments; Springer: Cham, Switzerland, 2017; pp. 139–195.
J. Clin. Med. 2020, 9, 3287 17 of 20

47. Kang, J.M.; Cho, Y.-S.; Park, S.; Lee, B.H.; Sohn, B.K.; Choi, C.H.; Choi, J.-S.; Jeong, H.Y.; Cho, S.-J.; Lee, J.-H.;
et al. Montreal cognitive assessment reflects cognitive reserve. BMC Geriatr. 2018, 18, 261. [CrossRef]
48. Valenzuela, M.J.; Sachdev, P. Brain reserve and dementia: A systematic review. Psychol. Med. 2006, 36,
441–454. [CrossRef]
49. Dupuis, K.; Pichora-Fuller, M.K.; Chasteen, A.L.; Marchuk, V.; Singh, G.; Smith, S.L. Effects of hearing and
vision impairments on the Montreal Cognitive Assessment. Aging Neuropsychol. Cogn. 2015, 22, 413–437.
[CrossRef]
50. Rossetti, H.C.; Lacritz, L.H.; Cullum, C.M.; Weiner, M.F. Normative data for the Montreal Cognitive
Assessment (MoCA) in a population-based sample. Neurology 2011, 77, 1272–1275. [CrossRef]
51. Park, M.J.; Kim, D.J.; Lee, U.; Na, E.J.; Jeon, H.J. A Literature Overview of Virtual Reality (VR) in Treatment
of Psychiatric Disorders: Recent Advances and Limitations. Front. Psychiatry 2019, 10, 505. [CrossRef]
52. Zucchella, C.; Sinforiani, E.; Tamburin, S.; Federico, A.; Mantovani, E.; Bernini, S.; Casale, R.; Bartolo, M.
The Multidisciplinary Approach to Alzheimer’s Disease and Dementia. A Narrative Review of
Non-Pharmacological Treatment. Front. Neurol. 2018, 9, 1058. [CrossRef]
53. Hegarty, M.; Montello, D.R.; Richardson, A.E.; Ishikawa, T.; Lovelace, K. Spatial abilities at different scales:
Individual differences in aptitude-test performance and spatial-layout learning. Intelligence 2006, 34, 151–176.
[CrossRef]
54. Van Schaik, P.; Martyr, A.; Blackman, T.; Robinson, J. Involving Persons with Dementia in the Evaluation of
Outdoor Environments. CyberPsychology Behav. 2008, 11, 415–424. [CrossRef]
55. Weniger, G.; Ruhleder, M.; Lange, C.; Wolf, S.; Irle, E. Egocentric and allocentric memory as assessed by
virtual reality in individuals with amnestic mild cognitive impairment. Neuropsychologia 2011, 49, 518–527.
[CrossRef]
56. Serino, S.; Morganti, F.; Di Stefano, F.; Riva, G. Detecting early egocentric and allocentric impairments
deficits in Alzheimer’s disease: An experimental study with virtual reality. Front. Aging Neurosci. 2015, 7, 88.
[CrossRef] [PubMed]
57. Cushman, L.A.; Stein, K.; Duffy, C.J. Detecting navigational deficits in cognitive aging and Alzheimer disease
using virtual reality. Neurology 2008, 71, 888–895. [CrossRef] [PubMed]
58. Lee, J.-Y.; Kho, S.; Yoo, H.B.; Park, S.; Choi, J.-S.; Kwon, J.S.; Cha, K.R.; Jung, H.-Y. Spatial memory impairments
in amnestic mild cognitive impairment in a virtual radial arm maze. Neuropsychiatr. Dis. Treat. 2014, 10, 653.
[CrossRef] [PubMed]
59. Plancher, G.; Tirard, A.; Gyselinck, V.; Nicolas, S.; Piolino, P. Using virtual reality to characterize episodic
memory profiles in amnestic mild cognitive impairment and Alzheimer’s disease: Influence of active and
passive encoding. Neuropsychologia 2012, 50, 592–602. [CrossRef]
60. National Collaborating Centre for Mental Health. Dementia. In Dementia: A NICE-SCIE Guideline on
Supporting People With Dementia and Their Carers in Health and Social Care; British Psychological Society:
Leicester, UK, 2007.
61. Sikkes, S.; De Lange-de Klerk, E.; Pijnenburg, Y.; Scheltens, P. A systematic review of Instrumental Activities
of Daily Living scales in dementia: Room for improvement. J. Neurol. Neurosurg. Psychiatry 2009, 80, 7–12.
[CrossRef]
62. Bucks, R.; Ashworth, D.; Wilcock, G.; Siegfried, K. Assessment of activities of daily living in dementia:
Development of the Bristol Activities of Daily Living Scale. Age Ageing 1996, 25, 113. [CrossRef]
63. Johnson, N.; Barion, A.; Rademaker, A.; Rehkemper, G.; Weintraub, S. The Activities of Daily Living
Questionnaire: A validation study in patients with dementia. Alzheimer Dis. Assoc. Disord. 2004, 18, 223.
64. Tarnanas, I.; Schlee, W.; Tsolaki, M.; Müri, R.; Mosimann, U.; Nef, T. Ecological validity of virtual reality daily
living activities screening for early dementia: Longitudinal study. JMIR Serious Games 2013, 1, e1. [CrossRef]
[PubMed]
65. Dulau, E.; Botha-Ravyse, C.R.; Luimula, M.; Markopoulos, P.; Markopoulos, E.; Tarkkanen, K. A virtual
reality game for cognitive impairment screening in the elderly: A user perspective. In Proceedings of the 2019
10th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), Naples, Italy, 23–25
October 2019; pp. 403–410.
J. Clin. Med. 2020, 9, 3287 18 of 20

66. Zygouris, S.; Giakoumis, D.; Votis, K.; Doumpoulakis, S.; Ntovas, K.; Segkouli, S.; Karagiannidis, C.;
Tzovaras, D.; Tsolaki, M. Can a virtual reality cognitive training application fulfill a dual role? Using
the virtual supermarket cognitive training application as a screening tool for mild cognitive impairment. J.
Alzheimer’s Dis. 2015, 44, 1333–1347. [CrossRef] [PubMed]
67. Zygouris, S.; Ntovas, K.; Giakoumis, D.; Votis, K.; Doumpoulakis, S.; Segkouli, S.; Karagiannidis, C.;
Tzovaras, D.; Tsolaki, M. A preliminary study on the feasibility of using a virtual reality cognitive training
application for remote detection of mild cognitive impairment. J. Alzheimer’s Dis. 2017, 56, 619–627.
[CrossRef] [PubMed]
68. Karr, J.E.; Graham, R.B.; Hofer, S.M.; Muniz-Terrera, G. When does cognitive decline begin? A systematic
review of change point studies on accelerated decline in cognitive and neurological outcomes preceding
mild cognitive impairment, dementia, and death. Psychol. Aging 2018, 33, 195–218. [CrossRef]
69. Bohil, C.J.; Alicea, B.; Biocca, F.A. Virtual reality in neuroscience research and therapy. Nat. Rev. Neurosci.
2011, 12, 752–762. [CrossRef]
70. Farias, S.T.; Harrell, E.; Neumann, C.; Houtz, A. The relationship between neuropsychological performance
and daily functioning in individuals with Alzheimer’s disease: Ecological validity of neuropsychological
tests. Arch. Clin. Neuropsychol. 2003, 18, 655–672. [CrossRef]
71. Castelvecchi, D. Low-cost headsets boost virtual reality’s lab appeal. Nature 2016, 533, 153–154. [CrossRef]
72. Kennedy, R.S.; Lane, N.E.; Berbaum, K.S.; Lilienthal, M.G. Simulator sickness questionnaire: An enhanced
method for quantifying simulator sickness. Int. J. Aviat. Psychol. 1993, 3, 203–220. [CrossRef]
73. Clay, F.; Howett, D.; FitzGerald, J.; Fletcher, P.; Chan, D.; Price, A. Use of Immersive Virtual Reality in
the Assessment and Treatment of Alzheimer’s Disease: A Systematic Review. J. Alzheimer’s Dis. 2020, 75,
1–21. [CrossRef]
74. Geerlings, M.I.; Deeg, D.J.; Schmand, B.; Lindeboom, J.; Jonker, C.; Breteler, M.M.B. Increased risk of mortality
in Alzheimer’s disease patients with higher education? A replication study. Neurology 1997, 49, 798–802.
[CrossRef]
75. Lampit, A.; Hallock, H.; Valenzuela, M. Computerized cognitive training in cognitively healthy older adults:
A systematic review and meta-analysis of effect modifiers. PLoS Med. 2014, 11, e1001756. [CrossRef]
[PubMed]
76. Hill, N.T.M.; Mowszowski, L.; Naismith, S.L.; Chadwick, V.L.; Valenzuela, M.; Lampit, A. Computerized
Cognitive Training in Older Adults With Mild Cognitive Impairment or Dementia: A Systematic Review
and Meta-Analysis. Am. J. Psychiatry 2017, 174, 329–340. [CrossRef] [PubMed]
77. Barnes, D.E.; Yaffe, K.; Belfor, N.; Jagust, W.J.; DeCarli, C.; Reed, B.R.; Kramer, J.H. Computer-based cognitive
training for mild cognitive impairment: Results from a pilot randomized, controlled trial. Alzheimer Dis.
Assoc. Disord. 2009, 23, 205–210. [CrossRef] [PubMed]
78. Schwenk, M.; Sabbagh, M.; Lin, I.; Morgan, P.; Grewal, G.S.; Mohler, J.; Coon, D.W.; Najafi, B. Sensor-based
balance training with motion feedback in people with mild cognitive impairment. J. Rehabil. Res. Dev. 2016,
53, 945–958. [CrossRef]
79. Williams, B.; Doherty, N.L.; Bender, A.; Mattox, H.; Tibbs, J.R. The effect of nintendo wii on balance: A pilot
study supporting the use of the wii in occupational therapy for the well elderly. Occup. Ther. Health Care
2011, 25, 131–139. [CrossRef]
80. Manera, V.; Petit, P.-D.; Derreumaux, A.; Orvieto, I.; Romagnoli, M.; Lyttle, G.; David, R.; Robert, P.H. ‘Kitchen
and cooking,’ a serious game for mild cognitive impairment and Alzheimer’s disease: A pilot study. Front.
Aging Neurosci. 2015, 7, 24. [CrossRef]
81. Lee, G.-H. Effects of Virtual Reality Exercise Program on Balance, Emotion and Quality of Life in Patients
with Cognitive Decline. J. Korean Phys. Ther. 2016, 28, 355–363. [CrossRef]
82. Hwang, J.; Lee, S. The effect of virtual reality program on the cognitive function and balance of the people
with mild cognitive impairment. J. Phys. Ther. Sci. 2017, 29, 1283–1286. [CrossRef]
83. Htut, T.Z.C.; Hiengkaew, V.; Jalayondeja, C.; Vongsirinavarat, M. Effects of physical, virtual reality-based,
and brain exercise on physical, cognition, and preference in older persons: A randomized controlled trial.
Eur. Rev. Aging Phys. Act. 2018, 15, 10. [CrossRef]
84. Sakhare, A.R.; Yang, V.; Stradford, J.; Tsang, I.; Ravichandran, R.; Pa, J. Cycling and Spatial Navigation in an
Enriched, Immersive 3D Virtual Park Environment: A Feasibility Study in Younger and Older Adults. Front.
Aging Neurosci. 2019, 11, 218. [CrossRef]
J. Clin. Med. 2020, 9, 3287 19 of 20

85. Ben-Sadoun, G.; Sacco, G.; Manera, V.; Bourgeois, J.; König, A.; Foulon, P.; Fosty, B.; Bremond, F.;
d’Arripe-Longueville, F.; Robert, P. Physical and cognitive stimulation using an exergame in subjects
with normal aging, mild and moderate cognitive impairment. J. Alzheimer’s Dis. 2016, 53, 1299–1314.
[CrossRef] [PubMed]
86. Kober, S.E.; Wood, G.; Hofer, D.; Kreuzig, W.; Kiefer, M.; Neuper, C. Virtual reality in neurologic rehabilitation
of spatial disorientation. J. Neuroeng. Rehabil. 2013, 10, 17. [CrossRef] [PubMed]
87. Man, D.W.; Chung, J.C.; Lee, G.Y. Evaluation of a virtual reality-based memory training programme for
Hong Kong Chinese older adults with questionable dementia: A pilot study. Int. J. Geriatr. Psychiatry 2012,
27, 513–520. [CrossRef] [PubMed]
88. Fasilis, T.; Patrikelis, P.; Siatouni, A.; Alexoudi, A.; Veretzioti, A.; Zachou, L.; Gatzonis, S.S. A pilot study and
brief overview of rehabilitation via virtual environment in patients suffering from dementia. Psychiatriki
2018, 29, 42–51. [CrossRef] [PubMed]
89. Belghali, M.; Chastan, N.; Cignetti, F.; Davenne, D.; Decker, L.M. Loss of gait control assessed by
cognitive-motor dual-tasks: Pros and cons in detecting people at risk of developing Alzheimer’s and
Parkinson’s diseases. Geroscience 2017, 39, 305–329. [CrossRef]
90. Liao, Y.-Y.; Chen, I.H.; Lin, Y.-J.; Chen, Y.; Hsu, W.-C. Effects of Virtual Reality-Based Physical and Cognitive
Training on Executive Function and Dual-Task Gait Performance in Older Adults With Mild Cognitive
Impairment: A Randomized Control Trial. Front. Aging Neurosci. 2019, 11, 162. [CrossRef]
91. Dockx, K.; Bekkers, E.M.; Van den Bergh, V.; Ginis, P.; Rochester, L.; Hausdorff, J.M.; Mirelman, A.;
Nieuwboer, A. Virtual reality for rehabilitation in Parkinson’s disease. Cochrane Database Syst. Rev. 2016, 12,
CD010760. [CrossRef]
92. Eggenberger, P.; Schumacher, V.; Angst, M.; Theill, N.; de Bruin, E.D. Does multicomponent physical exercise
with simultaneous cognitive training boost cognitive performance in older adults? A 6-month randomized
controlled trial with a 1-year follow-up. Clin. Interv. Aging 2015, 10, 1335–1349. [CrossRef]
93. Foster, P.P.; Rosenblatt, K.P.; Kuljiš, R.O. Exercise-induced cognitive plasticity, implications for mild cognitive
impairment and Alzheimer’s disease. Front. Neurol. 2011, 2, 28. [CrossRef]
94. Huang, K.T. Exergaming Executive Functions: An Immersive Virtual Reality-Based Cognitive Training for
Adults Aged 50 and Older. Cyberpsychol. Behav. Soc. Netw. 2020, 23, 143–149. [CrossRef]
95. Manera, V.; Ben-Sadoun, G.; Aalbers, T.; Agopyan, H.; Askenazy, F.; Benoit, M.; Bensamoun, D.; Bourgeois, J.;
Bredin, J.; Bremond, F.; et al. Recommendations for the Use of Serious Games in Neurodegenerative
Disorders: 2016 Delphi Panel. Front. Psychol. 2017, 8, 1243. [CrossRef] [PubMed]
96. Farshid, M.; Paschen, J.; Eriksson, T.; Kietzmann, J. Go boldly!: Explore augmented reality (AR), virtual
reality (VR), and mixed reality (MR) for business. Bus. Horiz. 2018, 61, 657–663. [CrossRef]
97. Carmigniani, J.; Furht, B.; Anisetti, M.; Ceravolo, P.; Damiani, E.; Ivkovic, M. Augmented reality technologies,
systems and applications. Multimed. Tools Appl. 2011, 51, 341–377. [CrossRef]
98. Chatzopoulos, D.; Bermejo, C.; Huang, Z.; Hui, P. Mobile augmented reality survey: From where we are to
where we go. IEEE Access 2017, 5, 6917–6950. [CrossRef]
99. Albrecht, U.V.; Folta-Schoofs, K.; Behrends, M.; von Jan, U. Effects of mobile augmented reality learning
compared to textbook learning on medical students: Randomized controlled pilot study. J. Med. Internet Res.
2013, 15, e182. [CrossRef]
100. Kucuk, S.; Kapakin, S.; Goktas, Y. Learning anatomy via mobile augmented reality: Effects on achievement
and cognitive load. Anat. Sci. Educ. 2016, 9, 411–421. [CrossRef]
101. Lehman, S.; Graves, J.; Mcaleer, C.; Giovannetti, T.; Tan, C.C. A mobile augmented reality game to
encourage hydration in the elderly. In Proceedings of the International Conference on Human Interface and
the Management of Information, Las Vegas, NV, USA, 15–20 July 2018; pp. 98–107.
102. Rohrbach, N.; Gulde, P.; Armstrong, A.R.; Hartig, L.; Abdelrazeq, A.; Schröder, S.; Neuse, J.; Grimmer, T.;
Diehl-Schmid, J.; Hermsdörfer, J. An augmented reality approach for ADL support in Alzheimer’s disease:
A crossover trial. J. Neuroeng. Rehabil. 2019, 16, 66. [CrossRef]
103. Wolf, D.; Besserer, D.; Sejunaite, K.; Riepe, M.; Rukzio, E. cARe: An Augmented Reality Support System for
Dementia Patients. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and
Technology Adjunct Proceedings, Berlin, Germany, 14–17 October 2018; pp. 42–44.
J. Clin. Med. 2020, 9, 3287 20 of 20

104. Aruanno, B.; Garzotto, F.; Rodriguez, M.C. HoloLens-based Mixed Reality Experiences for Subjects with
Alzheimer’s Disease. In Proceedings of the 12th Biannual Conference on Italian SIGCHI Chapter, Cagliari,
Italy, 18–20 September 2017; pp. 1–9.
105. Park, E.; Yun, B.J.; Min, Y.S.; Lee, Y.S.; Moon, S.J.; Huh, J.W.; Cha, H.; Chang, Y.; Jung, T.D. Effects of a Mixed
Reality-based Cognitive Training System Compared to a Conventional Computer-assisted Cognitive Training
System on Mild Cognitive Impairment: A Pilot Study. Cogn. Behav. Neurol. 2019, 32, 172–178. [CrossRef]

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional
affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).