Polytechnic University of the Philippines

Sta. Mesa, Manila

Graduate School

Written Report: McNemar Test

PSY 641 Statistics in Behavioral Science

Submitted by: Arian R. Ferrer

Submitted to: Dr. Elmer G. De Jose
Introduction:
According to Statistical Solution (2018), McNemar’s test was first published in
a Psychometrika article in 1947. It was created by Quinn McNemar, who was a professor
in the Psychology and Statistics department at Stanford University. This non-parametric
(distribution-free) test assesses if a statistically significant change in proportions have
occurred on a dichotomous trait at two time points on the same population. It is applied
using a 2×2 contingency table with the dichotomous variable at time 1 and time 2.

Rationale:
McNemar's one degree of freedom chi‐square test for the equality of proportions
appears frequently in the analysis of pairs of matched, binary outcome data (Y1i, Y2i). An
assumption underlying this test is that the responses from pair to pair are mutually
independent (Eliasziw and Donner, 1991). According to Emerson (2006), In certain
applications, however, the pairs may represent repeated measurements on the same
experimental unit, and hence this assumption is violated. If response were equal in the
two group, discordant pairs should be equally likely to be in either order.

–Condition on the number of discordant pairs Intuitively, the number of discordant pairs
does not contribute much information as to which group does better.

– Under the null hypothesis, the discordant pairs should be equally likely to be in either
the “b” or the “c” cell of the contingency table.

Proponent:

Quinn Michael McNemar (1901–1986) McNemar’s test is applied when an

experiment involves a binary outcome observed on matched pairs or repeated measures
on the same subject (e.g., before and after). Results are summarized in a 2.2 contingency
table. McNemar was a renowned psychologist in the field of IQ testing10,11,12. He
published ‘The Revision of the Stanford-Binet Scale’ (IQ test) in 1942. He held
Professorships in statistics, psychology and education (Htoon, Chan and Allen, 2012).

Assumptions:

According to Stephanie (2017), The three main assumptions for the test are:
1. You must have one nominal variable with two categories (i.e. dichotomous variables)
and one independent variable with two connected groups.
2. The two groups in your the dependent variable must be mutually exclusive. In other
words, participants cannot appear in more than one group.
3. Your sample must be a random sample.

Sample Computation:
In order to run a McNemar test, your data should be placed into a 2×2 contingency
table, with the cell frequencies equaling the number of pairs. For example, a researcher
is testing a new medication and records if the drug worked (“yes”) or did not (“no”). A table
is set up with the count of individuals before and after being given the medication
(Stepahie, 2017).

The cell labels a-d are in blue:

Cells b and c are used to calculate the test statistic; these cells are called “discordant.”
The McNemar test formula is:

For the set of data above, we have:

= (100-10)2 / (100 + 10)
= 902 / 100
= 73.63
Journals/Publications that used McNemar’s Test:

Journal 1: Developing Effective Communication Materials on the Health Effects of

Climate Change for Vulnerable Groups: A Mixed Methods Study

According to Kreslake, Price and Sarfaty, (2016) Individuals with chronic health
conditions or low socioeconomic status (SES) are more vulnerable to the health impacts
of climate change. Health communication can provide information on the management of
these impacts. This study tested, among vulnerable audiences, whether viewing targeted
materials increases knowledge about the health impacts of climate change and strength
of climate change beliefs, and whether each are associated with stronger intentions to
practice recommended behaviors.

Methods:

Low-SES respondents with chronic conditions were recruited for an online survey
in six cities. Respondents were shown targeted materials illustrating the relationship
between climate change and chronic conditions. Changes in knowledge and climate
change beliefs (pre- and post-test) and behavioral intentions (post-test only) were tested
using McNemar tests of marginal frequencies of two binary outcomes or paired t-tests,
and multivariable linear regression. Qualitative interviews were conducted among target
audiences to triangulate survey findings and make recommendations on the design of
messages, (Kreslake, Price and Sarfaty, 2016).

Results:

Respondents (N = 122) reflected the target population regarding income,

educational level and prevalence of household health conditions. (1) Knowledge.
Significant increases in knowledge were found regarding: groups that are most vulnerable
to heat (children [p < 0.001], individuals with heart disease [p < 0.001], or lung disease [p
= 0.019]); and environmental conditions that increase allergy-producing pollen (increased
heat [p = 0.003], increased carbon dioxide [p < 0.001]). (2) Strength of certainty that
climate change is happening increased significantly between pre- and post-test (p <
0.001), as did belief that climate change affected respondents’ health (p < 0.001). (3)
Behavioral intention. At post-test, higher knowledge of heat vulnerabilities and
environmental conditions that trigger pollen allergies were associated with greater
behavioral intention scores (p = 0.001 and p = 0.002, respectively). In-depth interviews
(N = 15) revealed that vulnerable audiences are interested in immediate-term advice on
health management and protective behaviors related to their chronic conditions, but took
less notice of messages about collective action to slow or stop climate change.
Respondents identified both appealing and less favorable design elements in the
materials, (Kreslake, Price and Sarfaty, 2016).

Conclusions:

According to Kreslake, Price and Sarfaty, (2016) Individuals who are vulnerable to
the health effects of climate change benefit from communication materials that explain,
using graphics and concise language, how climate change affects health conditions and
how to engage in protective adaptation behaviors.

Journal 2: Children Attribute Moral Standing to a Personified Agent

According to Freier (2008), The relationships that children are likely to develop with
personified technologies. The paper presents evidence that children, eight and nine years
of age, consciously judge interactions with a personified software agent to be not only
social but also moral in nature. The results also suggest that specific design decisions
can have a dramatic influence on children’s judgments regarding the moral standing of
personified technology. A discussion follows with a cautious argument that children’s
moral attributions to a personified technology are a good thing for healthy child
development, while emphasizing that further research is needed in order to fully
understand the implications.

Research Questions:
According to Freier (2008), The research questions that guide this study follow
directly from the prior discussion, specifically with respect to the conventional and moral
domain distinctions in developmental psychology. The questions are as follows:
(1) will children generalize their knowledge of domain distinctions to the context of human-
agent interactions? and,
(2) what role does the design of the personified technology play in promoting or hindering
this generalizing behavior on the part of children?

Method:
60 children, 30 males and 30 females, between the ages of eight and nine years
(mean of 9.02 years), were recruited to participate in this study. The race-ethnicity of the
sample follows: Caucasian-American (85%), Asian-American (8%), Hispanic- or Latino-
American (5%), and Alaskan- Native-American (2%). Children were randomly assigned
to one of two study conditions, stratified by gender to guarantee gender balance.
Sessions lasted approximately 20-40 minutes. During the session, parents or guardians
were asked to wait in another room. Prior Experience with Technology. Children were
asked about their prior experiences with technology. All children (100%) in the study had
at least one computer in the home, and half (50%) used the computer very often as
opposed to not very often. The majority of children (77%) had access to the internet in
their home and over half of those children (59%) accessed the internet very often. Over
half the children (57%) had some sort of gaming platform in their home (e.g., game
consoles such as Playstation, or hand held game systems such as Nintendo DS) and
over a third of those children (38%) played with the gaming platform very often, (Freier,
2008).

RESULTS
In this section, results are presented for the criterion judgment questions and
justifications relating to both the researcher-agent interactions as well as the hypothetical
human-human interaction scenarios (Freier, 2008).
Justification Category Definition and Examples:
Physical-Material Physical-Material refers to mechanical or technological features or
processes. Also refers to material maintenance (e.g., “it wouldn’t be
ok because you bought it and you didn’t take care of it”), functioning
(e.g., “the television can’t talk back to real people”), and financial or
 physical resources (e.g., “you wouldn’t want to do that unless you
want to waste a bunch of electric energy”).
Personal-Psychological Personal-Psychological refers to psychological or behavioral
responsiveness (e.g., “Judith didn’t seem to react much”),
perception and the senses (e.g., “the person can’ hear you”),
agency (e.g., “Judith wanted to play the real way”), and emotion
(e.g., “computers don’t have feelings yet”).
Social-Conventional Social-Conventional refers to concepts of a social, cultural, or
normative nature, including authority (e.g., “you would get in
trouble”), rules and laws (e.g., “those are the rules”), and norms
(e.g., “then everyone would start putting triangles on the board”).
Moral Moral refers to physical welfare (e.g., “it could break like bones and that”),
psychological welfare (e.g., “it hurt her feelings”), concepts of a
deontic moral nature (e.g., “it’s not really fair to the kids”), and
virtues (e.g., “it’s not good to say bad words”).

Conclusion:
According to Freier (2008), Children are constantly grappling with the difficult
problem of determining what is and what is not appropriate action in response to the
constraints of their environment. Actions, from a Piagetian perspective, are not only
behavioral activities, but also include the active processes of thought and knowledge
construction. Children judge, reason, and behave, that is they act, with a generally
unspecified goal of attaining equilibrium amongst their internal concepts of the world, and
between those internal concepts and the external world’s projection by way of their
senses. Fundamental changes in the constituents of children’s environments may have
significant implications for child development. The introduction of technology that acts
social can and does influence children’s social and moral thought. It is important,
therefore, to conduct further research in this area to understand the larger implications
and guide future design towards beneficial ends. For the purposes of this paper, the focus
was on an empirical study of children’s judgments and reasoning regarding the distinction
between social-conventional and moral actions while interacting with a personified,
virtually embodied, computational agent. Further research in this area is needed to
understand the potential impact and inform social robot and personified technology
designers of the potential positive and negative influences that their design decisions may
have on children’s social and moral development.
References:

Eliasziw M. and Donner A. 1991, Application Of The Mcnemar Test To Non‐

Independent Matched Pair Data,
https://onlinelibrary.wiley.com/doi/pdf/10.1002/sim.4780101211, First
published: December 1991

Emerson, S. 2006, Lecture 16: Two Sample Inference for Correlated Response Data,
University of Washington November 27, 2006

Freier G., 2008, Children Attribute Moral Standing to a Personified Agent Rensselaer
Polytechnic Institute, 110 8th St., Sage 4202, Troy, NY, USA

Htoon, Chan, Allen 2012, Behind Every Familiar Statistical Test is a Famous
Statistician, Duke-NUS Graduate Medical School, Singapore, Yong Loo Lin
School of Medicine, National University of Singapore, Singapore Eye Research
Institute, Singapore, Proceedings of Singapore Healthcare Volume 21
Number 3 2012

Statistic Solutions, 2018, https://www.statisticssolutions.com/non-parametric-analysis-

mcnemars-test/

Stephanie, 2017, McNemar Test Definition, Examples, Calculation,

http://www.statisticshowto.com/mcnemar-test/ October 12th, 2017

Kreslake M., Price K., and Kreslake M., et al. 2016, Developing Effective Communication
Materials On The Health Effects Of Climate Change For Vulnerable Groups: A
Mixed Methods, BMC Public Health (2016) 16:946 DOI 10.1186/s12889-016-
3546-3

Freier G., 2008, Children Attribute Moral Standing to a Personified Agent Rensselaer
Polytechnic Institute, 110 8th St., Sage 4202, Troy, NY, USA