1

IB PSYCHOLOGY INTERNAL ASSESSMENT

AN EXPERIMENT TO INVESTIGATE HOW EFFORT HEURISTIC AFFECTS

THE JUDGEMENTS OF QUALITY

(A REPLICATION OF KRUGER ET AL (2004))

Personal IB candidate code: krj740

Submission Date Session: May 2024

Group candidate: krj734, krj736, krj957

Word Count: 2200

 2

TABLE OF CONTENTS

INTRODUCTION ______________________________________________________ 3

EXPLORATION _______________________________________________________ 5

ANALYSIS ___________________________________________________________ 8

EVALUATION _________________________________________________________ 9

REFERENCES ________________________________________________________ 11

APPENDIX ___________________________________________________________ 13
 3

INTRODUCTION

In the study of psychology, there are three different approaches that are deeply
intertwined with brain functioning: biological, sociocultural, and cognitive. This
internal assessment will investigate the cognitive approach, focusing on the effects
of heuristics, the dual processing model, and judgment quality. According to Evans
Jonathan, there are two-phase theories of human inference put forth: heuristic
processes and analytic processes (Evans, J. S. B., 1984). Evans proposed that an
individual uses heuristic processes to select information that is pertinent to the given
scenario. Deconstruction of this broad concept: heuristics can be known as mental
shortcuts, which allow people to make decisions and solve problems in a quick and
efficient way (Tversky Kahneman, 1982). General rules of thumb for decision-making
would be generated to make those solutions fast and correct for certain contexts
(MSEd, 2022). However, the reliance on heuristics might hinder the making of
decisions or justifications about more complicated issues. Thus, cognitive biases,
which are systematic fallacies in reasoning that influence people's decisions and
judgments, might be involved in the process of making choices (MSEd, 2022). The
effort heuristics describe that individuals are more inclined to assign the monetary
value of an object and evaluate its quality as higher if that piece of work involves
more endeavor. This means that subjectivity of effort might occur as we ignore other
crucial aspects of judgments but prioritize the time invested. This could potentially be
attributed to the fact that it indicates that people's evaluations of objects or
anticipated outcomes can be influenced by information about the production process
in addition to the object's features. This implies that effort heuristics are intrinsically
contextual, and people may take this into account when evaluating objects (Ziano et
al., 2023).

The study being replicated is Kruger et al. (2004), which examined the influence of
the effort heuristic on object judgment. They believed in the tendency to evaluate the
quality of an object if it was perceived as involving more effort. This phenomenon is
in line with cognitive dissonance theory, which holds that people justify their efforts
by thinking that the results are valuable and the products are of high caliber (Ziano et
al., 2023).
 4

Kruger and his colleagues (2004) discovered evidence to bolster the idea that
individuals occasionally utilize the amount of work put into something as a proxy for
its quality. Kruger divided individuals into three distinct experiments. Each
experiment was separated into two conditions: low-effort heuristics and high-effort
heuristics. Three separate studies were conducted using various stimuli: a piece of
armor, poetry, and painting. Each experiment had low-effort and high-effort
conditions, with the two conditions differing in the amount of time it took for the
objects to be created, corresponding to each condition. Participants were being
asked to judge the quality of objects based on a numerical scale from 0 to 99.
Results across all 3 experiments indicated that participants tended to perceive an
item as superior when they perceived a greater level of effort invested in its creation.

Our study aimed to investigate the role of effort in determining the quality of an
artwork. The artwork used is called ‘Composition VII’ drawn by Wassily Kandinsky.
Since we will only be carrying out one of the three experiments that Kruger et al.
(2004) conducted, this study will be partially replicated. Because effort is a major
factor in determining an object's value, we decided to investigate this notion. And this
is applicable in the real world as well. For example, in a school setting, teachers may
evaluate students' work more favorably if they feel that they have worked very hard.

Independent Variable: Two conditions are used for this experiment: the high-effort
heuristic at 3 months effort to complete drawing and the low-effort heuristic at 7
hours.

Dependent Variable: The quality judgment of the artwork as measured by the

participants’ responses on the Likert scale from 1 (low) to 10 (high).

H1 (Research hypothesis): The average quality rating of a painting completed in 3

months will significantly surpass that of a painting completed over a 7-hour period,
as measured on a 10-point Likert scale (1 indicating very low quality and 10
indicating very high quality).
 5

H0 (Null hypothesis): There will be no significant difference in the perceived quality

ratings between the high-effort heuristic (a painting completed over a 3-month
period) and the low-effort heuristic (one completed in 7 hours), as assessed using a
10-point Likert scale.

The reason why we separate the conditions into 7 hours and 3 months is because in
real-world situations, such as the classroom, students may believe that half of an
average artwork that consists of thick layers, which requires six months to draw, may
emphasize the significance of deft execution in producing high-quality art. This
comprehension, albeit shorter than the seven-hour condition, can offer insightful
information about how students view the creative process. (How Long Does It Take
to Finish 1 Painting, accessed in 2023)

EXPLORATION

Research Design

An independent measures design was used to carry out this investigation. Various
participant groups were exposed to various experimental situations; in other words,
participants only had one chance to engage in the particular condition of a specific
experiment. In order to prevent carryover effects, it is therefore crucial to eliminate
knowledge of the genuine purpose if they are in the same location for an interview.

Sampling Technique

To collect participants, we used convenience sampling, choosing participants based

on their availability. This was accomplished by contacting IB teachers whose
students were accessible throughout that study period and could participate in the
experiment. We choose an entire class group; therefore, not every member would be
chosen at random. This method was selected since it was convenient, allowing us to
approach those who were willing to engage in a relaxed and adaptable manner. But
since there were individual variances among the subjects, we took 20 students as
largest sample group possible that we could collect to control the problem.
 6

Participants

Our participant group was made up of 20 GCSE Grade 10 Business students, 10

males and 10 females, studying in an international school through convenient
sampling. Since the sample didn't include students enrolled in the IB Psychology
course, none of them had any prior experience with psychology. This may lessen
data variability and facilitate independent variable's actual impact. To counteract the
effect of age, the group was made up of both boys and females between the ages of
15 and 16. Each member of the group was fluent in English and came from variety of
cultural backgrounds, including America, Korea, and Vietnam. This could make the
results more broadly applicable and reduce any bias that could result from
homogeneity, but it can be harder to account for potential confounding variables.

Controlled Variables

Variables that were kept controlled in our experiment to achieve the best possible
results include:

● All participants received identical paper copies of the consent, debriefing,

painting, and question. This meant we were able to regulate the
circumstances fairly.
● The environment of the experiment: Test circumstances include sitting apart,
not conversing, and exam-like surroundings. To avoid distracting participants
with loud noises, we ensured that they were all in a quiet room.
● Standardized instructions were read to all participants from a pre-written
sheet. This was done to avoid any differences in the word choice of the
instructions, with concise and clear language used.
 7

Materials
● Consent Form (Appendix 1)
● Print out of the artwork ‘Composition VII’ attached with the questionnaire
(Appendix 5)
● Questionnaire based on the Likert scale from 1 to 10 attach with the artwork
printout (Appendix 4)
● Pen
● Debriefing form (Appendix 2)

We carefully selected these trustworthy materials to enhance conditions similar to

the original study, which improves measurement consistency and boosts confidence
in the findings with effective confounding variable control.

Procedure

In the experiment, we lined up participants and led them into the classroom to sit at
tables arranged in sequential order, marked with odd and even numbers using
markers. The low-effort heuristic group was assigned to the participants with odd
numbers, and the high-effort heuristic group was assigned to the participants with
even numbers. The high-effort group was shown to a different, ready-to-use room.
Standardized instructions (Appendix 3) outlining the experiment were given to both
groups, and they were also given the chance to sign consent papers (Appendix 2)
attesting to their voluntary involvement, confidentiality, and right to withdraw. When
everything was set up, participants received copies of the artwork and
questionnaires, and they had 10 seconds to respond.

After completing all consent papers and questions, we gave the participants the
debriefing forms to ensure they were comfortable. Upon signing the debriefing form,
the participants received gratitude for their time. This procedure was processed at
the same time in both circumstances. We will safeguard and preserve their data in
our sharing group document so that only our partners are permitted access. This
means that participant data is private.
 8

ANALYSIS

Descriptive Statistics

From the experiment, we have gathered the data from two conditions on the Likert
scale from 1 to 10. We chose to calculate the mean as a focus for the central
tendency and standard deviation to show how dispersed the data are relative to the
average mean. As seen in this table, the mean rating for the quality of the artwork in
low-effort condition was 5.3 out of 10. Nevertheless, the mean amount for the
high-effort condition shows off the higher score of 7.4. Regarding these results, it
proved that effort invested in work correlates with its quality. The more effort
participants put in, the better the quality of an object (Appendix 3).

Low effort condition High effort condition

Mean Rating for quality of 5.3 7.4

the artwork

Standard Deviation of 1.34 0.84

quality of the artwork

Table 1: Mean Rating and Standard Deviation of 2 conditions

Graph 1: Arithmetic Mean for the Quality Rating of the Artwork | Created using Microsoft Excel
 9

Inferential Statistics

From the raw data we collected, I used a Mann-Whitney U test to see the differences
between the two independent groups when the dependent variable is ordinal but not
normally distributed (Mann-Whitney U Test in SPSS Statistics, accessed in 2024).
The results showed the U-value was 91.5, which is higher than the critical value of p
< 0.05, which was 23 (Appendix 7). Additionally, the z-value was -3.1 and the p-value
was 0.001, indicating that 0.1% of the population I have would receive the results at
random, which is extremely uncommon based only on chance. In other words, the
results gained from the experiment have been 99.9% manipulated by the
researchers.

This then allows us to reject the null hypothesis and accept the operationalized
hypothesis that participants in the high-effort heuristic condition are more likely to
rate the drawing as having a higher quality based on high effort.

EVALUATIONS

The results demonstrate statistically significant agreement with the experimental

hypothesis since the drawing in high effort condition received higher quality rating for
the effort put into it. The mean outcomes for the two conditions had a significant
difference, therefore suggesting that effort heuristic plays a vital role when producing
the quality rating. Also, the experiment was highly controlled in good condition. The
difference in time frame and choice of painting has led to a good result due to its
abstraction; therefore, avoid participant bias because they might not know the value
of the artwork stimuli.

The use of the independent measures design allowed us to decrease order effects
and raise external validity, which improved the generalizability of my findings by
looking at other participant groups. The Mann-Whitney U indicates that the study
was well-controlled, thus able to regulate confounding variables while limiting the
experiment's potential for generalization.
 10

Furthermore, the group of participants is appropriate to improve the accuracy of the

outcome because none of them are really interested in the arts, so their collective
artistic expertise and sketching experiences may help to mitigate participant bias.

However, with regard to simple questionnaire that was included in the survey and
that went straight to the study's objectives of "take 7 hours to complete" and "take 3
months to complete,” which lacked of solid artistic background, it could easily let the
participants know the true purpose of the experiment, which would otherwise be
hindered by their lack of integrity when asking questions. Additionally, there is no
obvious connection between the favorite rating on a Likert scale of 1 to 10, since I
did not specify the favorite level for each number. Another constraint that results in
the research's accuracy being compromised is the size of the study sample. This
limits the extent of the findings that can be generalized to further study as all
participants share the same educational background (Business).

Hence, to improve the study, I will construct the experiment to become more natural
due to the hasty procedure by giving them more flexible time to observe the artwork
and respond rather than a strict 10-second limit. Also, provide a more natural
questionnaire with the artwork storyline or background to avoid participant bias.

Based on this investigation, we can say that the effort heuristic is crucial to how
15–16-year-old IB students think and rate the quality in relation to their perspective.
 11

REFERENCES

Critical Values of the Mann-Whitney U (Two-Tailed Testing). (n.d.). UMas Boston

OpenCourseWare. Retrieved February 27, 2024, from
https://ocw.umb.edu/psychology/psych-270/other-materials/RelativeResourceManag
er.pdf

Evans, J. S. B. (1984). Heuristic and analytic processes in reasoning. British Journal

of Psychology, 75(4), 451-468.

How long does it take to finish 1 painting? – Studio Amsterdam in Maitland, NSW.
(n.d.).
https://www.studioamsterdam.com.au/how-long-does-it-take-to-finish-1-painting/

Kruger, J., Wirtz, D., Van Boven, L., & Altermatt, T. W. (2004, January 1). The effort
heuristic. Journal of Experimental Social Psychology.
https://doi.org/10.1016/s0022-1031(03)00065-9

Mann-Whitney U Test in SPSS Statistics | Setup, Procedure & Interpretation | Laerd

Statistics. (n.d.).
https://statistics.laerd.com/spss-tutorials/mann-whitney-u-test-using-spss-statistics.p
hp#:~:text=Introduction,continuous%2C%20but%20not%20 normally%20 distributed.

MSEd, K. C. (2022, November 8). What Are Heuristics? Verywell Mind.

https://www.verywellmind.com/what-is-a-heuristic-2795235

MSEd, K. C. (2022, November 7). What Is Cognitive Bias? Verywell Mind.

https://www.verywellmind.com/what-is-a-cognitive-bias-2794963

Tversky, A. & Kahneman, D. (1982). Judgment under uncertainty: Heuristics and

biases. In D. Kahenman, P. Slovic, & A. Tversky (Eds.) Judgment under uncertainty:
Heuristics and biases. (pp 3-20). Cambridge, UK: Cambridge University Press.

Roberts, D. R. A. F. (n.d.). Standard Normal Distribution - MathBitsNotebook(A2).

MathBits.com (MathBitsNotebook.com) - Fred and Donna Roberts.
https://mathbitsnotebook.com/Algebra2/Statistics/STstandardNormalDistribution.html
 12

Ziano, I., Yeung, S. K., Lee, C. S., Shi, J., & Feldman, G. (2023, January 1). “The
Effort Heuristic” Revisited: Mixed Results for Replications of Kruger et al. (2004)’s
Experiments 1 and 2. Collabra. https://doi.org/10.1525/collabra.87489
 13

APPENDIX

Appendix 1: Consent form

Letter of Consent

Hello, we are experimenting as part of our internal assessment for our IB Psychology class. We
are

investigating anchoring bias. We would like to ask you to be part of our experiment.

If you agree to take part in this experiment, you should know that:

● All data that we obtain will be kept confidential and anonymous.

● You may stop participating in this experiment at any time.
● You shall receive information about the nature of the experiment and our results after
we have obtained results.

The experiment will take about 80 minutes to complete.

If you agree, we ask that you sign the form below and fill in the following information relevant to
our experiment.

Gender: Mother tongue: Age:

I, _______________________________________, understand the nature of this experiment

and I agree to participate voluntarily. I permit the researchers to use my data as part of their
experimental study.

Signature:

Date: __________________
 14

Appendix 2: Debriefing form

Briefing notes/standardized directions:

Good morning. Thank you for participating in this experiment.

First, I will give you a paper to sign. This is called an informed consent letter. You should know
that you are entitled to withdraw from the experiment in case you change your mind. Don’t
hesitate to ask questions if you have any. After you have signed the paper, I will read some
instructions about the experiment to you.

Now, you will have 10 seconds to view and rate the painting. Please read the description
carefully.

Your time starts now!

Debrief:

Dear participants, thank you for your participation in our cognitive process experiment. Your
involvement is essential to the success of our research. Before we proceed, we would like to
provide you with some information about the experiment

We aim to investigate how anchor effects affect judgment. As well as to justify the presence
of the Dual Process Model, which explains your system 1 thinking will be based on the first
piece of information given, the length that took to complete the painting. For example, there
are two groups, one group receives a length of 7 hours, and the other receives 3 months, we
hypothesize that as 3 months is a higher length, you will give a higher value. Thus the given
first piece of information is your anchor.

Your participation and responses will remain confidential. Your personal information will be
kept private, and the data collected will be used solely for research purposes

We appreciate your time and effort in participating in this study. Your contribution contributes
to our understanding of memory processes and learning strategies. Once again, thank you
for your cooperation.
 15

Appendix 3: Raw data

Condition 1: Low-Effort (7 hours)

Participant Rating of Quality out of 10

1 7

2 3

3 6

4 4

5 6

6 4

7 5

8 5

9 6

10 7

Mean 5.3

Standard Deviation 1.34

Condition 2: High-Effort (3 months)

Participant Rating of Quality out of 10

1 7

2 8

3 8

4 6

5 7

6 8

7 7

8 9

9 7

10 7

Mean 7.4

Standard Deviation 0.84

 16

Appendix 4: Questionnaire
 17

Condition 1 with low effort heuristic (7 hours)

 18

Condition 2 with high effort heuristic (4 hours)

Appendix 5: Piece of Art

 19

Appendix 6: Inferential Statistics | Calculate from http://vassarstats.net

 20

Appendix 7: The Bell Curve of Normal Distribution