Bui Thanh Huong
Bui Thanh Huong
Bui Thanh Huong
GRADUATION PAPER
HÀ NỘI – 2018
ĐẠI HỌC QUỐC GIA HÀ NỘI
TRƯỜNG ĐẠI HỌC NGOẠI NGỮ
KHOA SƯ PHẠM TIẾNG ANH
HÀ NỘI – 2018
DECLARATION
I hereby state that I: Bùi Thanh Hường, being a candidate for the degree of
Bachelor of Arts (English teaching Education) accept the requirements of the
College relating to the retention and use of Bachelor’s Graduation Paper deposited
in the library.
In terms of these conditions, I agree that the origin of my paper deposited in the
library should be accessible for the purposes of study and research, in accordance
with the normal conditions established by the librarian for the care, loan or
reproduction of the paper.
Signature
……………………………
May 2018
ACKNOWLEDGEMENTS
i
ABSTRACT
With the introduction of the Project 2020, students at most universities in
Vietnam have to pass the graduation proficiency assessment. The understandings of
the test effects on learning and teaching, or washback, are essential for the teachers,
students and test administrators to gain desirable test outcomes. However, little is
known about test’s washback, particularly washback on students in Vietnam. This
study, hence, investigate the perception of students of a graduation proficiency
assessment and the washback of the test on students’s test preparation strategies
through questionnaires and interviews. The results show that students have positive
perceptions towards the graduation proficiency assessment but the test exerts little
effect on their test preparation. Although students of different years share the
optimistic attitudes toward the test, they tend to prepare for the test differently.
While the seniors study more test-like materials, the freshmen are likely to study
general English tasks.
ii
Table of contents
ACKNOWLEDGEMENTS ...................................................................................... i
ABSTRACT ............................................................................................................. ii
LIST OF ABBREVIATIONS .................................................................................. v
LIST OF FIGURES AND TABLES ....................................................................... vi
CHAPER I: INTRODUCTION ................................................................................ 1
1. Statement of research problem & questions ..................................................... 1
2. Scope of research .............................................................................................. 2
3. Significance ....................................................................................................... 2
4. Design of the study ........................................................................................... 2
CHAPTER 2: LITERATURE REVIEW ................................................................. 4
2.1. Issues in language assessment........................................................................ 4
2.2. Washback in language testing and assessment .............................................. 4
2.2.1. The definitions of washback .................................................................... 4
2.2.2. The nature of washback ........................................................................... 6
2.2.3. Theoretical framework of washback ....................................................... 7
2.3. English language proficiency tests ............................................................... 11
2.3.1. Communicative language proficiency ................................................... 11
2.3.2. Major proficiency tests .......................................................................... 11
CHAPTER 3: METHODOLOGY .......................................................................... 16
3.1. Context ......................................................................................................... 16
3.2. Research questions ....................................................................................... 16
3.3. Participants and selection of participants ..................................................... 17
3.4. Data collection method ................................................................................ 18
3.4.1. Questionnaire ......................................................................................... 18
3.4.2. Interview ................................................................................................ 20
3.5. Data collection procedure ............................................................................ 21
3.6. Data analysis ................................................................................................ 23
CHAPTER 4: RESULTS AND DISCUSSION ..................................................... 26
4.1. Students’ attitude toward the graduation proficiency assessment (GPA).... 26
iii
4.1.1. Students’ decision to take the GPA ....................................................... 26
4.1.2. Whether the test would motivate students to improve their English ..... 27
4.1.3. Whether the test would raise students’ English proficiency ................. 29
4.1.4. Whether the test would measure students’ proficiency accurately ....... 30
4.1.5. Whether the test’s results would be useful in supporting job application
......................................................................................................................... 32
4.1.6. Test anxiety ............................................................................................ 34
4.2. GPA washback on students test preparation for the graduation proficiency
assessment ........................................................................................................... 36
4.2.1. The general English studies ................................................................... 36
4.2.2. Test-specific English activities .............................................................. 38
PART V: CONCLUSION ...................................................................................... 48
5.1. Summary of the findings and discussion ..................................................... 48
5.1.1. Students attitude toward the test ............................................................ 48
5.1.2. Washback on students’ test preparation ................................................ 49
5.2. Conclusion and implication ........................................................................ 50
5.3. Recommendations ........................................................................................ 50
5.4. Limitations of the study ............................................................................... 51
5.5. Suggestions for further research .................................................................. 51
REFERENCES ....................................................................................................... 52
APPENDIX ............................................................................................................ 56
Appendix 1 .......................................................................................................... 56
Appendix 2 .......................................................................................................... 61
Appendix 3 .......................................................................................................... 66
Appendix 4 .......................................................................................................... 68
Appendix 5 .......................................................................................................... 70
iv
LIST OF ABBREVIATIONS
FYSY First year and second
GPA Graduation Proficiency Assessment
MCQs Multiple choice questions
MoET Ministry of Education and Training
SPSS Statistical Package for the Social Sciences
TOEFL Test Of English as a Foreign Language
TYLY Third year and last year
VNUH Vietnam National University Hanoi
VSTEP Vietnam Standardized Testing of English Proficiency
v
LIST OF FIGURES AND TABLES
Figure 2.1. A basic model of washback (Hughes, 1993) ......................................... 8
Figure 2.2. A washback model of students’ learning (Shih, 2007, p.151) ............. 10
Figure 3.3. Interview participants ........................................................................ 22
Figure 4.1. Reasons to take the GPA...................................................................... 27
Figure 4.2. Students’ perception toward test motivation ....................................... 28
Figure 4.3. Students’ perception toward test function of raising English
proficiency .............................................................................................................. 29
Figure 4.4. Students’ perception of the GPA’s accuracy ....................................... 31
Figure 4.5. Students’ perception toward the test usefulness .................................. 33
Figure 4.6. Respondents’ test anxiety .................................................................... 35
Figure 4.7. Students general English learning activities ........................................ 37
Figure 4.11. Time to prepare before the test .......................................................... 40
Figure 4.14. Test preparation strategies ................................................................. 43
Figure 4.15. Time for each test preparation activity .............................................. 45
Figure 4.16. Difference between two groups ......................................................... 46
vi
CHAPER I: INTRODUCTION
1. Statement of research problem & questions
The impacts of language assessment, in other words, test washback, have
seen an increased interest in the last few decades (Alderson & Wall, 1993;
Bailey, 1996; Hughes, 1989; Caine, 2005). The research contexts spread from
Canada, Brazil, Germany, and Greece to Japan, China, and Hong Kong. A large
number of studies have emerged, especially on large scale tests such as TOEFL
and IELTS (Alderson & Hamp-Lyons, 1996 and Green, 2007), Hong Kong
Certificate of Education Exam in English (HKCEE) (Cheng, 2004), General
English Proficiency Test (Pan & Newsfields, 2012) . The findings of the degree
of washback, however, vary overtime; while some scholars concluded that there
was almost no washback in their studies (Watanabe, 2014; Shih, 2007), others
stated the strong effect of washback (Ferman, 2004). Thus, it can be indicated
that the washback might be different in each context.
In context of Vietnam, in recent years, the status of foreign languages is
considerably heightened thanks to the implementation of National foreign
language 2020 project, which requires that “by 2020, most Vietnamese students
graduating from secondary, vocational schools, colleges and universities will be
able to use a foreign language confidently in their daily communication, their
study and work (…)” (Prime Minister, 2008). Therefore, universities and
colleges have to select a Graduation Proficiency Assessment (GPA) for their
students. One of the tests used for university exit is known as “Vietnam
standardized test of English proficiency (VSTEP), a high-stake test which is
likely to exert certain impact on test users (e.g. students, teachers and
practitioners). However, there are a few studies of GPA or VSTEP washback on
students, particularly the non-English majored ones.
To fill such gap, this study aims to investigate the washback on non-
English majored students’ preparation strategies for the graduation proficiency
assessment (GPA) and their attitudes toward the test in VNUH, particularly at
the School of Law.
1
In brief, this study will address the following questions:
1. What are students’ perceptions toward the English graduation proficiency
assessment?
2. To what extent does the graduation proficiency assessment influence
students’ preparation strategies(from students’ perspective)?
3. Are there differences in the test preparation strategies of students in
different years?
2. Scope of research
This research targets students at the School of Law, Vietnam National
University, Hanoi (VNUH). To graduate from this school, all students are
required to pass the GPA level of B1 or B2; or in other words, they have to pass
the level 3 or level 4 of Vietnamese Standardized Test of English Proficiency
(VSTEP). The alternatives namely TOEFL, TOEIC, Cambridge Test, or IELTS
certificates are also accepted. Moreover, students can make decision to take any
examination as long as they have an equivalent level 3 or 4 VSTEP certificate
which satisfies the language graduation requirement.
For time limited, this study will only look at the washback on students’
preparation for the test and their attitude toward the GPA.
3. Significance
Despite being conducted at a small scale, not to be generalized, the study
would partly fill the literature gap of washback on students. By targeting non-
English major students, the research may raise their awareness of different test
preparation strategies as a good source for better learning. Simultaneously,
teachers, practitioners and other stake holders might have a better understanding
of their students’ attitude and learning in order to enhance the quality of the tests,
teaching and learning activities. This study would also be useful for other
researchers concerning language assessment, particularly washback area.
2
Chapter 1: Introduction – presents the rationale, aims research questions,
significance, scope, and design of the study
Chapter 2: Literature review – discusses the theoretical framework and studies
of washback and English language proficiency tests
Chapter 3: Methodology – demonstrates the context and research design of the
study
Chapter 4: Findings and discussion – shows the comprehensive of data and a
discussion on the findings of the study
Chapter 5: Conclusion – provides a summary of the findings, conclusion,
recommendation, limitations, and future direction for the study.
3
CHAPTER 2: LITERATURE REVIEW
2.1. Issues in language assessment
Historically, language testing has evolved and expanded over 30 years.
The 1980s witnessed a shift from language tests which focused on discrete-point
format to the communicative language testing; besides, the field also expanded
to Second Language Acquisition areas. The next decade carried on the
expansions in numerous areas namely “research methodology”, “practical
advances”, “factors that affect performance on language test”, authenticity,
“ethical issues and consequences of test use.” (Bachman, 200, p.4). Of all those
issues, the study of washback, a facet of test consequences, until now, has still
been discussed worldwide for its importance. According to some researchers, the
test impact could “governs and determines people future’s education” (Shohamy,
2001 as cited in Brown & Abeywickrama, 2010, p.22). Due to the considerable
power and authority of the tests, many policy-makers may use them to administer
the educational system or promote particular behaviors (Shohamy et al., 1996).
2.2. Washback in language testing and assessment
2.2.1. The definitions of washback
Testing has been utilized in education and employment area for such a
long time for a multitude of purposes namely measuring test-taker’s proficiency,
playing the gatekeepers’ roles, or motivating students. On account of the
powerful role of test in different fields, there is a notion that testing has an
influence on learning and teaching. Before 1990s, the common terms in language
testing to refer to the belief about the testing and teaching/ learning relationship
were “curricular alignment” (Linn, 1983), which refers to the relationship of the
testing content and the designed curriculum; “measurement-driven instruction”
(Popham, 1987), which holds the view that “testing should drive curriculum and
thereby teaching/ learning”; “systemic validity” (Frederiksen & Collins, 1989),
which refers to changes in curriculum and instruction to “foster the development
of the cognitive skills that the test is designed to measure (Frederiksen & Collins,
1989, p.27). In 1993, for the first time, Alderson and Wall official introduced the
4
term “washback” with proven evidence from empirical studies. Since then, more
attention has been paid to washback with different definitions.
Washback or backwash in language assessment is generally known as
“the effect of testing on teaching and learning” (Hughes, 1989, p.1). Despite the
prevalent of washback in applied linguistic field, the term is hardly seen in
dictionaries. However, some dictionaries as the New Webster’s Comprehensive
Dictionary and the Collin Cobuild Dictionary can be found including backwash
which is defined as “the unwelcome repercussions of some social action” and
“unpleasant after-effect of an event or situation” respectively. These definitions
consist of an interestingly negative connotation which refers to the adverse
relationship between testing and teaching/ learning discussed later. (Cheng et al.,
2004). Alderson & Wall (1993) assume that washback renders “teachers and
learners to do things they would not necessarily otherwise do” (p.117). Messick
(1996) refers to washback as “the extent to which the introduction and the use of
a test influences language and teachers to do things they would not otherwise do
that promote or inhibit language learning” (p.241). “The connection between
testing and learning” is how Shohamy et al. (1996) defines the term.
In terms of scope, washback is divided into narrow and broad scope. In
the narrow scope, washback is more frequently refer to the effect of the test on
teaching and learning. Both Alderson and Wall (1993), Bailey (1996) and
Messick (1996) agree that “teachers and learners do things they would not
necessarily otherwise do because of the test (Alderson and Wall, 1993, p. 117).
In the broad scope, washback is just considered as a facet of consequential
validity which “encompasses all the consequence of the test”, covering the
concerns of its accurate intended-criteria measurement, its influence on the test
preparation; and its “social consequences of a test’s interpretation and use”
(Brown & Abeywickrama, 2010, p.34). Bachman and Palmer (1996) utilizes the
term impact to refer to consequential validity, possibly “more broadly
encompassing the many consequences of the assessment prior and after a test
administration” (p.34). Therefore, washback is considered as one dimension of
impact (Hamp-Lyons, 1997). For illustration, Bachman and Palmer (1996)
5
consider test impact into two levels: micro- and macro- levels. The impacts on
individuals namely test-takers or teachers are at the micro level. At the macro
level, the test can affect the whole educational system and society.
In this study, the interpretation of “washback” in narrow scope will be
adopted: washback at the micro level which is the effects of the test on
individuals. More specifically, the effect of VSTEP on students in the aspects of
test preparation and attitudes.
2.2.2. The nature of washback
Regarding the nature of washback of a test, most studies mention the
dimension of value such as positive or negative. However, according to
Watanabe (2000), there are four other dimensions namely specificity (general or
specific), intensity (strong or weak), length (long or short) and intentionality
(intended or unintended) (Cheng at al., 2004, p.20). In this minor thesis, only the
value and specificity of washback are discussed.
2.2.2.1. Positive versus negative
Alderson & Wall (1993) considers washback as a neutral term which
could be beneficial or negative. The positive washback would have beneficial
impacts such as promoting teaching learning activities, encouraging positive
attitudes toward the test and improving motivation (Alderson & Wall, 1993).
Shohamy (1993) also states that the test has positive washback if it forces
students to learn more such as listening more carefully or taking the study more
seriously. On the other hand, negative washback refers to the test anxiety and the
fear of poor performance; hence, focus on excessively on the skills tested
(Tsagari, 2011). Similarly, teachers might be afraid of the test results that they
may just focus on the test or narrow the curriculums. Another feature of negative
washback is that students review their notebooks and find books which are
related to the test (Damankesh & Babaii, 2015). In short, the positive washback
encourages the derisable changes while the negative washback bring undesirable
ones.
6
2.2.2.2. General versus specific
This dimension of washback indicates that the washback may be general
or specific. The general impact may be produced by any tests (i.e. students study
harder), whereas, if the washback is specific, it can be seen in only one aspect or
a type of test. Didi & Ridha (2011) state that the English national Examination
(ENE) has specific washback because the teachers focused more on reading
skills which were greater part of the test rather than communication skills.
7
Figure 2.1. A basic model of washback (Hughes, 1993)
Despite showing the clear relation among the test and others elements
namely participants, process and products, the model does not include different
aspects affecting learning and teaching process.
Basing on model of Hughes (1993), especially participants and product,
Bailey (1996) construct his model comprising of two parts ’washback to the
learners’ which refer to “the effects of test-derived information on test-takers and
and “washback to the programme” which are washback to “teachers,
administrators, curriculum developers, counselors, etc.,”. (p.264)
Some examples of the “wash aback to the learners” are suggested by
Bailey (1993) as following:
1) Practicing items similar in format to those on the test.
2) Studying vocabulary and grammar rules.
3) Participating in interactive language practice (e.g., target language
conversations).
4) Reading widely in the target language.
5) Listening to non-interactive language (radio, television, etc.).
6) Applying test-taking strategies.
8
7) Enrolling in test-preparation courses.
8) Requesting guidance in their studying and feedback on their
performance.
9) Enrolling in, requesting or demanding additional (unscheduled)
test-preparation classes or tutorials (in addition to or in lieu of
other language classes).
10) Skipping language classes to study for the test.
(Bailey, 1996, p.264)
Nevertheless, the model does not indicate how different individual is
affected by the test. Alderson and Wall (1993)’s model which is considered more
in details (Pan, 2014) proposed the “Washback Hypothesis” taking a different
approach. There are “15 possible hypotheses regarding washback” as following:
1) A test will influence teaching.
2) A test will influence learning.
3) A test will influence what teachers teach; and
4) A test will influence how teachers teach; and therefore, by extension
from (2) above:
6) A test will influence how learners learn.
7) A test will influence the rate and sequence of teaching; and
8) A test will influence the rate and sequence of learning.
9) A test will influence the degree and depth of teaching; and
10) A test will influence the degree and depth of learning.
11) A test will influence attitudes to the content, method, etc., of
teaching and learning.
12) Tests that have important consequences will have washback;
and conversely
13) Tests that do not have important consequences will have no
washback.
14) Tests will have washback on all learners and teachers.
15) Tests will have washback effects for some learners and some
teachers, but not for others.
(Alderson and Wall, 1993, p. 120-121)
The washback on students are now demonstrated in more aspects namely
how and what learners learn, the rate, sequence, degree, and depth of learning
and the attitude to content and method of teaching.
Shih’s study (2007) built a new and more detailed model on washback on
students with a thought that the previous models are not suitable any more. The
model shows the variety of test components which might exert impacts, the
different aspects on students’ learning and psychology and other factors such as
extrinsic and intrinsic ones. As can be seen from the chart, washback of the test
9
on students learning includes content of learning, total time on learning, learning
strategies, learning motivation and test anxiety. However, how each aspect of
learning as mentioned is affected by the test is not demonstrated.
10
In this research, the model of Shih (2007) is adopted as it is more students
focus. Five aspects of washback on students learning and psychology are also
concurred by Ferman (2004) (cited in Cheng at al., 2004)
2.3. English language proficiency tests
To study washback, different examinations are investigated. As most of
them are language proficiency tests, the following part concentrates explaining
this test type and give some typical examples.
2.3.1. Communicative language proficiency
Communicative Language Testing (CLT) is an original approach toward
language testing, which focuses on the authenticity and communicative purposes
of the test. Proficiency test “is designed to measure people’s ability in a
language, regardless of any training they may have had in that language” (Brown
and Abeywickrama, 2010, p.11). Besides, it is also stressed that the test is not
restricted to any course or skills. Therefore, a communicative language
proficiency test examines the overall English of test-takers for communicative
purposes.
Most communicative language proficiency tests have summative results
which present a single score with 2-3 sub scores. These sub scores are less likely
to give any feedback or serve diagnosis function; hence, students can only
interpreting their scores by looking at the rubrics.
11
employed to decide the international students’ ability of English in academic
context.
The table below is the demonstration of the TOEFL iBT test structure.
The test includes four sections Reading, Listening, Speaking and Writing allotted
in about four hours. Each skill makes up 30 scores to set the total score of 120.
Detailed information of each skill follows.
Table 2.3. The structure of the TOEFL iBT Test
Section Number of Testing time Score Scale
items/ tasks
Reading 36–70 60–100 minutes 0-30
Listening 34–51 60–90 minutes 0-30
Break 10 minutes
Speaking 6 tasks 20 minutes 0-30
Writing 2 tasks 50 minutes 0-30
Total Approximately 4 0 - 120
hours
13
Shohamy et al.(1996) investigate the washback of two different tests
namely ASL and EFL from perspectives of teachers, students and language
inspectors. The results show two different pictures. While the ASL has almost
no effect on the teaching and learning activities, test preparation and time
allotment, the EFL has tremendous washback. Although the teachers show
negative attitudes toward the two tests’ quality, the tests are still in need for
learning promotion.
Pan (2014) examines the washback of TOEIC & GEPT as graduation
requirements by doing survey, asking question and observing two groups
including non-exist requirement students and exist requirement students. When
looking at the aspect of degree/ depth of learning, attitude toward methods of
learning and some learners but not others being influenced” (Alderson & Wall,
1993), Pan divides the questionnaire into three parts types of test-preparation
activities, type of language skill-building activities, viewpoint of the GEPT and
TOEIC. The results show that although the performance and motivation of the
exist requirement group higher, the learning methods of two group are not
significantly different.
Damankesh & Babaii (2015) investigate a highschool final examination
and explore the washback on students’ test-taking and test-preparation strategies.
The results show that the examination influences students’ learning behaviors by
getting them take certain types of test-taking and test-preparation strategies. The
washback is partly negative as some strategies have adverse effect on students’
creativity and inhibit their learning. On the other hand, the test has some slightly
positive influences which foster the students’ cognition and attention.
In 2017, Green investigates the effectiveness of IELTS preparation class
on students’ writing scores. Students had to take two writing tests prior and after
the preparation course. In addition to the test instrument, there were two
questionnaires to gain data of participants’ different background and the “process
and outcome variables.” It was concluded that the test preparation brought no
apparent benefit to students’ test scores. Another study on TOEFL (Alderson &
14
Hamp-Lyons, 1996), however, brings an opposite result that the test had
influence on both what and how teachers teach.
Different communicative proficiency tests have been studies all over the
world from international levels such as IELTS and TOFEL, national levels such
as ASL, EFL and GEPT to the local tests such as highschool examination. The
results are varied among the studies. At the same scale, while TOFEL had
washback on teachers on both how and what they teach, the IELTS preparation
did not have apparent washback. Therefore, the test washback might not be
affected by the test nature only. In order to understand washback, specific
contexts need investigating. In the context of Vietnam, nevertheless, there are
only few studies.
2.4.1. The studies of washback in Vietnam
In Vietnam, there are three studies of washback, only one of which is
about the VSTEP, the newly designed test in Vietnam. The other two studies
international examinations such as TOEFL (Nguyen , 1997 & Barnes, 2016)
Nguyen (2017) study the washback of VSTEP on first year students at
ULIS, VNUH. The two aspects of behaviors and attitudes were examine by likert
scale questionnaires. It is found that the test had strong washback on learning
content, learning methods and affective conditions.
However, no study of washback in Vietnam has focused on non-English
majored students.
Chapter summary
This chapter has reviewed the language testing context and given an
insight view about washback definition, nature and different models. This study
adopts the narrow definition of washback, studies washback value and specificity
and applies the model of Shih (2007). Two typical language proficiency tests
were introduced followed by a number of studies worldwide and in Vietnam.
There are a multitude of washback research on teachers but few studies have
studied the washback on students particularly in Vietnam context.
15
CHAPTER 3: METHODOLOGY
3.1. Context
In Vietnam, the status of English has recently heightened thanks to the
Project 2020 by Prime Minister (2008). To achieve the goals that most
Vietnamese when graduating from universities and colleges could use English
in Dailey conversation, the VSTEP 5 levels was issued. Most university students
have to pass certain levels of the test in order to graduate from schools. Other
compatible international certificates such as IELTS, TOEFL, TOEIC are also
accepted.
The School of Law, VNUH, also adopts the VSTEP as the graduation
proficiency assessment (GPA). The VSTEP level 3 or the alternative
international tests is the precondition for students to graduate from the school.
Only fast-track students have to achieve the level 4. As the certificates are valid
in two years, most students consider taking the test in the second half of their
students’ life. It should also be noted that these students can take the test at any
time they find convenient.
3.2. Research questions
This study examines 3 research questions:
1. What are students’ perceptions toward the English graduation proficiency
assessment?
By answering this question, the researcher penetrates how students
perceive the test and if they are negatively or positively influenced. Their
perceptions towards the GPA may also explain their behaviors.
2. To what extent does the graduation proficiency assessment influence
students’ preparation strategies (from students’ perspective)?
This question aims to investigate students’ reflection/perspectives
regarding their behavior during the preparation phase to see whether the GPA
exerts washback on them, whether those impacts are strong or week, negative or
positive, general or specific.
16
3. Are there differences in the test preparation strategies of students in different
years?
The question is raised to explore whether the first-year students and senior
study English differently for the GPA.
3.3. Participants and selection of participants
The population size is approximately 1400 students including around 700
first-year and second year students and roughly 700 third-year and fourth-year
students at the School of Law, VNUH.
Participants are students at the School of Law, VNUH. They are not major
in English nor studying English for Special Purposes. The School of Law offers
three General English courses from level A1 to B1 for mainstream students and
one extra B2 course for students in fast-track division. However, students can
choose to take the courses or not at any stage of their university life. While
English course taking is flexible, the graduation proficiency assessment is
compulsory. Students at mainstream program have to reach B1 level by the time
they graduate and it is particularly B2 level for fast-track students. The School
identifies students’ levels by VSTEP certificates or other international
certificates namely TOEIC, IELTS and TOEFL. As a result, students have to
take the tests before graduating. First-year and second year students are newbies
to the School; thus, they still have a plenty of time ahead to prepare for the test.
In comparison, the third-year and four-year students are those who have limited
time for test preparation.
Regarding sampling design, the study involves stratification of the
population before selecting the sample. The population is divided into two strata
based on the school year with the characteristics briefly described above. The
first group comprises of the first and second year students while the third and
fourth year students are in the other stratum. The ratio of two groups is roughly
1:1. For each stratum, cluster sampling is used. The law students study in
different classes in Hall P1 and P2 in G Building, hence, some classes will be
selected randomly for data collection.
17
The sample size was 123 students considering the population size 1400
students, confidence level 95%, confidence interval ±8.
In 123 responses, 82.9% students (102 students) report to take or have
taken the GPA at school while 17.1% consider to sit alternative proficiency tests
namely IELTS and TOEIC. Those who do not take the GPA will be excluded
from the latter half of the study which examines students’ test preparation
behavior. In 102 students responding to take the test, there are 57 first year and
second year students making up 55.9% and 45 third and fourth year students
accounting for 44.1%.
18
The face-to-face questionnaire is chosen as its advantages in
administration. Face-to-face questionnaire gives the researcher to meet
participants in person to explain the purpose of the study as well as to answer
any pop-up questions. Besides, a higher rate of response will be guaranteed in
comparison with online and mail questionnaires.
The questionnaire is adapted from Stoneman (2005). Questions related
to students’ self-rating and perspective toward the test, respondents’ out of class
activities and test preparation are raised in the survey. The questionnaire
comprises of four parts as following:
Part 1 asks for demographic information. Questions about the name,
phone, and email are not numbered and students only need to leave information
if they are willing to join the interview round. Question 1’s purpose is to find out
whether students take the GPA or not. All the questions done by non-GPA test-
takers will be excluded. As the purpose of this paper is to find the washback of
GPA, if students take other tests, it is unlikely that they have any influences.
Question 2 collects students’ school year which helps the researcher to divide
the participants into two groups as presented above in order to answer research
question 3.
Part 2 helps to collect washback on attitudinal aspects. There are 7
questions from question 3 to 9. Question 3 asks for the reasons why students
choose GPA but not other tests. The next two questions are students’ self-rating
of their English proficiency and test anxiety which might explain students test
preparation later. The next four questions seek for students’ perspectives of GPA
as a source of motivation, as a means to raise students’ standard, as a tool to
measure students English proficiency correctly and a support for job application.
Part 3 includes 2 questions to find out students’ learning activities outside
classes; to be more specific whether students learn English by watching the TV
program, listening to music, or speaking with native speakers. The second
question aims at examining the average time students spend on doing those
activities weekly
19
Part 4: includes 4 questions which are to investigate washback on
students’ behavioral and attitudinal aspects.
Table 3.2. Questionnaire Items
Part Content Item
Part 1: - demographic information (name, phone,
Background and email)
information - GPA test-taking decision Q1
- students’ school year Q2
Part 2 - reasons to take the GPA (believes toward Q3
Attitudinal aspects the test)
- self-rating proficiency Q4
- respondents’ level of test anxiety Q5
3.4.2. Interview
As participants cannot be directly observed, interview is a reasonable
method to collect participants’ historical information (Creswell, 2014) such as
the process that students prepare for the test.
Semi-structured interview is also utilized because there are disadvantages
in the use of questionnaires. Firstly, it is argued to be insufficient in
20
understanding some information (i.e. behaviors, feelings). Secondly, the
questionnaire includes limited questions developed from a fixed scheme, so they
may have some missing important information. In order to overcome these
weaknesses, semi-structured interview is put into use.
The interview questions are aimed not to draw generalization but to
provide essentially qualitative explanations for certain findings in the
questionnaire. The interview scheme is divided into 3 parts. Part 1 – identify
students test taking status and the reasons they choose this examination. Part 2
asks questions about students’ perspectives toward the GPA and why they hold
those opinions so as to answer Research question 1. Students perspectives toward
test importance, test results, test motivation, test accuracy and test anxiety, as
well as their motives, are collected in this part. Last but not least, behavioral
aspects of washback such as time allotment, learning strategies, content, and
materials are covered in part 3.
3.5. Data collection procedure
Step 1: Designing the survey questionnaire and interview questions
Both questionnaire and interview questions are designed based on
washback theory on students’ test preparation and modified from previous
studies. While the questionnaire depends heavily on Stoneman’s paper
(2005), the question list is adapted from both Stoneman (2005) and Shih
(2007).
As the participants are not major English, the questionnaire and interview
questions are translated into Vietnamese. By doing that, not only is
misunderstanding, or misinterpreting shunned, but also the respondents are able
to express themselves more precisely and thoughtfully.
Step 2: Piloting and revising instruments
The questionnaire was piloted by 7 students as presented in chart
below.
21
English
1 student
majored
Piloting 4th
year students VSTEP test-
3 students
taker
Non English
major
Non VSTEP
test-taker 3 students
22
answer pop-up questions. However, some queries were still clarified while
the researcher was in the class.
Interview
20 questionnaire respondents who provided the phone numbers were
contacted. Only 10 of them replied but three students are not available; hence,
there were 7 participants for the interviews.
Two participants were interviewed face to face; the conversations were
noted down and recorded. The remaining were interviewed via the Internet
with the help of Facebook video call, no recording tape was provided in the
application so the research made attempt to take as much note as possible.
Step 4: Collecting, quantifying and qualify data
Quantitative data
Quantitative data was collected and analyzed by Anova to answer to
research question 3, the descriptive statistics and frequency were applied to
answer question 1 and 2.
Qualitative data
Data was reviewed and coded. Each participant was coded as
GPA_number (i.e. GPA_1)
3.6. Data analysis
Step 1: Processing quantitative and qualitative data
Firstly, the researcher codes data. For the questionnaire, data were coded
in SPSS program. Regarding the interview, data were put in different themes.
Secondly, the internal reliability of the questionnaire was checked by
using Cronbach's alpha estimation in SPSS program. However, the unordered
categorical data such as the school year or reasons to take the GPA were not
examined by this software. As the purposes and type of questions in each part
were different, the internal reliability testing of each part was processed
separately. The result is shown as below.
Part 1: Students’ perspectives toward the GPA
23
Table 3.4. Reliability Statistics part 1
Reliability Statistics
.654 .659 6
Part 2: Students’ general English activities (this part filters the cases of non-
GPA test-takers)
Table 3.5. Reliability Statistics part 2
Reliability Statistics
.633 .644 16
The open-ended question “How many hours do you spend doing those
activities?” was excluded from the test, nearly a quarter of respondents do not
give answer and the answers vary wildly.
Part 3: Students’ specific test preparation (this part filters the cases of non-
GPA test-takers)
Table 3.6. Reliability Statistics part 3
Reliability Statistics
.704 .588 15
At all parts, the Cronbach’s Alpha is larger than 0.6, which indicates that
the questionnaire is acceptable (Stephanie, 2014). The statistics shows that the
questionnaire was sufficiently reliable for analysis.
Step 2: Interpreting quantitative and qualitative data
Items are categorized following related questions
- Research question 1
24
- Research question 2 and question 3
To answer question 1 and 2, various types of questions were employed in the
questionnaire including multiple choice questions (MCQs), rating scales and
open-ended questions. The frequency of each option was calculated and
presented in percentage terms and/ or numerical terms. Data were then
demonstrated by means of tables, pie charts and bar chart.
The research question 3 is to identify the differences between the group of
novice students and the seniors; hence, one-way ANOVA test was applied. The
extent of the difference is decided by ANOVA Sig. of which the value is 0 < Sig.
< 1. If ANOVA Sig. < 0.05 or Sig. = 0.05, there is a difference and vice verse.
Phạm (2016), in his article, presented in great details how to find ANOVA Sig.
and how to interpret it.
Step 3: Finalizing the report
The data which were computed and illustrated in the previous steps were
interpreted. By analyzing data, students’ perspective towards the test and their
reflection toward test preparation were discovered.. Besides, the explanations for
those views might be revealed from interviews with students. Additionally, by
comparing the test preparation activities of the respondents, the differences
between two groups were identified.
Chapter summary
This chapter described the methodology applied in this study with the
specific context at the beginning. A mixed method using both survey and
interview was employed in order to get both the overall trend and deep
understandings of the issues. The sample was first stratified, then the
questionnaires were delivered to each stratum following clustering samples.
There were 123 responses in which 7 respondents took part in the interview
round. The procedure of collecting, processing and analyzing data were also
thoroughly presented.
25
CHAPTER 4: RESULTS AND DISCUSSION
4.1. Students’ attitude toward the graduation proficiency assessment
(GPA)
This section presents the results on students’ perspectives toward the
GPA and their test anxiety based on the questionnaire. Additionally, further
findings from interview data are discussed. As described in the methodology
chapter, research question 1 “What are students’ attitude toward the graduation
proficiency assessment?” can be answered by six questionnaire items:
- Why do you take the VSTEP instead of IELTS, TOEFL, TOEIC or other
compatible tests as the graduation test? (Q3)
- Do you think the graduation proficiency assessment (GPA) will motivate
students to improve their English? (Q6)
- Do you think the GPA will raise the proficiency of English of university
students? (Q7)
- Do you think GPA will measure students’ English proficiency accurately? (Q8)
- Do you think the GPA test result will be useful in helping you find a job? (Q9)
- How do you feel about having to take the GPA? (Q5)
The answers for these questions are presented below.
4.1.1. Students’ decision to take the GPA
Respondents stated their reasons to choose GPA by answering an open-
ended question. This type of question helps the scholar to collect numerous
26
opinions, but its disadvantage is the low response rate; in return, there were only
57 answers which are categorized into 7 groups as in figure 4.1 below.
Regarding the nature of the test, many respondents (31.2%) believed that
the GPA was the easiest proficiency test in comparison with TOFEL, IELTS,
and TOEIC. Besides, the GPA was perceived to be convenient for registration
and be at reasonable price for non-English majored students at the School of
Law.
25,0
20,0 14,0
15,0 12,3
10,0 7,0
5,0 1,0 1,0
0,0
Reasons
27
motivate students to improve their English?” (Q6). The finding shows that more
than a half of respondents (56.3%) consented to the motivating function/ impact
of the GPA, whereas just more than a fifth (22.7%) showed the opposite
opinions; the rest accounting for 21% had no opinion. This result indicates a
positive trend among the students of the Law School that the GPA would
motivate students to learn English. The subsequent interview gave some
explanations for this thinking.
No;
22,70%
Yes;
Neutral; 56,30%
21%
28
“I find the GPA not too challenging; besides, I am confident with my English
level, so I don’t spend much time to learn it” GPA_3
In short, most respondents found the test motivate them to improve their
English; whereas, some students considered the test unimportant and
unchallenge to stimulate learning.
4.1.3. Whether the test would raise students’ English proficiency
Respondents answered yes/ neutral/ no to the question “Do you think the
VSTEP will raise the proficiency of English of university students?” The table
below displays the distribution of students’ perspectives toward the raising
proficiency function of the test. Overall, the result shows enthusiastic responses.
63.1% participants answered positively while only 17.5% answered “no”.
17,5 Yes
Neural
19,4
63,1 No
It is quite surprising that most interviewees did not agree with the
findings. Many respondents including the low and higher English competence
ones claimed that their English level did not improve at all owing to the test.
“I find the test easy so I merely spend time for it. I think my English proficiency
does not raise at all.” GPA_3
“Getting B1 is quite hard for me so I go to an English center in which I am
provided with help by teachers who are familiar with the test format. I do not
learn the basic knowledge such as grammar, vocabulary, and skills but I am
given the templates to speak and write. However, they are unpractical as I cannot
speak or write for communication purposes. Therefore, I do not think the GPA
raise my competence at all.” GPA_4
29
Even though many students insisted on gaining no more knowledge from
learning for the test, throughout the interview, the researcher could spot their
minor improvements.
“In the speaking test, I understood most phrases and ideas that the teachers
said, but I could not speak fluently. When I got out of the room, numerous ideas
flashed through my mind. I regretted not telling them.” GPA_5
This student found herself know nothing in English when first entering
the university. If she could speak a little English even it was not natural, it should
be considered as one of her improvements.
In short, while the questionnaire data demonstrates a vast number of the
respondents perceive the test as a means to raise students English proficiency,
interview participants showed pessimistic views toward the issue. The cause for
this contrast might be that the interviewees did not see their progress in language
learning.
30
Perceive the test as an accuracy tool to
measure students' proficiency
Missing ; 9%
Yes; 30,10%
Yes
No
Neutral; 35% Neutral
Missing
No; 27%
31
“I think reading, listening, and writing tests are quite okay, but I am not
sure about speaking skills, the scores vary following the examiner.” GPA_2
“I don’t think multiple choice questions can assess students’ levels
accurately, as they can choose randomly.”
Others believed that the overall score did not reflect the test-taker English
proficiency.
“I know many people at my university who got scores equivalent to B1 or
even B2 level but cannot speak or write in English as the description of those
levels. For example, if she gets a B1 certificate, she should be able to talk about
common topics or have simple conversations, right? My friend just can greet like
hi, or goodbye.” GPA_3
“I don’t think my score reflects my current level at all. You know, I got
an extremely low score in listening and considerably high score in reading;
overall, I passed the GPA. I know my final score is the average score of four
skills, but I feel like I am not really qualified. My listening score is too low to B2
level. I think the test designers should give a restriction to each skill. For
example, all my skills have to above 3 or 4, I don’t know, to pass the B2 level.”
GPA_4
To summarize, the non-English major students did not evaluate whether
the GPA was accurate or not but gave their opinions toward the test. Although
the distribution seems to equal among “yes” and “no”, the students’ perspectives
on the test’s downside should be taken into serious consideration.
32
respondents outnumbered the opposite side 1.5 times. On that account, there is a
slight tendency that respondents perceived the test usefulness positively.
Yes
39,80%
39.9% No
26,80%
Neutral
Missing
25,20%
33
as IELTS, TOEFL or TOEIC rather than the GPA. Hence, I don’t consider the
GPA useful.” GPA_1
The interviewee also pointed out that many employers might be skeptical
about the test result.
“I cannot use the GPA result to apply for a job, as I don’t think the
employers would trust the test score.”
In summary, respondents tend to find the GPA beneficial for job
application. Although the test might be helpful for recognizing respondents’
English proficiency, not many recruiters would concern and accept this test. The
respondents suggested that the GPA should be more prevail and more accurate
to support students in finding jobs.
34
Test anxiety
40 36,6
35
30
25,2
22,8
Percent (%)
25
20
15
10 8,1
4,9
5 2,4
0
extremely fairly neutral fairly extremely missing data
worried worried confident confident
States of feeling
35
4.2. GPA washback on students test preparation for the graduation
proficiency assessment
Since the study aims to examine students who have taken the GPA, those
who sit for other tests were excluded. There were only 102 questionnaires
remaining. In studying the washback on students test preparation, The dimension
of general/ specific following the paper of Cheng et al. (2004) and (Stoneman,
2005) was applied
4.2.1. The general English studies
A multi-response question and an open-ended one were employed to
investigate (1) types of activities, and (2) the time spent on these activities of
respondents. These questions help to identify the general English activities of the
students.
The multi-response questions asked respondents to choose the activities
that they apply to study English such as watching English TV programs, listening
to English radio/ song, reading English books/ newspaper/ magazines, writing
diaries in English and Speaking English with teacher. This list of activities was
adapted from Stoneman’s (2005) questionnaire. In 102 responses, two
respondents stated to play games in order to study English, this information is
not shown in the chart.
As it can be seen from the chart, the three most common activities
are related to receptive skills such as listening to English radio/ songs (79.4%),
watching TV programs (63,7%) and learning English on the Internet (54.9%).
Productive skills activities including speaking and writing are less commonly
applied (<20%). The extend that each student take up general activities is
presented by how many ways or methods that each employs. The table below
show the number of English learning activities taken up by the respondents
ranking from one to eleven activities
36
General English learning activities
Writing diaries in English 4,9
Writing e-mail in English 5,9
Joining English speaking clubs 7,8
Write to English speaking friends 9,8
Speaking English with native speakers 13,7
Speaking English with teacher 19,6
Reading English books/ newspaper/… 23,5
Keeping a vocabulary notebook 28,4
Going to English tutorial schools 30,4
Studying English dictionaries 34,3
Learning English on the Internet 54,9
Watching English TV programs 63,7
Listening to English radio/ song 79,4
0 10 20 30 40 50 60 70 80 90
.
Table 4.9 below shows how many hours each week that students spend
time learning English. It should be noted that this table is not highly reliable but
just for reference since many students complaint that they could not remember
or count the exact how many hours they study English a week. As their major is
37
not English, they do not learn it continuously and constantly. As a result, many
blanks were left and many respondents may have made up a number to satisfy
the researcher. In 102 responses, solely 76 students stated the numbers of hours
they devoted to learning which ranges enormously from 0 to 40 hours. Although
in most cases, students claimed to study 2 hours weekly, the average number was
much higher at 5.9 hours. This figure is later compared with the statistic of
students’ test specific preparation in the next part.
Table 4.9. Numbers of hours per week for General English activities
Valid 76
N
Missing 26
Mean 5.90
Median 4.00
Mode 2
Std. Deviation 6.832
Minimum 0
Maximum 40
Sum 449
This finding goes in line with the study of Stoneman on the Hong Kong’
students out of class activities. (Stonean, 2006). Additionally, very few students
claimed to continue those activities when preparing for the test.
4.2.2. Test-specific English activities
The researcher asked 4 questionnaire items from question 12 to 15 to
investigate how students prepare for the test. (Appendix 2). Question 12 enquired
students about their preparation status particularly whether they have taken any
steps or not. The next three questions asked about the time, learning strategies
and the content when students study for the test either in the past, at the present
or in the future as they take the GPA at different times. Question 13 gave the
respondents three options regarding the number of hours they spend studying the
test per week: 0 -2 hours, 2 – 5 hours, and more than 5 hours. Students learning
strategies such as seek help from teacher, access the GPA’s website, study with
friends or self-study were investigated in question 14. Question 15 helped the
researcher identify to what extent the respondents studied the test-like materials.
38
4.2.2.1. Test preparation status
Regarding test preparation status, the chart 4.10. below compare if
students have prepared for the test between two groups. Overall, the third and
last year students had a clearer view of their provision with 54.8% “yes” answer
compared with 11.3% “yes” answer of the counterpart. A third of respondents in
the first and second year students group did not do any preparation while the
figure was only more than 40% for the other group. The finding is consistent
with the hypothesis that the first and second year students tend to have less
preparation than the third and fourth year students. The ANOVA test also
approved the difference (Welch Sig. = 0.00) between the two group: the former
group are unlikely to have taken the studying (mean = 1.87), while the latter tend
to have made several preparations (mean = 1.44).
80 75,5
70
60 54,8
50 42,9
Percent
40
30
20 11,3 13,2
10 2,4
0
Yes No No option
Test preparation status
First and second year students Third and four year students
39
60
51,4
50
Percent (%) 40
32,1
30
18,9 18,6
20 13,2 13,2
11,3 12,8
7,1
10
2,9 4,3 5,7 2,8
5,7
0 0
0
0 1 week 1 week - 1 month - 2 months 6 months more no option
or less 1 month 2 months -6 - 1 year than 1
months year
Time
First and second year students Third year and fourth year students
Missing 9 27 26.5
41
English activities could be used to study for the test. However, they all concured
that when the test was closer, more test specific activities would be employed.
4.2.3. Test preparation strategies
Question 14 contains four sub-questions which ask whether respondents
(a) seek for help from English teacher (b) access the GPA website (c) study in
group with friends (d) self-study to find out what students’ learning strategies
are. The number of learning strategies are counted and presented in the table
4.2.1_4 below. The choices range from 0 methods to maximum 4 methods while
2 is the most common selection. On average, the participants take up about 1.7
learning strategies (mean = 1.696; SD = 1.21).
Table 4.13. Number of learning strategies taken by respondents
Valid 102
N
Missing 0
Mean 1.6961
Median 2.0000
Mode 2.00
Std. Deviation 1.20879
Minimum .00
Maximum 4.00
Sum 173.00
25 1.0000
Percentiles 50 2.0000
75 3.0000
Regarding the test preparation strategies, the figure 4.2.2_4 below
illustrates the percentage of different methods taken by two group of students.
There is no remarkable distinction between two group according to ANOVA test
Sig. > 0.05).
42
160
140
120
Percent (%) 100 88,9
80
60 51,1
53,3
40
31,1 59,6
20 33,3
21,1 12,3
0
seek help from access the GPA study with friends self-study
teachers website
Test preparation strategies
First and second year students Third and last year students
44
Never Few
littletimes
time Many
muchtimes
time A lot of time
PRACTICE WRITING
ESSAYS 26,7 36 30,7 6,7
PRACTICE WRITING
LETTERS
28 38,7 30,7 2,7
PRACTICE SPEAKING
PART 3 FORMAT 30,7 38,7 30,7 0
PARCTICE SPEAKING
PART 2 FORMAT 33,8 40,5 24,3 1,4
PRACTICE SPEAKING
PART 1 FORMAT
10,7 40 41,3 8
PRACTICE LISTENING
MCQS 29,7 32,4 36,5 1,4
PRACTICE READING
MCQS
32 29 36 2,7
PRACTICE PAST
PAPER TES 34,7 41,7 16,7 6,9
“My friend told me to read and listen for MCQs in the books FCE,
practice writing letters and essays following the IELTS Trainer. Without her, I
don’t know what to study”. GPA_3
Other reason was that the respondents also spent time on learning general
English namely grammar and vocabulary
“Even though the test doesn’t include grammar and vocabulary tasks, I
still feel an urge to improve these areas so as to get a higher score.”
Regarding the difference between two group, the ANOVA test only
showed different means in “practice reading MCQs and practice speaking by
choosing one option of three”. However, the stacked bar 4.16 below compares
test preparation activities between first year & second year students (FYSY) and
45
third year & last year students (TYLY) showing the dissimilar tendency in most
activities.
120
100 2,8 0 5,1 2,9 0 8,3 7,7
11,1 16,7
80 22,2 11,1 25,7
46,2 30,6
38,9 53,8 51,3
60 33,3 31,4
50
40 33,3 47,2
20,5
20 41,7 44,4 40 33,3
27,8 20,5 20,5 13,9 7,7
0
Practice TYLY Practice TYLY Practice TYLY Practice TYLY
past reading listening speaking
paper MCQs MCQs part 1
test (FYSY) (FYSY) format
(FYSY) (FYSY)
Never Few
Littletimes
time Many
Muchtimes
time A lot of time
120
100 2,8 0 0 0 2,8 2,6 5,6 7,7
8,3 25
80 39,5 35,9 22,2 22,2
38,5 38,5
38,9
60 38,9 38,9 33,3
40 42,1 38,5 38,5 38,5
20 50 38,9
36,1 25,6 36,1
18,4 20,5 15,4
0
Parctice TYLY Practice TYLY Practice TYLY Practice TYLY
speaking speaking writing writing
part 2 part 3 letters essays
format format (FYSY) (FYSY)
(FYSY) (FYSY)
For the mentioned activities, there is an apparent trend that the first year
and second year students who “never” practiced the test approximately
outnumbered the counterpart. For instance, in practice speaking part 2, up to a
half of FYSY respondents never did it, the figure was only 18.4% for the TYLY
students. As a result, the time that the seniors did the test-like materials was over
that of the FYSY. To be more specific, while more FYSY students spent “little
time” for the test, the TYLY had a tendency to practice the test “much time”.
These findings indicate that the GPA has exert specific washback on the
test-takers; the closer the test is, the more specific test preparation activities.
However, the washback is just minimal since respondents do not make long-term
effort to prepare for the test; various TYLY students study for the test in two
46
months or less while most FYSY respondents even have no plan for the
provision. In addition, that students do not devote “a lot of time” for preparation
infers that the test do not induce a high degree of washback. This is consistent
with the study of Shih (2007), in which students prepare for the test but with little
effort; thus, he concludes that the General English Proficiency Test (GEPT) has
limited washback on students.
To conclude, students spend “little time” on the materials which are
similar to the test. Perhaps, they have limited understanding of the test format or
they are likely to learn the basic knowledge. There is different inclination
between two groups of students that the seniors make more effort to prepare for
the test. Consequently, the GPA has little washback on students’ test preparation.
Chapter summary
This chapter has reported and interpreted quantitative and qualitative data
to answer three research questions. Research question one is answered by
analyzing 6 attitudinal questionnaire items. The findings show that in general,
students have positive thinkings toward the test. For research question 2 and 3,
respondents who has no intention to take the GPA is excluded from the study as
the aim of these questions are identify students’ test preparation activities and
the difference between the freshmen and seniors; six behavioral questions are
studied. Quantitative data are presented in tables, pie charts and bar charts for
analysis; then the findings are further described by interpreting qualitative data.
47
PART V: CONCLUSION
The previous chapter has discussed the findings and interpretation
from quantitative and qualitative data to answer three research questions.
This conclusion chapter is going to summarize the main findings, evaluate
the study and give some recommendations for the students and teachers at
the researched school. Besides, the limitation of the study and some
suggestions for further studies are also put forward.
5.1. Summary of the findings and discussion
5.1.1. Students attitude toward the test
Most students at the School of Law take the GPA before graduating
instead of registering other proficiency tests for several reasons. Firstly, the test
is considered undemanding, convenient and economical to take. Besides, some
respondents have little knowledge of the school policy and reckon the test as a
compulsory one.
In general, most respondents have positive thinking toward the GPA.
Regarding the test as a tool to motivate and raise students’ English proficiency,
the majority of the respondents show optimistic responses. Most students feel
encouraged to study English to pass the test so that they could meet the school’s
requirement, cover their English scores, and recognize their English levels. In
terms of test usefulness, there is a slight tendency toward the positive as
numerous respondents reckon that English plays a critical role in job finding
process and the GPA could show those recognitions. With regards to attitude to
test accuracy, no trend could be drawn from the survey.
Although the GPA seems to get credit for the mentioned reasons, some
respondents hold negative views toward the test. For few interviewees, the
test goes again their wants, does not change high proficiency students as well as
motivate low competent ones. Likewise, there are several claims that either the
easiness or the practicing to the test do not raise students’ English levels. The
test reliability and accuracy are also questioned for marking speaking may differ
from different examiners, MCQs might cause the random choices, and some find
48
their friends unqualified but still pass the exam. As the unpopularity of the test
among the recruiters and the seem to low of value certificate, a couple of
respondents do not regard the test useful for their job application.
For test anxiety, a multitude of respondents arenervous before the test
because of the unpopular format and test-taking pressure. However, most of them
do not experience the extreme worry as the GPA is consider not too demanding.
5.1.2. Washback on students’ test preparation
With regard to the test preparation behaviors, the general English learning
activities are studied. In average, the respondents engage in a modest number
of activities, only three to four out of thirteen ones in the survey. Time spent for
those activities vary from 0 to 40 hours per week. However, those behaviors are
claimed to lessen when the test is closer. Regarding the test specific preparation
behaviors, there is a larger number of the third year and last year
respondents taking steps to prepare for the GPA than the first year and
second year students. While the former has made effort for test preparation in
approximate 2 months, most of the students in the latter group are uncertain
about the study plan. Despite those differences, the two groups appear to have
the same the test preparation strategies. A vast number of students study for
the test by themselves for saving money purpose. In the second place is seeking
for help from English teachers and accessing the GPA website which provide
learners with more knowledge of the test. While English center offers students
with excessive test materials, the GPA website is perceived to provide poor
content for provision.
In terms of test-like content for preparation, few respondents practice
the test-like materials and they are likely to spend little time studying
instead of a great quantity of time. There is a clear difference between two
groups of students shown by the report. Wh ile up to a half of the FYSY
respondents have never practice as the test format, the figure is much smaller in
the TYLY group. As a result, the latter group study more test similar tasks
mentioned. Additionally, while more FYSY students practice the test only “little
time”, the counterpart spends more time for test-like preparation. In short, the
49
seniors have more preparations and do more activities close to the test, while the
freshmen are likely to make little attempt toward the GPA. It can be inferred that
the GPA has little washback on students’ preparation.
5.2. Conclusion and implication
This study aims at investigating the washback back of GPA on students’
test preparation and their attitude toward the test. To achieve this aim, the
researcher has initially reviewed the literature of washback and the proficiency
test to obtain both theoretical and empirical background. The frameworks of
Cheng et al. (2004) and (Shih, 2007) are then adopted. Following that, the
methodology has presented in details the context, participants and research
search design including how the data are collected, analyzed and interpreted.
Both survey and interview are employed to answer research questions. The
findings and discussion about students’ attitude toward the test and washback on
students’ test preparation are then presented in chapter 4.
5.3. Recommendations
The understanding of washback on students’ test preparation and
students’ attitude toward the test is of great help for students, teachers, and test
administration. From the findings, the researcher proposes some minor
suggestions as following:
Firstly, students should build up their language proficiency gradually so
that they would not end up cramping for exam and consequently studying just
test-like materials.
For teachers, besides test-similar materials and templates, they should
also provide students with more practical exercises, so after taking, the students
could apply the knowledge in daily situations.
Last but not least, test administrators should take the feedbacks of the
students into serious consideration. The complaints such as the GPA is
insufficiently popular to support the application process, the test seems to be
unreliable and invalid to students and the materials for the GPA is particularly
rare should be fully resolved.
50
5.4. Limitations of the study
Although the study has been carefully carried out, limitations are
unavoidable. Firstly, owing to time limit, the study was conducted on a small
population size, only the non-English major at the School of Law. Therefore, to
generalize results for a larger group, there should be more participants. Secondly,
this study collected second hand data, opinions of the students, which may not
reflect the actual performance. Hence, for further study, the observations or
experiences method could be employed. Finally, some other aspects and
dimensions of washback have not been concerned for the limitation of the
research.
5.5. Suggestions for further research
Due to the limitations of this research, there are rooms for further
studies. The following are some suggestions:
- Washback on students on larger scale with multiple methods
- Washback on teachers and others participants
- Washback on all level of Vietnamese educational system
51
REFERENCES
.
Bachman, L. F. (2000). Modern language testing at the turn of the century:
assuring that what we count counts. Language Testing, 17(1), 1–42.
Bailey, K. M. (1996). Working for washback: a review of the washback
https://doi.org/10.1177/026553229601300303
Manchester.
Erlbaum Associates.
Damankesh, M., & Babaii, E. (2015). The washback effect of Iranian high
https://doi.org/10.1080/09695940701272880
295–303.
đến bậc 5 cho đối tượng sau trung học phổ thông (Vietnamese
ĐHQGHN.
SPSS.
Shohamy, E., Donitsa-Schmidt, S., & Ferman, I. (1996). Test impact revisited:
221.
Kong.
54
Sukyadi, D., & Mardiani, R. (2011). The Washback Efect of the Englsih National
Examination (ENE) on English Teachers' Classroom Teaching and
Students' Learning. k@ta, 96-111.
Tsagari, D. (2011). Washback of a high-stakes English exam on teachers’
55
APPENDIX
Appendix 1
WASHBACK OF THE PROFICIENCY ASSESSMENT AS EXIT
REQUIREMENT ON NON-ENGLISH MAJOR STUDENTS AT A LAW
SCHOOL
FELTE, ULIS, VNU
Questionnaire
I am Huong from QH2014.F1.E3 of ULIS, VNU, Hanoi and I am carrying a
research on “Washback of the proficiency assessment as exit requirement on
non-English major students at a law school” which gives a closer look on how
law students prepare for the graduation proficiency assessment (GPA) and their
attitudes influenced by the test. Your response to this questionnaire is of great
help for the final product, which might assist test-designers develop a fairer and
more valid test. I hope you provide trustworthy answers as it is only way to
ensure the validity of the study. Your information will be kept confidential under
any circumstances. Thank you very much for your contribution!
There is a following interview to discuss the exit test further. The participants
will get 50.000 if they finish both the questionnaire and the interview. In case,
you would like to attend the interview, please leave your email and phone
number below. The researcher will contact you as soon as possible.
I appreciate your help in completing both the questionnaire and the interview,
thank you once again.
Your sincerely,
Bui Thanh Huong
56
PART 1: BACKGROUND INFORMATION
Your name: ……………………………………………........……
Phone number: ………………………………Email: ……………………
1. Did you/ do you intend to take the graduation proficiency assessment (GPA)
offered by your university?
Yes No
2. You are: First year student Third year student
Second year student Fourth year student
PART 2: PERSPECTIVES TOWARD THE GRADUATION
PROFFICIENCY TEST (GPA)
3. Why do you take the VSTEP instead of IELTS, TOEFL, TOEIC or other
compatible tests as the graduation test?
………………………………………..………………………………….……
4. How do you rate your overall English language ability? (Please tick the box
in the scale below)
very poor excellent
1 2 3 4 5
5. How do you feel about having to take the VSTEP? (Please tick the box in the
scale below)
extremely extremely
worried 1 2 3 4 5 confident
6. Do you think the graduation proficiency assessment (GPA) will motivate
students to improve their English?
No Yes No option
7. Do you think the VSTEP will raise the proficiency of English of university
students? (please tick one box)
No Yes No option
8. Do you think VSTEP will measure students’ English proficiency accurately?
(please tick one box)
57
No Yes No option
9. Do you think the VSTEP test result will be useful in helping you find a job?
(please tick one box)
No Yes No option
PART 3: GENERAL ENGLISH ACTIVITIES
10. Which of the following out-of-class activities do you do? (Please tick the
box/ boxes that apply) 0. is an example
0. Watching English TV
programs
7. Speaking English with teachers
Others……………………………………………………………………………
11. Approximately how many hours per week do you normally spending on
doing those activities outside class? _____________________ hours
PART 4: TEST PREPARATION
12. Have you done anything to help you to prepare for the test? (Please tick the
box)
NO
YES _ Please specify the activity or activities ………………………
………………………………………..............................……………
13. a. How long before the GPA did/will/do you start preparing for the test?
(give a number and circle year/ month/ week/ day) i.e. 2 weeks
58
……………………………………………………… year/ month/ week/ day.
b. How many hours per week did/will/do you study for the GPA?
15. Do you do these activities when preparing for the VSTEP? (Please tick the
box/ boxes that apply) 0. is an example
0. I practice the past paper test
1. I practice the past paper test
2. I practice items similar in the test format
a. I practice reading MCQs questions
If yes, where __________________________________________
b. I practice listening MCQs questions
If yes, where __________________________________________
c. I practice speaking familiar topic about myself and my surrounding
(study, work, family, friends, hobbies, sports, fashion, etc.)
d. I practice speaking by choosing an option in three and giving reasons
e. I practice speaking by developing ideas for a particular topic (e.g.
benefits of reading, etc.)
e. I practice writing letter
d. I practice writing essays (discussion, opinion, problems and
solutions, advantages and disadvantages essays)
I study vocabulary for common topics in the test
59
--------------------------------------This is the end----------------------------------------
60
Appendix 2
TÁC ĐỘNG CỦA BÀI ĐÁNH GIÁ NĂNG LỰC TIẾNG ANH CHUẨN
ĐẦU RA (CĐR) ĐỐI VỚI SINH VIÊN KHÔNG CHUYÊN KHOA LUẬT
PHIẾU THĂM DÒ Ý KIẾN
Các bạn thân mến,
Tôi là Hường, sinh viên khóa QH2014.F1.E3 trường Đại học Ngoại Ngữ -
ĐHQGHN. Tôi đang tiến hành một nghiên cứu về “Ảnh hưởng của bài đánh
giá năng lực Tiếng Anh chuẩn đầu ra đối với sinh viên không chuyên”. Đề
tài đưa ra cái nhìn cụ thể hơn về sự chuẩn bị của sinh viên khoa Luật cho kỳ thi
Đánh giá Năng lực tiếng Anh chuẩn đầu ra (CĐR) và nhận thức của sinh viên về
kỳ thi. Câu trả lời của bạn đóng góp một phần không nhỏ cho sự hoàn thiện
nghiên cứu này, có thể giúp đỡ những người thiết kế bài thi phát triển bộ đề công
bằng và chính xác hơn. Tôi mong bạn có thể cung cấp câu trả lời chân thật nhất
vì đó là cách duy nhất để đảm bảo tính chính xác cho nghiên cứu này. Thông tin
của bạn sẽ được giữ bí mật trong mọi trường hợp. Cảm ơn sự đóng góp của bạn!
Sau phiếu thăm dò ý kiến, sẽ có một buổi phòng vấn sâu hơn về bài thi chuẩn
đầu ra. Người tham gia sẽ được nhận phần quà tương đương 50.000 vnd nếu
hoàn thành cả phần khảo sát và phỏng vấn. Trong trường hợp bạn muốn tham
gia buổi phỏng vấn, hãy để lại địa chỉ email và số điện thoại bên dưới. Tôi sẽ
liên lạc để xếp lịch với bạn sớm nhất có thể.
Tôi rất trân trọng sự giúp đỡ của bạn trong quá trình khảo sát và phỏng vấn.
Xin cám ơn bạn một lần nữa.
Thân ái,
Bùi Thanh Hường
61
PHẦN 1: THÔNG TIN CĂN BẢN
Tên bạn là (không bắt buộc)……………………………………………………
Số điện thoại: …………………………….. Email: …………………………
1. Bạn có định thi/ đã thi CĐR ở trường cho yêu cầu tốt nghiệp không?
Có Không
2. Bạn là: Sinh viên năm nhất Sinh viên năm ba
Sinh viên năm hai Sinh viên năm tư
PHẦN 2: THÁI ĐỘ CỦA SINH VIÊN ĐỐI VỚI BÀI THI CHUẨN ĐẦU
RA
3. Tại sao bạn chọn thi CĐR thay cho thi IELTS, TOEFL, TOEIC hay bài thi
khác cho yêu cầu tốt nghiệp? (Nếu bạn chọn bài thi khác, hãy nêu rõ đó là bài
thi gì, tại sao bạn lai chọn)
…………………………………………….…………………………………
4. Bạn tự đánh giá khả năng Tiếng Anh của mình như thế nào? (Hãy đánh dấu
V vào ô trống bên dưới)
6. Bạn có nghĩ Kỳ thi Đánh giá Năng lực Ngoại ngữ (CĐR) sẽ tạo động lực để
sinh viên cải thiện Tiếng Anh? (hãy tích vào ô trống)
62
8 Bạn có nghĩ bài thi CĐR đánh giá trình độ tiếng Anh của sinh viên một cách
chính xác? (hãy tích vào ô trống)
0. Watching English TV
programs (Xem chương trình
7. Speaking English with
teachers
TV) (Nói Tiếng Anhh với giáo viên)
8. Speaking English with native
1. Watching English TV
speakers
programs
(Nói Tiếng Anh với người bản
(Xem chương trình TV)
xứ)
2. Listening to English 9. Going to English tutorial
radio/music schools
(Nghe radio/ nhạc Tiếng Anh) (Đến trung tâm Tiếng Anh)
3. Reading English books/ 10. Joining English speaking
newspaper/ magazines clubs
(Đọc sách/ báo/ tạp chí Tiếng (Tham gia câu lạc bộ Tiếng
Anh) Anh)
11. Learning English on the
4. Writing diaries in English
Internet
(Viết nhật ký Tiếng Anh)
(Học Tiếng Anh trên mạng)
5. Write to English speaking
12. Studying English
friends
dictionaries
(Viết/ chat với bạn người bản
(Học từ điển Tiếng Anh)
xứ)
13. Keeping a vocabulary
6. Writing e-mail in English
notebook
(Viết email bằng Tiếng Anh)
(Giữ một quyển sổ từ vựng)
63
Khác: ................................................................................................................
11. Một tuần bạn dành bao nhiêu thời gian để học Tiếng Anh?
……………………… giờ.
PHẦN 4: HOẠT ĐỘNG HỌC TIẾNG ANH CỤ THỂ ĐỂ CHUẨN BỊ
CHO CĐR
12. Bạn đã làm gì để chuẩn bị cho kỳ thi CĐR chưa? (hãy tích vào ô trống)
CHƯA
RỒI
13. a. Bạn dành bao nhiêu thời gian để chính thức chuẩn bị cho CĐR?
(hãy đưa ra một con số và khoanh vào năm/ tháng/ tuần/ ngày)
………………………………………. năm/ tháng/ tuần/ ngày. (Ví dụ:
2 tuần)
b. Bạn dành bao nhiêu tiếng một tuần để chuẩn bị cho CĐR?
15. Bạn học những mảng gì để chuẩn bị cho CĐR? Đánh giá thời gian bạn
thực hiện các hoạt động dưới đây bằng việc chọn mức độ từ 1 (không bao giờ)
đến 4 (rất nhiều thời gian)
1 – không bao giờ
2 – một ít thời gian
3 – nhiều thời gian
64
4 – rất nhiều thời gian
1. Luyện tập đề đã thi. 1 2 3 4
2. Luyện tập câu hỏi giống trong định dạng đề thi. 1 2 3 4
a. Đọc và trả lời câu hỏi trắc nghiệm.
Tài liệu nào (nếu 1 2 3 4
có)……………………………………………
b. Nghe và trả lời câu hỏi trắc nghiệm.
Tài liệu nào (nếu 1 2 3 4
có)……………………………………………
c. Luyện nói các chủ để thân thuộc: về bản thân và
về môi trường xung quang (học tập, công việc, gia 1 2 3 4
đình, bạn bè, sở thích, thể thao, v.v.)
d. Luyện nói bằng cách chọn một trong ba lựa chọn
1 2 3 4
rồi giải thích lý do
e. Luyện nói bằng cách phát triển các ý cho một chủ
1 2 3 4
đề (vd lợi ích của việc đọc, v.v)
f. Luyện tập viết thư 1 2 3 4
g. Luyện tập viết luận (thảo luận, vấn đề và giải
1 2 3 4
quyết, lợi ích và tác hại)
65
Appendix 3
The interview questions – English version
I. BACKGROUND
1. Do/ did you take the VSTEP? Why?
2. When did you/ will you take the test?
II. ATTITUDE TOWARD THE TEST
2. At that time, did you consider VSTEP as an important test?/ Do you consider
VSTEP as an important test?
No (why not)
Yes (important in what way? And how important on a 1 (+) to 5 (+++)
scale)
3. At that time, who did you think were interested in the VSTEP test scores/
test results?
Why were they interested/ not interested?
Yourself
Parents
University
Employer
4. Did/ do you feel motivated by the test to put more effort into improving your
English?
No (why not)
Yes (how motivating on a 1 (+) to 5 (+++) scale)
5. Did the GPA psychologically influence you during your preparation for the
GPA? If yes, how, were/ are you worried, or were/ are you confident? (1 to 5
scale, 1 = extremely worried, 5 = extremely confident) Why?
6. Did/ do you think VSTEP assess students’ English levels accurately? Why?
III. LEARNING PREPARATION
a. Timing
7. How long did/ will you spend to prepare for the test in total? Per week?
8. Do you think it was/ is enough? Why?
66
b. Content/ Method
9. Tell me through the process and stages of how you prepared/ prepare for the
VSTEP:
When did you did what?
Who you went to?
What resources you used?
10. What would you otherwise do if you did not have to pass the GEPT as one
of the degree requirements? Would you take the GEPT? Would your test
preparation be different?
67
Appendix 4
Các câu hỏi phỏng vấn
I. BỐI CẢNH
1. Bạn có tham gia thi VSTEP không? Tại sao?
2. Bạn đã làm bài kiểm tra khi nào?
II. THÁI ĐỘ ĐỐI VỚI KỲ THI
2. Vào thời điểm đó, bạn có coi VSTEP quan trọng không?
• Không (tại sao không)
• Có (quan trọng theo cách nào? Và tầm quan trọng của thang điểm 1 (+) đến 5
(+++))
3. Vào thời điểm đó, bạn nghĩ ai quan tâm đến điểm thi / kết quả thi VSTEP?
Tại sao họ quan tâm / không quan tâm?
• Bản thân bạn
• Cha mẹ
• Trường đại học
• Nhà tuyển dụng
4. Bạn có nghĩ bài kiểm tra thúc đẩy sinh viên nỗ lực hơn trong việc cải thiện
tiếng Anh không?
• Không (tại sao không)
• Có (cách thúc đẩy tỷ lệ 1 (+) đến 5 (+++))
5. GPA có ảnh hưởng tâm lý đến bạn trong quá trình chuẩn bị không ?
không? Nếu có, làm thế nào, là / bạn đang lo lắng, hoặc là / bạn có tự tin? (Tỷ
lệ 1 đến 5, 1 = cực kỳ lo lắng, 5 = cực kỳ tự tin) Tại sao?
6. Bạn có nghĩ rằng VSTEP đánh giá chính xác trình độ tiếng Anh của học sinh
không? Tại sao?
III. CHUẨN BỊ HỌC
a. Thời gian
7. Bạn đã dành bao nhiêu thời gian để chuẩn bị cho bài thi? Mỗi tuần?
8. Bạn có nghĩ rằng đó là / là đủ? Tại sao?
b. Nội dung / Phương pháp
68
9. Cho tôi biết qua quá trình và các giai đoạn về cách bạn chuẩn bị / chuẩn bị
cho VSTEP:
Khi nào bạn đã làm gì?
Bạn đã đến ai?
Bạn sử dụng tài nguyên nào?
10. Bạn sẽ làm gì nếu bạn không phải vượt qua GEPT như một trong những
yêu cầu mức độ? Bạn có tham gia GEPT không? Chuẩn bị kiểm tra của bạn có
khác không?
69
Appendix 5
VSTEP Format
VSTEP Listening Reading Writing Speaking
Format
Time 40 minutes 60 minutes 60 minutes 12 minutes
Number 3 sections 4 passages 2 writing 3 parts
of 35 MCQs 40 MCQs tasks
questions
Task Test-takers Test-takers Task 1: Part 1: Social
listening to reads for Write a interaction
short passages about 120 – word Test-taker
conversation, different letter. answers 3-6
instruction, topics, with Task 2: questions in
announcement, total Write 250 two different
dialog and approximately – word topics.
discussion; 1900-2500 essay on a Part 2: Solution
then answer words; then given topic discussion
MCQs answer MCQs Test-taker
after each text receives a
situation with 3
solutions. He/
she has to give
opinion on the
best solution
and criticize
other two.
Part 3: Topic
development
Test-taker
speaks about a
70
given topic,
using given
ideas or his/ her
own ideas. Part
3 ends with
some
discussion
questions.
Purpose Assess Assess Assess Assess
different different letter different
listening reading writing speaking skills:
subskills from subskills from skills and Social
level 3 to 5: level 3 to 5: persuasive interaction,
listen for reading for writing discussion and
details, listen details, reading skills topic
for main ideas, for main ideas, development
listen for reading for
opinion and opinions,
purpose of attitudes of the
speakers and writers, infer
infer information
information. and guess the
word meaning
from the text,
71