SSLA, 20, 1736. Printed in the United States of America.

RESEARCHING VOCABULARY THROUGH A WORD KNOWLEDGE FRAMEWORK

Word Associations and Verbal Suffixes

Norbert Schmitt
University of Nottingham

Paul Meara
University of Wales at Swansea

This study examines how two types of word knowledge, word associations and grammatical suffix knowledge, change over time both receptively and productively. Ninety-five secondary and postsecondary Japanese students were tested on three word associations and inflectional and derivational suffixes for each of 20 verbs, once near the beginning of their academic year and once near the end. The results showed their average vocabulary gain was 330 words. The students showed rather poor knowledge of the allowable suffixes for the verbs, especially the derivative suffixes. Likewise, the subjects did not show very good mastery of the verbs word associations. Even for verbs rated as known, the students as a group were able to produce only

Research such as the present paper relies on the cooperation of a number of people. We thank the students of Notre Dame Womens University and Kyoto Nishi High School for their participation and Rob Waring and Renee Hawkins for allowing access to their classes. Stephanie Swoll translated the test instructions into Japanese. A more detailed account of this study can be found in Schmitt (1995b). Norbert Schmitt is completing his Ph.D. studies at the University of Nottingham. His main research interests are L2 vocabulary acquistion and testing. He has recently coedited Vocabularly: Description, Acquisition, and Pedagogy for Cambridge University Press. Address correspondence to Norbert Schmitt, Dept. of English Studies, University of Nottingham, Nottingham NG7 2RD UK; e-mail: [email protected]. Paul Meara is Director of the Wales Applied Language Research Unit at Swansea University of Wales and is a committee member of the British Association for Applied Linguistics. His three bibliographical volumes on vocabulary research have become the standard reference works in the field. His e-mail address is [email protected].
1997 Cambridge University Press 0272-2631/97 $7.50 + .10

17

18

Norbert Schmitt and Paul Meara about 50% of the word associations possible on the test as judged by native speaker norms. Word association knowledge and suffix knowledge were shown to correlate with each other and with total vocabulary size. The subjects overall had from 19 to 25 percentage points more receptive knowledge than productive knowledge.

In the last 20 or so years, there has been a growing realization that total language proficiency consists of much more than just grammatical competence. Concurrently, there has been an increasing awareness that there is much more to knowing a word than just learning its meaning and form. Richards (1976) made the first attempt to list the different types of knowledge that are necessary to fully know a word. Elaborating on Richards list, Nation (1990, p. 31) developed a list of various types of knowledge that one must possess both receptively and productively in order to have complete command of a word:
1. The spoken form of a word. 2. The written form of a word. 3. The grammatical behavior of the word. 4. The collocational behavior of the word. 5. How frequent the word is. 6. The stylistic register constraints of a word. 7. The conceptual meaning of a word. 8. The associations a word has with other related words.

If all of these types of word knowledge are mastered, then a speaker should be able to use that word in a nativelike and fluent manner. This is not to say that native speakers have full command of each of these types of word knowledge for every word they are familiar with; indeed, they are likely to have only mastery of a limited number of word knowledge categories for a large percentage of words in their lexicon. For most native speakers, there are a large number of low-frequency words of which they would have receptive, but not productive, knowledge of the above word knowledge types. For all but the best known words, the different word knowledges are likely to exist at varying positions on a receptive to productive plane. L2 learners will also have different mastery of the various kinds of word knowledge, with formal, grammatical, and meaning aspects probably learned first, and some other aspects, such as collocational behavior and register, perhaps never being mastered at all. In listing the types of word knowledge, Richards and Nation have provided an initial framework through which to view vocabulary acquisition and use. At this point the framework is purely descriptive and does not have the power to explain either the processes of acquisition for the different kinds of word knowledge or the mechanisms by which they interrelate. This is partially because there is currently no generally accepted model of how vocabulary is acquired (Meara, 1984, in press) that could tie the different strands of vocabulary research together. The result has been isolated research on the various individual kinds of word knowledge but with

Word Knowledge

19

no vision of how they interrelate. The various kinds of word knowledge must be interrelated: for example, frequency of use is one of the determinates of where a word lies on the formal to vernacular continuum, the strong level of collocation in idioms causes them to have a meaning different than if the words were analyzed separately, and associations most often belong to the same syntactic class. Yet, to our knowledge, no previous research has explored the interrelationships among the various types of word knowledge. There have been a number of discussions about what makes a word easier or more difficult to learn (Ellis & Beaton, 1993; Higa, 1965; Hulstijn, 1994; Laufer, 1990, 1994; Nation, 1990; Singleton, 1994), but none of them have explicitly linked different kinds of word knowledge. This is unfortunate, because a better understanding of these interrelationships could go a long way toward developing an explanatory model of vocabulary acquisition. As this is one of the first exploratory steps toward using the word knowledge framework as a research rationale, it was impossible to design a study that could capture all of the word knowledge categories, given the limitations of our current knowledge. One example of these limitations concerns our knowledge of intuitions of frequency. Although studies have shown that native speakers can give subjective estimates of frequency for long lists of words that correlate fairly strongly with objective word count data (Carroll, 1971; Shapiro, 1969; Tryk, 1968), these studies do not shed much light on what is known about the frequency of each individual word. Likewise, word frequency is a major effect in psycholinguistic experiments, but this does not tell us about an individuals intuitions of word frequency. Given such limitations, it was decided to limit the current study to two types of word knowledge: grammatical knowledge and word association knowledge. Grammatical knowledge about a word can embrace a variety of things, from its word class and where it normally occurs in a sentence, to its morphological characteristics. This last category was chosen as the specific type of grammatical knowledge to be studied here because of its importance in the formation of word families (employ employer, employee, employment, employable) and thus vocabulary size. Also, the other kinds of grammatical knowledge are more likely to be mastered in the beginning stages of learning a new word. Affix knowledge seemed a suitable category for subjects who were at a high enough level to deal with the second aspect studiedword associations. Word associations are the links that connect or relate words in some manner in a persons mind. A common way of eliciting them is to have a tester give a prompt word and have the subject say the first word that comes to mind. Meara (1983) summarized the work done on word associations in a second language and concluded that L2 learners tend to give more varied responses than native speakers, even though their vocabularies are smaller. He also noted that learners tend to produce clang associations, nonrelated but similar-sounding words (reflecteffect), instead of the semantically related responses that adult native speakers typically produce. Overall, L2 associations are often very different from those of native speakers (see also Kruse, Pankhurst, & Sharwood Smith, 1987; So derman, 1993). This study was designed with two purposes in mind. The first was to examine two individual vocabulary knowledge components, affix knowledge and word association

20

Norbert Schmitt and Paul Meara

knowledge, and also to explore how they relate to each other, to overall vocabulary size, and to general language proficiency. The rationale for assuming that word association and verbal suffix knowledge are related is based on the following: Greater knowledge of suffixes would coincide with a larger vocabulary because it would facilitate access to more members of a words family. This larger vocabulary size would make more words available for associations. This makes it more likely that an appropriate word family member is available for use as an association (nature might associate with agreeable but not with the more common agree). At the same time, it would not lead to a great increase in the number of concepts to be associated. In this case, agree and agreeable represent different concepts, but the members of most word families share the same underlying meaning, as in the case of the word family deriving from employ. Evidence that native speakers do not normally give the members of a target words family as association responses (Postman & Keppel, 1970) indicates that associations tend to be related to concepts, not the form of words. Thus, one would expect there to be an interrelationship between association knowledge and suffix knowledge (more word forms) but that it would be relatively weak (word forms concepts). If this study can demonstrate empirically that these two types of word knowledge are indeed interrelated, it would suggest that research into understanding the mechanisms of that relationship (and those of other, more transparently related kinds of word knowledge) would further our knowledge of lexical acquisition and use. The second purpose was to try to design a study that included desirable elements of research that are not often seen in vocabulary research. First, most research to date has focused on either receptive or productive knowledge. However, because productive and receptive knowledge of a word clearly differ in some aspects (Melka Teichroew, 1982), it was decided to measure both aspects. A second important element ignored in most vocabulary studies is the incremental nature of vocabulary acquisition. Nation (1990, pp. 4345) surveyed a number of studies that substantiate the intuitive assumption that words are not learned completely from one exposure (see also Hulstijn, 1992; Nagy, Herman, & Anderson, 1985). Nevertheless, most research designs use a single dichotomous vocabulary measurement based on indicated knowledge of a single word meaning. These measurements do not really give adequate information about the range of mastery of the target word: from barely knowing it receptively to being able to use it productively with stylistic and collocational appropriateness. In addition, one-off studies do not necessarily give a very good indication of how individuals acquire vocabulary knowledge. Longitudinal studies are much more likely to give an accurate picture of what it means to learn a word. A third element is the distinction between how many words are known (vocabulary size) and how well individual words are known (depth of knowledge). Many studies concern themselves only with vocabulary size, but using a word knowledge framework requires us to explore depth of knowledge by examining how well different word knowledge components are known. A fourth element is that important information about individual behavior is often lost in group analyses. In addition to discovering group trends, it is important to look at individual results, to see what is happening

Word Knowledge

21

in the individual, for in the end we are attempting to learn about individual behavior, not group behavior. This study will take all of these elements into consideration in its exploration of word association and verbal suffix knowledge.

METHODOLOGY Creating the Study Instrument

Because the main purpose of the study was to study the learners depth of knowledge of vocabulary, rather than their vocabulary size, it was necessary to choose carefully a limited number of prompt words, following the cautions given by Morgan and Bonham (1944) and Meara (1983). Verbs were chosen as the class of words to be used, mainly because they take the widest range of suffixes. An additional consideration was that verbs as a class are somewhat neutral in terms of difficulty, being neither the easiest nor the most difficult grammatical class of words (Morgan & Bonham, 1944). A list of 50 verbs was compiled from the Brown frequency list (Francis & Kuc era, 1982) using the following criteria as a guideline: Each verb took a variety of suffixes; the verb and its related noun did not use an identical form (i.e., to smile, a smile); some suffixes from the higher Bauer and Nation (1993) difficulty levels, such as -ee, -ive, and -al, were represented; verbs of varying frequency were included; and the verbs required, at most, a one-letter change to take a suffix (indicate indication). An example of a verb that would not qualify under the last criterion is describe; when adding the suffix -ion, more than one letter must be changed to achieve description. Twelve subjects from one of the target classes were asked to rate how well they knew each of the 50 verbs. This information was used to choose mainly verbs that were partially known, but also a few that were relatively well known and a few that were not known very well at all. A final list of 20 prompt words was arrived at by choosing verbs at these familiarity levels that also conformed to as many of the preceding criteria as possible. Next, an inventory of the most common suffixes compatible with the chosen verbs was compiled. It consisted of suffixes from five of the Bauer and Nation (1993) difficulty levels: Level 2 suffixes -ed, -ing, and -s; Level 3 suffixes -able, -er, and -ly; Level 4 suffixes -ion and -ment; Level 5 suffixes -age, -ance/ence, and -al; Level 6 suffixes -ee and -ive; and one suffix not explicitly mentioned in any of the levels, -ure. With the exception of -age, all the suffixes were allowable with two or more of the verbs. This meant that three inflectional suffixes were included (-ed, -ing, and -s), whereas the rest were derivative suffixes used to change the base verb into another word class. Baseline data were obtained from 18 native speakers (all English EFL teachers in Japan) to determine which suffixes were allowable for each word. They were given the list of prompt verbs and asked to produce all acceptable inflectional and derivative suffixes for each verb. Because there was some disagreement in the results, especially concerning suffixes like -ee and -able, only suffixes given by at least half of the native speakers (nine) were accepted. In addition, 16 other verb and suffix combinations were accepted, despite the lack of native speaker consensus, because

22

Norbert Schmitt and Paul Meara

they were explicitly listed in Websters Ninth New Collegiate Dictionary (1987). To develop the association baseline data, 30 native speakers (also English EFL teachers in Japan) were asked to give three word associations for each of the 20 prompt verbs, producing 8590 word associations for each. Based on these baseline data, the test instrument was constructed (see the Appendix). It consisted of a productive section, a receptive section, and two vocabulary-size tests. The productive section listed the 20 prompt words, followed by a blank on which to write any suffixes the subjects thought were allowable and then three blanks on which the subjects were to give three word associations. The instructions made clear that all of the prompt words were verbs. A 4-point Lickert scale (03) was also attached to each prompt word to enable subjects to indicate how well they knew the verbs. The 4 points on the scale were described in the following way: 0 = I dont know this verb, 1 = I think I might have some sense of what this verb means, 2 = I think I know this verbs meaning, but I am not sure, and 3 = I know this verb. The receptive section was similar, except that a list of possible suffixes followed each prompt verb, from which allowable suffixes were to be circled. The receptive association component for each prompt verb consisted of the three most frequent native speaker associations from the baseline data, along with a totally unrelated distractor, set in a randomized order. To measure the subjects overall vocabulary size, the Levels test (Nation, 1990; Nation, 1983) was added to the test instrument. It is a vocabulary-size test that evaluates subjects at each of five levels: at the 2,000, 3,000, 5,000, and 10,000 word frequency levels as well as measuring academically based English. The proportion of correctly answered items at each level (out of 18 possible) is taken to be the proportion of total words known at that level. The totals for each of the five levels are added together to achieve an estimate of overall vocabulary size. In addition, members of the high school class took a computerized vocabulary-size test called the EVST (Meara & Jones, 1990) and the TOEFL language proficiency test (Educational Testing Service, 1987).1

Subjects
The subjects for this study consisted of three intact classes of Japanese students. Two of the classes were made up of English majors from a prestigious Japanese womens university including one freshman class (36 subjects) and one sophomore class (31 subjects). The subjects were 1820 years old and had studied English for approximately 6 years in junior and senior high school. The third was a senior class of 28 high school students (24 females and 4 males, 1718 years old). Most of the high school students were studying to enter a college of foreign languages and had had about 5 years of prior English instruction. Five of the 95 subjects were absent for the second administration of the test, whereas others did not satisfactorily complete every section of the test instrument, resulting in a range of usable responses from 86 to 95. Although the intention was to include subjects of three different English proficiency levels, the groups ended up being very similar, with the high school students generally falling between the freshman and sophomore university

Word Knowledge

23

students in vocabulary size (as measured by the Levels test). The subjects overall could be roughly categorized as intermediate, with their mean vocabulary-size score being 3,900 words (minimum 2,333, maximum 5,733, SD 751). TOEFL (Educational Testing Service, 1987) scores were available for the high school students. The mean score was 437 (minimum 390, maximum 527, SD 29.62). The university students received three 90-minute classes with native-speaking instructors per week for the 13 weeks of study between the T1 and T2 measurements. In addition, they took literature and linguistics classes that involved reading in English but listening in Japanese. The high school students received about 14.5 hours of instruction per week in English-medium classes, plus any additional conferencing or project work.

Procedure
The test was administered to the three intact classes of Japanese students near the beginning of their school year (T1). In all cases, the students were shown examples of how to complete the test before beginning and told that blind guessing would decrease their score. They were instructed to complete the test sections in order and not to go back and work on previous sections. This was necessary to ensure that they would not transfer suffixes or associations from the receptive section to the productive section. The tests were carefully proctored to ensure that this did not happen. The subjects were to write acceptable suffixes and three associations on the blanks on the productive section. On the receptive section, they were to circle acceptable answers. They were not told how many nativelike receptive associations there were, but they were asked to circle any word they thought was related to the prompt word. The students received their vocabulary-size scores but were not given back the test instrument. Near the end of their school year, the subjects were given the same identical test (T2), using the same procedure as in the T1.

Scoring
For each prompt verb on the production task, the subjects were asked to supply three associations and all allowable suffixes. This means there were always three nativelike productive associations possible, whereas the number of nativelike suffixes varied from three to seven, depending on the verb. The receptive task was the same, except that there was also a distractor association. The subjects responses were compared against the native speaker baseline master key for scoring. If a word association submitted on the productive task seemed nativelike but was not on the key, its appropriateness was judged by two native speakers. If both found it acceptable, it was scored as nativelike and added to the list. There were 281 such associations added to the original list of 821, making a total of 1,102 nativelike associations. On the receptive tasks, it was possible for students to guess. Inflectional suffixes were allowable for all prompts, so it was the derivative suffixes that were idiosyncratic to each verb. When the tests were analyzed, it was found that few unacceptable derivative suffixes were circled. In cases where unacceptable derivative suffixes were

24

Norbert Schmitt and Paul Meara

Table 1. Summary of subjects vocabulary size and TOEFL scores

Vocabulary Size T1 Mean SD Number Minimum Maximum 3,900 751 95 2,333 5,733 T2 4,230 780 88 2,667 6,756 Difference +330* +334 +1,023 T1 437 29 28 390 527 TOEFL Scores T2 471 37 26 397 553 Difference +34* +7 +26

*p < 0.001, paired t-test.

indicated, they were usually the only noninflectional suffixes marked. Therefore, the situation did not often arise in which subjects could get credit for guessing on the receptive suffix task. On the receptive association task, if a distractor was circled, it was subtracted from the number of nativelike responses given. For example, if a student circled two nativelike associations and the distractor, their score for that particular word would be 1 (2 1 = 1). As it happened, few test instruments showed evidence of guessing. Not many distractors were chosen, but when they were, they were usually the only answer picked or one of two picked, so the resulting score was still zero. All 1 scores (only the distractor picked) were changed to zero.

RESULTS Subjects Vocabulary Size

Table 1 shows that the average gain from T1 to T2 as measured by the Levels test was about 330 new words. The lowest score in the group improved by almost exactly the average number of words, 334, whereas the highest score jumped 1,023 words. (The lowest and highest scoring subjects were different from T1 to T2.) Whether an increase of 330 words over the course of the better part of a school year is an achievement or a disappointment is a matter of personal judgment. The figures are from an EFL situation where Japanese students receive virtually no English input outside of the classroom, and what gains these subjects made probably stemmed almost directly from their formal language study. Moreover, other evidence indicates that an increase of this magnitude is fairly typical for an intensive EFL language course that does not focus specifically on vocabulary (Milton & Meara, 1995). An advantage of the Levels test is that it gives information about vocabulary change in a number of different frequency levels. The combined group scores showed a significant (p < 0.05) improvement on all five levels of the test. The subjects as a group increased their vocabulary size by 106 words in the 10,000 level, whereas they only gained 22 words in the 3,000 level. In the other three levels, they gained 7080 words. These results are not unexpected, as the more words one knows in a level, the fewer words there are left to learn. However, it is interesting to note the subjects gained more words in the most frequent level (2,000-word level) than at

Word Knowledge

25

the 3,000-word level, even though it might have been assumed that they knew the words in the 2,000 level better. Considering the groups T2 average vocabulary size was 4,230 words, this means they were filling in the gaps in their knowledge of more frequent words as well as gaining new lower frequency words. This is a good illustration that learners do not know all (or even most) of the words at the more basic higher frequency levels before they begin learning rarer words at lower frequency levels. In other words, it seems that word frequency by itself is not a reliable index to the likelihood of a word being known. The subjects as a group did not seem to learn new words at any single word frequency level; rather, the learning seemed to be spread out relatively evenly among the five frequency levels measured. There was also no trend for students with higher vocabulary sizes making better gains or for students with lower sizes making poorer gains. The general results give an impression of the group gains, but what is interesting is the individual results that are hidden in the group totals. A very surprising feature of individual scores was the number of decreasing T2 scores. Of the 88 subjects who took both T1 and T2 testings, 25 (28%) showed a decrease in the estimate of their overall vocabulary size. These decreasing scores occur in each of the three class groups. Decreases also occur in each of the five frequency levels examined. In fact, a surprising 67 out of 88 subjects (76%) had at least one frequency level where they had a decreasing vocabulary score. Of the 63 subjects who increased their overall vocabulary size over the school year, 43 (68%) had at least one frequency level with a negative score. It is possible that these decreasing scores are due to performance issues, rather than an attrition of underlying competence or knowledge. Still, the results suggest that we must modify our impression of the mental lexicon as something steadily increasing in size during the school year to one in which at least some of the vocabulary is in a state of flux. Similarly, the fact that small decrements often appear in repeated measures studies indicates that vocabularies seem to exhibit a certain degree of flux over even relatively short periods of time (Meara & Rodriguez Sanchez, 1993). Words do not seem to be learned smoothly from one frequency level to the next, and even high-frequency words that are learned seem to be forgotten. This dynamism has yet to be taken fully into consideration by modern testing methods.

Verbal Suffix Knowledge

The results from the preceding section indicate that most subjects increased their vocabulary size. Table 2 illustrates that the subjects as a group increased their knowledge of verbal suffixes as well, although the increases could be labeled as modest, with the highest one being a 25 percentage point increase in the production of -ment.2 The rate of change was not always consistent between the productive and receptive tasks. The subjects as a group increased their productive knowledge of three suffixes (-ion, -ly, and -able) without improving the corresponding receptive knowledge and improved their receptive knowledge of -s, -age, and -ive without increasing their productive knowledge of those suffixes. This indicates that, at least

26

Norbert Schmitt and Paul Meara

Table 2. Change in knowledge of individual suffixes for 20 prompt verbs from T1 to T2 (N = 88)
Productive Results Suffix -ed -ing -ment -s -ion -ly -ence -ee -age -er -ive -able -al -ure All suffixes T1 (%) 80 56 51 49 27 26 25 16 9 6 4 3 1 1 42 T2 (%) 86* 63* 76* 43 36* 39* 37* 29* 15 14* 6 9* 2 6* 47* Receptive Results T1 (%) 89 72 78 84 45 53 48 31 37 19 38 29 9 7 62 T2 (%) 92* 76* 86* 89* 46 57 58* 44* 46* 25* 28* 30 10 14* 66*

*p < 0.05, Wilcoxon signed ranks test.

as a group, there is not necessarily a one-to-one correspondence between increases in productive and receptive knowledge. With the exception of the inflectional suffixes and -ment, the subjects as a group did not show very good mastery of the suffixes. The receptive suffix total score seems low considering that inflectional suffixes alone made up 57% of the total possible (60/106). It might have been expected that the students would easily select all the rule-based inflectional suffixes and several derivative suffixes besides to push the total percentage known much higher than 62 or 66%. If the derivative suffixes are analyzed alone, only 15% were given on the productive section. Even inflectional suffixes were not particularly well mastered on the productive task at 59% correct. These figures lead to the conclusion that the subjects as a group have a rather weak awareness of derivative suffixes and their use, although they also lack convincing mastery of even inflectional suffixes. Looking at the results in Table 2, it is possible to see a clear ordering of the suffixes Japanese students were both able to produce and recognize in this study. Three of the top four suffixes were inflectional suffixes. This is not surprising because inflectional suffixes can be generalized because they are rule-based, whereas the relationship of derivative suffixes to individual words is largely idiosyncratic. This ordering of suffix percentage known seems to be rather stable, as all correlations are 0.92 (p < 0.001) or higher between productive and receptive versions of the same testing, between T1 and T2 testings for the productive results, and between T1 and T2 testings for receptive results. This stability led us to check whether the suffixes

Word Knowledge

27

Table 3. Change in association knowledge from T1 to T2

T1 Ma Productive associations (N = 89) Receptive associations (N = 86)
a

T2 SD 6.41 9.99 Ma 22.82 36.59 SD 6.21 9.63 pb .001 ns

19.62 35.38

Number of nativelike associations given (60 possible). bPaired t-test. ns = not significant.

fell into an implicational scale. They did not, because there was simply too much variation in what derivative suffixes individual subjects knew. Nevertheless, if any of the derivative suffixes were present, then the inflectional suffixes were almost always present as well.

Word Associations
Compared to verbal suffix knowledge, association knowledge is transparently related to understanding of conceptual meaning. Given the increase of vocabulary size of the group (based on a test that used meaning to determine whether a word was known or not), the association scores should rise as well. The results in Table 3 do not totally bear this out. On average, each subject was able to produce 3 more associations out of 60, although there was no statistically significant change in their receptive ability. The figures do not look very impressive, but this is one case in which it is essential to look at the behavior of individual subjects. There was a pattern of considerable individual variation. Of the 86 subjects who completed both T1 and T2 association tasks, 33 subjects showed increases in both their productive and receptive associative knowledge of the target verbs from T1 to T2. Twenty-four showed an increase in their productive knowledge with a drop or no gain in their receptive knowledge. Conversely, there were 14 subjects who increased in their receptive knowledge but had a decrease or no gain in their productive score. Fifteen others had poorer scores or no gain both productively and receptively at T2. No clear pattern emerged that could explain this lack of consistency between productive and receptive association knowledge change. There does not seem to be any relationship between the relative number of productive associations known on the T1 test and the size of increase between T1 and T2. This variability was particularly noticeable in the receptive association category. Of the 44 subjects who started with above-average receptive association knowledge (37 or greater) on the T1 test, exactly half, 22, suffered a decrease in score on the T2 test, whereas 22 either remained the same or managed an increase. These data provide evidence that, just as individual learners vary in the rate they learn words, they also vary in how they acquire nativelike associations for those words. Of course, the association results can best be interpreted according to how well the prompt words were known to the individual subject. This would almost certainly

28

Norbert Schmitt and Paul Meara

Table 4. Productive association knowledge according to self-assessment ratingsa

0 Subject 1 2 3 4 5 6 7 8 9 T1 0:4 0:6 0:5 0:3 0:1 0:1 0:4 0:2 0:1 T2 0:3 0:5 0:1 0:4 0:1 : 0:4 0:2 0:1 T1 2:6 0:4 0:2 : 4:2 1:6 0:3 1:4 0:4 1 T2 5:6 0:3 0:2 0:3 0:1 4:7 0:3 1:2 : T1 1:2 : 0:2 8:6 12 : 7 3:2 3:5 8:4 2:2 2 T2 1:1 2:3 1:1 5:3 5:3 0:1 2:4 0:1 1:5 T1 12 : 8 10 : 10 20 : 11 10 : 8 16 : 10 15 : 10 12 : 8 14 : 9 18 : 12 3 T2 18 : 10 13 : 9 27 : 16 25 : 10 28 : 15 16 : 12 12 : 9 21 : 15 15 : 14

a In each column, the right-hand figure indicates the number of words placed by the subject in the self-assessment category. The left-hand figure indicates the total number of nativelike associations produced for these words. Up to three associations are possible for each word.

vary among the 20 prompt verbs, because the study design deliberately tried to include some words likely to be known, some unknown, and some partially known. This was done to permit an analysis of the relationship between how well the prompt words were known (according to the self-evaluation rating scale included on the test instrument) and suffix and association knowledge. Although it would have been desirable to test or interview the subjects on the 20 prompt words after the main test to confirm their self-evaluations, there was no opportunity to do this. The prompt words for each individual test first needed to be divided into one of the four self-evaluation ratings categories (03). Because this was a very timeconsuming manual process, the productive tasks of a random subset of nine tests were analyzed by totaling the suffixes and associations for all the words in each category. The productive suffix results showed an unremarkable steady improvement, but the productive association results proved more interesting (Table 4). There are several notable features of the data in this table. The first is that for words rated as unknown (0), there was not one nativelike association given, on either the T1 or the T2. If a person does not know a word, it is almost impossible for them to guess a nativelike word association for it. Nevertheless, if they have partial knowledge of a word, their word associations can reflect this. This suggests that vocabulary tests utilizing word associations may have promise (Read, 1993). There is little difference in the association knowledge between words rated as 0 and 1, but words rated as 2 are beginning to be matched with some nativelike associations (see p. 22 for a description of the ratings). The major surprise is that subjects were only able to supply about 50% of possible associations for words they rated as known (3). In real terms this means that they were able to give an average of 1.5 nativelike associations out of three possible for each verb. If the subjects truly had mastery of the verb, it should not have been too difficult to give acceptable associations for it. It must be concluded that the subjects did not have very high levels of associative knowledge, even for words they thought they knew. There are two likely explana-

Word Knowledge

29

Table 5. Correlations of word association, suffix, vocabulary size, and TOEFL scores
Associations Productive T1 Vocabulary size (N = 88) TOEFL score (N = 28) T2 Vocabulary size TOEFL score 0.49* 0.48* 0.62* 0.66* Receptive 0.39* ns 0.61* 0.51* Derivative Suffixes Productive 0.27* ns 0.35* ns Receptive 0.37* ns 0.41* ns

*p < 0.05, Pearson product-moment. ns = not significant.

tions for this. First, the subjects may have overestimated their knowledge of the prompt verbs and they should have rated them as partially known. Second, the verbs were known in the traditional definition of knowing their meaning but that other kinds of word knowledge were not mastered. Both possibilities are likely to be true. Previous schooling probably taught subjects to believe that if they knew a words meaning, they knew that word. Thus, they rated words as known that they do not have competent associative knowledge for.3 This suggests that students need to be made aware of the different kinds of word knowledge it requires to truly master a word.

The Relationship Among Suffix Knowledge, Association Knowledge, Vocabulary Size, and Language Proficiency
In the introduction it was stated that different kinds of word knowledge must be interrelated. Although this is a strong intuition that can be logically argued, it has not been empirically tested. This study offers the opportunity to compare the various vocabulary measures, to see whether they do indeed correlate. First, Table 5 shows the correlation results among association knowledge, derivative suffix knowledge, vocabulary size, and general language proficiency. We can see that word association knowledge does indeed have some relationship with both overall vocabulary size and general language proficiency, as measured by the TOEFL test. The productive association correlations are particularly robust. This may be because, in order to give productive associations, a word must be known relatively well. This unaided productive knowledge correlates at about the same level as receptive association knowledge with vocabulary size but considerably higher when matched with more general language proficiency, as measured by the TOEFL test. As association knowledge is only one of several kinds of word knowledge, it could not be expected that the correlations would be any stronger than indicated. However, given the strength of the correlations on both the T1 and T2 tests, we can conclude that an ability to form nativelike associations is definitely related to both vocabulary size and general language proficiency. There were weaker, but still significant, correlations between receptive deriva-

30

Norbert Schmitt and Paul Meara

Table 6. Suffix and word association correlations

Derivative Suffixes T1 Productive (N = 94) Receptive (N = 94) T2 Productive (N = 89) Receptive (N = 89) Productive Associations ns 0.37* 0.36* 0.36* Receptive Associations ns 0.41* 0.37* 0.49*

*p < 0.05 Pearson product-moment. ns = not significant.

tive suffix knowledge and vocabulary size. Because inflectional suffixes were known by the majority of subjects, it was found that derivative suffixes provided a much better measure of the subjects suffix knowledge. The results show that derivative suffix knowledge does fit with vocabulary size in the way we would expect, considering that familiarity with derivative suffixes can help provide access to the other members in a word family. Somewhat surprisingly, in no case did derivative suffix knowledge correlate significantly with the TOEFL language proficiency measures. Next, let us examine whether there is empirical support for the statement that different kinds of word knowledge are related. Table 6 shows the relevant results. The results show that derivative suffix knowledge and word association have at least a weak relationship with each other. The figures are consistent and suggest that derivative suffix knowledge and word association knowledge correlate in the 0.30.5 range. This is not surprising, and one might suppose that correlations of this magnitude might be obtained when comparing any of the word knowledge types with each other or with measures of vocabulary size or general proficiency. The point is that this is the first time that the interrelationship of different kinds of word knowledge has been empirically demonstrated, at least in a second language context. This opens up the possibility of a new line of research that examines how the different types of componential word knowledge relate to and build upon each other to make up the sum of what people know about words. Previously, very few studies have attempted to measure and compare receptive and productive knowledge of a word. In her survey of receptive versus productive vocabulary, Melka Teichroew (1982) found that the difference between the two may be less than apparent. She cited some studies that claim the difference is really rather small; one estimates that 92% of receptive vocabulary is known productively. Takala (1984) suggested the figure may be even higher. As Table 7 illustrates, this study, with its particular set of assumptions, found that the difference between receptive and productive mastery of suffix and association knowledge is in the area of 1925 percentage points. It is important to remember that this is for intermediate students (average TOEFL score of 437 for the high school students). The difference between productive and receptive performance may vary considerably according to a students level of language proficiency, with beginning students more likely to have a wider range between the two. It is also interesting to note that some individual subjects actually performed better productively than receptively.

Word Knowledge

31

Table 7. Percentage of acceptable productive responses versus acceptable receptive responses (N = 94)
Productive T1 Suffixes Associations T2 Suffixes Associations 41.83 32.70 46.94 38.03 Receptive 62.09 58.33 65.92 60.99

DISCUSSION
The preceding analysis has shown that suffix and association knowledge are related to each other, as well as to vocabulary size and general second language proficiency. It has also shown that the subjects in this study do not have anything near nativelike mastery of these two types of word knowledge. Complementing these findings, the tests were also viewed holistically to search for information that might be missed in a strictly statistical appraisal. Most of the following observations concern the word association task, generally supporting Mearas (1983) conclusions about L2 learners word associations. 1. A number of prompt words were mistaken for other words with a similar orthographical or phonological form, causing mistaken associations:
(1) (2) (3) (4) (5) (6) discloseshop, door, open disclosefar disclose-distance, home, station quoteseparate, half, cake stimulatesame, equal haltmelting, water, refrigerator

It seems obvious that the subjects recognized cut instead of quote, similar instead of stimulate, and hot instead of halt. It is interesting to note that close, the verb (inverse of open), and close, the adjective (not distant), both must have been the source of the nonnative-like associations for disclose. In many cases disclose was rated as a known word, further strengthening the argument that there may be discrepancies between self-evaluation of knowledge of a word and actual mastery of associative knowledge. It seems that subjects can easily mistake a lower frequency word for a higher frequency word with a similar form if they are not careful. It also supports the caution Clarke and Nation (1980) give about using word parts to guess meaning. In other cases, the given associations seem to be related to the prompt words only on a basis of form (clang association):
(7) commitconnection, community, cooperate (8) reflecteffect, affect

32

Norbert Schmitt and Paul Meara

Meara (1983) concluded that this is a common feature among beginning learners associations. Later, when words become better known, their associations tend to become meaning-based. These examples indicate that the subjects had very little understanding of the prompt words, even though Example (7) was rated as 2 and Example (8) as 1. 2. Quite often, word associations showed a degree of conceptual correctness, but they were simply not nativelike:
(9) (10) (11) (12) denyinvitation, opposite, accept denyrefuse, invitation completewhole, moon commitcrime, purchase

These subjects seem to have acquired enough linguistic knowledge to produce some word associations that broadly imply an understanding of the basic concepts attached to the verbs, but other responses are certainly not complete or nativelike. This argues for the view that overall knowledge of a word should be seen as being incrementally acquired, because knowledge of a words meaning does not guarantee that other types of word knowledge, collocational knowledge, for example, are completely mastered. 3. Many of the associations reflected what might be called classroom English:
(13) (14) (15) (16) completeanswer, notes, paragraph completesentence, homework, report usepen, book, pencil separateclass, school, friends

Although it is normal to have a certain amount of the language in a classroom focus on the classroom environment itself, this shows the desirability of making sure that as much language as possible reflects its use in the real world. Use of authentic materials with topics not involving education can help ensure that students learn the majority of their language about nonclassroom-based vocabulary, avoiding the situation in which students know words like pencil, assignment, and study well, but little else. This should also help ensure that students develop more nativelike associations. 4. The preceding examples show that word associations not only can indicate whether a word is known or not, but that they also can often show the causes of nonnative-like responses. It would be unwise, however, to leave the impression that this is always true. In some cases, the association responses seem so strange that it is difficult or impossible to guess exactly what the subjects were thinking of as a stimulus word or what type of associations they were trying to make:
(17) acquaintaccept, promise, do (18) inheritclever, study, dictionary

5. In the course of preparing the baseline data, it became obvious that native speakers also vary in their knowledge of words. For example, in the suffix task,

Word Knowledge

33

native speakers exhibited a considerable amount of variation in their judgments of the acceptability of suffixes like -er, -ee, and -able with the prompt verbs. Evidently, native speakers do not share exactly the same intuitions about the morphology of the English language, and many find combinations of words and suffixes acceptable that are not commonly considered possible by prescriptive grammar books and dictionaries. It is true that this is fringe variation, because native speakers for the most part share a common core knowledge of the words. Still, native speaker knowledge varies to some degree, and the implications of this for L2 learning have not yet been adequately explored.

CONCLUSION
This research is a preliminary investigation of word knowledge using a framework first put forward by Nation (1990). As such, it remains extremely tentative and exploratory. Our view, however, is that exploratory work of this sort has an important role to play in developing explanatory theories out of descriptive frameworks like those of Nation (1990) and Richards (1976). What they have done is to recognize that a void exists in the vocabulary studies field and to formulate their intuitions in writing so that others can evaluate and refine those initial formulations. This research has empirically shown that there is a relationship between at least two of the kinds of word knowledge they isolated. It has also shown that suffix and association knowledge have a relationship to the overall size of the learners vocabulary and general language proficiency as measured by the TOEFL test. The learners did not have very good mastery of either association or suffix knowledge, indicating we need to learn more about the different kinds of word knowledge and how to effectively promote their learning. It is hoped that this study will inspire other research in this area. To learn about the depth of our students knowledge of words, we are going to have to integrate several things into our future vocabulary research. First, because vocabulary is learned incrementally, we must develop measures that capture partial knowledge of words, perhaps by using productive and receptive matching tasks in each study. Second, the results from this research have shown the wide variations in individual vocabulary learning. Future studies should examine this variability in addition to looking for group trends. Third, we must remember that there are a number of word knowledge competencies that make up total knowledge of a word. Hopefully, research that takes all these things into consideration will enable us to better understand the complexities of how people learn words. (Received 2 May 1996)
NOTES
1. For further details on the testing aspects of this study, see Schmitt (1995a). 2. The Wilcoxon signed ranks test was used because many of the individual suffixes were not normally distributed. 3. They did not have competent suffix knowledge for words rated as known either. The percentage of suffixes known was generally under 50%. However, this is not as surprising as the association results because suffix knowledge is not as transparently connected with meaning.

34
REFERENCES

Norbert Schmitt and Paul Meara

Bauer, L., & Nation, I. S. P. (1993). Word families. International Journal of Lexicography, 6, 253279. Carroll, J. B. (1971). Measurement properties of subjective magnitude estimates of word frequency. Journal of Verbal Learning and Verbal Behavior, 10, 722729. Clarke, D. F., & Nation, I. S. P. (1980). Guessing the meanings of words from context: Strategy and techniques. System, 8, 211220. Educational Testing Service. (1987). TOEFL test and score manual. Princeton, NJ: Author. Ellis, N. C., & Beaton, A. (1993). Psycholinguistic determinants of foreign language vocabulary learning. Language Learning, 43, 559617. Francis, W. N., & Kuc era, H. (1982). Frequency analysis of English usage: Lexicon and grammar. Boston: Houghton Mifflin. Higa, M. (1965). The psycholinguistic concept of difficulty and the teaching of foreign language vocabulary. Language Learning, 15, 167179. Hulstijn, J. H. (1992). Retention of inferred and given word meanings: Experiments in incidental vocabulary joint (Eds.), Vocabulary and applied linguistics (pp. 113125). London: learning. In P. J. L. Arnaud & H. Be Macmillan. trange ` re au cours de la lecture: Ses Hulstijn, J. H. (1994). Lacquisition incidente du lexique en langue e avantages et ses limites. Acquisition et interaction en langue e trange `re, 3, 7796. Kruse, H., Pankhurst, J., & Sharwood Smith, M. (1987). A multiple word association probe in second language acquisition research. Studies in Second Language Acquisition, 9, 141154. Laufer, B. (1990). Why are some words more difficult than others?Some intralexical factors that affect the learning of words. International Review of Applied Linguistics, 28, 293307. Laufer, B. (1994). Appropriation du vocabulaire: Mots faciles, mots difficiles, mots impossibles. Acquisition et interaction en langue e trange `re, 3, 97113. Meara, P. (1983). Word associations in a foreign language: A report on the Birkbeck Vocabulary Project. Nottingham Linguistic Circular, 11, 2938. Meara, P. (1984). The study of lexis in interlanguage. In A. Davies, C. Criper, & A. Howatt (Eds.), Interlanguage (pp. 225235). Edinburgh, Scotland: Edinburgh University Press. Meara, P. (in press). Towards a new approach to modelling vocabulary acquisition. In N. Schmitt & M. McCarthy (Eds.), Vocabulary: Description, acquisition, and pedagogy. Cambridge: Cambridge University Press. Meara, P., & Jones, G. (1990). The Eurocentres Vocabulary Size Tests 10KA. Zurich, Switzerland: Eurocentres. Meara, P., & Rodriguez Sanchez, I. (1993). Matrix models of vocabulary acquisition: An empirical assessment. In Proceedings from the CREAL Symposium on Vocabulary Research (pp. 2426). Ottawa, Ontario, Canada: University of Ottawa. Melka Teichroew, F. J. (1982). Receptive versus productive vocabulary: A survey. Interlanguage Studies BulletinUtrecht, 6, 533. Milton, J., & Meara, P. (1995). How periods abroad affect vocabulary growth in a foreign language. I.T.L., 107108, 1734. Morgan, C. L., & Bonham, D. N. (1944). Difficulty of vocabulary learning as affected by parts of speech. Journal of Educational Psychology, 35, 369377. Nagy, W. E., Herman, P. A., & Anderson, R. C. (1985). Learning words from context. Reading Research Quarterly, 19, 304330. Nation, I. S. P. (1990). Teaching and learning vocabulary. Rowley, MA: Newbury House. Nation, I. S. P. (1983). Testing and teaching vocabulary. Guidelines, 5, 1225. Postman, L., & Keppel, G. (1970). Norms of word associations. New York: Academic Press. Read, J. (1993). The development of a new measure of L2 vocabulary knowledge. Language Testing, 11, 355371. Richards, J. C. (1976). The role of vocabulary teaching. TESOL Quarterly, 10, 7789. Schmitt, N. (1995a). An examination of the behaviour of four vocabulary tests. In D. Allan (Ed.), Entry points: Papers from a symposium of the research, testing, and young learners special interest groups (pp. 3439). Whitstable, England: IATEFL. Schmitt, N. (1995b). Verbal suffix and word association knowledge of Japanese students. Unpublished masters thesis, University of Wales, Swansea. Shapiro, B. J. (1969). The subjective estimation of relative word frequency. Journal of Verbal Learning and Verbal Behavior, 8, 248251. Singleton, D. (1994). Introduction: Le ro le de la forme et du sens dans le lexique mental en L2. Acquisition et interaction en langue e trange `re, 3, 327. So derman, T. (1993). Word associations of foreign language learners and native speakersA shift in response

Word Knowledge

35

type and its relevance for a theory of lexical development. In H. Ringbom (Ed.), Near-native proficiency bo, Finland: A bo Akademi. in English. A Takala, S. (1984). Evaluation of students knowledge of English vocabulary in the Finnish comprehensive school (Rep. No. 350). Jyva skyla , Finland: Institute of Educational Research. Tryk, H. E. (1968). Subjective scaling of word frequency. American Journal of Psychology, 81, 170177. Websters ninth new collegiate dictionary. (1987). Springfield, MA: Merriam-Webster.

APPENDIX
SAMPLES OF THE TEST ITEM FORMATS
Productive 1 19 Receptive 1 14 use quote -able -age -al -ance/ence -ed -ee -er/or -ing -ion -ive -ly -ment -s -ure -able -age -al -ance/ence -ed -ee -er/or -ing -ion -ive -ly -ment -s -ure computer ground employ famous tool say speak person use 0 1 2 3 halt 0 1 2 3

36
LIST OF PROMPT VERBS WITH THEIR ALLOWABLE SUFFIXES AND THE RECEPTIVE ASSOCIATIONS AS THEY APPEARED ON THE TEST INSTRUMENT
-ing -s -ly -ment -ive -ive -ive -s -s -s -s -s -s -ing -s -ment -s -s -s -er/or -ing -s use agree separate arrange stimulate indicate reflect proceed disclose deny refer commit employ develop continue complete quote acquaint halt inherit -ure -s -ee -s -ing -s -able -able -able -able -ed -ed -ed -ed -able -able -able -ed -able -ed -al -ed -able -ance/ence -ed -able -age -ed -ed -ed -er/or -er/or -er/or -ing -ed -al -al -ee -ed -er/or -ed -ing -ed -ed -ing -ance/ence -ed -ing -ing -er/or -ing -ing -ing -s -ing -ed -ance/ence -ing -ee -ing -ing -ion -er/or -ing -s -ed -er/or -ment -ion -ing -ion -ion -ion -ure -s -ing -ed -ment -er/or -ment -s -ly -ing -s computer noise turn fix energy tell busy meat information lie job promise use grow trick carry ground meet army school employ same divide flowers half point mirror go wing mix effort sin clear laugh ahead finish famous friend stop death tool contract apart cool excite show light continue tell refuse direct decide job film proceed work say know brain house speak opinion divorce marriage drugs lovely think forward secret hide recommend flat worker idea go on end person zone guard money