Reproduction of Prose Passages
- Author:
Rupert Clendon Lodge,
University of Minnesota, and Joseph Leonard Jackson, University of Alberta.
Introduction.
These tests were given to students of the University of Alberta, in various classes, during the last days of January and the first days of February, 1916. The students tested were not, for the most part, members of psychological classes, and none of them were practised in taking such tests. In giving the test by the group method, we wished to see how far it could be used as a reliable index of ability in college studies, when dealing with groups such as the Freshman and Sophomore classes, sex groups, and age groups. We did not attempt (as Travis2 did) to discover whether the test gave satisfactory results for individuals. For in order to gauge the intelligence of individuals it is generally agreed3 that one kind of test alone is insufficient, and that a series of different tests give better results; and in the second place, tests given to classes are not given under the clinical conditions desiderated by Binet and those who follow his methods;4 while finally, even under favorable clinical conditions, Goddard finds the tests for “fifteen” and “adult,” of which the present test is a modification, unreliable.5
1See “Reproduction of Short Prose Passages: a Study of two Binet Tests,” by A. Travis, Philadelphia, The Psychological Clinic, vol. ix, 1915, pp. 189-209. Miss Travis suggests for use with university students three new passages (203, 204), but has not apparently worked with them herself. Her paper furnishes the material for the present investigation, in which the passages suggested are used for the first time. 2 See Travis, op. cit., pp. 191, 192, 193, 201, et al.
3 Cf. e. g. “A New Scale of Mental and Physical Measurement for Adolescents, and Some of its Uses,” by Helen Thompson Woolley. Jour. Educ. Psych., vol. vi, 1915, p. 521.
4 Cf. e. g. “In practice the examination should be conducted in a qu it place, the child being taken alone and as free from distractions as possible.” Henry H. Goddard, “Binet’s Measuring Scale for Intelligence,” The Training School, vol. vi, No. 11, 1910. Cf. also “The Binet Measuring Scale of Intelligence. What it is and how it i> used,” by the same, p. 6.
‘Fifteen years’ and ‘Adult’ have not proved reliable” Henry H. Goddard, “Standard Method for Giving the Binet Test,” The Training School, vol. x, April, 1913, last paragraph of article.
The students tested are grouped as follows: Class Men Women Total Senior and Graduate. Junior Sophomore Freshman Matriculant1 Totals. 20 24 36 42 13 135 5 3 15 15 6 44 25 27 51 57 19 179
Procedure.
The tests were given by R. C. L., who, on entering the classroom, distributed sheets of paper, one to each student, and gave the following instructions: “Please write, at the head of your paper, your name, age in years and months, class, and the time. I am going to read a prose passage. You will be given three minutes in which to write out as much as you can of the sense of the passage. The actual wording does not matter. Is there any one who does not understand? “2
After answering questions asked by students, in the course of which the additional information was always given (1) that the papers were required by the department of psychology for purely scientific purposes, and (2) that the results would not in any way prejudice the academic standing of the students, R. C. L. then read the first passage slowly, and the students were subsequently allowed precisely three minutes for the reproduction of the sense of what they had heard. At the end of the three minutes they received the command “Stop?at once please!” The instructor proceeded: “I will now read a second passage in a similar way, and you will reproduce the sense of it in the same manner.”
1 The Matriculant Class is from Alberta College South, a theological college affiliated with the university and situated on the university campus.
2 The instructions given are more explicit than those in the experiment reported by Travis (190); this was necessary, as our students were not taking the test as “a regular class-room exercise” (189). It will be noticed that the instructions were to reproduce the sense, rather than the wording, of the selections. This was done (1) in order to make the test call forth the same general direction of effort from all the students, and (2) to make it a test for “intelligence” rather than, as Binet puts it, a “memory span for ideas” (see Travis, 201). We also imposed the time-limit of three minutes in order to equalize the time-factor as far as possible, whereas in the experiment reported by Travis, they were allowed to take as long as they pleased (190). This, while correct when trying to obtain results for individuals under clinical conditions (cf. Goddard, “Standard Method for Giving the Binet Test,” p. 2, The Training School, vol. vi, 1913, No. 2, April), seemed to us unadvisable when dealing with groups, as in our investigation. Miss Travis informs us (1) that as “a regular class-room exercise,” the students were allowed to take as long as they pleased, and (2) that there “proved to be no such difference … between the maximum and minimum times as to make it advisable to limit the time in any way.” Our conditions were different.
We did not think it necessary to allow “a few minutes for rest and relaxation,” but proceeded at once to read the second passage, and after that had been reproduced, the third, in a similar manner. The students were then asked, if men, to make a large M, and if women, to make a large W, at the foot of the sheet; after which the papers were collected and class-work was resumed. The whole time taken, from entering the room to having the papers collected, occupied about eighteen minutes.
Marking.
For marking the papers, two methods were adopted, which, following Travis, we shall call “qualitative” and “quantitative.” The qualitative method, suggested by Travis as likely to be preferable for the first two selections of prose,1 ranks the papers with reference to the following qualities: “Judgment in organizing and condensing material, terseness and vigor of expression, and finally, what is hard to describe but easy to recognize, an effect of skill and mastery in the execution of the task.”2 The quantitative method consists in rating the papers “on points,” i. e. in giving unit credit for each idea reproduced. The chief difference is that the quantitative method allows nothing for “coherence,” while the qualitative method allows no credit for detached words, which reproduce, perhaps, ideas, but not their connection. For the results obtained by the qualitative method R. C. L. is solely responsible, while J. L. J. carried through much of the quantitative rating. The two methods were applied independently of one another.
The qualitative method contains two parts. The papers were ranked, on the qualitative basis explained above, into five groups, A, B, C, D, E. This was done independently for Selection I, then for Selection II, and then for Selection III. The mark for each selection is called a “score,” and for 179 students there are accordingly 537 scores in all. For the purpose of comparing the different class-groups, sex-groups, etc., in which the numbers were unequal, the number of scores was expressed in percentages, so that the groups could be compared in respect of their percentage of A scores, B scores, etc. This represents the first part of the method. In the second part, as cases arose in which one class seemed to counterbalance a weakness in A scores with a superiority in B scores, and thus made this method of comparison difficult, it seemed advisable to devise some means whereby the average score of one group could 1 Cf. Travis, pp. 203, 204. 2 Travis, p. 193. be compared directly and arithmetically with the average score of another group. For this purpose, numerical values were assigned to A, B, C, etc., in each selection. In order (1) to avoid arbitrariness, and (2) to achieve a basis of comparison with the “quantitative” results, the numerical values assigned have precisely the value of the average A, B, C, etc. obtained in the application of the quantitative method. The exact values assigned were as follows: Selection I. . Selection II. Selection III. 34.610 62.465 43.056 B 24.825 45.425 34.273 14.034 27.747 27.673 D 7.690 16.240 20.227 E 1.672 6.327 12.500
The significance of these numerical values will be understood when it is realized that the average mark which would be assigned a place in grade A by the quantitative method (34.610) represents precisely the reproduction of 34.6 out of 100 ideas which might possibly have been reproduced in that selection. It is not claimed that every paper to which the score A is given by the qualitative method reproduces exactly 34.6 out of a possible 100 ideas; but treated on the average, as is necessary for group results, it seemed less arbitrary to give to every qualitative A the average mark for the corresponding grade obtained by the quantitative method.1 Results Obtained by the Qualitative Method.
(a) Class groups. Class Freshman… Senior Sophomore.. Junior Matriculant. 5.2 9.3 2.0 2.6 1.7 B 16.2 13.3 15.7 14.1 14.0 57.2 49.3 51.0 43.6 26.3 D ? 13.9 17.3 20.3 18.0 42.1 E 7.5 11.0 11.0 21.8 15.8 This table shows the distribution of scores according to classes. The Seniors obtain the highest percentage of A scores, the Freshmen 1 In the paper reported by Travis, the data were ranked as if they were college quiz papers, in five groups D, G, P, N, and F, with the percentages usual at Pennsylvania. On the basis of this standard Travis finds that so many of the better students “make a poor showing” that she tends to regard the tests as “unreliable” (pp. 191, 192, 208). This method is incorrect. ” The final appeal in all questions of doubt should be, not to what the examiner might expect … but rather to the fact as to what normal children do under such circumstances.” (Goddard, “Standard Method, etc.,” p. 4.) It was for this reason that we adopted as our numerical values averages derived from what the students actually performed, rather than the “college quiz” standard, which in this case, is (of course) arbitrary. of B and C, the Matriculants of D, and the Juniors of E scores. The classes are arranged in order of achievement, though difficulty was experienced as to whether the relatively large Senior score in A does not offset the Freshman results in the other grades. To solve this difficulty, the numerical values adopted above were substituted for the actual scores, and the general average scores of the classes, each class being compared with the other students grouped together, are as follows:
Class Freshman… Senior Sophomore.. Junior Matriculant. Average Score 23.714 23.334 22.053 20.550 17.787 P. E. 3.76 5.86 4.06 4.44 2.43 Average Score of Rest 21.132 21.604 21.919 22.360 22.176 Difference 2.582 1.730 0.134 -1.810 -4.389 These figures show the same results as those obtained above, but rather more clearly for purposes of general comparison. In fact, the second tabulation tends to express the results a little more exactly in the present case, for it is more arbitrary to treat all A scores as equal, whether obtained in I, II, or III, than to treat them as of the different average values discovered by the quantitative method. (6) Sex groups.
It will be noticed that, while approximately equal in respect of C scores, the women preponderate in B and D, the men in the extreme grades, A and E. This preponderance is fairly steadily maintained in the various classes, for which the results are as follows: Class Matriculant Freshman. Sophomore Junior Senior…. Totals.. Women 4.4 2.2 2.3 B 5.6 15.6 17.8 33.3 26.7 17.0 27.8 62.2 44.4 16.7 60.0 48.8 D 66.7 13.3 24.4 50.0 13.3 26.3 4.4 11.1 5.4 Men 2.6 5.5 1.9 2.8 11.7 4.7 B 18.0 16.4 14.8 12.5 10.0 14.5 25.6 55.5 52.7 45.9 46.7 49.1 D 30.8 14.0 18.5 12.3 18.3 17.7 23.0 8.6 11.1 23.6 13.3 14.0
These results indicate that the general result obtained by comparison of the sex-groups as wholes is fairly consistently maintained in detail. The men tend towards the extremes in greater proportion than the women, while the women are almost all to be found in the middle grades. But though this difference of distribution is strikingly displayed in this form of marking, it is not easy to see whether the men or the women “do better.” As Travis, like most investigators in this field, insists that the women do better, it becomes necessary to transform these marks into their numerical equivalents, in which case we find the results to be as follows:
Class Average Female Average Male Difference in Favor of Female Senior Junior Sophomore.. Freshman… Matriculant. Totals.. 25.948 21.679 22.666 24.359 17.038 24.664 P. E. 3.73 22.664 19.552 21.769 24.057 17.891 21.669 P. E. 4.77 3.284 2.127 0.897 0.302 -0.853 2.995
On the whole, these figures indicate that the results for the sex groups are in a line with similar results on the part of other workers in the field of tests.
Age groups.
As eight of the students omitted to fulfil the instructions respecting their “age in years and months,” only 171 students could be compared from the point of view of age. It was decided to compare those below the age of twenty-five with those of twenty-five and upwards, and also to compare students between the ages of eighteen and twenty-four (inclusive) with a group consisting of those above and those below those limits. By the second part of the qualitative method the results are as follows: Age 16-24.. 25-41.. 18-24.. 16-17 25-41J Number of Students 116 55 101 70 Average Score 22.906 20.708 22.819 21.493 P. E. 2.54 2.93 4.13 4.34
Difference
2.198 (in favor of 16-24) 1.326 (in favor of 18-24) These results indicate that, taken in groups, students below the age of twenty-five tend, on the average, to obtain a higher score than students above that age.
The students were also compared in “time-groups,” to discover whether morning tests gave higher scores than afternoon tests. With the material at our disposal, the results were negligible. Results Obtained by the Quantitative Method.
By the quantitative method, marks were assigned to the papers in such a way as to express the percentage of possible work accomplished.1 Further, (1) for purposes of comparison with the “qualitative” results, and (2) in order to make clear the distribution of scores, a small group of the highest scores was called A, a larger group of the next highest scores was called B, the largest group of scores, distributed closely about the average, was called C, and D and E corresponded, as closely as possible to the number of scores called B and A respectively. The number of A scores, B scores, etc., was then added in the various class groups, age groups, etc., and expressed in percentages, precisely as was done with the similar marks in the qualitative method. While there are differences in detail between the two methods, the average mark for which an A or a B is given is precisely the same in both methods, so that a direct comparison of the results should be possible. (a) Class groups. Class Senior Freshman Sophomore. . Junior. Matriculant. 12.5 10.9 12.4 9.9 5.3 16.6 18.4 13.0 18.5 7.0 38.9 43.1 39.2 37.0 43.9 16.6 16.1 16.9 14.0 14.0 15.3 11.5 18.3 19.8 29.8 Average Score 24.399 23.994 23.563 21.576 16.897 Average of Rest 22.616 22.519 22.387 23.028 23.574 Difference 1.783 1.475 1.176 -1.452 -6.677
Like the qualitative scores, these figures leave it doubtful whether the Freshmen or the Seniors are to be ranked first; but the general averages, which in the application of the quantitative method are rigorously exact, show that the Seniors are to be placed first. While by the qualitative method the Freshmen were decidedly first, as a group, by the quantitative method, the Seniors receive a slightly higher score. The general proportion between the other classes is much the same by both methods.
Sex groups.
These figures show that, on a quantitative estimation, the women not only preponderate in the three middle grades, but also 1 For a detailed discussion of the quantitative method, see Appendix.
Class Matriculant Freshman.. Sophomore. Junior Senior Totals.. Women 8.9 17.7 20.0 11.3 B 5.5 22.2 17.7 22.2 33.3 19.7 55.5 42.2 37.7 44.4 40.0 42.4 D 22.2 17.7 15.5 22.2 6.7 16.7 E 15.9 8.9 11.1 11.1 9.8 Men 7.7 11.6 10.2 11.1 10.5 10.6 B 7.7 17.0 11.1 18.0 12.3 14.1 38.5 43.4 39.8 36.1 38.6 40.0 D 10.3 15.4 17.6 13.9 19.3 15.8 E 39.9 12.4 21.3 20.8 19.3 19.5
that they tend to secure more A scores on the whole, and especially in the Senior class. Certain differences between the results obtained by the two methods stand out more clearly when we compare the average scores obtained by the application of the quantitative method:
Class Total scores. Senior Sophomore.. Matriculant. Freshman… Junior Female 23.99 30.68 26.85 17.68 24.58 20.82 Average Male 22.49 22.83 22.20 16.54 23.78 23.43 Difference in Favor of Female 1.50 7.86 4.65 1.14 0.80 -2.61
Here we notice, not only that the differences between the average scores made by the sexes in the various classes are greater, but also that in the descending order of the classes, the Sophomores, who ranked third qualitatively, now rank second, and that the Juniors, who ranked second qualitatively, are now placed last, while the Freshmen and Matriculants have changed places. These and other differences between the two methods of marking the same papers appear clearly by such a comparison.
(c) Age Groups. Age 16-14. 25-41. 18-24. 16-17 25-41 Number of Students 116 55 101 70 Average Score 25.120 20.163 25.383 21.717 Difference 4.957 (in favor of 16-24) 3.666 (in favor of 18-24) These figures bear out the results obtained by the qualitative method. Conclusions from the Results Obtained by Both Methods. Without as yet raising the question of the validity of conclusions obtained from these results, it is possible to draw tentatively certain definite conclusions. Where there is a difference amounting to a contradiction between the results obtained by the two methods, we have decided to prefer the “qualitative” results.1 Our conclusions are as follows:
(a) Class groups. While it is far from easy to decide between the Freshmen and the Seniors, we are inclined, on the basis of the results before us, to consider the Freshman class as the more intelligent on the whole. This superiority seems to us to rest mainly on “natural ability,” for a first-year class could hardly claim to have “superior training” as compared with men who have been trained in the institution for four years. The relatively high score of the Seniors, coupled with the fact that they appear yet higher when estimated on a basis which pays regard to quantity rather than quality of the reproduction, seems to us to indicate that the superiority of the Seniors to the Juniors, Sophomores, etc., is largely a matter of superior training, though here too it seems to us that the Seniors have slightly greater “natural ability” than any of the other classes, with the exception of the Freshmen.
In the case of the Sophomores and Juniors, it appears to us that the Sophomore class is decidedly more alert and active, while the work of the Junior class indicates a considerable degree of laziness (cf. the large number of E scores), especially as regards the men students. In regard to “intelligence” we are unable to draw any sharp distinction; the difference appears to us to depend chiefly on persistence as opposed to laziness and unwillingness to throw themselves into work which was not to “count” from an academic point of view.
The difference between the matriculant class and the matriculated students, in spite of one or two excellent matriculant papers, appears to us to be clearly established; in this connection it is interesting to note that, while by the quantitative method all the other classes received a relatively higher score than when judged qualitatively, the matriculant class alone is seen to be still more inferior, an inferiority presumably due largely to absence of training.
(b) Sex groups. In order to understand the results for the sex groups, it appears to us necessary to take together the two main 1 See Appendix.
conclusions to which the results by both methods point. If we consider the comparison by averages, we should tend to say, like Travis,1 that the women “do better;” but if we consider the comparative distribution of the scores, especially the A and E scores, it looks as though the men do both better and worse; i. e. they seem to exhibit greater excellence (A), and also a considerable degree of laziness (E). Consequently, although the women “do better” in the average, we cannot claim that they are more “intelligent.” They seem to be much nearer the general average, whereas the men tend proportionately more towards the extremes, A and E. In the light of this result, it is necessary to conclude that the women are more persistent and less variable as a group, but that the men are not only more lazy, but also more “brilliant;” on the whole, they tend to attain higher results when they do try, though in many cases (as indicated by the proportion of E scores), they were not easily induced to try.
(c) Age groups. From the results obtained by both methods, we conclude that, if these tests are at all reliable in dealing with groups, students will, on the whole, do slightly better in college studies if they are below the age of twenty-five. The fact that the 16-24 group did slightly better than the 18-24 group appears to us to be due to the fact that the individual students under eighteen are rather more able than average individuals of that age. On the whole, then, we conclude that students below the age of twentyfive will tend to get the most out of their lectures and reading, at least so far as ability to reproduce the sense of what they have heard is concerned.
General Conclusions.
(1) Validity of the above conclusions. We are informed by the Controller of Examinations that the Freshman class is considered the ablest “first year” which the university has known for many years; that of the other classes, the Senior year is considered the ablest, the Sophomore year the next in order along the lines of ability approved of in colleges, and that the Junior year, as far as the men are concerned, is not as successful, either in respect of academic attainments or in the mysterious qualities which go to make up “class spirit.” We are further informed by the same authority that our general conclusions with regard to sex and age groups are in fair general agreement with what the university administration has come to believe on the basis of examinations. From this infor]C?. Travis, p. 202. mation it would appear that, in dealing with groups of students, the test gives sufficiently accurate results to warrant its use as an instrument of diagnosis. But, for the reasons stated in the introduction, as well as on account of the magnitude of the probable error, we do not recommend its use for mental diagnosis in dealing with individual students: we know of no test which can be used for this purpose when given by group methods.1
(2) Comparison with the conclusions of Travis. There are two fundamental differences between the paper of Travis and the present investigation which make any comparison of their results difficult. In the first place we used material suggested, but not used, by Travis: the conclusions numbered (1) and (5) in her paper, therefore, dependent, as they are, on material already employed for testing purposes, which is in many respects different from the material suggested by her as more desirable, are incapable of comparison with any results obtained by us. In the second place, Travis is throughout attempting to use tests given by the group method in order to test individuals, and it is from this point of view that we find her stating that her ” conclusions are negative.” This point of view is particularly apparent in the conclusions numbered (1), (2), and (4). The only conclusion remaining, which is numbered (3), viz. that “women do better in these tests then men,” would from our point of view require to be qualified by the words “on the average,” and would require to be yet further qualified by insisting on the fact that the men reach a higher standard on the one hand, and on the other exhibit a greater degree of laziness, which is what accounts for the women’s apparent superiority in respect of the average.
Appendix. Discussion of the Quantitative Method and its Value.
The passages used in this experiment were suggested by Travis in her article, pp. 203, 204.2 For the purpose of scoring by points, we followed the method of analysis into essential ideas,3 and gave equal credit for each idea thus reproduced. Selection I was analysed into twenty-six ideas, as follows:
To counterfeit (5) perpetually
the hand of God (6) without warrant
is the boldest of all forgery. (7) but his own fantastic surmise,
(4) He who takes upon him (8) to unfold 1 Travis seems to reeognizc this towards the end of her paper (see p. 208, conclusion No. 4). 2 See Travis, pp. 203, 204. 3 See Travis, pp. 192, 193. REPRODUCTION OF PROSE PASSAGES. 139 (9) the secret (18) of those thoughts (10) and unsearchable mysteries (19) reprobate (thoughts) (11) of high providence (20) that would wrest (12) is likely for the most part (21) the sword (13) to mistake (22) of justice (14) and slander them, (23) out of God’s hand (15) and approaches (24) and employ it (1G) for the most part (25) more justly (17) to the madness (26) in their own conceit. As there are thus twenty-six possible points, the score, expressed in percentage form, is 3.85 per cent for each idea reproduced. The average total score was 14.034 per cent, which was accordingly allotted the grade C. In the grading, in accordance with the normal curve of distribution,1 the grades were assigned as follows:2
E per cent 0.0-3.85 D per cent 7.69 c per cent 11.54-19.23 B per cent 23.08-26.92 A per cent 30.76-50.00 This method of scoring, in giving an identical mark for each and every idea reproduced, seems to presuppose that each and every idea is equally hard to reproduce, and therefore deserves equal credit. This is an a priori notion, not borne out by an examination of what the students actually achieved. The exact frequency of reproduction of the different ideas was: Frequency per 100 Students 35.19 34.64 31.84 21.79 20.11 18.98 17.32 16.75 Number of Idea 2 26 1 11 3 20, 22 21 23 Frequency per 100 Students 16.19 13.38 11.73 10.61 9.49 6.69 5.28 4.47 Number of Idea 10 24 9 8 14 7 17 18, 25 Frequency per 100 Students 3.35 2.79 2.23 1.67 1.12 0.56 Number of Idea 4, 12 15 6 13, 16 19 5
These results indicate that such ideas as “hand of God,” “conceit,” “counterfeit,” “providence,” and a few others were most 1 Travis (192) seems to approve of Finkelstein’s marking system; but according to this (“The Marking System in Theory and Practice,” 191.3, p. 16) “the theoretically ideal curve of … college marks is … not the probability curve, but the skewed curve with the mode to the right of the middle of the abscissa. We have followed Boring in thinking this unjustified, and have retained the probability curve. (See E. G. Boring, “The Marking System in Theory,” Pedagog. Seminar, vol. xxi, 1914, esp. p. 227.) 2 The average attained in each of these grades has been given in dealing with the qualitative method above140 THE PSYCHOLOGICAL CLINIC. readily apperceived and reproduced by the students. But the small figures (only 35 per cent for the highest) show further that no one idea impressed itself on much more than a third of the students. There is thus a considerable degree of variation, and by this method of grading it is possible for two students to secure a similar score, in which, however, the points reproduced by the one consist of relatively essential ideas, while those reproduced by the other may consist of relatively unessential ideas, incoherently strung together. This is an inadequacy inherent1 in the “quantitative” method, as will be seen more clearly when we consider certain specimen papers: Specimen Paper I. He who seeks (4) to understand and explain (8) the thoughts and actions of God (? 11) attempts a very hard task. His attempt is sure to end in a frenzy (17). His thoughts (18) are very similar (15-16) to those of a reprobate (19). He seems as though he were trying to take (20) the sword (21) of justice (22) out of the hand of God (23). Specimen Paper II. To counterfeit God (1-2) is the most presumptuous act of man. To think that man can attain to His thought is only to show conceit (26), especially to imagine that one can do more wisely and more justly (25). The quantitative grading of these papers would assign I to grade A, and II to grade C. And yet it is clear to the “qualitative judgment” that II is distinctly superior. Qualitatively, in fact, after a careful comparison with the other papers, II was assigned to grade A, and I received the grade C. The one paper shows a real understanding of the “sense of the passage,” while I indicates only a mediocre appreciation of the content of what was read. Selection II was analysed into thirty-four “essential ideas,” as follows:
Exposing (11) and also that (heating power)
his thermometers (12) of the region beyond
to the colors (13) the extreme red.
successive (colors) (14) Then,
of the spectrum (15) drawing a line
solar (spectrum), (16) straight (line)
Sir William Herschel (17) to represent
determined (18) the length of the spectrum,
the power of each (color) (19) he erected perpendiculars
(10) heating (power) (20) at various points 1 In Travis’ paper a similar difficulty was experienced, but it seems to have been solved arbitrarily: cf. “a score of 81 per cent … strictly should be 79 per cent, but some papers with 11 points were better than others, and some were worse, so that in ranking the papers by percentages the seven papers with 11 “points each were distributed over the range from 81 per cent down to 76 per cent inclusive” (p. 194). REPRODUCTION OF PROSE PASSAGES. l4l (21) to represent (28) which showed (22) the intensity (29) at a glance (23) calorific (intensity) (30) the manner in which (24) existing at those points. (31) the heat (25) Uniting the ends (32) was distributed (26) of all his perpendiculars, (33) in the spectrum (27) he obtained a curve (34) solar (spectrum). As there are thus thirty-four possible points, the score, expressed in percentages, is 2.94 for one idea reproduced, 5.88 for two ideas, and so on. In accordance with the normal probability curve, the grades were assigned as follows: E per cent 0.0-11.76 D per cent 14.71-17.65 C per cent 20.59-38.24 B per cent 41.18-50.0 A per cent 52.94-100
The averages attained in these grades have been given above, in dealing with the qualitative method. The exact frequency of reproduction of the thirty-four different ideas was: Frequency per 100 Students 66.36 57.54 54.18 53.06 51.38 45.25 44.70 42.34 39.67 37.43 Number of Idea 2 1 6 18 27 4 23 7, 16 33 26 Frequency per 100 Students 34.07 33.51 31.84 30.16 27.37 25.69 24.58 24.02 23.46 20.11 Number of Idea 12 34 11, 28 5 31 10 3, 20 25 8 17 Frequency per 100 Students 18.44 17.98 15.64 14.53 12.82 10.61 8.94 6.69 6.14 Number of Idea 32 19 21 13, 30 15 29 9, 24 22 14
These results indicate that such ideas as “thermometers,” “exposing,” “solar,” “length of spectrum,” “he obtained a curve,” and a few others made the most pronounced impression on the students. We further notice that the five ideas mentioned impressed themselves on more than one-half of the students, while no one idea failed to impress less than six per cent of the students tested. From this one might be tempted to infer that the content of selection II came more within the grasp of university students than did the more abstract and “theological” selection I. To some extent this may perhaps have been the case. But an experiment which we made in order to test this indicates that another factor was still more prominent. A group of twenty students was tested by having selection II read first and selection I second. The results were as follows:
Selection Read First Number of Students Tested Average Score for I, per cent Average Score for II, per cent 20 159 87.65 82.67 47.24 105.13
The difference shown by these figures between a prose passage taken first and a prose passage taken second thus appears to be, in one case 40.41 per cent and the other 22.46 per cent, or, on the average for 171 students, 36.62 per cent, on the general basis chosen for comparison. The difference of 36.62 per cent on this basis is not due only to the order in which the passages are read: undoubtedly other factors also enter in, but we have not been able to discover any method for evaluating and discounting the other factors. In order to illustrate (1) the general inadequacy of the quantitative method in cases where its results conflict with “qualitative’ scoring, and (2) the apparently greater ability to reproduce the sense displayed in dealing with selection II, two specimen papers are given which to some extent correspond to the similar papers given as specimens of work done on selection I. Specimen Paper III. By exposing (1) his thermometers (2) to the sun, Sir Wm. Hershaw (? 7) found the heat (10) of each of colors (9). By drawing a line (15) to represent (17) the length of the ray (? 18) from ultra red to ultra violet (? 12, 13), and by erecting perpendiculars (19) which represented (21) the intensity (22) of each ray (? 24), he was able to draw a curve (27) representing (28) the different colors (? 3) of the spectrum (33).
Specimen Paper IV. Sir William Herschel (7) exposed (1) thermometers (2) to different colored rays (3), obtaining readings for each (8 or 9?), and even for those beyond (12) the red ones (? 13). He then (14) drew a straight line (15, 16) and erected perpendiculars (19) to mark the different colors of the rays in order, making the height of the perpendiculars proportional to the temperature produced. By drawing a curve (27) through the tops of the perpendiculars, he showed (28) graphically the calorific heat of the different rays.
From the purely quantitative point of view, III received the grade A, and IV the grade C; and yet it is obvious to the qualitative judgment that IV is far superior in reproducing the sense of the passage. From the qualitative point of view, IV received A, while III received C. Ill in fact shows understanding neither of the experiment nor of the object of the “curve,” while IV, in spite of the phrase “calorific heat,” indicates a thorough grasp of both the really essential points.
Selection III was analysed into fifty-five “essential ideas,” as follows: (1) The Normans (29 (2) are a people (30 (3) cunning (31 (4) and revengeful. (32 (5) Eloquence (33 (6) and dissimulation (34 (7) appear to be their qualities (35 (8) hereditary (qualities). (36 (9) They can stoop (37 (10) to flatter, (38 (11) but unless they are curbed (39 (12) by the restraint (40 (13) of law, (41 (14) they indulge (42 (15) the licentiousness (43 (16) of nature (44 (17) and passion. (45 (18) Their princes (46 (19) affect the praise (47 (20) of munificence (48 (21) popular (munificence). (49 (22) The people (50 (23) observe the mean (51 (24) or rather (52 (25) blend the extremes (53 (26) of avarice (54 (27) and prodigality, (55) (28) and in their thirst As there are thus fifty-five possible points, the score, expressed in percentages, is 1.82 for one idea reproduced, 3.04 for two ideas, and so on. The average score was 27.673 per cent, to which accordingly the grade C was assigned. The grades were assigned to the different percentages as follows: eager (thirst) of wealth and dominion, they despise whatever they possess and hope whatever they desire. Arms and horses, the luxury of dress, the exercises of hunting and hawking, are the delight of the Normans, but on occasions pressing (occasions) they can endure with patience incredible (patience) the inclemency of every climate and the toil and abstinence of a life military (life). E per cent 0.0-14.54 D per cent 16.36-21.82 C per cent 23.64-30.91 B per cent 32.73-36.36 A per cent 38.1S-52.73 144 THE PSYCHOLOGICAL CLINIC. The exact reproduction frequency per hundred students, of the fifty-five ideas was: Frequency per 100 Students 99.44 84.36 73.18 68.16 54.74 “46.36 45.25 43.00 42.45 41.90 36.88 36.32 35.19 34.07 33.51 Number of Idea 1 2 43 47 55 14 3 18 10 41 51 54 9, 26 13 11, 15 Frequency per 100 Students 32.30 31.84 31.28 30.16 29.60 28.49 26.81 25.14 24.02 23.46 22.35 21.79 21.23 20.11 18.98 Number of Idea 39 50 32 33, 42 38 22 52 34, 35, 46 45 40 28 17 4,6 27 7 Frequency per 100 Students 18.44 17.32 16.19 15.64 15.09 13.97 12.26 11.18 7.80 6.14 5.59 5.23 4.47 3.35 2.23 Number of Idea 37 20, 25 53 19 12 30, 31 5, 21 36 8, 48 16 23 29 49 44 24
These figures show that such ideas as “the Normans,” “delight,” “ability to endure,” “military (life),” “indulgence,” and a few others made a very considerable impression, as judged by the percentage of students reproducing these ideas. This appears to indicate (1) that the subject was well within the comprehension of the students, and especially (2) that the students were now well practised. Both factors were present, but the test does not provide any way of deciding definitely which factor played the larger part. Two specimen papers are added, in order (1) to illustrate a typical difference between the quantitative and qualitative markings, and (2) to show the kind of work done by the students.
Specimen Paper V. The Normans (1) are a cunning (3) race (2). But unless curbed (11) by law (13), the Norman will indulge (14) in acts of licentiousness (15). So the Norman always despises (32) that which he has (33) in hoping (34) for more (? 35). Dress (39), hunting (41) and hawking (42) occupy the mind of the Norman (44) a great deal (? 43). But, having these properties, the Norman can live under (47) any climatic conditions (51 and ? 50), and the rigors (52 or 53) of a …
Specimen Paper VI. The Normans (1) are a cunning (3) and revengeful (4) people (2). There is this difference in the quality of upper and lower classes, that the rulers (18) show a desire to obtain the approval of the people (19 and 21), while the people (22) are lovers of sport. They tend to luxuriousness, but on occasions (45) can endure (47) the rigor (50) of any climate (51) to which military (55) exigencies may expose them.
From the purely quantitative point of view, V is assigned the grade A, and VI the grade C. But, when judged qualitatively V shows considerable weakness. The connection between the different sentences “But … So …” is verbal only: there is no inner thought-connection. In VI, on the contrary, the general sense is well understood, though a certain condensation of the thought is evident: e. g., “they tend to luxuriousness, but . . seems to sum up both the “luxury of dress” idea and the “indulgence in licentiousness of nature and passion” notion. Qualitatively considered, V was given C, and VI the grade A, after careful comparison with the other papers.
Conclusions as to the value of the quantitative method. (1) The specimen papers show that the quantitative method can easily assign a higher grade to an inferior paper.1 (2) The quantitative method assigns higher value to “rote memory” than to intelligent condensation, coherence, and vigor of expression.
(3) Where there is disagreement between the results of the two methods, the qualitative method is likely to be more nearly correct. (4) The quantitative method has some value, in that it enables us to assign different values to A, B, C, etc., in the different selections in accordance with the different degree of accomplishment in those selections. These differences are due to (a) intrinsic differences with regard to difficulty of reproduction, and (b) the effect of practice, which gives a lower degree of accomplishment in the selection read first and an increased degree of accomplishment in the passages read later. Without the use of the quantitative method all A, B, C, etc., scores would have to be treated as equal, whether assigned in selection I, in II, or in III. (5) The value of the quantitative method is merely supplementary to results obtained by the qualitative method. 1 Cf. Tra-vis, p. 194, quoted in note 19.
Disclaimer
The historical material in this project falls into one of three categories for clearances and permissions:
Material currently under copyright, made available with a Creative Commons license chosen by the publisher.
Material that is in the public domain
Material identified by the Welcome Trust as an Orphan Work, made available with a Creative Commons Attribution-NonCommercial 4.0 International License.
While we are in the process of adding metadata to the articles, please check the article at its original source for specific copyrights.