The Significance of Test Results in Predicting Efficiency in Garment Machine Operating
- Author:
Katherine Treat
Vocational Adjustment Bureau, New York City* The chief purpose of the experiment1 described in this report was to determine if the aptitude of girls entering a class in garment machine operating could be measured.2
In outlining the experiment, it was felt that if significant relationships were found the test could be used in place of an actual try-out on the machines to weed out probable successes from probable failures among further applicants for such trade training. When significant relationships between test results and ability to learn the trade were actually found in the course of this investigation, it was decided to experiment further with the tests to determine their validity with trade school groups slightly older chronologically and of a higher intellectual status than the original group. The results for all of the groups included in this investigation are pooled in this report.
MAKE-UP OF THE GROUPS STUDIED IN THIS INVESTIGATION
1. Experimental Group. This was the first group studied in the course of the investigation. It consisted of 101 subnormal girls who passed through the class in garment machine operation conducted * The Vocational Adjustment Bureau is a social service organization, specializing in the study of problem girls, especially in relation to vocational guidance and employment. The maladjustments of these girls are often due to subnormal mentality or psychopathic tendencies which may result in social and industrial inadequacy. By taking into account the entire situation surrounding the girls, and helped by the studies of the Research Department, an effort is made to place each girl where she may develop to the limit of her capacity. The Bureau maintains an experimental therapeutic workshop, (VAB WORKSHOP), to which nervous and mentally disturbed girls are sent for economic rehabilitation.
1 Devising the team of tests was part of a larger experiment. For a complete report of the whole experiment see Unger, Edna W.: Vocational Training for Subnormal Girls, An Experiment in the Garment Machine Operating Trade, Journal of Personnel Research, Vol. V, No. 6, Oct., 1926, pp. 243-255. A reprint of this article may be obtained from the Vocational Adjustment Bureau.
2 “To learn” was defined as to learn to operate the power machine and to produce work reaching or exceeding the average standard set up by the machine operating industries. TESTS IN GARMENT MACHINE OPERATING 219 by the Vocational Adjustment Bureau from July 1923 to March 1925 and for whom complete records were available. The chronological ages of these girls ranged from 12 to 19 years, eighty percent of the group being between 14 and 16 years of age.
In mental age the girls included in the Experimental Group ranged from approximately 8 to 11 years. Two-thirds of the group were included between the mental ages of 7 years, 9 months and 10 years. None of the girls in this group had an I.Q. above 70, inasmuch as girls with Standford-Binet I.Q.’s above this were not admitted into the class. As a matter of fact, 50 percent of them were either in ungraded classes at the time of admission to the class in garment machine operating or had previously attended such classes. The tests used in this investigation were standardized on the first 45 girls of this group3 and then given to the girls who entered subsequently.4
In the instruction in garment machine operating given to this Experimental Group the atmosphere of the shop rather than that of the class room was maintained. Trade standards were adhered to as closely as possible in every phase of the work. The class was, however, designated purely as a testing class. Each girl received approximately 90 hours of training, extending over a period of six weeks.
2. Trade Extension Classes. The second group consisted of girls from Trade Extension School Classes. The Trade Extension Classes represent a branch of the Manhattan Industrial High School, a public trade school of New York City. They are for (1) girls over 14 years of age who have not completed the course at elementary school but wish to take trade training, and (2) those girls who are unable to complete the trade course at Manhattan Industrial High School because they find the trade training of that school too difficult. One of the courses offered in the Trade Extension Classes is training in power machine operating. For admission to the class the girl must meet the requirements of a machine operating test requiring one hour per dayfor five days. This test is an actual try-out 3 Treat, Katherine: A Team of Psychological Tests for Mentally Deficient Power Machine Operators. Unpublished thesis, Columbia University, New York, 1924. 4 Treat, Ivatherine: Tests for Garment Machine Operators. The Personnel Journal, Vol. VIII, No. 1, June, 1929. on the machines which, however, has not been standardized. There exists, however, this, eliminating factor, not present in the case of either of the other two groups.
The training corresponds rather closely to that used in the Bureau’s group, except that some handwork and stitching on foot driven machines is given. Members of this group had received approximately 90 hours training in operating when the tests were given November 11-18, 1926.
The girls in the Trade Extension Class ranged in chronological age from 14| years to V7years, with two-thirds of the group between 15 and 17 years.
The girls ranged in mental age from 8 years to about 15 years, with two-thirds of the group between 11 and 14 years mental age. The Otis “B” Test for General Intelligence was used to find the mental age, and the results therefore are not perhaps altogether comparable with those of either of the other groups, in which different tests were used.
3. East Side Continuation School. In accordance with the laws of the State of New York, “children between 14 and 17 years of age who are regularly employed must attend continuation or part time school for 4 hours per week, unless they have completed the four year high school course. If out of employment such children shall attend such schools not less than 20 hours per week.”5 One of the courses offered for girls in the East Side Continuation School is power machine operating. There is no preliminary requirement for admission to this class. The training received is individual in character, because the personnel is so constantly shifting that little regular class work can be given. The girls furnish the material for whatever simple articles they wish to make on the machine, and after learning control of power from the teacher, they try to make the article.
From the point of view of length of training and experience, the group was very mixed, this being the third year in continuation school for some girls and the first week for others. Many of the girls were working, and some had been working for over a year at machine operating jobs in various industries. No estimate could be made of the number of hours training for these girls, though no girl was in5 An Abstract of Laws Governing School Attendance and the Employment of Children in New York State, Bureau of Women in Industry, Department of Labor, Sept., 1926. eluded in this study unless she had had training for one month or more involving at least twelve hours of work on the power machine. The girls in this third group ranged in chronological age from 15 to 17 years with over 90 per cent of the group between 16 and 17 years. This third group was older chronologically than the other two groups, but there was much overlapping in all of the groups. The mental age ranged from approximately 1years to almost 18 years with two-thirds of the group between 10| and 14| years. The Otis Self-Administering Test, Intermediate Grade, Form A was used to determine the mental age.
Total Group. The combined group consisted of 242 girls with complete records, made up of the 101 girls in the experimental group, 40 in the trade school group, and 101 in the continuation school group. They had had from 12 to 90 or more hours of training in power machine operating.
The combined group ranged in chronological age from a little over 12 years to 19 years, with the average at 15 years and 10 months and over two-thirds between 15 and 17 years.
Since three separate tests were used for measuring general intelligence, no comparison of the mental ages or Intelligence Quotients could be made. An inspection, however, shows the two trade school groups to be within approximately the same general range and the experimental group to be lower in range of general intelligence. Standards of accomplishment in power machine
OPERATION
1. Experimental Group. The criterion with which each test was compared for the first group of 45 girls, on which the original battery of tests was standardized, consisted of a single mark made up from an average of the actual marks given day by day. A five point scale?poor, poor+, fair, fair-]-, and good,?was employed in marking. All work was rated on six points, viz:?accuracy, speed, neatness, handling, attitude, and grasp of directions. In the final mark the two items of accuracy and speed were weighted by giving each twice the value of the other items. The final weighting was obtained by adding the weighted score and dividing by 8. This weighted score is known as the “objective work rating.”
For the whole group of 101 girls, however, this “objective work rating” was modified by a subjective rating. This second rating was called the “judgment rating.” In this, the absence of the incentive of remuneration, competition, pressure, strain, etc., was taken into account by the instructor in judging the chance of success for each individual as a future operator. Some who rated well in actual accomplishment had to be rated subjectively as possible failures in industry for such reasons as nervousness, inability to concentrate, quarrelsomeness, need of constant prodding, etc., while others who rated slightly below average on the basis of actual accomplishment, were considered possible successes for such reasons as personality, failure to put forth full effort in training because of lack of financial compensation, etc. Since the purpose of this experiment was not only to measure the trainability of the subnormal girl as a machine operator, but also to estimate her probable success as an operator in industry, it was decided to use the second or subjective “judgment rating” as the final criterion. On this “judgment rating” Criterion, 21 percent of the girls were rated as probable successes, 21 percent as doubtful successes, and 58 percent as probable failures.
2. Trade Extension Classes. The Criterion with which each test and the combined tests were compared in the case of this group was as nearly the same as that for the Bureau group as could be expected from different raters, different instructors, a slightly different training course, and a slightly different group of girls. The weekly machine operating marks were copied from the girls’ report cards and averaged. The five point scale of poor, poor-]-, fair, fair--, and good was used in rating the following: work, speed, attitude, and grasp of directions. In the final score the two items ” work” and “speed” were weighted by giving each twice the value of the other items. The final weighting was obtained by adding the weighted scores and dividing by 6. This weighted the two items ‘work” and “speed” a little more than for the Bureau group, but it was felt to give the nearest comparable Criterion. Each girl was then given a “judgment rating” representing the combined opinion of her four instructors with reference to probable success or failure in the trade. This “judgment rating” represented the final Criterion. 3. East Side Continuation School. For this school the same plan of marking was adhered to as for the Bureau group. The final “judgment rating” constituted the Criterion. Only one instructor judged the marks in this school.
Table I shows the distribution of machine operating ratings (“judgment ratings”) on probable successes, doubtful successes, and probable failures, for each of the three groups and for the total group.
Table I Showing Percentage Distribution of “judgment ratings”?Successes, Doubtfuls, and Failures?for the Separate Groups and the Combined Group. Group Successes Doubtfuls Failures Experimental 22 21 57 Trade Classes 20 52.5 27.5 Continuation School. . 31 46 23 Total or Combined.. . 25 36 39
THE TESTS AND THEIR ADMINISTRATION
The conditions for testing were in every case quite good. The individual tests for the Continuation School group were given at a table in the same room where the operating class was being conducted. There was considerable confusion in this room, but the testing was done in a far corner.
A preliminary battery of IS tests was tried out on the first 45 girls of the Experimental Group. The five tests which showed the highest correlation with the “objective work rating” were chosen for the final team of tests. They were then given to the remaining 56 girls in the Experimental Group, thus covering the entire group of 101 operators. The final team included the following tests: Triviming Test.6 This is test 11 of the series, Girls’ Mechanical Assembly Tests, Institute of Educational Research, Teachers’ College, and was given as a group test to small groups. It involves cutting out a design between a path of double lines, the path becoming narrower as it progresses. The excellence of the performance depends upon the girl’s ability to cut between the lines without touching either of them. Porteus Maze Test.7 This is a test in which the subject is instructed to trace her way through certain mazes from a fixed start to a fixed end. There are eleven mazes in a scale of increasing difficulty designated by ages from 3 to 14 inclusive. The test age score achieved by the subject is expressed in terms of mental age. This test requires prudence, foresight, practical judgment, perhaps a certain degree of emotional stability, and has been found to correlate with industrial success.8
6 Toops, Herbert A.: Tests for Vocational Guidance of Children from Thirteen to Sixteen, Teachers’ College, Contributions to Education, No. 136, Oct., 1924, pp. 123, 125.
7 Porteus, S. D.: Porteus Tests, Vineland Revision, Publications of the Vineland Training School at Vineland, N.J., No. 6, Sept. 1919. 8 Glenn, Irene: A Report on the Correlation of Mental Tests with Academic and Manual Subjects, Journal of Educational Psychology, Vol. Ill, p. 496.
Paper Folding Test.9 This is a simple test involving the process of folding five 4X4 sheets of yellow paper into smaller pieces according to a designated pattern. The subject is told to fold, as quickly and neatly as possible, her five sheets in just the way the Examiner illustrates. The papers are scored for time and quality, but only the quality score was considered in this investigation. The subjects were hurried in order to determine the quality of work they could do under pressure. It is a simple test to comprehend, and after the first paper is folded becomes a test of manual dexterity requiring precision, neatness, and speed rather than a test of comprehension. Card Sorting Test. This test consists in arranging two packs of playing cards into suits. Four cards are removed in order to make an even 100 cards. The cards are always presented to the subject in the same order, and she is told to arrange them into the four suits which had been previously pointed out. To make a good score requires apperception, neatness, precision, speed and the ability to learn quickly and remember which cards go into the individual piles. Tapping Test.10 The apparatus for this test consists of a block of wood about 5 inches by 8 inches in which there are holes arranged to make an equilateral triangle. The holes are metal-lined. Behind each hole is a small metal counter. The board is supported to stand at an angle of 45 degrees. There is a metal stylus with a wooden handle. The subject is told to take the stylus and insert it into the holes in rotation as quickly as possible. A trial practice period of 15 seconds with each hand is given. Then the total number of taps made by the right hand and registered by the counter on the back of the board in 30 seconds is averaged with the total number made by the left hand in 30 seconds. This average represents the score. This test requires coordination of hand and forearm, and a sense of rhythm.
When applied to the whole group of 101 girls the first three tests were shown to be the most valuable, so when the team was given to the Trade Extension Classes and to the East Side Continuation School the least valuable test, namely Tapping, was omitted, and the fourth test, Card Sorting, was tried out on the Trade Classes group and on 77 of the Continuation School girls, a number considered sufficient to estimate the value of this test. For the Experimental Group of 101 girls correlations between test scores and “judgment ratings” were computed by the Toops’ 9 Porteus, S. D.: Ibid.
10 Scott, Walter Dill: Personnel Management. A. W. Shaw & Co., 1923. revision of the Pearson correlation formula.11 Toops’ multiple ratio correlation formula12 was applied to these raw correlation coefficients, on both raw scores and team scores, to select the battery of tests which would yield the maximum prediction of the Criterion. The results for the two other groups, namely the Trade Extension Classes and the East Side Continuation School were subjected to the same analysis by the same method.
Finally the results of these three groups were plotted together in a correlation scattergram, using different symbols for each group. An inspection of the scattergram (Figure I) showed the three groups to be nearly homogeneous in content, and to make valid the computations of correlations, and the selection of a team of tests on the basis of the entire group of 242 cases.
The resulting multiple ratio correlation on the combined groups of 242 subjects was .61 ?.03, quite comparable with the correlation of .06 ? .04 on the Experimental group13 alone, and is considerably higher than that of .45 ? .09 on the Trade Extension Classes group and that of .31 + .06 on the East Side Continuation School. These correlations are much lower than that of .84+ .09 figured by the Rank Difference Method obtained on the original 45 subjects from the Experimental group with the work rating as Criterion. As in the case of the separate groups, it was found that a combination of Porteus Maze, Paper Folding and Trimming yielded practically as high a degree of prediction as would be obtained by adding the other two tests of the team, namely Card Sorting and Tapping. The above correlations are computed from the raw weighted scores of these three tests and the Criterion, i.e. the “judgment rating” of the ability to operate an electric power machine. The raw score weights for the combination of the scores of these three tests into one combined weighted score were obtained by dividing the three weights for each of the tests by their respective standard deviations. The weights are as follows: Paper Folding X 5; Porteus MazeX2; and TrimmingXl.
11 Toops, Herbert A.: Eliminating Pitfalls in Solving Correlations’ Journal of Experimental Psychology, Vol. IV, No. 6, Dec., 1921. 12 Toops, Herbert A.: As yet unpublished.
13 In a report of this experiment, “Vocational Training for Subnormal Girls,” by Edna W. Unger (quoted above) the correlation of the total team scores with the criterion is reported as .59 ? .05, whereas it is reported in this paper as .66 ? .04. The original multiple of the total scores was obtained by combining tests according to the original weights used, and the latter multiple by combining the single tests and building up the correlation the .07 points.
Figure 1 Criterion Scores and Team Tests (Weighted Raw Scores) Showing Critical Score for Combined Group of 242 Subjects. Judgment Ratings Test Scotgs 12 3 4 5 56-60 52-56 48-53 44-48 40-44 36-40 32-36 28-32 24-28 80-2416-20 13-16 8-12 M-)’ ? H{i IIM HH II HU? WV W 9 ? O O A A WW AAA A A4 A A 0 ? ??? 9 &5> O It ? ? d & o ? ? (it ? ? on a i? I & 9*9 9 ? O III ? oo o I A ? 0 9 9 9 1SJ2 9 6## ? 9 ? ? 111 ?4 OJ AA w HIM 0?9 o ? m 0 9 A ? 9 t U 0> 9 0 mi a o o o 1|9? ? 99 mi a 9 9 IHM 09 ll ? 9 O ?? 99 9A. ? o A 9 A dA ? oo No-? 50 45 87 44 IS 242
Key: | Experimental Group 1&2, Probable failures A Trade Extension Classes 3, Doubtful successes ? East Side Continuation 4&5, Probaole successes School Key: [ Experimental Group A Trade Extension Classes ? East Side Continuation School 1&2, Probable failures 3, Doubtful successes 4&5, Probable successes TESTS IN GARMENT MACHINE OPERATING 227 Table II shows a comparison between the coefficients of correlations for the team tests and the Criterion for each of the separate groups and for the Total Group. In every case the correlations for the Experimental group, or the group for which the tests were originally intended and upon a portion of which they were standardized, were highest. The correlations were next highest for the combined or Total Group. Table II Coefficients of Correlation between the Criterion and Team Tests for the Various Groups. Group Porteus Maze Paper Trimming Total 8 Tests No. Folding Experimental 5S + .05 .52 + .05 .53 ?.05 .66+.04 101 Trade Classes 22+ .11 .45+ .09 .23+ .01 .45 + .09 40 Continuation School. .27 + .07 .26+ .07 .27 ?.07 .31 ?.06 101 Total 3 Groups 52?.03 .52?.03 .47?.03 .61?.03 242 Development of Critical Score for the Elimination of Failures This method14 involves plotting test scores against “judgment ratings” as for a correlation chart, and then drawing an arbitrary line so that those making desirable test scores for the Criterion will be included.
Experimental Group. For this group, when the raw scores were plotted against the “judgment ratings” it was found that a score of 32 or above for the three tests as a critical score eliminates approximately 63 percent of the failures, 5 percent of the doubtful successes, and 5 percent of the probable successes.
Trade Classes. A critical raw score of 32 for the three tests eliminates 45 percent of the failures, 24 percent of the doubtful successes, and no probable successes in this group.
East Side Continuation School. In this case, a critical raw score of 32 eliminates 24 percent of the probable failures, 7 percent of the doubtful successes, and no probable successes.
Total Group. A critical raw score of 32 for the three tests used eliminates 44 percent of the probable failures, 3 percent of the doubtful successes, and 3 percent of the probable successes for the total group. This critical score, therefore, fails to eliminate 56 percent of the failures. These failures occurred, it would seem, not so much because of absence of the specific motor abilities required by the tests, as either by reason of the fact that a number of the elements de14 Bills, Journal of Applied Psychology, Vol. V, 1921. manded in machine operating were not measured by the tests or that perhaps certain other necessary qualities classifiable under the head “personality” were not involved in the examination.
An inspection of the records of these failures revealed that some were decidedly institutional cases (especially from the Bureau’s Experimental group), so hopeless was the prognosis for success at even the lowest grade of unskilled work. A considerable number were sufficiently unstable emotionally to interfere with the proper concentration on their work. Some showed a decided lack of interest in their work. It is also probable that for the more intelligent girls in the two trade school groups the tests were not sufficiently discriminative of manual dexterity. In the case of no single test used alone in predicting the Criterion, did the critical score method prove effective as a means of eliminating prospective failures for any of the three groups or for the Total Group.
An Analysis of Other Eliminating Factors In considering other possibilities, besides the team of tests as a means of eliminating failures, none were found to be so valuable as the tests.
Chronological Age. In the narrow range of each of the separate groups, there is little relationship between chronological age and the ability to run an electric power machine, but for the combined groups this relationship is somewhat more marked. For the Experimental group the correlation was .016 ? .07, for the Trade Classes .088?.01, for the Continuation School .004+ .07, and for the total or combined group it is raised to .26 ? .04. However, even this correlation of .26 is hardly of significance. Intelligence Quotient. Since a different general intelligence test was used for each of the three separate groups, no attempt was made to pool these results. In each case, however, the relationship between general intelligence and the Criterion is positive, being .35 + .05 for the Experimental group, .30+ .10 for the Trade Classes, and .33 + .06 for the Continuation School. There is an average correlation of .33 for the three groups, representing a slight but consistent positive relationship between general intelligence as measured by the tests and the ability to run an electric power machine.
It is interesting to note in this connection that a critical score of I.Q. 50 eliminates 24 percent of the failures in the Experimental group when the individual Binet is used. An I.Q. of 80 as a critical score eliminates only 9 percent of the failures for the Trade Classes when the Otis “B” Test is used. An I.Q. of 65 as a critical score eliminates 33 percent of the failures (and no probable successes) for the Continuation School group when the Otis Self-Administering Test, Intermediate Grade, Form A is used. This last represents a better elimination of failures for the Continuation School than the team of tests, the best critical score in this team eliminating only 24 percent of the probable failures.
Summary and Conclusions
1. The team of three tests referred to in this report is effective in eliminating future failures among applicants for classes in power machine operating. It is most effective when used for that type of girl on which the tests were originally standardized, namely the subnormal.
2. The team is most effective when the raw scores are used and given the following weights: Paper FoldingX5, Porteus MazeX2, and TrimmingXl.
When this procedure is followed with a mixed group the best critical score eliminated 44 percent of the probable failures, 3 percent of the doubtful successes, and 3 percent of the probable successes. When applied to the group whose Intelligence Quotients are below 70 on a Binet scale the same critical score eliminates 63 percent of the probable failures, 5 percent of the doubtful successes, and 5 percent of the probable successes.
3. Although there is no point of emotional instability at which it can be said that a girl can or cannot learn operating, it is obvious that no unstable girl should attempt to learn the trade.
4. From the evidence from the Experimental group, and other evidence which the Bureau has gathered,15 no girl should be admitted to a machine operating class who tests below a mental age of 8 years on a Stanford-Binet scale.
5. The tests were not found to be sufficiently discriminating for use with the two trade school groups, especially when the groups are considered separately. The results show that when the tests are subjected to use and analysis on the groups differing from that group on which they were standardized, they are not as selective as for the original group.
15 Unger, Edna W: op. cit. It is possible that if other tests, requiring a higher degree of intelligence and perhaps finer manipulative ability were added to these three, the resulting combination might be valuable in eliminating future failures among the candidates for these classes. At present if for any reason the try-out period in the Trade Classes were to be eliminated, the tests would be a valuable substitute for it. Such a substitute would probably be more valuable than the results in this paper show, for the group then would be more heterogeneous. If the class in machine operating in the Continuation School were to be overcrowded and a preliminary selection required, the group test of general intelligence alone would be a helpful eliminating factor, the battery tests described in this report serving probably as a good secondary eliminating factor.D
Disclaimer
The historical material in this project falls into one of three categories for clearances and permissions:
Material currently under copyright, made available with a Creative Commons license chosen by the publisher.
Material that is in the public domain
Material identified by the Welcome Trust as an Orphan Work, made available with a Creative Commons Attribution-NonCommercial 4.0 International License.
While we are in the process of adding metadata to the articles, please check the article at its original source for specific copyrights.