# requiring a math skills unit results of a randomized experiment[精选推荐pdf]

1Requiring a Math Skills Unit: Results of a Randomized Experiment Susan Pozo and Charles A. Stull* Research spanning three decades supports what many experienced instructors of economics have long concluded – math matters. Students with greater mathematics preparation attain higher test scores in introductory economics (Cox, 1974; Reid ,1983; Lumsden and Scott, 1987; Anderson, Benjamin and Fuss, 1994; Ballard and Johnson, 2004). While all levels of competency seem to explain performance, Charles Ballard and Marianne Johnson (2004) find “mastery of extremely basic quantitative skills is among the most important factors for success in introductory microeconomics.” Furthermore, research shows that mathematical competency reduces anxiety in economics classes (Benedict and Hoag, 2002). To the extent that anxiety may interfere with the cognitive process, an effective mechanism to correct for these deficiencies is desirable. The research supports that there may be simple methods economics instructors can use to improve students’ learning. Common techniques include assigning a math chapter in the text, completing a math unit at the university skills center, or completing a computer unit that reviews and tests basic math skills. These alternatives, however, require effort from all students, including those possessing good math skills. Consequently, many instructors make these math assignments optional, while particularly encouraging those with weaker math skills to complete them. But this procedure is also problematic, as students most in need of the math review are often least likely to put forth effort when there is no tangible reward. An alternative strategy is to give students a grade incentive to complete a math skills program. In this paper we report on the results of a controlled experiment with random assignment which tests whether giving a grade incentive to complete a math 2skills unit results in higher overall achievement in introductory economics. We find that students provided with the incentive get higher exam scores. The achievement gain is most noticeable for students lower in the grade distribution. Students with the weakest backgrounds and therefore with the greater marginal gains from completing the math unit are more likely to derive the benefits from that effort. I. Experimental Design A. Subjects The experiment was performed with students enrolled in two sections of principles of macroeconomics taught by the same professor during spring 2004 at Western Michigan University, a large regional university located in Kalamazoo. Informed consent from each student was obtained on the first day of class as specified by the protocol reached with the university s Human Subjects Institutional Review Board. The assignment to treatment or control group was determined by randomization (described below) and not by class section. Two students did not consent to having their scores used in the analysis, and 6 students added the course too late, preventing them from completing the introductory mathematics unit. Hence, 273 students were in the experiment. B. Random Assignment All students were randomly assigned to be in the treatment or the control group.1 Students with odd social security numbers (total of 157) were assigned to the treatment group, while students with even social security numbers (total of 116) were assigned to the control group. Re-examination of official university records confirms these two classes contain more students with odd numbers. This difference is not beyond what one would expect from random chance and the difference in group size does not affect our statistical analysis. 3C. Treatment Students in both the experimental and control group were asked to complete an online diagnostic math test during the first week of class. This test consisted of 28 questions covering numerical calculations, graphs, units of measurement, area, and simple algebra. Following the due date, students received their scores and information about math deficiencies. Students in the experimental group were reminded that this test score would serve as their math grade, but they could improve their math score by working through the appropriate online tutorials and taking a post-review test. The students were reminded that the higher of the two test scores would be used in computing final grades. Hence, students with math deficiencies were provided an incentive to improve math skills while proficient students needed bear no further costs. Students in the control group were strongly encouraged to complete the math diagnostic test, to work through the appropriate tutorials and to take the post-test to gauge their comprehension of the material. They were informed that neither the pre-review nor post-review score would be used in their final grade. D. Course Structure and Grading Students’ course grades were determined by performance in three areas: i) weekly homework units, ii) two midterm exams, and iii) a comprehensive final exam with weights of 0.3, 0.4 and 0.3 respectively. For the control group, the highest 10 of 13 homework scores were used to compute students’ homework grade. For the experimental group, the highest 9 out of the same 13 homework scores were used with the math score (higher of diagnostic or post review tests) counting as the 10th homework. Individuals in the experimental group who did not take the diagnostic or post test earned a “0” for the math score. 4Recall that both groups had equal access to the math unit. The only difference is that students in the control group could not use their math score as one of the 10 homework scores, while the experimental group students were required to use the math score. Hence the two groups serve to compare two common pedagogies in economics: providing an optional math review or requiring a math review unit. E. Final Sample The number of subjects in the experimental and control groups were reduced by 12 and 8 respectively because these students officially dropped the course during the semester. In addition, 2 from each group unofficially dropped the class resulting in 143 and 106 in treatment and control groups respectively. II. Evaluation What impact did the mathematics unit have on the overall class performance of students who were graded on it? Prior research suggests that graded homework causes students to expend more effort and attain higher exam scores relative to non-graded homework (Grove and Wasserman, 2006). Here, requiring the math unit appears to lead to more effort. While all students were strongly encouraged to complete the math unit, students in the control group exerted little effort as more than half earned a score of 0. In contrast, for the experimental group the bulk of the scores were over 50 percent indicating substantially more effort. Did grading the math unit elevate overall class performance? Examination of the histograms of the (weighted) average of midterms and final exam scores for the two groups of students (see Figure 1) suggests that the treatment affected the distribution for the experimental group. The range of scores is narrower and the left tail appears to be compressed relative to the control group, tempting us to conclude there was improvement in performance owing to the treatment. Formal assessment can be obtained by testing the hypothesis that the mean score for the experimental group is equal to or less than the mean score for the control group; that is, H0: cEμμ≤, against the alternative HA: cEμμ>. A one-tailed test is appropriate in this case, since we would endorse the math unit requirement only if the mean scores improved. We consider several measures of class performance. The total score in the class is the simplest measure, but it may be imperfect because it includes the homeworks which differ by experimental design. An alternative is to use a weighted average of the midterm and final exams. The overall mean effect of the experiment is shown in the “midterm Stull: Kalamazoo College, Kalamazoo, MI 49006. We are grateful to Paul Romer, Michael Murray, and William Bosshardt for comments and discussion. 1. Assignment was pre-determined by the researchers and not contingent on participation. Students did not know to which group they were assigned when consenting or declining to participate. Refusal to participate did not impact on course requirements as specified by group assignment. Refusal to participate only affected our ability to include the student’s score in this analysis. 2. Since the gain in scores is taking place mostly among weaker students, it is unlikely that a Hawthorne effect exists. Table 1 Descriptive Statistics and Hypothesis Testing Midterms & Final Midterms Final Total class E C E C E C E C X 67.6 65.6 70.8 68.2 63.4 62.2 68.8 65.9 S 10.36 12.58 11.23 13.49 11.7 13.8 9.8 11.8 SE 0.87 1.22 0.94 1.31 0.97 1.33 0.82 1.1 CEXX− 2.00 2.60 1.2 2.9 t (prob) 1.35 (0.09) 1.64 (0.05) 0.71 (0.24) 2.00 (0.02) F (prob) 1.48 (0.02) 1.44 (0.02) 1.40 (0.03) 1.45 (0.02) E refers to the experimental group and C to the control group. S and SE are the standard deviation and standard error. t and prob correspond to tests of the null and alternative hypotheses: CEHμμ≤:0and CEAHμμ>:. F and prob refer to H0: σ2E ≥ σ2C and HA: σ2E at the 10, 5 and 1 percent level of significance. CpQE pQCpQE pQCpQ12Table 3: Differences in quantiles (standard errors in parentheses) CpEpQQ - p Midterms Final Total score 0.05 12*** (3.5) 1.3 (3.4) 7.9*** (2.9) 0.10 5** (3.2) 2.7 (2.4) 6** (3.3) 0.15 5** (2.8) 1.4 (2.7) 5.5*** (2.0) 0.20 4* (2.9) 1.3 (2.9) 4** (2.0) 0.25 4** (2.3) 1.3 (2.4) 4.9** (2.0) 0.30 3* (2.1) 1.3 (1.9) 4.6** (2.4) 0.35 2.5 (2.2) 2.7* (1.9) 3.2* (2.5) 0.40 2 (2.0) 2.6* (1.9) 1.9 (2.1) 0.45 1 (1.8) 2.7* (2.0) 2.1* (1.6) 0.50 2 (1.9) 2.0 (2.1) 2.5** (1.3) 0.55 2 (1.9) 1.3 (2.5) 2.6** (1.3) 0.60 1.5 (2.0) 0 (2.5) 3.2** (1.6) 0.65 1 (2.4) 1.3 (2.4) 2.1 (1.9) 0.70 0 (2.6) 0 (1.9) 1.6 (1.8) 0.75 0 (2.5) -1.3 (1.7) 0.8 (2.1) 0.80 1 (2.0) 0 (1.6) 0.4 (2.1) 0.85 1 (1.5) 0 (2.3) 1 (2.1) 0.90 0 (2.2) -4 (3.6) 0.7 (2.1) 0.95 0 (2.8) -1.3 (4.5) -1.9 (2.6) *, **, and *** denote rejection of the null hypothesis ≤ in favor of the alternative > at the 10, 5 and 1 percent level of significance. E pQCpQE pQCpQ13Figure 1 Histograms of Midterms & Final Scores 14Figure 2 Quantile-Quantile Plots 15