Abstract
We describe the findings from a randomized evaluation of a one-year kindergarten math enrichment program, the High 5s program. The program was designed to provide small-group math enrichment in a fun, club-like format to children who had received enriched math instruction the prior year. Participants included 655 kindergarten students in 24 low-income schools in New York City. Students were randomly assigned to either the “business as usual” control group or to participate in the High 5s math clubs, which met outside of class in small groups with a trained facilitator three times per week. The High 5s program produced a positive impact on kindergarten math skills.
Introduction
Over the last few decades there has been a heavy emphasis on increasing literacy skills among low-income children, with federal and state initiatives designed to ensure that all children can read by grade 3. State and federal dollars have been spent to improve reading curricula, hire reading coaches and provide tutoring and small group support for struggling readers. However, much less emphasis has been placed on improving the early math skills of students in low-income schools. Kindergarten classes typically devote less than one hour per day to math compared with over 1.5 hours for literacy. Moreover, kindergarten math instruction is often very basic, covering topics that students know when they enter kindergarten such as simple counting and shape recognition. [1]
This is true despite research having shown that math skills are highly correlated not only with later math achievement but also with later reading achievement, high school completion, and college attendance. [2]
A possible response to this situation would be small-group interventions designed to build young children’s math skills. Such interventions have a long history in literacy, and a strong research base suggests that small group literacy instruction is effective. [3] In the area of math, there are few well-developed programs to provide supplemental math support or enrichment to young children, let alone rigorous research to determine whether such programs are effective. [4]
Here we report findings from research on such a program that one of us (Robin Jacob) helped design and evaluate. The study suggests that small group math instruction for kindergarteners is a promising strategy for improving early mathematical competency.
The High 5s Program
The High 5s program was developed in the context of a larger MDRC project to evaluate Building Blocks, a 30-week, pre-K math curriculum designed to take into account children’s natural developmental progression in math. [5] Developed by researchers at the University of Michigan with support from MDRC and the developers of Building Blocks, the goal of High 5s was to provide a consistent instructional approach and alignment of content from the pre-K math curriculum to kindergarten. Importantly, High 5s was designed as a hands-on program to foster interest in math, critical thinking, and broad knowledge of mathematical concepts including not only numeracy, but geometry, patterning, and measurement as well.
Students in the High 5s program met for 30 minutes three times a week in “math clubs,” which took place either before or after school or during lunch. The clubs met for roughly 28 weeks from October through May. Activities in the clubs are delivered in a game-like format and are intended to be fun, engaging, interactive, and developmentally appropriate.
Each club includes 3-4 children working with a trained facilitator. Most facilitators had a BA degree, but limited formal teaching experience. They were paid a salary commensurate with that of a paraprofessional teacher in the New York City public schools (around $25 per hour, depending on experience).
In the study reported here, facilitators received a substantial amount of training and supervision over the course of the year. [6] They received 16 days of training before clubs began and an additional eight throughout the school year. In addition, supervisors from Bank Street College of Education provided ongoing support in weekly meetings that included 4-5 facilitators with one supervisor. These meetings included support regarding logistics, curriculum review, reflection about student learning, and guidance and training in small-group management. Supervisors also met individually with facilitators regularly and provided coaching in the field as needed.
The study
Students were eligible to participate in the High 5s program if they had attended one of the 24 public preschools that participated in the Making Pre-K Count project described above and the student stayed in the same school between pre-K and kindergarten. These schools served predominantly low-income, Black and Hispanic students. With the exception of one, all schools had fewer than 50% of their students scoring proficient or above in reading and mathematics at the end of third grade. On average, roughly 50% of the students were Hispanic and 43% were Black. All schools had at least 70% of students who were eligible for free or reduced price lunch (FRPL) and nine of the 24 schools had 100% of students with FRPL eligibility.
Individual children were randomly assigned within schools in the fall of the kindergarten year to receive the High 5s program in addition to their regular kindergarten math instruction (n=320), or to a “kindergarten-as-usual” control group (n=335). [7] The researchers were able to obtain student outcome data from 613 of the 655 students who were randomly assigned as part of the study, and attrition was similar for the program and control groups. There were no statistically significant differences in measured baseline demographic characteristics between children in the High 5s and the kindergarten-as-usual groups.
Kindergarten classrooms contained students from both program and control groups as well as other students who were not part of the study. To reduce the likelihood that control students would be exposed to any High 5s instruction, the clubs were held in pre-K classrooms or other multi-use spaces (e.g., a resource room), and not in kindergarten classrooms. In an effort to keep teachers informed, kindergarten teachers were provided with a few example activities from the clubs, but otherwise teachers did not have access to the High 5s curriculum. Most of the interactions that High 5s facilitators had with teachers were brief, and mostly focused on dropping off or picking up students from the classroom.
Consistently, one of the key challenges of education interventions is implementation. Using a combination of daily logs completed by facilitators and observations of the clubs conducted by external staff, researchers concluded that the program was implemented with a high degree of fidelity. Student attendance was 87 percent, which is quite high for a supplemental program conducted outside of the typical school day. Logs indicate that 93 percent of scheduled club sessions were completed, and that facilitators followed the intended pacing of the curriculum and activities. According to external observers, the majority of facilitators met or exceeded standards for quality instruction. More generally, observers reported that the facilitators had a good rapport with students and created a positive instructional climate.
Student achievement effects
The study analyzed two different measures of math achievement: the Woodcock-Johnson applied problems subscale and the REMA-K. The Woodcock-Johnson is a widely used standardized assessment of mathematical thinking that provides a global measure of math ability. The REMA-K was adapted from an assessment designed by the developers of the Building Blocks Pre-K program. [8 ] The REMA-K is a longer and more discriminating measure of mathematical knowledge, with questions designed to assess discrete math skills. For example, the REMA-K asks “How many pennies are left if I have 5 and cover these 3?” while, the Woodcock-Johnson asks “How much money do I have if I add three pennies and two nickels together?” The latter requires multi-step addition as well as knowledge regarding the value of different coins, while the former is a simple one-step subtraction problem.
As shown in in Figure 1, High 5s had a positive effect on one of two measures of student math skills. Students in High 5s scored 0.19 SD higher than control students on the REMA-K, which is equivalent to roughly two-and-a-half months of learning on the assessment. There was a positive but not statistically significant impact on the Woodcock-Johnson Applied Problems assessment (effect size = 0.09). [9]
Somewhat surprisingly, the study did not find a significant effect on children’s math attitudes. [10] Researchers speculate that the reason for this is that when children answered survey items about their views of math they were thinking about their regular math class. Qualitative research suggests students greatly enjoyed the activities in the clubs. The following exchange, as reported by one of the facilitators during the last week of the clubs, captures this difference:
While filling out their “why we like math” page, the children [in this club] all concluded that they didn’t like math. I [the facilitator] said that was strange because they’ve been doing math in High 5s all year and were loving it and were so happy. They clarified that they like math in High 5s but they don’t like it in school. R explained in his words that “In school, you do math and you be quiet and look down at your paper. They just tell you that you’re wrong. And then nobody talks to you. It’s just wrong and you have to be quiet. But in High 5s we have you. You never say we did it wrong and we all talk and figure it out and then nobody’s wrong. That’s why I’m happy when I do math in High 5s.”
Mechanisms
The research identified several potential mechanisms through which High 5s may have enhanced children’s math skills. First, students in the High 5s program received substantially more math instruction than control students. Kindergarten teachers spent an average of 52 minutes per day on math, for a total of about 4 hours and 20 minutes per week. High 5s added an additional 75 minutes of math instruction – an increase of roughly 30 percent.
Second, the instructional approach and content in High 5s differed considerably from the standard math instruction observed in kindergarten classrooms. In math classes in the study, students spent 83 percent of their time in whole-group instruction or seat work, and most activities involved either workbooks or no materials at all. In contrast, High 5s featured small group interactive activities with a variety of manipulatives. In addition, High 5s exposed students to somewhat more advance and a wider range of mathematical topics relative to the classroom math instruction.
Finally, the instructional climate in clubs differed substantially from the climate in classrooms. As illustrated by the child’s quote above (and consistent with the more general qualitative research findings), students enjoyed the math activities in High 5s. External observations of the clubs and the regular math classes indicate that facilitators in the clubs were somewhat more likely to ask open-ended questions, encourage mathematical reflection, and differentiate instruction.
Conclusions
Small group reading instruction is ubiquitous in elementary schools. This type of intervention provides the opportunity for more tailored, individualized instruction, which may help to motivate as well as instruct students. It also provides greater opportunities for conversation and interaction among teachers and students. Prior research has demonstrated the efficacy of this approach for improving students’ early literacy skills, yet small group instruction is used much less frequently for math.
This study provides some preliminary evidence that small group instruction may be a promising approach for math instruction as well. Students who participated in the High 5s small group math instruction made statistically significant and substantively meaningful gains on one of two measures of math skills. Researchers will continue to follow students who participated in the study through elementary school to assess whether the gains made by the children who participated in High 5s are sustained or fade over time.
Although the program was delivered by facilitators who had little formal teaching experience and who were paid a salary commensurate with that of a paraprofessional staff member, the High 5s model tested in this study was resource intensive. [11] Because clubs were only offered outside of regular instructional time (before school, after school, or during lunch), each facilitator could only run a few clubs per week and facilitators had to travel from school to school to serve students, which added both time and transportation costs. Facilitators were also offered substantial training and support to implement the clubs.
The researchers are now working to develop a model for such small-group math instruction that requires fewer resources and could be more easily scaled. For example, one might conduct small groups in classrooms themselves or via pull-out services, using paraprofessional staff already employed by schools.
If a more cost-effective model can be developed, small group math instruction may be a promising approach for elementary schools to consider in the future.
— Robin Jacob and Brian A. Jacob
Robin Tepper Jacob is an associate research professor at the Institute for Social Research and the School of Education at the University of Michigan. She is also co-director of the Youth Policy Lab, a partnership between the Survey Research Center and the Gerald R. Ford School of Public Policy. Brian Jacob is the Walter H. Annenberg Professor of Education Policy, Professor of Economics, and Director of the Center on Local, State and Urban Policy (CLOSUP) at the Gerald R. Ford School of Public Policy. Joseph Ryan is Associate Professor School of Social Work and Co-Director Child and Adolescent Data Lab at the University of Michigan.
This post originally appeared as part of Evidence Speaks, a weekly series of reports and notes by a standing panel of researchers under the editorship of Russ Whitehurst.
The author(s) were not paid by any entity outside of Brookings to write this particular article and did not receive financial support from or serve in a leadership position with any entity whose political or financial interests could be affected by this article.
Notes:
1. Engel, M., Claessens, A., & Finch, M. A. 2013. Teaching students what they already know? The (Mis) Alignment between mathematics instructional content and student knowledge in kindergarten. Educational Evaluation and Policy Analysis, 35(2), 157-178; Engel, M., Claessens, A., Watts, T., & Farkas, G. (2016). Mathematics Content Coverage and Student Learning in Kindergarten. Educational Researcher.
2. Jordan N., Kaplan D., Ramineni C., Locuniak M. 2009. Early math matters: Kindergarten number competence and later mathematics outcomes. Developmental Psychology, 45(3), 850–867; Duncan, Greg J., Chantelle J. Dowsett, Amy Claessens, Katherine Magnuson, Aletha C. Huston, Pamela Klebanov, Linda S. Pagani, Leon Feinstein, Mimi Engel, and Jeanne Brooks-Gunn. 2007. “School Readiness and Later Achievement.” Developmental Psychology 43, 6: 1428-1446; Duncan, Greg J., and Katherine Magnuson. 2009. “The Nature and Impact of Early Skills, Attention, and Behavior.” Paper presented at the Russell Sage Foundation Conference on Social Inequality and Educational Outcomes, New York City.
3. Slavin, Robert E., Cynthia Lake, Susan Davis, and Nancy A. Madden. 2010. Identifying What Works for Struggling Readers: Educator’s Guide. Best Evidence Encyclopedia. Center for Data-Driven Reform in Education, Johns Hopkins University. Website: www.bestevidence.org.
4. Among older children, there is evidence that one-on-one or small group tutoring can be effective; e.g. Ritter, G. W., Barnett, J. H., Denny, G. S., & Albin, G. R. 2009. The effectiveness of volunteer tutoring programs for elementary and middle school students: A meta-analysis. Review of Educational Research, 79(1), 3–38; Fryer, R. 2011. Injecting successful charter school strategies into traditional public schools: A field experiment in Houston. (Working Paper 17494). Retrieved from National Bureau of Economic Research website: http://www.nber.org/papers/w17494. There is also evidence that computer-aided math instruction, which students engage with individually, can improve math achievement; e.g. Steenbergen-Hu, Saiying, and Harris Cooper. 2013. “A Meta-Analysis of the Effectiveness of Intelligent Tutoring Systems on K–12 Students’ Mathematical Learning.” Journal of Educational Psychology 105(4): 970-87; Cheung, Alan CK, and Robert E. Slavin.. 2013. “The Effectiveness of Educational Technology Applications for Enhancing Mathematics Achievement in K-12 Classrooms: A Meta-Analysis.” Educational Research Review 9(1): 88-113.
5. Making Pre-K Count was developed by MDRC in partnership with the Robin Hood Foundation, Overdeck Family Foundation, the Heising-Simons Foundation, and other funders as part of the Robin Hood Early Childhood Research Initiative to design and test interventions aimed at improving outcomes for children living in poverty in New York City. For more information about the project, including the published reports, see here.
6. Bank Street College of Education provided training, supervision, and support for the facilitators in this program.
7. Among the children who planned to stay in the same school between pre-K and kindergarten, 738 parents (99% of eligible children) gave consent at the end of pre-K for their student to participate in the second phase of the study. Some students who were assigned to the High 5s program group did not participate in the program or discontinued their participation before the end of the year. All students who were assessed at the end of kindergarten, regardless of their participation, were included in analyses.
8. An item-by-item analysis of the REMA-K showed that fewer than 10% of the items on the test were aligned with specific Building Blocks or High 5s activities.
9. Children in the High 5s study had all participated in Making Pre-K Count and therefore had all been exposed to the Building Blocks math curriculum in preschool.
10. The study found no significant effects on language or executive functioning skills.
11. For a detailed discussion of costs, see the High 5s implementation report.
