Executive Summary
Education researchers and policymakers are increasingly interested in tracking students’ long-term outcomes, such as high school graduation, college enrollment, college graduation, and earnings in the labor market. But the obvious fact that a long time has to pass before long-term outcomes can be measured means that such evidence is dated by the time it is available. The program being examined may have gotten significantly better or worse.
A recent example is the Florida Tax Credit (FTC) scholarship program, which provides private-school tuition scholarships to low-income students. An Urban Institute study released earlier this fall found that participating in FTC increased students’ college enrollment rates by six percentage points. But this study was based on students who first participated in FTC between 2004 and 2010, and the program has nearly tripled in size since then.
This report provides new evidence on how participation in FTC has evolved over this recent period of rapid expansion, and what that means for its likely effects on participating students’ college enrollment rates. I calculate the number and share of FTC students attending different groupings of private schools, which have differential track records in terms of increasing students’ rates of college enrollment.
I find that schools where most students use FTC scholarships, which have smaller positive effects on college enrollment relative to public schools, grew much more rapidly than schools with smaller percentages of students on scholarships, which have larger positive impacts. But I also find that non-religious schools, which do not appear to boost college enrollment, have not expanded more rapidly than Catholic schools, which do. These findings suggest avenues for future research on the optimal design of private school choice programs.
How FTC affects college enrollment rates
Earlier this fall, my Urban Institute colleague Daniel Kuehn and I published a new study that examines, for the first time, the impacts beyond high school of a statewide private school choice program. We examine the Florida Tax Credit (FTC) scholarship program, which provides private school tuition scholarships to children from low-income families (defined as those making less than 185 percent of the federal poverty level, which is the same eligibility requirement as for a free or reduced-price lunch). [1]
The FTC program is effectively a means-tested voucher program, but it is called a tax credit scholarship program because rather than being funded directly by the government it is supported by corporate donations to non-profit organizations (which distribute the scholarships). These donations are reimbursed by the state through corporate tax credits.
In our study, we compare the enrollment rates at public colleges in Florida of 10,330 FTC students to those of non-participating students who initially attended the same public schools and had similar demographics (language spoken at home, country of birth, race/ethnicity, disability status, age, and free lunch participation) and test scores (in math and reading) prior to participation.
We find that attending a private school through FTC increases the rate at which participants enroll in college by 6 percentage points, or about 15 percent. For example, Figure 1 shows that 45 percent of students who first participated in FTC in elementary or middle school attended a public college in Florida within two years of expected high school graduation, compared to 39 percent of matched non-FTC students.
Figure 1. Enrollment rates at Florida public colleges, FTC participants and matched sample of non-participants
The effect of FTC participation is concentrated in community colleges, with smaller and not consistently significant effects of enrollment in four-year universities. This pattern of results is similar for students who first participated in FTC in elementary/middle school or in high school. Additionally, we find larger effects for students who participated in FTC for longer periods of time, although most students (61 percent) only remained in the program for one or two years. Estimated effects on associate degree attainment rates are not consistently positive.
This overall pattern of results masks significant heterogeneity by the type of private school attended. Figure 2 shows that Catholic and non-Christian religious schools (which enrolled 15 and 5 percent of FTC students, respectively, in 2010) have the largest positive impacts on college enrollment, non-Catholic Christian schools (the broadest group, at 62 percent of FTC students) have smaller but still substantial impacts, and non-religious schools (enrolling 17 percent of students) have a zero average effect. [2] All of these comparisons are relative to students who did not participate in FTC, who generally attended public schools. None of the groupings of private schools examined had a negative estimated impact relative to public schools—some just had a larger positive effect than others.
Figure 2 also shows that schools with smaller percentages of their students on scholarships have larger positive impacts on college enrollment rates. For example, the impact of attending a private school where less than 25 percent of students receive an FTC scholarship (which was the case for 32 percent of all FTC students in 2010) is 2-3.5 times as large as attending a private school where the majority of students are on scholarships (which were attended by 30 percent of FTC students).
Figure 2. Effect of FTC participation on enrollment in a Florida public college (relative to non-participants), by type of private school attended
How has FTC evolved since 2010?
The Urban study examines data on Florida students who first participated in FTC between 2004 and 2010. But total enrollment in the program has nearly tripled since then, from less than 35,000 in 2010 to more than 100,000 today. The program’s effect on today’s participants may differ due to changes in which private schools participate in the program, which students participate, and the quality of the public schools that FTC students would otherwise attend.
In particular, skeptics of private school choice programs worry that lax government regulation (compared to the public sector) will allow too many low-quality schools to operate. The Florida program requires participating schools to meet federal nondiscrimination requirements and state and local health and safety codes, and to administer a state-approved norm-referenced test to their FTC students (the results have been made publicly available for schools with 30 or more gain scores since 2010-11). [3] But the comparability of test scores across different tests and student populations is limited.
A recent series of articles by the Orlando Sentinel highlighted problems at some schools that participate in the program, describing Florida’s choice system as “so weakly regulated that some schools hire teachers without college degrees, hold classes in aging strip malls and falsify fire-safety and health records.” [4]
Are these problems isolated exceptions or evidence of a growing problem in the FTC program? Little systematic evidence exists on how the program has evolved as it has expanded, and we won’t know the college enrollment outcomes of more recent FTC participants for many years. But it is possible to examine whether the mix of schools attended by FTC students has changed, and whether those changes have favored groups of schools that have been more or less successful in the past.
I address this question by analyzing student-level data provided by Step Up for Students, the non-profit organization that administers the program, and information on private schools in Florida provided by the Florida Department of Education. The private school data are available beginning in 201
I examine the two broad categorizations of private schools which were found to vary in terms of their average impacts on college and enrollment rates: by religiosity and denomination, and by the share of students who received FTC scholarships. These groupings are broad, and likely average across significant differences in student outcomes across individual schools. [5]
As noted above, Catholic and other (non-Christian) religious schools tend to have the largest estimated impacts on college enrollment, followed by non-Catholic Christian schools and non-religious schools (with the latter having impacts that are not distinguishable from zero). If FTC participation has shifted to favor non-religious schools, we might expect the overall effectiveness of the program to decline.
But this is not the case. The types of schools attended by FTC students have changed little over the last six years of the program. Figure 3 shows that participation in the program has grown across all categories of schools. The share of students attending non-religious schools (the only group to have no significant effect on college enrollment) has only increased by two percentage points since 2010 (from 17 to 19 percent).
Figure 3. Number of FTC students, by school type, 2010-2016
The Urban study also finds evidence that schools that serve a smaller percentage of FTC students have larger positive impacts on college enrollment than schools where a majority of students are on scholarship. We might be concerned that, over time, there will be more private schools with a majority of students on FTC and that these schools might not be the highest quality (as evidenced by the fact that families do not attend if they have the pay the full price).
Figure 4 shows that the number of students attending schools where most students are on scholarships increased by about 49,000 between 2010 and 2016, a period when total participation increased by 65,000 students. The percentage of FTC students attending majority-FTC schools doubled over this period (from 30 to 60 percent). Put another way, about 76 percent of the growth in FTC enrollment over this period occurred at majority-FTC school
Figure 4. Number of FTC students, by private school’s FTC percentage, 2010-2016
Conclusion
This analysis indicates that participation in the FTC program has shifted toward schools with weaker track records of improving student outcomes, according to one broad categorization (share of students on scholarships) but not another (religiosity and denomination of school). This suggests that the average effect of FTC participation may decline, if the pattern of college enrollment effects from prior years persists.
This report provides less guidance on the broader issue of the ideal level of government regulation in private school choice programs. There is significant disagreement about how to balance the goal of keeping (or weeding) out low-performing schools with the possibility that regulations will discourage high-performing schools from participating. [6]
The Florida program is more regulated than other tax credit scholarship programs, but less regulated than most voucher programs, according to a 2013 Fordham Institute study. [7] The Fordham study counted the number of requirements for participating schools, with the goal of measuring regulatory burden. But less is known about how the content of regulations affects the number and quality of participating schools. The burden imposed by some regulations may well be justified by the protections they provide for students and taxpayers.
Research on private school choice is much better equipped to measure the effects on participants’ outcomes than to offer guidance on policy design. The main reason is that there are only a handful of programs with track records that can be compared. For example, the Florida program has a positive track record in terms of college enrollment whereas the Louisiana voucher program has a negative track record in terms of test scores. [8] Comparing just these two programs, it’s impossible to tell if this is the result of differences in program design, outcomes measured, the maturity of the program, or something else.
Careful study of variation in effects across different private schools within a given program may offer a path forward to drawing broader lessons from individual programs. The Florida study shows that some broad categories of private schools have more positive effects than others. A more fine-grained analysis might reveal more about the types of private schools that produce weaker or stronger outcomes for their students. For example, perhaps the choice of curricula, types of teachers employed, or length of the school day and year are associated with better or worse outcomes. [9]
Longer-term outcomes are especially desirable for such a line of inquiry, as there is greater agreement on the value of enrolling in college or finding success in the labor market than in performance on a particular test. Identifying the kinds of private schools that boost these outcomes could enhance policymakers’ ability to design private school choice programs that expand disadvantaged children’s access to high-quality educational opportunities.
— Matt Chingos
Matthew M. Chingos is a Senior Fellow at the Urban Institute.
This post originally appeared as part of Evidence Speaks, a weekly series of reports and notes by a standing panel of researchers under the editorship of Russ Whitehurst.
The version of this article released on 11/30/2017 underestimated the increase from 2010 to 2016 in the number and share of students attending private schools where a majority of students participate in the Florida Tax Credit (FTC) scholarship program. In the present version, released on 12/5/2017, Figure 4 was updated to correct the miscalculation, and the report’s conclusions were updated to reflect the corrected data. Figure 2 was also updated but the overall pattern of results reported did not change.
The author(s) were not paid by any entity outside of Brookings to write this particular article and did not receive financial support from or serve in a leadership position with any entity whose political or financial interests could be affected by this article.
Notes:
1. Children whose family incomes rise slightly can remain eligible, and the eligibility threshold was increased beginning in 2016-17.
2. The enrollment shares by sector reported in this paragraph and the following paragraph are based on the data used in this report, not the Urban study. Among the 5 percent of FTC students who attended a non-Christian religious school in 2010, 49 percent were at an Islamic/Muslim school, 28 percent were at a Jewish school, and 23 percent were at a school that selected the “other” category when asked to identify its denomination. In 2016, the breakdown was 37 percent Islamic/Muslim, 39 percent Jewish, and 24 percent other (and the overall share in the non-Christian religious category was 6 percent).
3. http://www.fldoe.org/schools/school-choice/facts-figures.stml
4. http://www.orlandosentinel.com/features/education/os-florida-school-voucher-investigation-1018-htmlstory.html
5. Previous research has found that FTC students’ average gains on norm-referenced tests vary substantially across individual private schools in Florida (http://www.fldoe.org/schools/school-choice/facts-figures.stml).
6. https://www.redefinedonline.org/2016/03/louisiana-voucher-debate/
7. https://edex.s3-us-west-2.amazonaws.com/publication/pdfs/20130129-School-Choice-Regulations-Red-Tape-or-Red-Herring-FINAL_7.pdf
8. http://www.uaedreform.org/louisiana-sseep-evaluation/
9. For parallel evidence on charter schools, see https://scholar.harvard.edu/files/fryer/files/dobbie_fryer_revision_final.pdf.
