A recent report from GEMS Education Solutions, a London-based education consultancy, titled “The Efficiency Index: Which education systems deliver the best value for the money,” is admirable in its aim.
Using the Programme for International Student Assessment (PISA) scores from the Organization for Economic Cooperation and Development (OECD), the report ranks which countries get the best bang, in terms of student outcomes, for the government buck. Given that, as GEMS says, government budgets are limited and they must spend efficiently, this should theoretically be an interesting exercise to help governments better allocate resources.
But the report does not produce a set of rankings from which we can learn meaningful lessons because the report itself has at least one significant flaw. The implication of the report—that we can learn lessons from countries with high student achievement per government dollar spent because they must have high-quality, efficient public schools—is therefore not necessarily true.
For at least one of the countries at the top of its index—Korea—the way in which its government spends money and how its schools are run are scarcely responsible for the country’s academic results. Korea boasts high achievement scores despite of its public schools, not because of them.
The introduction to the report acknowledges this to an extent, as Lord Andrew Adonis, the former United Kingdom education minister, wrote about the “long queues of pupils at bus stops in Seoul on their way home after evening (private) school,” and Andreas Schleicher, the director for education and skills to the secretary-general at the OECD, wrote that “the apparent high degree of efficiency of the East Asian education systems may, at least in part, be due to significant household spending on out-of-school education that is not accounted for by the analysis.”
The authors of the report try to account for after-school tutoring, but, in classic econometric fashion, they dismiss it as a factor explaining PISA scores—even though Korea is a significant outlier on their own chart correlating PISA scores to the percent of students receiving tutoring (the chart shows a negative relationship between the two elements).
The underlying data also appears to be flawed. According to a report that the OECD published, a far higher number of students in Korea—nearly 75 percent—attend forms of after-school education (mostly in hagwons) compared to the roughly 60 percent figure in the GEMS report. For anyone who has spent any time in Korea studying the country’s schools, as I did during my recent Eisenhower Fellowship, the higher number seems far more accurate.
Furthermore, although the report correlates the percent of students receiving tutoring, it does not capture the intensity of time spent in those tutoring sessions or the spending on tutoring. In the case of Korea, this is a significant omission.
As I’ve written, Korean students learn in after-school hagwons until the wee hours of the morning in many cases, despite regulations designed to halt that practice. And in their public schools, particularly in high school and middle school, as the teacher lectures to a large class of students, many if not most students sleep. It’s hard to imagine that the teachers are so talented that they can cause their students to learn through osmosis. Had the authors spent more than a few minutes in Korean schools, they might have grasped this. To make the point in a more quantifiable way, although public spending on education in Korea is around 5 percent of GDP, the country spends a whopping 1.8 percent of GDP on private after-school programs—or 20 percent of household income, according to this recent Wall Street Journal article.
Of course, it could be that the lesson to be learned is that countries should change their cultures and economic circumstances, value education more, and spend more privately on education outside of school, thereby allowing the government to spend less on education while student achievement rises. Good luck with that.
Educational efficiency is certainly a critical consideration, as the GEMS report states. Indeed, as we write in our new book Blended, many think that blended learning will help control costs and boost achievement in schools. But the flaws in this report that purports to compare countries’ educational efficiency shed doubt on its ability to further that conversation.
-Michael Horn
This first appeared on Forbes.com
