"When the facts change, I change my mind. What do you do?"

The relevance of the title will become apparent, but first some context.

I've been researching a book, which has largely been on the benefits that increased competition and greater operation of market forces can bring to sectors such as health and education, which tend to use largely non-market methods to commission and distribute their services.

Along the way I've been looking at the literature on new private schools competing with the traditional public system schools - Sweden is often credited with giving this initiative its start, and it has been spreading elsewhere, notably in the UK ('academies') and the US ('charter schools'). And it's reached here, too, with the proposed launch of our version of the idea ('partnership schools').

So I've had to wade into the politicised jungle of quantitative research on how these schools have been performing.

With that as background, here's the news.

For a long time one of the key pieces of evidence on US charter schools was a 2009 study from the Center for Research on Education Outcomes at Stanford University, a body with the snappy acronym, CREDO. It was a big piece of work - the authors called it "the first time a sufficiently large body of student‐level data has been compiled to create findings that could be considered "national" in scope" (here and later, I'm quoting from the 2009 report's Executive Summary). It covered the learning gains in English and maths of 70% of the total student body then in charter schools, and compared them with demographically matched comparator students ('virtual twins') in the public school system.

One of the headline CREDO results was seized on by opponents and critics of charter schools: overall, 17% of charter schools were better than the public schools, 46% were much the same, but 37% were worse. The differences in actual outcomes were not large, but they were statistically significant. It was a widely quoted result and the study received broad coverage in educational circles everywhere: our own PPTA, for example, had a link on their website directly to it.

There was a range of other interesting findings, too, notably the differences in quality across charter schools ("tremendous variation in academic quality among charters is the norm, not the exception. The problem of quality is the most pressing issue that charter schools and their supporters face") and the fact that charters seemed to do better with students from more difficult backgrounds ("two subgroups fare better in charters than in the traditional system: students in poverty and ELL [English Language Learner] students"). As the report said, though, some charter success with sub-groups shouldn't be allowed to dominate the big picture finding: "greater attention should be paid to the large number of students not being well served in charter schools".

For people (like me) who believe that greater competition and better consumer choice are the first-best path to improved outcomes in many areas, this was, frankly, a disappointing though credible result.
CREDO said at the time that a follow-up research project was in the works, but then they went off the air, and over the years and from a distance I'd rather assumed they'd run into funding or other problems.

Fast forward to June 25 this year: CREDO reappeared with the second, even bigger report covering not just the 16 states of 2009 but another 10 states and New York City (looked at on its own for reasons we needn't bother with here), and using the school records of over 1.5 million charter school students.

Charter schools now outperform the traditional public schools (TPSs), mainly on the English side, with no real difference on the maths side. For English, charters are ahead 25% of the time, the same 56% of the time, and behind 19% of the time. For maths, it's technically a  small win for the TPSs though effectively a draw (charters ahead 29% of the time, behind 31% of the time, 40% no difference).

The 2009 result that charters were doing better with some disadvantaged groups came through again. As the 2013 Executive Summary puts it (p23), "Enrollment and persistence in charter schools is especially helpful for some students, particularly students in poverty, black students, and English language learners all of whom post significantly higher learning gains in both reading and math. Hispanic students are on par with their TPS peers in both reading and math. For students with multiple designations (such as being black and in poverty), the impacts of charter schooling are especially positive and noteworthy".

This, by the way, as I've posted before, is exactly what you'd expect from first principles: the people most likely to benefit from greater choice are those with the least opportunities now.

The report argues that one of the reasons for the better performance of charters is holding them to proper accountability and quality standards (8% of the original 2009 sample have closed), a conclusion I endorse, though I can't say I see the same principle of 'perform or get shut down' being invoked to the same degree to deal with the worst of the US public schools.

In any event, while it's hardly a knock-out win for proponents of greater choice and competition in education, it's a clear improvement on where we were four years ago. On the latest data and analysis, charter schools have a slight edge.

Hence the heading for this post: virtually every man and his dog in the educational establishment were keen to wave the first CREDO report around when it was critical of charters.

What are they doing now?