School league tables are worthless as a basis for choosing a school, and would be best left unpublished.

 According to two statisticians from Bristol, they offer little useful information for parents deciding which secondary school is best for their children, with confidence intervals so wide that only a handful of schools can be identified as better than the average school, or than one another.
 
The research, published originally in the Journal of the Royal Statistical Society: Series A, 172 (4) is summarised this week in Research in Public Policy. George Leckie and Professor Harvey Goldstein of the Centre for Multilevel Modelling at the University of Bristol take league tables apart and find there is very little worthwhile content.
 
League tables were introduced in the early 1990s and initially were based on the percentage of pupils achieving five or more GCSE passes with grades A* to C. But critics said this favoured schools with the brightest entry, so in 2002 it was changed to measure “value-added” – the amount by which a pupil had improved since entry into secondary school. A further twist in 2006 added family background, gender, and eligibility for school meals to the mix to try to account for social differences. This is called contextual value-added (CVA).
 
The two authors first compare CVA scores with the traditional “five passes” scores and show there is little correlation. Schools with high CVA scores have no increased likelihood of scoring well when measured by the number of A* to C grades at GCSE, and vice versa.
 
Then they look at the Government’s preferred measure, CVA (see their figure 2, below). The confidence intervals in this measure are so wide that for almost half the schools it is impossible to conclude that they are better than average, or than each other. Only schools at the top and bottom end are statistically distinguishable.
 
                                                                        
 
But even this is little help to parents because they are choosing a school now in order for their children to perform at their best five years from now. Parents choosing schools in 2008 (on the basis of the 2007 figures) would not know their children’s GCSE results until 2014.
 
So what actually matters to them is not whether than can choose a good school now from the league tables, but whether it will still be good in 2014. By comparing schools’ CVA rankings in 2007 with their rankings in 2002 (the greatest gap possible with existing data) they show that many schools improve, and many deteriorate, over a five year-period. A plot of the CVA scores schools are predicted to have in 2014 against what they actually had in 2007 (their Figure 4, below) shows such wide confidence intervals that almost all of them cross the zero line – which means their performance cannot be separated statistically. Only a very small number of schools can be distinguished with any reliability as better than average.    
   
                     
                      
So this implies that the league tables are a waste of time. Parents need to be aware that the tables “contain less information than official sources imply” and that they would be wise to put a lower weight on them in comparison with other sources of information.
 

“Publishing league table to inform parental choice of school is a meaningless exercise, as parents are using a tool which is not fit for that purpose. This is especially so as the media coverage rarely features confidence intervals.”

 
Scotland, Wales and Northern Ireland either never had or have abandoned publishing league tables. “Now seems a good time for England to follow suit, they conclude.
 
And the same arguments probably also apply to league tables in health and social services, where attempts are made to inform individual choices of instititution based on past performance. You have been warned.