Radio 4's Today programme this morning interviewed Professor Nutt, chairman of the Advisory Council on the Misuse of Drugs, about politicians’ undermining, and thereby undervaluing, the evidence on the harm done by drugs. 

The best-remembered example was Jacqui Smith’s proposed re-classification of cannabis from class C to class B. Well done, Today, for promoting the proper use of evidence.

But Today also interviewed the chair of the Prison Reform Trust about the practice of restorative justice for young offenders in Northern Ireland, which victims of crime valued. Many proper questions were put, but the answers were weakly evidenced. For example:

(1) Did restorative justice have any impact on the sentence the young offender would otherwise have received? Proper answer: no information, both because there was no randomized control group as in a prior, adequately powered formal experiment, nor are judges asked what sentence they would have given if the young offender had declined to face up to his/her victim(s) in a restorative justice conference, and undertaken to make amends.

(2) Does the estimated cost of £1,200 for a restorative justice conference add to, or substitute for, other criminal justice costs? Proper answer: we don’t know, as Northern Ireland presumably computed the cost of its restorative justice conferencing but, lacking a properly randomized control group, could not have costed its alternative sentencing because of the lack of like-for-like comparator. Actual answer glided away from the question, but suggested other criminal justice costs might have been many thousands of pounds, possibly for incarceration.

(3)    Re-offending rate for young offenders assigned to restorative justice compared to re-offending rate of like-for-like comparator? Proper answer: no like-for-like comparator and so unanswerable. Actual answer: 37 per cent re-offending rate over an unspecified follow-up interval (1-year, 2-years, 5-years?) for an unspecified number of young offenders who took part in restorative justice conferences in an unspecified period was compared to 75 per cent for young offenders who had been incarcerated (follow-up interval, number, period???). However, there is no guarantee, as the Today interviewer strived to convey, that the young offenders who took part in restorative justice conferences in Northern Ireland would all otherwise have been sent to prison.

(4) Experimental evidence on the impact and cost-effectiveness of restorative justice conferencing for young offenders? This last crucial question was not put – which put Today in the unenviable position of under-valuing evidence.

What was truly dire is that there was no mention of the Home Office’s landmark set of randomized controlled trials (RCTs) of restorative justice, some in young offenders, by which questions 1) to 3) were answerable. One of us (Sherman) co-directed these trials from 2001-2005, at £5 millions in taxpayer expense. An independent evaluation, published by the Ministry of Justice concluded that the return on investment in face-to-face victim meetings with offenders, as a supplemental cost to standard practice, was about £8 in the cost of crime prevented for every £1 spent on the meetings.

The Today discussion can perhaps be forgiven for not referring to such evidence when the Government itself has not published a press release about the results, merely posting a notice on a Ministry of Justice web page where it has sat without comment since June 2008.   

Taken together, these restorative justice trials randomized around 700 offenders who had pleaded guilty to a range of crimes – the range differed according to the sub-trial in which the offender was randomized. Ideally, for sufficient precision, each sub-trial should have randomized at least 400 offenders, not 100 for each to have had at least 50 per cent power in respect of modest reduction in 2-year re-offending rate, such as from 70 per cent to 60 pr cent. But, let us not make the best the enemy of the good.

 The elephant that stalked Today’s interview is the following.
 
Why, given that promising yet insufficiently precise results on efficacy and cost-effectiveness were obtained in a UK-landmark suite of RCTs on restorative justice, has the Ministry of Justice ignored its well-founded evidence?
 
The evidence suggested modest, not dramatic, impact; and cost-effectiveness, but with costs for restorative conferencing upfront. In times of financial restraint, costs upfront are never attractive and always harder to justify if effect-sizes are more modest than hoped-for. Of course, randomized trials should always be designed to detect plausible, not desired, effect-sizes.
 
Consistent with the UK’s need to grow its experimental evidence-base on restorative justice, and with financial constraint, would be for Ministry of Justice to continue down the path of randomization to restorative justice/conventional sentencing so that questions 1) to 3) continue to be answerable but on the basis of continued randomization in selected sub-trials.
 
Selection of which sub-trials to continue should be based on criminological expertise and a priori cost-effectiveness, not on p-values! For efficient deployment of the teams who manage restorative justice conferencing and to learn more quickly about conferencing but without compromise on like-with-like, the randomization ratio should be tilted away from 50:50 so that more offenders are randomized to restorative justice than to conventional sentencing, such as 60:40 or 67:33.
 
Clive Fairweather CBE, Scotland’s former Chief Inspector of Prisons for Scotland, gave a lecture in Cambridge last month on Healthy Prisons, in which he advocated that criminal justice should invest in young offenders: as future proofing. Similarly, last week’s Harveian Oration at the Royal College of Physicians by the Chief Executive of the Medical Research Council was entitled ‘Prevention is better than Cure’.
 
Sheila M. Bird is at the MRC Biostatistics Unit in Cambridge: Lawrence W. Sherman is Wolfson Professor of Criminology at Cambridge University.
 
(Interest declared: SMB chairs the Home Office’s Surveys, Design and Statistics Subcommittee. LWS co-directed the 2001-5 Home Office-funded randomized trials in restorative justice.)