The new Government has promised to establish a system in which the price paid for drugs represents the value they bring to patients. But it won’t be in operation before 2014, and in the interim, and in some cases after that, it may be necessary to negotiate “risk-sharing” agreements with the drug companies.

Under such deals, the companies will pay back part of the cost of the medicines if they fail to achieve their promise. But if the experience of the multiple sclerosis risk-sharing deal negotiated by the last Government in 2002 is any guide, the results are likely to be very disappointing.

In an accompanying article Sheila Bird draws attention to how the way in which the deal has undermined subsequent appraisals by the National Institute for health and Clinical Excellence (NICE) for MS drugs. Four papers published in BMJ.com today analyse the results of the deal, from different perspectives: health economists seeing it as a fiasco, neurologists taking a more benign view.

The actual results of the treatment with interferons and glatiramer acetate, published at the end of 2009 and analysed here by Sheila Bird, were appalling. Far from slowing progression of the disease, the drugs appeared to accelerate it. That means that to comply with the terms of the deal, the companies would have had to pay money to the NHS to use the drugs, rather than the NHS paying them.

Did this happen? Absolutely not. What actually happened was that the NHS went on paying close to the full price for the drugs in apparent disregard of the rules of the scheme. The committee responsible concluded that it was “premature at this stage to reach any decision about re-pricing the drugs without further follow-up and analysis”. So the whole basis of the scheme – linking prices to outcomes according to a protocol agreed at the start – was undermined.

According to the rules, price-setting reviews should have been held every two years. Had this actually happened, the money spent doing no good could have been spent somewhere else. In BMJ.com, Professor James Raftery of Southampton University calculates the scheme cost £50 million a year, or possibly twice as much, if the MS Society is correct in saying that 10,000 patients are getting the drugs through the scheme. “This may well be the most expensive, publicly funded, ongoing health related study in the UK, and probably anywhere, ever” he writes.

Those responsible for running the scheme mostly had vested interests in its success, including representatives of the four health departments, the MS Society, the MS Trust, and the Association of British Neurologists. Those who have to pay the bills – the primary care trusts – were not represented on the “independent scientific advisory group” which apparently made the decisions.

The scheme was also slow in publishing its results. Depending upon when the patients were recruited, the first analysis reported between two and five years after the data became available. Why? In another BMJ.com paper Christopher McCabe of Leeds University and colleagues concluded that this was because day-to-day running of the scheme was in the hands of the MS Trust which had campaigned for the funding of the treatments and appealed against the NICE ruling that they were not cost-effective.

If one went out looking for an example of how not to organise a risk-sharing trial, one could hardly have done any better. The two neurologists who do their best to defend the scheme, Neil Scolding of Frenchay Hospital, Bristol, and Alastair Compston of Cambridge University, admit the results were bad: “resoundingly negative” is Professor Compston’s phrase.

But Professor Scolding makes the sheer ineptitude of the trial its virtue. It is “just one study, unblinded, unrandomised, with an historical control group and using a measure of disability that is unfit for purpose”, he says, and therefore its results should not be taken seriously. This is breathtaking. The whole point of the scheme was to see if the drugs worked, according to a protocol agreed by all. They did not: but the companies who make them were not forced to pay any penalty.

Let’s hope the lessons are learned, but the attitude of the new Health Secretary, Andrew Lansley, towards end-of-life treatments for cancer suggests it may not have been. It’s awful to tell people there are no effective treatments for their condition, or that those available don’t deliver enough benefit to be value for money. But it is honest.

Today the Department of Health is still defending the scheme, on the grounds that it brought “many benefits to MS patients including better access to drugs”. Drugs that, according to the published results, were worse than no treatment at all. It is hard to disagree with Professor George Ebers, from Oxford University, a long-term critic, who said: “The scheme may have been well intentioned, but perhaps the public interest would be served by an independent inquiry.”