The Lancet, WHO and Caesareans: over to the Ombudsman
A formal complaint has been made to The Lancet’s Ombudsman over a paper on the risks of Caesareans that was the subject of a critical analysis by Straight Statistics.
Penny Christensen, Executive Director of Birth Trauma Canada, has asked for a review of the study, carried out by a team from the World Health Organisation. It concluded, on the basis of evidence which did not convince Ms Christensen (or me), that elective Caesareans without medical indication were almost three times as likely to cause maternal mortality and morbidity as vaginal births.
She seeks to establish two points: How did this study make it through The Lancet’s peer review process? And why has negative criticism of this study been largely censored by The Lancet? To which The Lancet itself has added a third, that in its own response she has been met with “obfuscation and resistance”.
To her complaint she attached the article from Straight Statistics, which provoked a strange response from a Lancet editor. He refused to consider the evidence presented on the grounds that it had not been peer-reviewed. “We are a scientific journal and as such prefer to see the scientific debate continued by reference to other academic articles that have been peer-reviewed” he said.
True, Straight Statistics makes no claim to be a scientific or medical journal but it is odd to claim that as a result its conclusions do not warrant consideration on their own merits. What matters is whether they are right or wrong, not whether they have passed through the laying-on-of-hands conferred by peer review. I hope The Lancet’s Ombudsman, Dr Charles Warlow, will not seek to hide behind this defence.
The study was carried out in Asia and the principal conclusion largely depended on a correction for confounding factors that appears implausible. The raw data showed that elective Caesarean without indication was safer than vaginal birth: but after correction, the paper concluded that it was less safe by a substantial margin. The original analysis is here: I will not repeat it, but since it appeared both on this website and in The Independent (below) there has been no response from either WHO or The Lancet.
WHO is opposed to elective Caesareans unless they are justified by medical conditions, though last year it admitted that it had little evidence to justify the advice it gives that Caesarean birth for whatever cause should not exceed 10-15 per cent. Its handbook on emergency obstetric care, updated in October 2009, now says the optimum rate is unknown.
It said: “Although WHO has recommended since 1985 that the rate not exceed 10-15 per cent, there is no empirical evidence for an optimum percentage or range of percentages, despite a growing body of evidence that shows a negative effect of high rates”.
Critics assert that this body of evidence, taken as a whole, does not justify the final words, and certainly the study published in The Lancet does nothing to justify them. It has already been strongly criticised by Andrew Weeks, of the School of Reproductive and Developmental Medicine at Liverpool, in a comment that The Lancet did publish. He said that the authors, led by Pisake Lumbiganon, had made “serious errors in their analysis of the data which completely change the conclusions of the paper”.
His criticisms focus on the failure of the team to analyse the data on the intention-to-treat principle, and the statistical corrections they made. For women, he said, the factors corrected for are part of their reality and an inseparable part of their overall risk. So the crude data, uncorrected for risk, present the best picture, and these show that both for babies and mothers elective Caesareans without indication are safer. The authors, in a response, rejected these criticisms.
Ms Christensen also made submissions to The Lancet, but these have not been published. She says: “I can recognise obstetrical bias when I see it, and the study by Lumbiganon is just that”.
Jonathan Bagley (not verified) wrote,
Thu, 12/08/2010 - 14:43
Refusing to engage with criticism that has not been peer reviewed appears to be becoming more and more common. The less than scholarly wing of "academia" has decided that it can intimidate Joe Public with this phrase. This has been happening in the claims regarding passive smoking. "Studies (using small samples) show" in the NEJM and the BMJ, for example, that heart attacks have decreased since smoking bans came into force. When informed non specialists point out that national statistics from the ONS and equivalent organisations don't bear this out, they are dismissed as they are not peer reviewed. Well, how on earth can an ONS table constitute an article submitted for publication? Another recent example is the controversy over the Spirit Level book about the effects of inequality. The authors have said they will not accept criticism unless it is peer reviewed. Below is a quote from the Equality Trust website.
"Almost all of the research presented and synthesised in The Spirit Level had previously been peer-reviewed, and is fully referenced therein. In order to distinguish between well founded criticism and unsubstantiated claims made for political purposes, all future debate should take place in peer-reviewed publications."
This becomes nonsensical when an argument is about whether Cuba has a greater life expectancy than the USA.
Piers Gaunt (not verified) wrote,
Wed, 25/08/2010 - 08:00
After reading your article in The Independent last Saturday titled 'Peer-reviewed journals aren't worth the paper they're written on' which must follow on from the above story I was torn between agreement with the article in general and dismay at the tone in which it was conveyed.
I am in firm agreement with the issues cited with respect to the peer-review processs. I also detest the notion that study owners are now refusing to respond to criticism unless it has been peer-reviewed. I just wanted to comment that, whilst the article raises the profile of the issue, it does so in such a negative manner that it ridicules articles in all peer-reviewed publications.
Personally I thought the aggressive nature of this article offered no alternatives to the peer-review process. It was a bashing of the current system without being progressive. Whilst I acknowledge that the issue needed to be raised it could have offered an alternative such as an interesting piece in the NYT - see link below.
http://dlvr.it/4CX5z
(Posted by Ben Goldacre on Twitter)