IF a leaflet given to women claimed that hormone replacement therapy reduced the risk of colon cancer by 50 per cent, but increased the risk of breast cancer by six in a thousand, what might they conclude? 

Unless they were astonishingly perceptive, they would conclude that on this basis the benefits of HRT exceeded its risks. But they would be wrong. The 50 per cent reduction in colon cancer would actually benefit fewer women than would be damaged by the six in a thousand increase in breast cancer.

The leaflet, which actually existed, is an example of a problem endemic in medicine. Journals, pamphlets, websites and patient leaflets systematically fail to present evidence in an even-handed way. They do not lie, but they conspire to deceive – and the effects are just as bad.

Typically, the benefits are presented as a percentage improvement, while the  harms are given in absolute terms. This makes the benefits look much bigger and the harms much smaller - even to clinicians who might be expected to know better.

The trick is called “mismatched framing”, and it is everywhere. One study of three major medical journals - the BMJ, The Lancet, and the Journal of the American Medical Association – found that when both benefits and harms were reported in studies published from 2004 to 2006, a third of the time they were not reported using the same metric.

Health information intended for public consumption is often much worse. The breast cancer leaflets distributed to women in the UK are an egregious example, failing to mention several of the serious harms mammographic screening can do.

They include overtreatment, an increase in mastectomies, the detection of anomalies that would never have led to cancer, and the anxiety and stress of being recalled for further tests. All these are described by Peter Götzsche and colleagues in BMJ (338, p446-448, 21 February 2009)

None of them is included in the leaflet given to British women, despite its title, Breast Cancer: The Facts. Nor are the benefits spelled out in a way that is easy to interpret. The leaflet says that breast screening saves 1,400 lives a year – a claim that is itself open to challenge – but nowhere does it say what that means for an individual deciding whether or not to accept the invitation to attend a mammogram. 

The answer is to present the data differently, and to use the same frames of reference for both the positives and the negatives. Gerd Gigerenzer, a psychologist and Director of the Centre for Adaptive Behaviour and Cognition at the Max Planck Institute for Human Development in Berlin, has clearly shown how this can be done. His ideas have gained some traction with the US Institute of Medicine, and the UK Academy of Medical Sciences is preparing a report on communicating benefits and harms, due out this summer.

What should it conclude? That medical reporting, in journals and elsewhere, is riddled with skewed presentations and that until clinicians get their house in order, everybody is being misled.

A common way of presenting benefits is as a percentage increase in survival. This looks mightily impressive. But it tells us nothing about the baseline risk. If that is small, even a 50 per cent reduction will be small, too. Compare the statement: “Mammography screening reduces the risk of dying from breast cancer by about 20 per cent” with the statement: “Mammography screening reduces the risk of dying from breast cancer by about one in a thousand - from 5 in 1,000 to about 4 in 1,000”. Both represent the same data, but the first looks like a huge benefit, the second rather a small one.

The answer, say Gigerenzer and colleagues in a report for the American Association for Psychological Science, is to present both benefits and harms in the same measures. His choice is natural frequencies (as in the second statement above) or their obverse, “number needed to treat”. In the breast screening example above – which probably exaggerates the benefits - the number needed to treat is 1,000. From this, women can instantly calculate that there is only a one in a thousand chance that their lives will be saved by attending screening, even if they do so for ten years, since all the data derives from studies of screening over a ten-year period.

Would this make fewer women attend screening? Almost certainly. In some countries, attempts to express the risks and benefits honestly have been deliberately obstructed because doctors are convinced that screening works. In 2006, the Austrian Association of Physicians asked their members to remove from their shelves a leaflet that sought to explain risks and benefits more honestly because they thought it would discourage women from coming. 

In the UK, obstruction has been subtler. Despite the fact that the deficiencies of the patient leaflets have been repeatedly pointed out by Peter Götzsche, Michael Baum, and others, little has changed. The head of the NHS breast screening service now says the leaflets will be “reviewed”. Don’t hold your breath.