Hooray! A proper survey of what doctors think of the Health Bill

Credit where it’s due: the Royal College of Physicians today reports the results of a properly-conducted survey of what its members and fellows think of the Health and Social Care Bill.

I’ve previously criticised the Royal Colleges of General Practitioners and of Psychiatrists for online polls of their members that generated very low response rates and from which anybody with a respect for evidence would hesitate to draw conclusions.

The physicians, by contrast, engaged the Electoral Reform Society and sent out questionnaires by post and e-mail (where known) to its 25,417 fellows and members, achieving a response rate of 35 per cent (8,878) which is respectable, certainly much better than the 9 per cent achieved by the RCGP and the 11 per cent by the psychiatrists.

The BMJ, which claims to be read by 100,000 doctors in the UK, did a similar online poll, with a response rate of less than 3 per cent – 2,710 (92 per cent) saying the Bill should be withdrawn, and 241 (8 per cent) saying it shouldn’t.

It’s interesting to compare results from the RCP’s survey with these online “voodoo polls”. The RCP found that 69 per cent favoured rejection of the Bill as it stands, with 6 per cent accepting it and 22 per cent neither completely accepting nor completely rejecting it.

When we look at the question most closely aligned with those asked by the RCGP and by BMJ – that is, should the Bill be withdrawn – 49 per cent favoured the call for withdrawal with 46 per cent calling for continued engagement to improve it. A further 4 per cent had no opinion.

This result is quite different from those of the RCGP and BMJ online polls, both of which scored more than 90 per cent in favour of calling for withdrawal of the Bill. The psychiatrists didn’t ask this exact question, but did ask whether respondents thought the bill “fundamentally flawed” and more than 80 per cent said yes.

The more nuanced response from the physicians is in spite of the fact that they may have more to lose from the bill than GPs, since it aims to put the GPs in charge of commissioning care, a prospect not all hospital doctors welcome.

The RCP’s president, Sir Richard Thompson, said: “We believe that this is the single biggest survey among the medical royal colleges, with the highest turnout, and while it shows there is a clear majority of RCP’s fellows and members who would personally reject the Bill, opinion is divided almost equally on whether the RCP should continue to critically engage or now call for the withdrawal of the Bill.”

The results, available in full on the RCP website, are a textbook example of why I rail constantly and unavailingly against self-selected online polls that draw opinion only from the strongly committed. These polls produce biased results, and the RCP deserves great credit for spending a bit of money to do it properly and showing what a difference it makes.

Update, 19 March

And since we're handing out bouquets, another one is due to the Readers' Editor of The Observer, Stephen Pritchard, who wrote in his column yesterday that the "poll" underlying a story in the paper last month was not to be trusted. 

The story said nine out of ten members of the Royal College of Physicians wanted the health bill withdrawn. But it came from a website, callonyourcollege.blogspot.com, set up to oppose the bill, and was a self-selecting open-access poll. Junk, in plain language. Stephen's column indicates that the one of the journalists responsible for the story is reluctant to accept this. 

But junk is what it was. It's no excuse to say that readers were given enough information about its provenance to work out for themselves it was junk. That's a journalist's job.