Builders and decorators regularly appear in lists of the ten least-trusted professions, along with journalists, politicians and estate agents. Nevertheless, a claim on August 4 by Which? (the trading name of what used to be the Consumers’ Association) that 2.5 million people in the UK have had disputes with builders and decorators in the past three years did seem to me slightly implausible.

That’s one in ten households. Since many households don’t entertain a builder or decorator for many years, it seems a high figure. It turns out to hang on precisely what you mean by a dispute.

Which? asked 2,545 people over 16 years of age a series of questions in an online poll. Weighting was applied to the responses so that they represented the adult population of the UK. No real problem with any of that.

The poll opened by asking if respondents had employed a builder or decorator in the past three years. A third had; the other two thirds hadn’t, so were excluded from answering any more questions. It’s unclear from Which?’s summary of the data whether the 2,545 respondents consisted only of those who answered yes to this question, or whether the results depended on the remaining one third, ie around 850 people.

Question 8 asked if respondents had had a dispute with the trader at any stage: not a disagreement, a discussion, a difference of opinion, or a negotiation. Building is a complex trade, with constant unexpected snags to be overcome. It’s unlikely any building project gets finished without some discussion of what should be done, how it should be done, and what it’s costing. The nature of building work is a constant dialogue between customer and tradesman. To categorise all such discussions as disputes and offer respondents no alternative formulation amounts to asking a loaded question.

Twelve alternatives were offered to categorise what the “dispute” amounted to. They included disputes over when to pay, delays over completion, disputes over tidiness, disputes over the quality of the work, tradesmen not turning up when they said they would, and so on. Totting up all these categories, the survey found that 19 per cent of those who had employed tradesmen had had “disputes” so categorised. Almost two thirds (64 per cent) were resolved by discussion. Of the rest, a third went to some form of outside arbitration, either a trade association or the courts.

If we define “dispute” as a disagreement that cannot be resolved amicably, or remains unresolved, then these are the only true disputes Which? uncovered. So instead of the headline number of 2.5 million, the true number of disputes was 900,000, of which 300,000 were sufficiently serious for the customers to seek redress through arbitration.  The other 600,000 might have left customers feeling aggrieved, but not enough to do anything about it, possibly because they may have recognised that the fault lay with them, and not the tradesmen they employed.  (A quarter had failed even to get a written quote before the work started, which left them very little room to protest about anything.)

The Which? survey got wide publicity, playing as it does into perceptions of cowboy builders roaming the country bodging  every job they tackle. Such cowboys exist; but so, alas, do cowboy surveys that win easy headlines by exaggeration.