The border controls row rumbles on, with no clarity so far on the nature of the pilot study authorised by ministers which, the Home Secretary has claimed, led to a 10 per cent increase in arrests.

It seems likely that this study lacked the objectivity that randomization brings to formally-designed experiments. In the House of Commons yesterday, questioners tried to elicit the evidential-basis for the claimed 10 per cent increase in arrests (not in arrest-rate). Temporarily, they seemed to wrong-foot even the Prime Minister - but only because his brief had not included the necessary detail.

If, indeed, the Ministerial pilot study was non-randomized, how was it designed? Answers to the following key questions would help.

Q1. If it is correct that the new targeting regime was authorised for six weeks in August to mid-September in 2011, what was the comparison period?

{Possible answers: the previous and subsequent 6 weeks in 2011; or the same 6-week period but in previous years, such as 2010, 2009, 2008, 2007}

Q2. What outcome measures were to be compared between the 2011-target-period and the control-periods against which performance would be compared?

{Possible answers: number of arrests; arrest-rate (number of arrests/estimated passenger throughput); arrests which, within 48 hours, resulted in no-further-action and entry was permitted (false positive signals); estimated mean queuing-time and 10th and 90th  percentiles of waiting-times [how waiting-time is measured is crucial for unbiased comparison]; number of weapon finds; and so on}

Q3. For each outcome measure, what were the results for the 2011-target-period and for each control-period?

{Possible answers: actual numbers - not just percentages, please. A 10 per cent increase from 20 to 22 is far from compelling but a 10 per cent increase from 2,000 arrests in the control period to 2,200 arrests in the 2011-target-period would be. Provided, of course, that passenger-throughput had not also increased by 10 per cent between the control and 2011-target-period, in which case the arrest-rate would not have altered. And, if passenger throughput had doubled in the 2011-target-period compared to the control-period, then a 10 per cent only increase in arrests would be cause for concern, not celebration}

If a Border-policy-change was enabled at all entry-points during a holiday-period of high passenger-throughput, arrests might have increased by 10 per cent merely because throughput did. What are the answers to the above not-so-simple questions?

Declaration of interest: SMB writes in a personal capacity. SMB is member of Home Office’s Surveys, Design and Statistics Subcommittee.