BBC cuts survey – A poor poll poorly reported
The BBC website reports:
A BBC-ICM poll on the effect of cuts to local government services made the prime minister think he “had died and gone to heaven.”
Responding to a question from Conservative backbencher Mark Pawsey who asked if it proved “you can get more for less” Mr Cameron said it showed “one of the many pillars of Labour’s policy that have collapsed today”, and he said “Let’s praise the BBC for once!”
Well, for once, we should criticise the BBC for a poor poll (pdf) and then reporting it badly.
The problem is that the BBC’s coverage highlights the findings that the poll has not measured well – which on the whole are good for the government – and almost completely ignored the more robust results, which are bad for the government.
The web coverage of the poll (by Mark Easton, who I should say I have always respected) starts by saying:
Many people in Britain think the quality of public services overall have been maintained or improved in the past five years despite government cuts, a poll for the BBC suggests.
The broadcast coverage that I heard had a pretty similar top line. This is no doubt what so pleased the Prime Minister.
- 40 per cent think services have got worse compared to 15 per cent who think they have got better
- 63 per cent say they are worried about cuts made to many public services (20 per cent disagree)
- 52 per cent are angry at cuts made to many public services (27 per cent disagree)
- 47 per cent think cuts to welfare and benefits have gone too far (37 per cent disagree)
Most challenging for the government given how much of the cuts have yet to come, only 27 per cent think spending needs to be reduced further compared to 54 per cent who disagree. That would be my top line finding.
These are all robust findings. It is a sample size of 1,000; and the questions are clear and do not lead. None are particularly good news for the government.
But this is not what dominates the BBC coverage. Instead they make two claims that are not as robust.
- The BBC ask people whether they think services have got worse (40%), better (15%) or stayed the same (42%). When reporting a question asking people to choose between two options with a further ‘neither one nor the other’ choice in the middle it is usually poor practice to lump this undecided group in with one of the others. But this is what the BBC have done by counting those who think services have stayed the same with those who think they have got better. The normal way of reporting this would be to highlight the 40 and 15 figures. People can see that a substantial group think ‘neither’ or don’t know as the quoted numbers do not add up to 100. It would have been just as bad to say 82 per cent of respondents think that services have got worse or stayed the same, even though the Prime Minister claims: “We have demonstrated in government that you can make reductions but improve services.”
- The section of the poll that deals with specific services also presents problems. This is not a good way of measuring people’s experiences of services. That would require a much more thorough survey of service users. Not all service users would have used the same services five years. Those who used libaries five years ago, but no longer do would not be counted as library users in the BBC sample. And worst of all they present a finding based on a sample of 35 – a tiny number for a survey. I would never allow that in a TUC release.
- And of course there are many services that are used by a very small proportion of the population but cost a lot and have been cut severely. These include services for vulnerable adults and children. People who have completely lost home care (most councils now only help those in the highest category of need) would, again. not count as service users as they can’t use a service that has been cut.
That’s not to say that none of these findings are of interest, but their presentation is skewed. 100 per cent of this reader is pretty cross.