From the TUC

BBC cuts survey – A poor poll poorly reported

10 Oct 2013, by in Politics

The BBC website reports:

A BBC-ICM poll on the effect of cuts to local government services made the prime minister think he “had died and gone to heaven.”

Responding to a question from Conservative backbencher Mark Pawsey who asked if it proved “you can get more for less” Mr Cameron said it showed “one of the many pillars of Labour’s policy that have collapsed today”, and he said “Let’s praise the BBC for once!”

Well, for once, we should criticise the BBC for a poor poll (pdf) and then reporting it badly.

The problem is that the BBC’s coverage highlights the findings that the poll has not measured well – which on the whole are good for the government – and almost completely ignored the more robust results, which are bad for the government.

The web coverage of the poll (by Mark Easton, who I should say I have always respected) starts by saying:

Many people in Britain think the quality of public services overall have been maintained or improved in the past five years despite government cuts, a poll for the BBC suggests.

The broadcast coverage that I heard had a pretty similar top line. This is no doubt what so pleased the Prime Minister.

Yet:

  • 40 per cent think services have got worse compared to 15 per cent who think they have got better
  • 63 per cent say they are worried about cuts made to many public services (20 per cent disagree)
  • 52 per cent are angry at cuts made to many public services (27 per cent disagree)
  • 47 per cent think cuts to welfare and benefits have gone too far (37 per cent disagree)

Most challenging for the government given how much of the cuts have yet to come, only 27 per cent think spending needs to be reduced further compared to 54 per cent who disagree. That would be my top line finding.

These are all robust findings. It is a sample size of 1,000; and the questions are clear and do not lead. None are particularly good news for the government.

But this is not what dominates the BBC coverage. Instead they make two claims that are not as robust.

  • The BBC ask people whether they think services have got worse (40%), better (15%) or stayed the same (42%). When reporting a question asking people to choose between two options with a further ‘neither one nor the other’ choice in the middle it is usually poor practice to lump this undecided group in with one of the others. But this is what the BBC have done by counting those who think services have stayed the same with those who think they have got better. The normal way of reporting this would be to highlight the 40 and 15 figures. People can see that a substantial group think ‘neither’ or don’t know as the quoted numbers do not add up to 100. It would have been just as bad to say 82 per cent of respondents think that services have got worse or stayed the same, even though the Prime Minister claims: “We have demonstrated in government that you can make reductions but improve services.”
  • The section of the poll that deals with specific services also presents problems. This is not a good way of measuring people’s experiences of services. That would require a much more thorough survey of service users. Not all service users would have used the same services five years. Those who used libaries five years ago, but no longer do would not be counted as library users in the BBC sample. And worst of all they present a finding based on a sample of 35 – a tiny number for a survey. I would never allow that in a TUC release.
  • And of course there are many services that are used by a very small proportion of the population but cost a lot and have been cut severely. These  include services for vulnerable adults and children. People who have completely lost home care (most councils now only help those in the highest category of need) would, again. not count as service users as they can’t use a  service that has been cut.

That’s not to say that none of these findings are of interest, but their presentation is skewed.   100 per cent of this reader is pretty cross.

6 Responses to BBC cuts survey – A poor poll poorly reported

  1. TC
    Oct 10th 2013, 7:03 pm

    And we wonder why the Government viewpoint dominates the ‘narrative’ not to mention that 75% of the announced cuts are yet to come…

    http://unisonactive.blogspot.co.uk/2013/10/bbc-spins-austerity-sunshine-story.html

  2. MD
    Oct 10th 2013, 8:44 pm

    Also the poll was “over the last 5 years”. How much of the improvements (or otherwise) could be attributed to policy from the previous administration.

    From personal experience, in our community, people are thrilled with the improvement in our local primary school, largely as a result of spanking new facilities built and paid for under the Building Schools for the Future initiative, axed by the current lot.

    Also, as a snapshot survey, we have no comparison of trends over time. Are less people seeing improvements now or more? Who knows?

  3. Ray McHale
    Oct 11th 2013, 12:17 am

    They concentrated on refuse collection and recycling, which are services experienced by everyone. However, improvements in these services and extensions to recycling have been driven by the penalties of the Landfill Tax – so indeed they have generally improved. But they represent a drop in the ocean compared to the cost of services for the elderly and adults with learning difficulties. As you say, those whose libraries have closed may well no longer be users. Over 4 years our Tory Council seems to have halved the people provided with meals on wheels. Basically they no longer even offer a service – just a subside for people to spend in the market, which mainly consists of the old private sector provider. But most people are not aware of this. Cuts to careers and youth services were not mentioned, nor to public and school transport, and housing is largely in the hands of the voluntary sector – like much social care provision. At least people were clear that many of these privatised care services are now cheap and nasty, based on low paid poorly trained staff who have to be incredibly flexible in their availability. This is a survey mainly of people’s impressions of how services are, rather than being based on real knowledge.

  4. Ben Soffa
    Oct 11th 2013, 8:31 am

    The presentation of the ‘meals on wheels’ score is even worse than you suggest.

    Because they use a net difference between those thinking there has been a change, the +12 improvement is based not on the sample group of 35 but actually just ten people nationwide.

    The raw ‘net: better’ score is 7 whilst the raw ‘net: worse’ score is 3 (with others thinking there’d been no change).

    I’ve complained at http://www.bbc.co.uk/complaints/ and suggest others might like to too.

  5. Denis Lenihan
    Oct 11th 2013, 12:55 pm

    Presumably if the poll was done, in say, the poorest 80 Boroughs, we would have seen a completely different result?

  6. Public service cuts: Has the government gotten away with it? | Flip Chart Fairy Tales
    Nov 14th 2013, 7:02 am

    […] Nigel Stanley at Touchstone noted, if you dig into the survey’s findings, they are not quite so positive. The majority of those […]