Straight Statistics change their tune as Ben Goldacre joins the criticism of the Sunday Times
Last Sunday I wrote a critique of a Sunday Times article about public versus private pay. Frankly I do not expect very much from the Sunday Times. What really irritated me was the claim that its facts had been validated by Straight Statistics – an organisation whose aims I admire and whose work I have quoted in the past.
I assumed that they had probably been done over by the Sunday Times, so I was scrupulous in drawing my piece to the attention of Nigel Hawkes and inviting him to reply.
I was grateful to him for doing so very speedily on a Sunday, but what he wrote in a comment to my post was a defence of the statistics used by the Sunday Times:
But he (ie Nigel Stanley) is simply wrong to argue that the Sunday Times has distorted and misused official statistics.
In his Bad Science column today Ben Goldacre also takes the Sunday Times to pieces.
He is unequivocal:
This was one of the most statistically misleading front page stories I have seen in a long time.
I’ve always been a big fan of Ben Goldacre, and emailed him a link to my post – though we have never met or communicated before. He has not responded to my email, and I’m not complaining as I’m sure he gets thousands.
But it is worth disclosing this as Ben is one of the directors of Straight Statistics. There is clearly some difference between this statement and that from Nigel Hawkes quoted above.
This sent me back to the Straight Statistics website to see if there had been any developments.
I find that on Friday – a full five days after the Sunday Times article originally appeared – Nigel Hawkes wrote:
The Sunday Times sought and was given permission to say that the figures had been validated by Straight Statistics. In the published version this was changed without permission to read that the analysis had been validated, a different matter.
This was what I thought might have been said last Sunday when I invited comment. After all it was precisely the claim that Straight Statistics had validated the analysis that was the major point of my post. The memo that Straight Statistics have now published is unobjectionable at least on pay. I could quibble with some points (eg pensions and the PWC paper), but it actually makes many of the same points that I did when I said that the Sunday Times distorted and misused ASHE statistics.
But even now they only say that the Sunday Times was wrong to say that they had validated the analysis. I still maintain – with Ben Goldacre – that the piece was an example of exactly what I thought Straight Statistics was set up to counter. Was I “wrong to argue that the Sunday Times has distorted and misused official statistics” or not?
I welcome the decision that Straight Statistics will publish their memos to journalists in future – though I think they are unwise ever to validate raw figures as it is always the analysis that makes the story.
If you use annual wage figures that mix up full and part time workers to compare pay, validating the figure is a meaningless thing to do unless you know the use to which it will be put.
In any case figures from ASHE do not need anyone to validate them. This is a well respected government survey – and while it is not perfect, it is as good as it gets. I certainly do not quarrel with the hourly median pay figures it contains.
Straight Statistics is a great idea, but the organisation has not had a good week. Perhaps it even had an internally argumentative one – I wonder why they have not opened up their post for comments.
Let us hope that it has also been an educational one.