The Sun Doesn't Always Shine on Sunday Morning TV

The Sun

The BBC Press Office has caught the Sun out over a 'stealth edit'. 

(The UK Press Gazette has the background to the story here).

The Sun piece originally claimed that Ofcom found that programmes like The Andrew Marr Show and The Sunday Politics have problems with balance when it comes to guest selection. The BBC says that Ofcom never said any such thing. And now the Sun seems to have acknowledged that (on the sly, so to speak):

Here's the 'Before'...

....and here's the 'After':

Ouch!

(Warning: This next sentence may contain sarcasm.)

The BBC, of course, never carries out 'stealth edits' to any of its online articles to cover up mistakes or embarrassments. 

*******

Rob Burley, AM editor. Not impressed.

But what of the Sun's original claims of anti-Brexit guest selection? 

The Sun's analysis found that The Andrew Marr Show has had 129 Remainers since last year’s referendum, compared with just 33 Leavers (84.6% compared to 15.4%) while The Sunday Politics had 78 Remainers compared to 37 Leavers (73.3% compared to 26.7%). 

Those are striking disparities. 

The first thing that puzzled me about those figures and percentages, however, is that they don't match. (Has anyone else spotted that? Or am I missing something?). Surely the AM percentages are actually 79.6% compared to 20.4%? And surely the SP percentages are actually 67.8% compare to 32.2%?

Still, the differences are clearly huge. Are they significant though? 

The BBC's defence is that the figures are spurious. (Andrew Marr called them 'fake news'). They issued a statement saying, "Not considering a Cabinet Minister or even the Prime Minister to be accountable for delivering the Government’s stated policy of Brexit is bizarre, especially when the Sun is willing to count Benedict Cumberbatch who talked about his acting career" and that “Nor was every politician interviewed asked about Brexit.”

Of course, any robust set of figures would have to only include interviews/press reviews where Brexit was raised. Did the Sun confine its analysis to just those interviews, or did they include every interview regardless of what was discussed? If they didn't narrow it down then they are guilty of giving us merely impressionistic findings rather than concrete ones, and the BBC has a point. I can't find the Sun's lists anywhere to check. (If you can, please let me know). 

Did the Sun include all those Remain-voting ministers who are now pushing for Brexit in their Remain figures? Where, for example, did they put Mrs May? Or Liz Truss? That was the BBC's first point and it is where things get tricky for such statistical studies post-referendum. Again, it would be interesting to see their lists. 

On the  Benedict Cumberbatch question, his inclusion would surely be relevant if and only if he was asked about/talked about Brexit as a known Remain supporter. The BBC says he only talked about his acting career during the Marr interview. Did he? Re-watching the interview today shows that the BBC is correct. Brexit never came up, so Benedict Cumberbatch should never have been included in the Sun's study. 

So yes, however much of a truth the Sun's analysis may point to its flaws appear fatal enough to me to render it pretty much dead in the water. You have to be fair-minded, ultra-precise and systematically show your workings for such a survey to hold water. (Not that the BBC's furious reaction suggests that they didn't take it seriously - or at least as a serious threat to their reputation!)

*******

P.S. The more I think about this and discuss it with others, and the more I see on Twitter about it, the more this half-baked Sun survey maddens me (in my usual mild way. Think of me as being like Droopy in that respect).

The BBC are correct about this one. The Sun's methodology was so sloppy and lacking in transparency that they've gifted the BBC an easy goal.

And, boy, from what I've seen on Twitter the BBC are certainly milking it!

Blogs like this one and sites like News-watch have made much of 'numbers' to demonstrate BBC bias - 'numbers' offered to their readers with total transparency, clear and (hopefully) watertight methodologies, a spirit of fairness and precision. When high-profile bunglers come along and botch their 'studies' then, alas, it risks giving the BBC all the justification in the world to slag off 'number-crunching' and 'clock-watching' - and I strongly suspect the BBC will be citing this rubbishy Sun 'study' for years to come.