Feathers flying and ad homs

News-watch has two new reports out today, and the Sunday Telegraph has reported their findings...

...and, my goodness, the BBC is not happy (to put it mildly).

News-watch's main findings are:

• that the BBC invited a third more pro-EU than Eurosceptic speakers to appear during the election campaign.
• that the BBC, during this period, has placed a heavy one-sided emphasis on the difficulties of withdrawing from the EU.
• that left-wing voices in favour of withdrawal from the EU were ignored for years by the BBC (of the 5,037 guests speakers on EU matters on Today between 2002 and 2015, only five were left-wing advocates of Brexit - 0.1% of the total speakers on the subject).

The BBC's response has been aggressive (to put it just as mildly): 

We do not recognise the allegations made by News-watch and to describe this as a 'report' would be a gross overstatement for what is a defective and loaded piece of work which wouldn't pass basic academic scrutiny.   

Across the election campaign we heard from a range of voices, provided our audiences with clear and balanced analysis and rigorously scrutinised the issues and this is quite simply as an obvious attempt by a lobby group to discredit the BBC when all we are doing is holding all politicians, no matter their view, to account.

I see that response as a ratcheting-up of the BBC's usual stonewalling whenever reports of this (unwelcome) kind come out - a ratcheting-up made all the easier by that absolutely dreadful Sun 'analysis' of 'BBC bias' on the Marr show and The Sunday Politics. As I wrote at the time:

The 'tally-ho' that went up from the BBC after that Sun survey also went up today. BBC types have been piling in on Twitter to rubbish the findings, despite not having read the reports in full - reports which News-watch will doubtless post in the coming days (as they always do). 

If you want to get the full flavour of the BBC's response today please explore the following Twitter thread, launched by blog favourite Rob Burley:

The "evidence" Rob has for asserting that News-watch's research is "unreliable" comes from one chap on Twitter who attempts to give it an impromptu fisking.

Peter the Would-Be Fisker initially goes for the ad hom approach (as did the BBC spokesman quoted above), checking out News-watch's website, putting two and two together and claiming it's partisan. 

Peter then picks one report and claims it lacks a methodology - even though the said report did have a clear methodology (on page 5) which Peter clearly failed to spot, probably in his headlong eagerness to quickly debunk News-watch.

Peter then says, "As far as I can see, none of these reports have methods section" - even though they actually all have clearly outlined methodologies. (Please check them for yourselves and you'll doubtless  all manage to "see" a good deal further than Peter here!).

Others, like the BBC's business reporter Joe Lynam, took a different ad hom route in trying to debunk News-watch today:

The snag with this one is that it's a very clever ad hom approach. It 'works'. 

As many BBC defenders on Twitter have said today, people who commission polls tend to get the results they want, or why commission the poll in the first place? The mere fact that pro-Leave, BBC-bashing MPs have commissioned this and got a result which proves them to be completely right about the BBC must mean that there's no smoke without fire. Monitoring groups can't be unbiased if they are commissioned. It stands to reason, surely?

Well, the obvious answer to that is that, no, it doesn't stand to reason at all. Not at all. 

The 'ad hominem' fallacy is, well, precisely that: a fallacy. The fact that a monitoring organisation (a pollster, an academic group, an independent monitoring unit) carries out a study on someone's behalf doesn't in itself invalidate anything. All that matters is the the quality and robustness of the evidence. If the evidence is offered transparently and holds up to scrutiny then all the 'ad homs' in the world will fail to refute it because the evidence is (or ought to be) independently verifiable (or refutable). 

Moreover, Joe Lynam & Co. invite another obvious riposte: The BBC's recent landmark impartiality studies, which largely 'proved' the BBC to be impartial, were commissioned by....guess who?...yes, by the BBC.

Aha! Does that invalidate the findings of all of those major BBC impartiality studies which found the BBC to be broadly impartial? By Joe's logic, yes, yes it does. 

And when the BBC commissions a bunch of ex-BBC types and pro-EU leftists and far-left activists at Cardiff University to review its entire output for impartiality and when they 'find' that the BBC has a bit of a right-wing, anti-EU bias but is basically sound and impartial, what's to stop people from 'the other side' playing exactly the same 'ad hom' card and rubbishing their findings on the 'no smoke without fire' principle that no-one-but-no-one puts aside their biases when publishing studies like this - least of all pro-BBC, pro-EU leftists commissioned by the BBC?

What's good for the pro-BBC goose is good for the anti-BBC gander, isn't it?

Now at this point a bit of necessary self-reflection is urgently needed because people on 'this side' are as guilty as anyone else of playing precisely the same game that Joe Lynam and the BBC spokesman quoted by the Telegraph have been playing here. We do it again and again.

And by that I also mean me. I partly rubbished that Cardiff University report (and others from Cardiff Uni) on exactly the same lines, so I'm no better than anyone else in that respect - except for the (blessed!) caveat mentioned below. (But, worse, that's far from the only time I've done it too).

So we all do it, and I don't blame Joe Lynam for trying it on here. We don't like or trust some statistic findings so we busily dig around and find 'Gotcha!' evidence that those responsible for those figures strongly hold a particular point of view and claim, abracadabra!, that their statistics must therefore be unreliable. 

It's so much easier to do that than to drill into the statistics themselves and, with an open mind, try to verify or refute them. 

Now in fairness to myself, I did also criticise the Cardiff survey on its merits too, condemning the extreme narrowness and randomness of its focus on just one week's output from two different years,  and comparing it to much better kinds of analysis, so my doubts about that Cardiff report remain. 

And that's what the BBC defenders ought to do here too, if they can. Do News-watch's findings stand up to scrutiny?

The fact that it's so much easier to take the 'ad hom' approach and try and find out who the researchers are, what their politics are, who backs them, who funds them, etc, and thus 'discredit' them, means, alas, that the 'ad hom' approach will always remain the first port of call for most people.

It's just the way it is (as Bruce Hornsby might put it).