Saturday 25 April 2015

Proving BBC bias - a case study (Paul Lewis on 'Money Box')



If there's one BBC presenter that everyone seems to like, it's Paul Lewis from Radio 4's Money Box.

Few people have a bad word to say about him, and he's someone who rarely, if ever, gets criticised for bias. He's a wonderful presenter, and a great credit to the BBC, and...

Evan Davis (interrupting): There's a 'but' coming, isn't there? What is it with you Paddington-hating bloggers and your hate-filled 'but's? Surely it's patriotic to be uncritical of the BBC? 

Shut it, Evan! 

But, yes, you're right for once. There is a 'but' coming, because Paul Lewis's interview with UKIP's economics spokesman Mark Hughes today provoked a certain amount of shock from UKIP supporters, both on Twitter (using the #bbcbias hashtag) and on BBC-bias-monitoring blogs. 

Here's a representative exchange on the subject from Biased-BBC:

MartinWW
Radio 4 – Moneybox. 25 April. Paul Lewis interviewed UKIP’s pensions spokesman at midday. One might have expected Lewis to have been his usual forensically-probing, but polite and even-voiced self. But, no. He was sharp-voiced, carping, hostile and argumentative and, furthermore, accused the spokesman of constructing policies to assist in his business. Lewis was more interested in finding fault than providing a fair platform for UKIP. Before this episode, I had considerable regard for Lewis, and it is sad that he reverted to the standard BBC stance towards UKIP.

  • Rob in Cheshire
    I must agree. Paul Lewis has always struck me as a decent and fair financial journalist, but his attitude towards the Ukip spokesman was terrible, the sort of constant interruption and clear bias one has grown to expect from Evan Davis and the like. A big fail for Money Box I’m afraid.

Is this just sour grapes from UKIP-supporting commenters, or is there something in their complaints about Paul Lewis's behaviour today? 

Well, it did seem to me to be an unusually sharp-toned interview from the Money Box presenter, though I have to say that I found it fascinating, and unusual - unusual in that it's extremely rare to hear a BBC interview that actually focuses on, and takes seriously, UKIP's policies, and is prepared to spend time digging into them.

********

All this is a preamble to a stocktaking piece about how we can actually prove BBC bias. 

A complaint based on the tone of a BBC interviewer tends to get nowhere, as the BBC will simply argue that judgements about tone are highly subjective and that the BBC interviewer in question's tone will have struck many others as just fine and dandy.

Plus they'd tell the complainant to check out that BBC interviewer's other (comparable) interviews, because (the BBC would say) they would show that the BBC interviewer in question either treats everyone like that or shifts his or her tone to match that of the interviewee (especially if the interviewee isn't answering the interviewer's questions).

The complainant could then trawl through all the recent episodes on the BBC i-Player and listen to them, noting the differences of tone. 

In the case of Paul Lewis's previous election interviews - with Danny Alexander of the Lib Dems, Stewart Hosie of the SNP and Labour's Ed Balls - that complainant might well find (as I did) that they were all splendid interviews but that none of them was as sharp and combative as the interview today with UKIP's Mark Hughes.

Would saying so in a complaint to the BBC get them to 'fess up to BBC bias? Not a chance of it. It's still all subjective, they'd reply.

Here then, perhaps, is where statistics might come in. 

A few years back (as old hands here will already know) I did a detailed study of over a thousand BBC political interviews, counting interruptions and then dividing the total by the length of the interview (as 11 interruptions in a 2-minute interview is, obviously, something very different to 11 interruptions in a 25-minute interview). The basis idea was: the higher the resultant figure (the 'interruption coefficient') the tougher, all things being equal, the interview. 

Putting that old idea back into practice on these Paul Lewis interviews produces the following results:

Mark Hughes, UKIP (25/4) - length: 9m 34s, 19 interruptions = I.C. of 2.0

Danny Alexander, Lib Dems (4/4)- length: [9m 59s, 14 interruptions = I.C. of 1.5

Ed Balls, Labour (18/4) - length: 12m 23s, 13 interruptions = I.C. of 1.1

Stewart Hosie, SNP (11/4) - length: 9m 44s, 6 interruptions = I.C. of 0.6 

That shows, if nothing else, that UKIP's Mark Hughes was interrupted almost twice as much, proportional to the length of the interview, as Labour's Ed Balls. And Stewart Hosie seemed to strike it lucky.

Now, the BBC would reply that all this proves is that the UKIP man was interrupted more than the Labour man. They would say (and I know this from personal experience) that this doesn't prove bias. 

They'd also advise you to take context into account: Was Mr Hughes being more evasive than Mr Balls? Was Mr Hughes trying to drag his answers out more than Mr Balls? 

You might, if you're fanatical enough, try to cancel out that 'context' cop-out by monitoring over a thousand interviews and deriving averages for each political party.

By so doing you could derive, as I did for some eight months from 2009-10, a list showing (a) party, (b) the number of interviews monitored and (c) the resultant average 'interruption coefficient'.

In descending order, from highest (toughest) to lowest (softest) the following list duly appeared, showing UKIP and the Conservatives to have been interrupted much more than Labour, Lib Dem or Green politicians. (UKIP, you will note, fared worst of all at the hands of BBC interviewers, interruption-wise):

UKIP (30) - 1.01
Conservatives (619) - 0.85
English Democrats (1) - 0.80
SNP (70) - 0.76
Sinn Fein (9) - 0.71
BNP (4) - 0.65
Plaid Cymru (11) - 0.65
DUP (10) - 0.62
Labour (1054) - 0.59
Liberal Democrats (333) - 0.44
Greens (16) - 0.26
TUV (2) 0.25
SDLP (3) - 0.20
UUP (2) - 0.15
Alliance (6) - 0.03
Respect (1) - 0 

Of course, it was entirely possible that BBC Complaints would still refuse to accept interruptions as a credible measure to monitor bias. They could do so - and did - by the simple act of denying they are a credible measure. 

I still think it is a highly suggestive and, if properly presented for checking, credible. (I posted details of every month's list on my old website). Maybe it should be brought back.

What counting interruptions lacks as a way of monitoring BBC bias, among other things (like a way of quantifying 'context'), is a set of complimentary measures. 

Inspired by David Keighley (of News-watch fame), what if a second measure might be found in working out how much of the interview the BBC interviewer talks for and comparing it with the amount of time the interviewed politician speaks for? 

Obviously, all things being equal, the more the BBC interview talks the less time his or her interviewee gets to talk and that, all things also being equal, would suggest that the BBC interviewer is seeking to dominate that interview - and, thus, that it's a tougher interview. 

That would be easy to do, readily re-checkable and hard to get wrong. And it would be very hard for the BBC to bring 'subjectivity' into it. 

(Plus, if you're a political animal who loves listening to interviews, it would be no hardship. And it would give those interviews an extra, personal edge).

How would that work with Paul Lewis's interviews so far then?

Well, here are the results of my analysis. The smaller the margin between the two scores for the interviewer and the interviewee (unless the former talks more than the interviewee!!), the tougher (all things being equal) the interview:
Mark Hughes, UKIP
PL= 203s (36.7%)
MH=350s (63.3%)
553s
Margin=26.6%
Ed Balls, Labour
PL=213s (28.7%)
EB=530s (71.3%)
743s
Margin=42.6%
Stewart Hosie, SNP
PL=185s (31.1%)
SH=409s (68.9%)
594s
Margin=37.8%
Danny Alexander, Lib Dem
PL=175s (29.1%)
DA=426s (70.1%)
601s
Margin=41%
That, as you'll doubtless have spotted, shows that Mark Hughes of UKIP got the 'toughest' interview and Ed Balls of Labour the 'softest' interview.

It also gives an extra edge to the relatively high 'interruption coefficient' for Danny Alexander, by showing that, in terms of the amount of time he got to talk, he didn't do too badly after all. 

By those two measures - counting interruptions (and dividing them by the length of the interview) and working out the balance of dominance in the interview by measuring the length of time the two protagonists spoke for - UKIP's Mr Hughes unquestionably came out worst on both counts. 

Add that to the 'subjective' sense of certain listeners that Paul Lewis's tone was unusually hostile to the UKIP spokesman, and you might - just might - have an argument the BBC will struggle to deal with.

Just one problem though: The BBC's online complaints form has a character limit. That makes explaining all of this and adding all the evidence frankly impossible. 

********

There's a fly in my ointment though - actually a whopping great bluebottle. 

Radio 4's excellent The Human Zoo, with Michael Blastland, did an election special a few weeks back. It examined how rational people really are, and how all manner of unconscious biases can make us impervious to reason yet prone to being influenced by all manner of non-logical things.

One of the programme's points was that people are staggeringly capable of ignoring or dismissing any stats they don't want to agree with - however watertight those stats may be.

And, worse, what might influence people where statistics fail are...anecdotes. People respond to anecdotes much more than they respond to statistics (if they don't like what those statistics are telling them).

So maybe it's better after all to stick with saying: "Paul Lewis is usually a credit to the BBC, but even he's not immune to BBC group-think, as shown by his much more hostile treatment of UKIP today. His interview with Mark Hughes proves the BBC are biased to the core against UKIP." 

Plus, that would have made for a much shorter post.

9 comments:

  1. Did you see Newswatch then? Election coverage, and how they try to achieve impartiality. It’s more or less a case of ‘we think we’re doing it about right’.
    http://www.bbc.co.uk/iplayer/episode/b05rhrkf/newswatch-24042015
    Samira:
    “perceptions obviously differ widely and there’s no doubt the BBC has a tricky balancing act to perform. It’s drawn up guidelines for this election.”

    Guidelines:
    “To achieve due impartiality, each bulletin, programme or programme strand...must ensure that the parties are covered proportionately over an appropriate period, normally across a week. Electoral support in the previous equivalent election is the starting point for making these judgments. However other factors should be taken into account when appropriate”

    “Well, the man in charge of overseeing whether there is balance and impartiality in the election coverage is the BBC’s chief political adviser Ric Bailey who’s with me now.”

    “Viewers might be surprised that there are guidelines for this election on proportion, because we’ve had a lot of letters from people who feel there’s no sense of any proportion worked out - for smaller parties and bigger parties.”


    (I can’t be bothered to transcribe the whole thing verbatim, but Ric Bailey is saying that they trust their editors to do what they do ‘in a fair way’.)“It’s about telling the story of the campaign, and using good editorial judgment, fairly.”

    “Are you counting ‘minutes’ for each party?”

    “I’m a really strong believer that you don’t achieve due impartiality by maths and by stopwatches - that’s what used to happen years ago, it’s no longer the case.[...]It’s not the whole picture[...]You’ve got to achieve a consistency of approach, a similar level of scrutiny across the different parties over time of which airtime is only one small part.”


    Samira:
    “Well people do feel airtime is important and if we look at the significant number of complaints we’ve had over the airtime given to Nicola Sturgeon - national coverage given that the SNP..........”

    “...........This is about telling the viewers the story of what’s happening in the election.”


    They did show those two clips - (Cameron on Marr and Farage with Davis) - but didn’t comment on them at all.

    It’s ‘tone’ rather than air-time, although I’d include interruptions under the heading on tone.
    But the BBC is obviously happy with their coverage.

    ReplyDelete
    Replies
    1. No, I meant to watch it but forgot.

      Well, if the BBC's main 'impartiality' man as regards this election says, “I’m a really strong believer that you don’t achieve due impartiality by maths and by stopwatches - that’s what used to happen years ago, it’s no longer the case", then I'm going to get my stopwatch and calculator right back out again.

      Delete
  2. Make that 'under the heading of tone'.

    ReplyDelete
    Replies
    1. I do!

      It’s almost as if Ric Bailey had seen David Keighley’s piece before speaking on Newswatch - but I don’t think he could have - unless it was published somewhere before April 25th Conservative Woman.

      Maybe they’re aware of your stopwatch / interruption coefficient stats - so they’ve had to come up with another line of defence? Or is that too conspiracy theory?

      I did wonder what he meant by “that’s what used to happen years ago, it’s no longer the case.”
      How and when did that “used to happen” I’d like to know, and why is it no longer the case?

      And, does he mean they used to time these things themselves, or does he mean his critics used to time them and base their criticism on those timings?? Either way, why is ”this no longer the case?”

      There’s no logic in it.

      Delete
  3. "That makes explaining all of this and adding all the evidence frankly impossible"

    Pretty much every aspect of the BBC complaints process is designed to make a complaint go away one way or other.

    And damning all ways is a specialty, always resulting in a case file closure.

    I've had them play dumb and require clarification, only to then claim that it's all expanded to a point where it exceeds their time commitment allotment.

    It must take a very special individual to turn up and clock in to that department each day, especially if they need to look into mirrors at any point.

    ReplyDelete
  4. I wonder whether there might be an objective measure of tone...pitch is often involved. A querulous tone tends to be high - with Evan Davis it's positively soprano when he doesn't like an interviewee.

    ReplyDelete
    Replies
    1. That would be useful. It might require the purchase of an electronic device though.

      Only dogs could hear some of his interruptions during that Nigel Farage interview.

      Delete
    2. There should also be some sort of variable for when some interruptions (or time the interviewer spends talking) to account for when it's necessary, like when the guest clearly dodges the answer.

      Delete

Note: only a member of this blog may post a comment.