Interruptions, interru...

The Unique Selling Point of my first BBC bias-related blog, Beeb Bias Craig, was its use of interruption coefficients (not my own invention, I hasten to add!). They provided a suggestive - if rather rough-and-ready - measure of how a politician fares at an interviewer's hands during any particular interview. 

It was an attempt to bring a degree of objectivity to blogging about BBC bias, which is usually highly subjective in nature.

Ignoring context and ignoring the content of the interview, it involved counting interruptions and timing the lengths of interviews (or the relevant parts of joint interviews), whilst using set criteria as to what a significant interruption is - so as to keep subjectivity at bay as far as possible, regardless of whichever party was being interviewed and my own feelings towards them.  

It was very simple to calculate. You simply divide the number of interruptions (made by the interviewer) by the length of the interview (or the time the interviewee gets to be questioned during a joint interview). The higher the interruption coefficient (I.C.) the tougher the interview. 

Much of the rough-and-ready element came from ignoring the context - such things as the fact that there are different kinds of interruption - supportive ones and challenging ones, for example - and that politicians sometimes gets cut off other politicians rather than the interviewer, thus cutting their time shorter than it might have been had the interviewer had his or her own way. 

Still, the idea was that if conducted systemically - i.e. EVERY interview with a politician on all the main BBC current affairs programmes over a long period - then such wrinkles would be ironed out and trends might (or might not) emerge. Not a single interview could be missed. tracked it for some nine months or so before deciding enough was enough! (It was enjoyable, as I like hearing political interviews, but very time-consuming.)

Using averages, it could then be seen whether particular parties were been treated differently. If the overall average showed that politicians from one party were (on average!) being interrupted far more often than those from another party, then that (to me) would strongly suggest bias. (Institutional  bias, you might say).

As a snapshot from late on, here's a post from the beginning of March 2010 (when Labour were still the government):

I think it's time to work out another super(duper)-average - i.e. the average I.C. for each political party since I began recording interruption coefficients last June.

So far 2194 interviews have been monitored.

The bias against right-of-centre parties (and nationalists) remains clear and unequivocal:

UKIP (30) - 1.01

Conservatives (619) - 0.85

English Democrats (1) - 0.80

SNP (70) - 0.76

Sinn Fein (9) - 0.71

BNP (4) - 0.65

Plaid Cymru (11) - 0.65

DUP (10) - 0.62

Labour (1054) - 0.59

Liberal Democrats (333) - 0.44

Greens (16) - 0.26

TUV (2) 0.25

SDLP (3) - 0.20

UUP (2) - 0.15

Alliance (6) - 0.03
Respect (1) - 0 

So, a UKIP spokesman is four times more likely to be interrupted by a BBC interviewer than a Green. A Conservative is twice as likely to be interrupted by a BBC interviewer than a Lib Dem. Labour - the government of the country! - is questioned less toughly than Plaid Cymru.

A number of UKIP and Conservative politicians were intrigued by these figures, which I used to e-mail to them. UKIP were especially enthusiastic.

You may not be surprised to hear that the BBC didn't share their enthusiasm. Nor did they accept the "damning" nature of these figures. Their Complaints Unit dismissed me crisply in this way:

In relation to your further concerns, ultimately our position is that no two interviews are the same. Each stands on its own merit and has a specific purpose. We don’t judge their success or failure by the frequency of interruptions nor by the tone of the questions or replies. What matters is that each interview is rigorous and fair and so cannot accept the premise that comparing the number of interruptions is any kind of indicator of the quality of the interview.

I'll let you make up your own minds about the merits of the BBC's response and, for that matter, of the whole enterprise.

Here's a comment from the latest Open Thread at Biased BBC, however, which strongly reminded me of the impulse that first prompted me to begin by epic trawl through the BBC's political programmes:

Newsnight Friday 3rd May. Stephanie ‘Two Eds’ Flanders discusses the council election results with UKIP’s Diane James and three LibLabCon stuffed shirts – including Chucka ‘two face’ Umnunna. Flanders starts by grilling Diane James for about 3 minutes, disparaging UKIP policies from their 2010 manifesto:  Flanders continually interrupts Diane, barely giving her a chance to respond to each attack before cutting her off and moving onto another attack.  Then Flanders turns to the LibLabCon stuffed shirts and says something like “when you listen to this lot (rudely gesturing at Diane) you must think what a load of rubbish right?”  She then provides Chucka with the best part of 10 minutes to waffle on without any interruptions atall. She never returns to Diane James for a response.Once the episode becomes available on iPlayer I will reply here with the stats on talking time and number of interruptions – but the segment just reeked of bias. This is how the BBC intend to play it:  Always ensure the UKIP speaker is outnumbered 4:1 by LibLabCon panellists (and presenter). Continually interrupt the Ukipper while allowing the LibLabCon speakers uninterrupted airtime to attack UKIP.

You can watch this for yourselves here (for the next seven days).

My figures for this would be:

Tom Brake, Liberal Democrat - I.C. of 2.8

Diane James, UKIP - I.C. of 2.0

Chuka Umanna, Labour - I.C. of 1.2

Bob Neill, Conservative - I.C. of 1.0

Diane James was interviewed for 3 minutes and interrupted 6 times. Bob Neill got 2 minutes and 10 seconds but was only interrupted twice. Chuka Umanna got the longest (3 minutes 34 seconds) while Tom Brake got the shortest (just 1 minutes 41 seconds), but both were interrupted 4 times. 

Here's part of the rough-and-ready aspect here. Tom Brake's short speaking time partly came about because Chuka Umanna interrupted him and took over. That boosted Tom's interruption coefficient. It also resulted in the pattern of contributions to the programme looking like this (given that Stephanie rightly let Tom Brake back in after Chuka's intterruption):

Diane James  

Bob Neill 

Chuka Umunna 

Tom Brake 

Chuka Umunna 

Tom Brake 

Rough-and-ready as such figures may be, especially with regards to a single interview, they do at least allow us to make certain observations about this particular interview. 

The Biased BBC reader was correct about the amount of time Diane got to speak but wildly over-estimated how long he had to endure Labour's Chuka for! (Listening to someone we disagree with/dislike probably tends to lead to this sort of false impression quite often. Listening to someone we agree with/like seems to make time  fly). The Biased BBC reader was also correct that Diane was interrupted most (in terms of sheer number of interruptions), but he neglected to figure in the amount of time she got in comparison to, say, Tom Brake. It was Tom who fared the worst in this regard, not Diane. He was interrupted more frequently. The Biased BBC reader was mistaken in stating that Chuka went uninterrupted, however, as this was far from being the case; indeed, all of the LibLabCon stuffed shirts were persistently interrupted by Stephanie Flanders. And as for not returning to Diane James for a response, well, that's certainly true; however, Bob Neill was similarly never returned to for a response (given that Chuka has attacked his party too). Maybe, until Chuka re-inserted himself back into the discussion so soon after Tom Brake's contribution had begun, no one would have been invited to have a second go? And, finally, as for having the LibLabCon trio on with UKIP's Diane and thus making it 3:1 (or 4:1) against, well, I suppose BBC's guidelines on impartiality would demand the presence of all three of those parties on the night after a set of election results came though. Wouldn't it have been wrong for the BBC not to have invited representatives from all three of the parties onto Newsnight?

Say what you like about the futility of counting interruptions for nine months and cast as many doubts as you like on statistics and their limitations, I still believe that stats can and do cast a clarifying light on things, as they've done here - if used in as objective a spirit as possible. 

Our subjective impressions are certainly prone to lead us astray - such as the Biased BBC reader's impression that Stephanie was "rudely gesturing at Diane" when she was saying "something like “when you listen to this lot (rudely gesturing at Diane) you must think what a load of rubbish right?”" Watch the i-Player (14.00-14.10) and you will see that no such rude gesture was made. The half-recalled quote wasn't that accurate either, making the question sound rather worse than it actually was: 

"Bob Neill, when you listen to this, I mean, although your prime minister's being very polite today, in your heart of hearts you still think they're not serious, they're a flash in the pan, don't you?" 

The frequent unreliability of witnesses is a widely-discussed feature of psychology. Those (including me) who watch and listen to BBC interviews on a regular basis are far from immune from this sort of unreliability. I heard something on Today recently as I was driving to work that made me gasp at its sheer, naked bias. On getting home that evening and re-listening (via the i-Player) it turned out that I'd make the same sort of mistake. Mark Mardell had not said what I thought he'd said (and what I'd probably expected him to say).  A post at Is...? was scuppered! We're probably into the terrain of confirmation bias again here.

Statistics and bare facts can help clarify matters, can't they? I think so. We may not like what they tell us, but that's probably our look out!

UPDATE

Now, this is interesting. The same Biased BBC reader has added his own stats on this interview. Good man! I will present them to you straight.

What this says about my former methodology (or the Biased BBC reader's methodology?) I'll leave you to decide:

Those stats from iplayer:

Diane James (UKIP):
Number of questions: 6
Interruptions: 3  ”Flat tax!” ” Two Tier Flat Tax”  ”I read your manifesto just 2 hours ago”
Total time spent asking questions & interrupting:  80secs
Total time allowed to respond: 100 secs

Tory Bob Neal
Number of questions: 3*
Interruptions: 0
Questioning: 30 secs
Answering: 100 secs
* Including this really tough opening question: “Bob Neal, when you listen to this (gesturing at Diane)…. I mean… although your prime minister’s being very polite today, in your heart of hearts you still think they’re not serious, they’re a flash in the pan, don’t you?”

Chucka:
Number of questions: 4
Interruptions: 1
Questioning: 45 secs
Answering: 170 secs

LibDem
Questions: 2
Questioning: 10 secs
Answering 65 secs
Interruptions: 3

So in summary, Diane got a grilling while Chucka and the Tory were given an open mic. Turns out Chucka only droned on for 170 secs not 10 minutes as I originally thought ( although it seemed like 10 minutes ). 

As Doyle pointed out, Diane James still came out looking the winner – despite the unfair treatment by Flanders.

UPDATE TO AN UPDATE:  OK, I won't entirely let you decide for yourselves!

To say that Bob Neill wasn't interrupted by Stephanie is wrong. For starters.

I'm sticking with my figures.

Still, the Biased BBC reader got some of the essentials right, such as the Lib Dem's failure to get "an open mic" - though he didn't exactly spell that out. That was left to us to infer.

He was also very honest in admitting his overestimation of how long Chuka had talked.

Note, however, that he didn't quote any of the challenging questions from Steph to Chuka or Tom.

Nevertheless, it's a wake-up call to me that even the matter of counting interruptions can result in different stats. Subjectivity remains.

Even though I'm sticking with my figures (and my tried-and-trusted methods), others clearly can hear things differently.

So, is it any wonder that the BBC can dismiss such statistics with such ease? How objective are such stats? What is an interruption? And, more to the point....what's the point?

"What's the point?" Hmm...