Inside the Westminster Accounts - how the database was created

If you’re an MP, filling in The Register is probably one of those tasks that sits alongside doing your tax returns or filing your expenses. A necessary evil.

It’s the system that ensures we all know how much money is being donated to members of parliament and who’s paying them for their second jobs. In other words, it’s important.

But that system is chaotic: a pile of information spread across far-flung corners of the internet.

When a scandal breaks, journalists and researchers scurry to look up the politician or company at the centre of the media storm. What they often find is a snapshot of politicians’ earnings and donations – MPs only declare their financial interests from the past year at any one time.

It means there’s no way to look at the system as a whole.

If you wanted to know how much your MP earned in the last few years or how many MPs have received funding from a particular organisation, there was little hope in finding this out easily. The Westminster Accounts, a collaboration between Sky News and Tortoise Media, changes that.

Scraping 650,000 entries – and hand-checking each line of data

The project involved bringing together the Register of Members’ Financial Interests, party political donation data from the Electoral Commission, and the Register of All-Party Parliamentary Groups (APPGS).

This might seem straightforward – it’s only three different things after all – but there’s a reason why it hadn’t been done before. It required both technical expertise and patience.

The first stage was building a computer programme to do the hard work, “scraping” more than 650,000 entries from official websites and bringing them into a single database.

This means the programme crawled the various parts of the internet where political earnings and donations had been reported, picking up things like who the MP involved was, which organisation was paying them, and for how long.

But computer programmes aren’t perfect. In fact, imagine a computer programme that gets tripped up every now and then when data isn’t formatted neatly, but which tends to pick things up correctly about 98% of the time. On the surface, that’s good and it might seem like most of the work is done.

But now imagine there are 650-odd MPs, each putting in a new item into their Register every six weeks or so. Do that over five years, and you’ve got almost 30,000 entries to analyse.

Now consider each entry can be broken into around five or six pieces of data – a company name, a date, an address, an amount paid, and so on. Putting that all together, you’re left with more than 175,000 data points. 98% of them might be correct, but that other 2% adds up to 3,500 errors. Not so good after all.

It meant hand-checking each line of data in spreadsheets, several times, by several people. And once that was done, doing it all again for good measure.

A day could be spent checking the records of one MP – or in the case of former Attorney General Sir Geoffrey Cox and his numerous engagements as a high-flying barrister – several days.

A season of spreadsheets, if you will – sometimes journalism isn’t quite like it is in the movies.

To give a sense of some of the flaws we were trying to spot, consider The Ministry of Foreign Affairs of Japan. You won’t see the ministry listed as a donor in our interactive tool as the donations were made before the last election, but it is a good example of the complexity involved.

The seven word name – ‘The Ministry of Foreign Affairs of Japan’ – appears 11 times in earlier versions of the Register, donating more than £110,000 to fund MPs’ travel to the country.

But there was another organisation listed in the initial data the computer spat out, also funding an MP for one of these trips. This time it had six words – The Ministry of Foreign Affairs, Japan. If you’re not careful, that one extra comma creates chaos.

One of the clever data scientists at Tortoise Media wrote some code to catch these instances and others like it, grouping all the payments together. But even when these inaccuracies get spotted, some tricky decisions have to be made.

How should organisations be grouped together? Should a local branch of a trade union donating to a political party be lumped in with donations made by the union’s central office? What about different newspapers paying politicians to write articles, but which are all part of the same media conglomerate? You can read the full methodology and decisions we took here.

What parliamentarians get up to with their outside earnings

Swimming through all the data, checking and double checking, does mean you get a feel for what parliamentarians are up to with their outside earnings. And there’s certainly a full range of roles. From carers and doctors in the NHS, to those political consultancy jobs which have come under so much scrutiny over the years.

But there are also the left-field entries. Kevin Brennan (guitar), Sir Greg Knight (drums), and Pete Wishart (keyboard) have reported payments for performing in their parliamentary rock band MP4. Scottish Conservative leader Douglas Ross moonlights as an assistant referee, which has included officiating Champions League matches for £1,400 a pop.

And while many MPs typically take on outside work in the legal profession or finance, there’s also a breadth of industry knowledge filtering into parliament.

Alister Jack, the Scotland Secretary, had a job with a fly fishing manufacturer before he entered government, which no doubt dovetailed nicely with his position as Chairman of the River Annan and District Salmon Fishery Board. And from a traditional pastime to a more modern one, Lord Vaizey declared a role as Chairman of the International e-games Committee.

The Westminster Accounts also allows us to calculate how much MPs are earning per hour and who the biggest earners are. Again, the breadth in this regard is striking.

Rehman Chishti, the MP for Gillingham and Rainham, scrupulously declared a £16 payment for two and a half hours work as a British Army Reservist. On the other end of the spectrum, Dr Liam Fox, the former Trade Secretary, has declared £30,000 of payments for no work at all.

He’s had his services retained by a PR firm since 2021 in exchange for his business advice, but it says in the Register that he’s worked no hours to date.

Sky News checked with Dr Fox, who confirmed the accuracy of his entry, but it does raise an additional issue – what if the Register is wrong? Wherever possible, we’ve decided to stick with whatever is in the public domain, so changes to the official reporting need to be updated before the Westminster Accounts will be.

And it’s a project that will carry on.

Every two weeks a new update to the Register of Members’ Financial Interests is released. And every quarter new Electoral Commission data gets spooled out. We know it’s unlikely to be perfect – even with a 99.99% accuracy rate in our checking, a mistake or two could slip through the net, which we’ll update when we catch it, so the spreadsheets aren’t finished yet. Long live the spreadsheet.

Read Entire Article