Lies, damned lies and statistics: Why we can’t trust numbers anymore
If we are serious about reform and achieving economic growth, we must first ensure that our understanding of our country is underpinned by accurate statistics, says Ben Sweetman
There are lies, damned lies and statistics, as the saying goes. But robust and reliable data is the foundation of good governance. If we are to have informed debates, if our policy interventions are to have the desired impact, and if we are to understand properly the developments taking place in our society, we must surely first ensure that the data we use to describe our country is accurate.
Having previously worked at the Office for National Statistics, and working now as a data analyst at Policy Exchange, I am more exposed to these challenges than most. Trusting that the data we use is accurate is absolutely critical for us to be able to confidently diagnose the public policy issues that we investigate on a daily basis.
Across a range of areas today, however, there is good reason to be concerned about our ability to produce and maintain high quality data. Several recent cases have lent weight to such concerns.
The Labour Force Survey (LFS), conducted by the Office for National Statistics, provides us with our official measures of employment and unemployment. These figures are integral for the Bank of England’s Monetary Policy Committee, who use them as an indicator of the labour market’s tightness in order to inform changes to the bank rate.
Inaccuracies
However, recently there have been major doubts over the accuracy of LFS findings, with the ONS having seemingly understated the number of people in employment by a substantial margin. Bank of England governor Andrew Bailey and Treasury Committee Chair Meg Hillier have also raised concerns about our country’s ability to deploy appropriate monetary and fiscal policy as a result of these accuracy issues.
There are also widely reported issues surrounding migration data. Recent ONS data revisions revealed that net migration reached a record high of 906,000 last year, 166,000 higher than first thought. And others have raised concerns about the granularity of the data released. Information on the amount of tax paid by nationality appears to have been deleted last year, some data on the immigration status of foreign nationals held in prisons has been withheld and there has even been a discontinuation of published information breaking down the population by nationality. Who took the decisions to stop collecting this data? Civil servants – or ministers? And why?
These data quality issues extend further across the work of the UK government.
A recent government consultation found statistical releases related to health and social care to be severely wanting: it found that data presented in “inconsistent formats”, was frequently duplicated, was difficult to access and navigate and was often simply not available.
Or take the way we assess the economic impact of new regulation. In many cases, Impact Assessments are poorly produced, often by junior members of a departmental team, and with little accountability when they prove inaccurate. Many regulations, including those retained by EU law are exempt from requiring an impact assessment. Many that ought to be accompanied by one are not. And so getting a grip of the effects of regulation on the economy is mighty hard.
So, what might explain these aggregate trends in the apparent reduction in data quality?
In the case of the Labour Force Survey, accuracy concerns stem from falling response rates and smaller sample sizes, which have led to increased volatility in the data. Fewer respondents mean that the LFS is less likely to be representative of the actual population.
There are potentially wider issues around the competency of some of those working in government data collection services. The ONS struggled to retain staff after moving to Newport, losing 90 per cent of its London-based employees, and wages are not as enticing as equivalent private sector roles.
And there is inevitably a political dimension. What incentives are there for the government to release data it holds on immigration flows that would probably cause it another headache? Likewise, with the scrutiny process for new regulations, if the government has already determined that it is going to introduce a rule, what incentives does it have to assess its economic impact properly on the economy?
Not only is unreliable data making it harder to govern effectively; it is also undermining trust in government itself. Improving statistical services is therefore not merely important for improving policy outcomes in the UK, but improving faith in democratic politics.
Not only is unreliable data making it harder to govern effectively; it is also undermining trust in government itself
There is much that the government can do proactively to improve the situation. Shortening the length of surveys, such that they can achieve their primary use-case effectively, should be a priority. The transition to an online survey for the LFS will also almost certainly improve response rates and is a positive development, though perhaps this change should have been adopted sooner. There should also be a concerted effort to improve the recruitment and retention of data analysts.
And finally, the government itself needs to take action to restore credibility and trust. The first place to start would be to publish data that they already collect openly and transparently. But also, they need to think about the incentives that exist within the system. Strengthening our existing challenge functions – for example, for when assessments are clearly erroneous – would incentivise people to produce data accurately the first time around.
The UK Government is notoriously inefficient. Public sector productivity remains eight per cent lower than in 2019, and five per cent lower than 1997, according to the ONS. The reality of our stagnant economy is no secret either. If we are serious about reform and achieving economic growth, we must first ensure that our understanding of our country is underpinned by accurate statistics.
Ben Sweetman is a Research Fellow at Policy Exchange