The issues there have been in recent years at the ONS sit at the intersection of three of my interest areas. There were technical problems concerning the data and statistics; change-management & morale issues within the staff team; and governance problems at the highest level. Despite that I haven’t followed the case closely.
There’s always something to be learned from failure, so I was looking around for a handy summary of this one and I couldn’t really find it. This is an attempt to draw together what I could find and bring out lessons for my benefit, and maybe for the benefit of connections on LinkedIn or wherever who have similar interests. This is too long for a LinkedIn post, so I'm reviving my blog for it.
Let’s start with some sources.
- There’s a recent report from the relevant Parliamentary Committee
- There’s a review conducted by a very senior retired civil servant
- Here’s a report on the ONS that the Office for Statistics Regulation prepared last year. The OSR is effectively the in-house regulator for the ONS as they are both parts of the UK Statistics Authority, and therefore answer to the same board.
- And here is an independent review of the UK Statistics Authority undertaken by an eminent professor, although it’s a couple of years old now and I accordingly found it less useful.
There’s also any number of articles in Civil Service World. I won’t link to these sources unless someone asks for some in comments. They’re easy enough to find with google if you want to look further.
So what happened?
The ONS was perceived to be one of the arms of government that worked well during the COVID pandemic. Commissioned by DHSC in April 2020, they had a programme running literally within days which gave generally high-quality and timely data on the prevalence of COVID-19 infection. At the same time, the organisation was struggling with the challenges that affect all data-collecting organisations, most acutely the continuous decline in people’s willingness to answer surveys which undermined key national statistics based on the Labour Force Survey in particular.
In March 2023 the Health Security Agency withdrew funding for the Coronavirus Infection Survey, which accordingly stopped (there was a partial replacement a little later). In October 2023 publication of statistics based on the Labour Force Survey was halted as the Survey no longer provided a robust basis for estimating unemployment statistics. This was again partially for financial reasons as additional budget for the LFS had been introduced during COVID and was then withdrawn. The non-financial reason was that the new methodology for collecting these data wasn’t sufficiently embedded yet.
Meanwhile in 2024 the ONS sharply scaled back the ambition of its Integrated Data Service project, and then in 2025 cancelled it altogether. The IDS was a project to bring together linked, anonymised datasets from across government for the use of all kinds of researchers. ONS failed to secure the support of the government departments holding those data, in part because they seemed to be cavalier about the privacy issues, and in part because they seemed to be vague about the benefits case. The attempt to refocus the programme in 2024 was a response to Professor Lievesley’s report (mentioned above) but it wasn’t enough. In the end, little or nothing was achieved at the cost of £228.7m.
Simultaneously with this, morale at the ONS went into severe decline. Obviously people had worked very hard to achieve those COVID outcomes, but they also benefitted from the COVID-era working from home. When the senior management tried to bring back on-site working the response was even more negative than you may have seen elsewhere, leading to a formal dispute which ran for two years from Spring 2024 to April 2026.
The National Statistician resigned in May 2025, and the chair of the board in July 2025.
Overall, you get the impression of an organisation which was good in a crisis, but not able to maintain focus on the planning and execution of projects either to collect data in established ways, or make successful change in ways of working. ‘Good in a crisis’ is ideal branding for a fire department, but not really what you look for in a statistical agency.
What were the reasons for this? Let’s start at the top.
The Chief Executive of the ONS is both responsible for running a multi-million pound agency, and also responsible for leading the whole of government’s professional statisticians. Likewise the board is responsible both for oversight of the ONS as an organisation, and for oversight of the effectiveness of the UK governments’s statistical enterprise. I’ve seen comment that this dual role makes these hard jobs to fill, but I’m not sure I agree that was really relevant here. The previous National Statistician was Professor Sir Ian Diamond, and the previous board chair was Sir Robert Chote. A glance at Wikipedia will tell you that these were exceptionally distinguished individuals. Sir Ian had run a university and a research council before taking on this role. It would be difficult to identify anyone with a stronger CV.
There’s much more scope, in my view, to challenge what these people were doing. There’s a really telling anecdote at paragraph 70 of the Parliamentary report. The ONS commissioned an external consultancy which was sharply critical of its governance, decision-making and senior management behaviours but the Board was never informed of the report, still less any follow-up action. The Chair found out about the existence of the report in a conversation with a staff member a year later - in about May 2024 - but took no action at all. The report was never seen or considered by the board. By this time, of course, the morale issues at ONS had gotten to the stage of formal industrial action. It is bad enough that a chief exec should treat his board with apparent contempt by reviewing governance without their involvement. It’s a good deal worse that the board then thought he was right to do so.
As well as overseeing the management of the ONS itself, the board also has a remit to safeguard the quality of official statistics. Nevertheless the Parliamentary report found that the board was not well-informed about the key issues with the new Labour Force Survey. So this wasn’t a case of a board that had become too focussed on one part of its remit at the expense of the other. Neither part was being done well.
So despite being impressive individuals of suitable experience, it seems that the senior management and board had not developed a culture of sharing issues openly, challenging meaningfully, or setting priorities effectively. This resulted in the crazy position of an organisation getting an average 6.5% real rate of budget growth every year since 2015/16 that still regarded itself as underfunded, and which failed to find the money needed for key projects like the Labour Force Survey.
It isn’t actually true that a fish rots from the head, but the metaphor is too useful to ever give up.
Looking in a bit more detail at the operational level, the most prominent failures relate to the Labour Force Survey and the Integrated Data Service.
The basic issue with the LFS was the falling response rate that affects almost all surveys almost everywhere. In 2017 a project was developed to replace the LFS, using administrative data where that’s possible and a new, shorter survey for those elements where a survey couldn’t be replaced. Responsibility for the new survey was split, and changed frequently. It proved difficult to secure all the administrative data hoped for, and difficult to resist new data requirements, so the transformed survey wasn’t shorter than the old one and responders giving up in disgust continued to be an issue. Old and new surveys were run in parallel for a while because they gave some different data and the ONS didn’t fully understand why. For a time, additional resources were put into LFS to try to collect more surveys, but that money was withdrawn in July 2023 leading to an immediate fall in response rates below a useable level. While the resources went back in fairly quickly, the LFS is a multi-wave survey using data from five quarters, so it takes a long time for even a single quarter’s worth of poor quality data to wash out.
The IDS is another tale of long-running muddle and indecision. When set up in 2020, the original purpose was to join up data from across government and make them available to researchers (both inside and outside government) in a simple, secure, anonymised way. This would replace the previous patchwork of different arrangements for getting access to data and enable a step-change in analysis. Sharing data creates risks, both to the privacy of data subjects and to the reputation of data owners, so it isn’t surprising that government departments were reluctant to share the data that they held.The IDS programme could not articulate clear benefits to those stakeholders who therefore remained unwilling to share data. These were fundamental issues that needed to be resolved before the project could be launched, yet somehow it lasted five years and spent £228.7 million. In mid-2024 the aspiration to provide data to non-government users was cut in an attempt to clarify the business case, but then in 2025 the whole project had to be abandoned. Obviously if stakeholders couldn’t see the benefits of the project, removing some of the benefits wasn’t the right way forward.
I think all of us in a university setting have worked on projects that made tons of sense at a very high, hand-wavy level, but were never pinned down enough for actual implementation, and where the requirements constantly changed underneath the project team. Most of those projects probably fell apart in less than five years, but still it is striking how these two case studies just repeat our own experience on a grander scale.
It is also striking that these project failures are also fundamentally failures of governance, planning, and prioritisation. This particular fish really did rot from the head.
Again, I doubt there’s anyone in my network who can’t already imagine what it must have been like for the staff working in an organisation like this, and who therefore struggles to see how morale got to be so poor. The formal basis of the long-running industrial action at ONS was in-office working, but there’s plenty of evidence from these reports about the frustration of working for an organisation which didn’t listen, didn’t set priorities, and didn’t always meet the standards of professional conduct you’d have a right to expect.
What can we learn? With modern technology, data start to seem weightless. You can download a huge wedge of NSS data in a few moments, and spin up any amount of analysis almost effortlessly. It’s easy to forget the amount of human labour that was needed to collect those data in the first place. People often tell me it should be easy for a university to know how many students it has. Everyone in my network probably shudders at that one because of the definitional issues around ‘student’, but how many even of us remember that alongside the definitional issues there are hundreds of staff, even at a smallish university, who need to be following processes and implementing policies every day if we want to get a meaningful number out of our database? If those policies and processes don’t work then, just like the LFS, we only have junk data.
The fundamental importance of governance is another learning point that won’t startle anybody. In this case we had all the form of effective governance, conducted by people who were clearly and obviously qualified for the job, yet it all turned out to be useless because the culture was wrong, and the governance was not focussed on the right issues. That feels like it hits close to home too.
