About the Office for National Statistics

 

The issues there have been in recent years at the ONS sit at the intersection of three of my interest areas. There were technical problems concerning the data and statistics; change-management & morale issues within the staff team; and governance problems at the highest level. Despite that I haven’t followed the case closely.

There’s always something to be learned from failure, so I was looking around for a handy summary of this one and I couldn’t really find it. This is an attempt to draw together what I could find and bring out lessons for my benefit, and maybe for the benefit of connections on LinkedIn or wherever who have similar interests. This is too long for a LinkedIn post, so I'm reviving my blog for it.

Let’s start with some sources. 

• There’s a recent report from the relevant Parliamentary Committee
• There’s a review conducted by a very senior retired civil servant
• Here’s a report on the ONS that the Office for Statistics Regulation prepared last year. The OSR is effectively the in-house regulator for the ONS as they are both parts of the UK Statistics Authority, and therefore answer to the same board.
• And here is an independent review of the UK Statistics Authority undertaken by an eminent professor, although it’s a couple of years old now and I accordingly found it less useful.

There’s also any number of articles in Civil Service World. I won’t link to these sources unless someone asks for some in comments. They’re easy enough to find with google if you want to look further.

So what happened? 

The ONS was perceived to be one of the arms of government that worked well during the COVID pandemic. Commissioned by DHSC in April 2020, they had a programme running literally within days which gave generally high-quality and timely data on the prevalence of COVID-19 infection. At the same time, the organisation was struggling with the challenges that affect all data-collecting organisations, most acutely the continuous decline in people’s willingness to answer surveys which undermined key national statistics based on the Labour Force Survey in particular. 

In March 2023 the Health Security Agency withdrew funding for the Coronavirus Infection Survey, which accordingly stopped (there was a partial replacement a little later). In October 2023 publication of statistics based on the Labour Force Survey was halted as the Survey no longer provided a robust basis for estimating unemployment statistics. This was again partially for financial reasons as additional budget for the LFS had been introduced during COVID and was then withdrawn. The non-financial reason was that the new methodology for collecting these data wasn’t sufficiently embedded yet.

Meanwhile in 2024 the ONS sharply scaled back the ambition of its Integrated Data Service project, and then in 2025 cancelled it altogether. The IDS was a project to bring together linked, anonymised datasets from across government for the use of all kinds of researchers. ONS failed to secure the support of the government departments holding those data, in part because they seemed to be cavalier about the privacy issues, and in part because they seemed to be vague about the benefits case. The attempt to refocus the programme in 2024 was a response to Professor Lievesley’s report (mentioned above) but it wasn’t enough. In the end, little or nothing was achieved at the cost of £228.7m.

Simultaneously with this, morale at the ONS went into severe decline. Obviously people had worked very hard to achieve those COVID outcomes, but they also benefitted from the COVID-era working from home. When the senior management tried to bring back on-site working the response was even more negative than you may have seen elsewhere, leading to a formal dispute which ran for two years from Spring 2024 to April 2026.

The National Statistician resigned in May 2025, and the chair of the board in July 2025.

Overall, you get the impression of an organisation which was good in a crisis, but not able to maintain focus on the planning and execution of projects either to collect data in established ways, or make successful change in ways of working. ‘Good in a crisis’ is ideal branding for a fire department, but not really what you look for in a statistical agency.

What were the reasons for this? Let’s start at the top.

The Chief Executive of the ONS is both responsible for running a multi-million pound agency, and also responsible for leading the whole of government’s professional statisticians. Likewise the board is responsible both for oversight of the ONS as an organisation, and for oversight of the effectiveness of the UK governments’s statistical enterprise. I’ve seen comment that this dual role makes these hard jobs to fill, but I’m not sure I agree that was really relevant here. The previous National Statistician was Professor Sir Ian Diamond, and the previous board chair was Sir Robert Chote. A glance at Wikipedia will tell you that these were exceptionally distinguished individuals. Sir Ian had run a university and a research council before taking on this role. It would be difficult to identify anyone with a stronger CV.

There’s much more scope, in my view, to challenge what these people were doing. There’s a really telling anecdote at paragraph 70 of the Parliamentary report. The ONS commissioned an external consultancy which was sharply critical of its governance, decision-making and senior management behaviours but the Board was never informed of the report, still less any follow-up action. The Chair found out about the existence of the report in a conversation with a staff member a year later - in about May 2024 - but took no action at all. The report was never seen or considered by the board. By this time, of course, the morale issues at ONS had gotten to the stage of formal industrial action. It is bad enough that a chief exec should treat his board with apparent contempt by reviewing governance without their involvement. It’s a good deal worse that the board then thought he was right to do so.

As well as overseeing the management of the ONS itself, the board also has a remit to safeguard the quality of official statistics. Nevertheless the Parliamentary report found that the board was not well-informed about the key issues with the new Labour Force Survey. So this wasn’t a case of a board that had become too focussed on one part of its remit at the expense of the other. Neither part was being done well.

So despite being impressive individuals of suitable experience, it seems that the senior management and board had not developed a culture of sharing issues openly, challenging meaningfully, or setting priorities effectively. This resulted in the crazy position of an organisation getting an average 6.5% real rate of budget growth every year since 2015/16 that still regarded itself as underfunded, and which failed to find the money needed for key projects like the Labour Force Survey.

It isn’t actually true that a fish rots from the head, but the metaphor is too useful to ever give up.

Looking in a bit more detail at the operational level, the most prominent failures relate to the Labour Force Survey and the Integrated Data Service.

The basic issue with the LFS was the falling response rate that affects almost all surveys almost everywhere. In 2017 a project was developed to replace the LFS, using administrative data where that’s possible and a new, shorter survey for those elements where a survey couldn’t be replaced. Responsibility for the new survey was split, and changed frequently. It proved difficult to secure all the administrative data hoped for, and difficult to resist new data requirements, so the transformed survey wasn’t shorter than the old one and responders giving up in disgust continued to be an issue. Old and new surveys were run in parallel for a while because they gave some different data and the ONS didn’t fully understand why. For a time, additional resources were put into LFS to try to collect more surveys, but that money was withdrawn in July 2023 leading to an immediate fall in response rates below a useable level. While the resources went back in fairly quickly, the LFS is a multi-wave survey using data from five quarters, so it takes a long time for even a single quarter’s worth of poor quality data to wash out.

The IDS is another tale of long-running muddle and indecision. When set up in 2020, the original purpose was to join up data from across government and make them available to researchers (both inside and outside government) in a simple, secure, anonymised way. This would replace the previous patchwork of different arrangements for getting access to data and enable a step-change in analysis. Sharing data creates risks, both to the privacy of data subjects and to the reputation of data owners, so it isn’t surprising that government departments were reluctant to share the data that they held.The IDS programme could not articulate clear benefits to those stakeholders who therefore remained unwilling to share data. These were fundamental issues that needed to be resolved before the project could be launched, yet somehow it lasted five years and spent £228.7 million. In mid-2024 the aspiration to provide data to non-government users was cut in an attempt to clarify the business case, but then in 2025 the whole project had to be abandoned. Obviously if stakeholders couldn’t see the benefits of the project, removing some of the benefits wasn’t the right way forward.

I think all of us in a university setting have worked on projects that made tons of sense at a very high, hand-wavy level, but were never pinned down enough for actual implementation, and where the requirements constantly changed underneath the project team. Most of those projects probably fell apart in less than five years, but still it is striking how these two case studies just repeat our own experience on a grander scale.

It is also striking that these project failures are also fundamentally failures of governance, planning, and prioritisation. This particular fish really did rot from the head.

Again, I doubt there’s anyone in my network who can’t already imagine what it must have been like for the staff working in an organisation like this, and who therefore struggles to see how morale got to be so poor. The formal basis of the long-running industrial action at ONS was in-office working, but there’s plenty of evidence from these reports about the frustration of working for an organisation which didn’t listen, didn’t set priorities, and didn’t always meet the standards of professional conduct you’d have a right to expect. 

What can we learn? With modern technology, data start to seem weightless. You can download a huge wedge of NSS data in a few moments, and spin up any amount of analysis almost effortlessly. It’s easy to forget the amount of human labour that was needed to collect those data in the first place. People often tell me it should be easy for a university to know how many students it has. Everyone in my network probably shudders at that one because of the definitional issues around ‘student’, but how many even of us remember that alongside the definitional issues there are hundreds of staff, even at a smallish university, who need to be following processes and implementing policies every day if we want to get a meaningful number out of our database? If those policies and processes don’t work then, just like the LFS, we only have junk data.

The fundamental importance of governance is another learning point that won’t startle anybody. In this case we had all the form of effective governance, conducted by people who were clearly and obviously qualified for the job, yet it all turned out to be useless because the culture was wrong, and the governance was not focussed on the right issues. That feels like it hits close to home too.

The new regulatory framework

It’s been quiet on this blog lately, as I haven’t managed to put together the time to start and finish a whole blog post. The new regulatory framework may not seem quite as newsworthy as it did when I started this post, but to be honest it was never very exciting anyway – the focus of this blog is on regulation and funding in English HE, so we only ever aspired to the lowest values of ‘newsworthy’.

The new regulatory framework for English Higher Education, to be honest, looks a bit of a mare’s nest. Here it is set out on one side of A4 courtesy of HEFCE’s recent publication on the subject.

Yet another post about UCAS data

UCAS have now published their data for applications to the 15 January deadline. By this stage, almost all school-leavers who are going to apply have applied, and a significant proportion of the Overseas and older applicants have also applied. Because the data are taken with reference to an actual deadline this dataset, unlike the earlier UCAS data releases, provides a reasonably firm basis for taking stock of this years’ recruitment position.

Marks out of Ten

A good habit I have picked up from Brad DeLong is to go back over one’s work from time to time and identify the things one has gotten wrong. He calls this ‘marking my beliefs to market’. Reading through the last year’s blog posts I can see a number of pieces about the UCAS recruitment cycle. I’ve already published my self-defence on that one here, and I continue to think that the 2012 recruitment cycle was a lot less than catastrophic.

On the other hand I posted a number of pieces on the degree awarding powers of the University (ex-College) of Law confidently predicting that they would not be transferred with the sale of the College’s teaching business, but I proved to be completely wrong about that. The Government decided that the degree awarding powers did not pertain to the Chartered corporation once called ‘the College of Law’, but pertained to the actual teaching business which the College used to run. Therefore a change of control of this business automatically meant a change in control of the degree awarding powers.

UCAS Data: 2013 Cycle

UCAS have now published their mid-December data on the 2013 cycle, and the press coverage has been mainly downbeat, perhaps because UCAS has not tried the faintly ludicrous ‘late surge’ spin that they put on the equivalent data last year.

Applying Student Number Controls to Alternative Providers with Designated Courses

Today is supposed to be the last day in human history. I felt it would be a fitting epitaph to spend a few of my last hours on the earth writing a thousand words on the BIS consultation on alternative providers and the SNC: an exercise in existentialist policy analysis, if you like.

If the world can last that long, the deadline for responses to the BIS consultation is 23 January 2013, so there will still be a little while to reflect on this when we get back from the Christmas break.

Clearing 2012: How wrong was I, and why?

Back in July, I wrote a little blog post confidently predicting that this year’s Clearing would be less interesting than was widely expected:

Somebody, somewhere will have a bad experience in Clearing because somebody, somewhere always does, but there is no reason to expect a pattern or trend. Those who do badly this year may do well next. Life staggers on much as before.

Mark Leach was kind enough to post it on WonkHE, so it gained rather more prominence than my typical blog post. Since then, anyone watching the press will have read about the unmitigated disaster that has rolled over institutions high and low. It seems I was too sanguine.

London Met Again

Last week’s post about London Metropolitan leaped almost instantly into my top-ten of all time, as measured by pageviews. Not that it takes a whole lot of pageviews to achieve that in my case, but it is enough to persuade me to write a follow-up post.

Most people will be aware that London Met has taken legal action against the UKBA. The form of this legal action will take is judicial review. Judicial review is a process which does exactly what it says on the tin: the decision taken by a public body (in this case the UKBA) is reviewed by a judge. There is no nonsense here with a jury. If the judge determines, on review, that the decision was wrong he (or she, but as this is an English judge we are talking about, almost certainly he) may quash it.

The business at London Met

There are so many issues in London Met’s unfortunate encounter with the UKBA that I scarcely know where to start: so much blame to be distributed, so many consequences to be guessed at. You have to eat the elephant one bite at a time.

Targets

Clearing is here now, and Jessica Guiver’s blog about targets – and what happens if you miss them – has got me thinking about this issue. A recruitment target is a number: it seems clear, precise and very measurable; but in fact a recruitment target is also a cultural artifact that can only be understood correctly in a particular context. Let me explain what I mean.

Broadly there are two approaches to targets. Some Vice Chancellors I have worked with in the past have seen targets as an essentially rhetorical device. Their role is to encourage the troops to try even harder and therefore it isn’t important that they can necessarily be achieved. It may even be detrimental to the intended incentive effect. Dull number crunchers like me tend to see targets as part of a rational planning apparatus in which we work out a budget and resource model robustly based on a realistic target number. Now whilst these approaches to targets can be contrasted in principle, in practice they are continually held in tension. Even the coldest technocrat does not actively wish to discourage recruitment. Perhaps there are some VCs out there so ardent for growth that they genuinely don’t care if the students all wind up in one department whilst the budget to teach them is in another, but they assuredly employ many people – both academic staff and accountants – who do.

Gearing up for Clearing

We are in the final stages, now of preparing for Clearing. My direct role in this process is a fairly limited one - I had a discussion yesterday about what budget code we should assign to the bottled water that keeps the Clearing team hydrated, but actually I don't even sign off the purchase order at the end of the day. Making Clearing happen is a major undertaking: the first morning puts a strain on out telephone and IT systems that nothing else in the year comes close to matching. This post isn't mainly about the mechanics of Clearing, though, because this is a planning blog. It is mostly about more of a macro-level issue: will we all get enough students to meet our budgets whilst staying below our student number control limits? The answer, of course, is no: but not in a very interesting way.

Degree Awarding Powers: Confusion reduced

A follow up to my post from earlier today. Whilst Education Investor repeats the BBC story that two private providers have gained degree awarding powers, a colleague from BIS has responded promptly and unequivocally to my email as follows:

I can confirm that the online list of recognised bodies is complete and accurate. The BBC story is incorrect in its statement that the second institution to be granted degree awarding powers this week is a private college. The other institution is a publicly-funded institution that receives HEFCE funding.

More confusion about Degree Awarding Powers

The news in the BBC and Times Higher that Regent's College has been granted degree awarding powers will come as a surprise to nobody likely to be reading a blog like this one. What came as a very severe shock to me, though, was this language in the BBC report:

'It [i.e. Regent's] is one of two private colleges being given degree-awarding powers this week....The identity of the second college has not yet been made public, with government officials saying it is up to that institution to make its announcement.'

There is nothing like this mentioned in the Higher story, so I very much hope it is a BBC reporter's misunderstanding (The Royal Academy of Music has recently been awarded DAPs, and as far as I can see from their website hasn't made a public announcement, but the RAM is certainly not a private college). BIS maintain a list of bodies with Degree Awarding Powers here. If it is really true that BIS is allowing private colleges with degree awarding powers to stay off this list until they wish to publicise their new powers, that would strike me as a very surprising thing indeed. It would call into question the Government's commitment to a 'level playing field' between public and private providers, not to say basic standards of openness in public administration.

I tried to speak to someone at BIS to gain clarification, but when she discovered I wasn't a journalist, the operator refused to put me through so I've had to fire off an email. When I get a response, I'll post an update.

Muhammad Saleem from Online Colleges sent me this infographic about MOOCs. I don't agree with everything it says (see my post here), but I thought it was interesting enough to be worth sharing.

Cash is fungible

Dewi Knight has an interesting post on WonkHE about the developing Welsh approach to HE. As Dewi highlights, part of the mood music which Welsh politicians use to accompany their policy proposals is the decisive rejection of market (and English) models. The Welsh Government even plans to take powers so that it can fund specific universities directly.

One of my occasional themes on this blog is that developments in different parts of the UK are not as different as they appear, and this is a good example. English Ministers emphasise the language of competition and markets, whilst Welsh Ministers use different language, but both are engaged in projects to reshape their respective HE sectors to the needs (as perceived by Ministers) of students and employers. Both are seeking to enhance their regulatory control over their respective sectors.

It is inconceivable that English Ministers could seek powers to fund institutions directly (even if they dared to bring forward an HE Bill at all...), but the pound a Welsh Minister grants to a Welsh institution will have exactly the same purchasing power as the pound an English student pays in state-subsidised fees when the English funding council decides to increase an institution's Student Number Control. Cash is fungible.

Changing Universities

On Friday I attended the Association of University Administrators' open forum Managing Change. This is the third of these annual events and interesting war stories are always exchanged. Unfortunately the Chatham House rule was in operation, so I can't discuss what I heard very freely.


What I will do is to post a link to the group's online resource Changing Universities, and encourage any readers who are involved in managing change in universities to consider getting involved in the group. If nothing else, the meetings provide a level of social support: it gives me reassurance to know that no-one else has completely resolved the issues that I can't resolve either.  

The BPP School of Health

Yesterday's news that BPP is branching out into a new professional area by launching a School of Health provides a good example of the way our HE system continues to evolve only haltingly towards a more open market, and some of the important barriers that still exist to private providers. The specific subjects offered are Psychology, Chiropractic and a little Nursing, as well as entry-level programmes designed for healthcare workers with very limited formal qualifications.

I'll take these in reverse order. As the existing health professions continue to evolve (nurses take on jobs formerly done by doctors, and become an all-graduate profession) there has been increasing emphasis on formal training and development for health care assistants and other support workers. London South Bank, for instance, offers a range of Foundation Degrees aimed at these workers and closely aligned with local NHS Trust. You will see from the materials I linked to that students are either in NHS employment, or offered NHS training contracts before they come on the course. So there's nothing very new in what BPP are offering, and their integration with employers (the key competitive advantage they have in e.g. Accountancy or Law as compared to the established universities) is not all that strong.

Chiropractic is another interesting choice, because it is a profession not regulated by the Health Professions Council. Psychology is regulated by the BPS, but unlike most existing University programmes (including private providers like Buckingham), BPP's programme doesn't seem to have BPS accreditation. No doubt BPP will gain accreditation in time, and develop increasingly strong NHS links, but we also see how the forest of poorly-coordinated regulatory bodies that surrounds the existing HE sector (and are a royal pain from a day-to-day management perspective) make it difficult even for respected providers like BPP to break open new areas.

Why haven't BPP chosen some easier targets? This little table shows you that:

JACS2 Subject Line	Applications	Accepts	Ratio of apps to accs (:1)
A1 - Pre-clinical Medicine	84,598	7,798	10.8
A2 - Pre-clinical Dentistry	12,712	1,196	10.6
B1 - Anatomy, Physiology and Pathology	30,466	3,315	9.2
B7 - Nursing	218,599	24,587	8.9
D1 - Pre-clinical Veterinary Medicine	8,485	971	8.7
B8 - Medical Technology	14,608	1,804	8.1
X1 - Training Teachers	58,374	7,388	7.9
B2 - Pharmacology, Toxicology and Pharmacy	32,849	4,539	7.2
L5 - Social Work	81,285	11,334	7.2
W4 - Drama	53,942	7,464	7.2
K1 - Architecture	31,062	4,379	7.1
Y Combs of soc studies/law with business	20,293	2,995	6.8
L1 - Economics	51,603	7,800	6.6
C3 - Zoology	9,991	1,544	6.5
B4 - Nutrition	7,139	1,150	6.2
C7 - Molecular Biology, Biophysics & Biochem	17,436	2,872	6.1
Total all subjects	2,847,012	492,030	5.8

Source: UCAS

These are 2011 data on subject areas with many applications, and high application-to-acceptance ratios. In other words the market is here. Drama is in there, where the barrier to entry is capital costs rather than regulation, but for the most part the dominance of health and related subjects is clear.

From David Willetts' perspective there is very little he can do about this, of course. The HPC, the BPS and the NHS don't answer to him.

Understatement

There are some significant differences between the cultures of US and UK journalism, as becomes pretty clear from even a quick scan of the headlines. Usually I think the British headline writers show more verve and humour, but sometimes the American approach wins out, as in this case where the Telegraph's story is reported in the Chronicle under the truly delicious title Foreign Students Said to Get a Leg Up in Admission to British Universities.

Gove lays a trap for the Russell Group

It has long been clear to most observers that Michael Gove is not truly a Conservative, but instead a leftist agent provocateur determined to heighten the contradictions inherent in Conservative educational policy until the whole structure collapses. If anyone had doubted this before, then his proposal to curry favour amongst the Tory grassroots by reversing the Thatcher government's key school reform surely marks the point at which he threw the mask aside.

Another post about Udacity

I posted about Udacity previously. At the time, I couldn't understand how anybody was proposing to make any money out of the project, and I see that more eminent people than me still can't. This reflects our old-economy idea that a business has to have revenue. I have since come to understand that in the internet economy it is far from unusual for businesses to have no revenue, in fact it is normal. The value of a company in this case is the group of users that it can bring to a better-established internet company, or the defensive value that the better-established company can gain by buying an upstart before it becomes a rival. In this context, Udacity (or any other learning provider) can make a fortune for its owners if it establishes a large user base and then sells out to Google or Facebook, even if no-one ever works out a way to cover the costs of the business in the meantime.


Understanding this helps me to place some other things in context too. Drop-out from these kinds of programmes is truly catastrophic if your comparator is a traditional university programme. Inside Higher Ed says that 'generally between 10 and 20 percent' of students make it to the final exam in an individual course, roughly equivalent to a UK module. In currently-existing British universities, by contrast, 86.9% of (Undergraduate) students successfully complete their (3+year) programmes and get an award. If your comparison is other internet experiences, though, getting 10-20% of your users as deeply engaged as to complete an entire course of study probably seems like an outstanding metric.


It follows from this, of course, that the learner is not the customer in this case. That may not be an issue: I don't use Facebook myself, but I do use Google and (obviously) Blogger, so perhaps my choice of illustration for this post shows a certain double standard. Besides, it isn't always clear that the learner is the customer in traditional education models either: this is sharply contested by some.


It also follows from this that Udacity is a bit less interesting than I previously thought. In my last post on the subject I was impressed by the audacity of taking both teaching and assessment out of the University offer, but now that I understand this as just one of many approaches to putting engaging content online so as to grow a significant user community it seems a lot less exciting, and a lot less relevant to my day-to-day work. If the experience is engaging, it will literally never matter if the assessment and certification 'issues' go unresolved; if the experience is not engaging it is also unlikely to matter whether assessment and certification can be resolved. 


So to go back to Paul Greatrix, I think the 'confusion' was probably more in his mind (and mine) than in the minds at Udacity, Coursera and the others.