The Office for Students consultation on quality regulation closes this week and much of the sector’s attention has understandably focused on the proposed changes to TEF and the implications of linking assessment outcomes to fee levels and growth permissions. These are significant matters deserving careful scrutiny. Yet reading David Kernohan’s recent mapping of what he calls the “shadow quality assurance system” prompted reflection on something that receives somewhat less attention: the parallel mechanisms through which quality is actually built, maintained and improved in practice.
External examiners, student complaints processes, professional body accreditation, periodic programme review, annual monitoring and peer observation of teaching. These systems operate largely below the radar of national policy debate. They generate no league tables and attract minimal media interest. Ministers do not announce reforms to external examining in parliamentary statements. Yet anyone who has worked within higher education providers knows that these mechanisms do work that outcome metrics cannot capture.
External examination operates through relationship and dialogue
Consider external examination. The system is, as QAA chief executive Vicki Stott observed this week, respected, popular and widespread. It operates through relationship and professional dialogue. An experienced external examiner notices things that no dataset could reveal: when assessment briefs have drifted away from learning outcomes, when marking has become inconsistent across a teaching team or when module content no longer reflects current professional practice. This noticing happens through careful reading of student work and through conversation with programme or course teams. It surfaces problems early, often before they manifest in the metrics that regulators monitor. The external examiner who flags concerns about assessment design in November may prevent the poor continuation rates that would only become visible in data two years later.
This early-warning function depends on several conditions that cannot be taken for granted. It requires examiners with sufficient expertise and standing to command respect. It requires institutions willing to hear unwelcome messages and act upon them. It requires time for the relational work through which trust develops between the examiner and the programme team. And it requires adequate recognition and remuneration. External examiner fees at many institutions have remained static for years, whilst expectations around reporting and regulatory compliance have grown. The pool of colleagues willing to undertake this work is also not inexhaustible.
Complaints data reveal where provision is under strain
Student complaints processes perform a different but equally important function. The individual complaint matters, of course, and institutions have clear obligations to address legitimate grievances. But the complaints data in aggregate tells a story about where provision is under strain. A cluster of complaints about feedback turnaround times on a particular programme signals something worth investigating. Rising complaints about placement experiences may indicate problems with employer partnerships that require attention. This intelligence only flows, however, if students trust the system enough to use it and if institutions have sufficient capacity to analyse patterns rather than simply processing individual cases. When complaints teams are cut or restructured, the early-warning function diminishes even if the formal process continues to exist.
Professional bodies hold programmes to standards that reflect practice
Professional body requirements introduce another layer of accountability that operates quite differently from regulatory oversight. The Nursing and Midwifery Council, the Engineering Council, the Solicitors Regulation Authority and their counterparts across the professions hold programmes to standards that reflect what practitioners actually need to know and do. Their requirements are granular in ways that institutional-level metrics cannot be. They mandate specific clinical hours, particular competencies and defined exposure to professional contexts. Programme teams preparing for professional body accreditation engage in detailed curriculum scrutiny that shapes what students experience in ways no TEF submission captures. This accountability is largely invisible at the level of institutional performance but profoundly consequential for graduate capability.
Quality enhancement happens through unglamorous daily work
Then there is the unglamorous daily work of quality enhancement. Annual programme monitoring requires teams to reflect systematically on what worked and what did not. Periodic review creates space for more fundamental questions about programme design and currency. Peer observation of teaching, when implemented well, builds pedagogic capability through collegial dialogue. These activities rarely feature in institutional marketing or regulatory submissions. They are not exciting. Yet they are the mechanism by which provision improves over time.
The risk of hollowing out what we cannot easily measure
The concern is not that the current regulatory focus on outcomes is wrong. Outcomes matter and institutions should be accountable for them. The concern is what happens to these shadow systems when resources are constrained and attention is diverted elsewhere. Quality offices have been restructured and reduced at many institutions in recent years. Administrative support for external examining has been cut. Time for the relational and reflective work that quality enhancement requires has been compressed as academic workloads intensify.
There is a risk, perhaps, that in pursuing quality through metrics, we inadvertently hollow out the mechanisms through which quality is actually built. The data that regulators examine reflects the cumulative effect of countless interactions, decisions and practices within institutions. If the systems that shape those interactions receive insufficient support, the outcomes we measure may eventually deteriorate in ways that metrics-focused regulation cannot easily address.
These are not comfortable questions and I do not pretend to have confident answers. Different institutions will recognise different pressures and possibilities in their own contexts. But as another national-level rethink of quality regulation takes shape, it seems worth asking what we know about the health of the systems that operate below the regulatory line of sight. Where external examination works well, what sustains it? Where complaints processes surface useful intelligence, what enables that? Where quality enhancement activities genuinely improve provision, what conditions make this possible?
The answers may matter more than we sometimes acknowledge.