A new benchmark of university admissions was published at the start of this week, and within a day it had become a ranking. There is no great mystery in that. A set of scores covering ninety-four institutions invites ordering, and the sector reads tables quickly because it has had a good deal of practice at doing so. Something was mislaid in the speed of the conversion, though. The interest of the exercise lies less in the positions it assigns than in the question it was built to ask, one institutions have seldom put to themselves in quite this form.
The comparison it makes is the unfamiliar part. Rather than measuring an intake against the population at large, or against the applications a provider happened to receive, it measures that intake against the pool of young people who achieved the grades the institution itself requires. Roughly seven in ten providers recruit a more disadvantaged intake than that pool would predict. Nearly a third do the reverse. Whatever one makes of the individual figures, the comparator is an analytical advance, because it isolates a stage of the process that has been difficult to examine from outside. Attainment is measured, application behaviour is studied and admission is counted, yet the distance between meeting a requirement and receiving a place has tended to remain the least visible part of the sequence.
What the measure can and cannot see
Two features of the exercise probably deserve more attention than they are yet to receive. The first is its vintage. The analysis follows a cohort who sat their GCSEs in 2015/16 and reached higher education by 2019/20, so it describes admissions judgement as it stood six or seven years ago, before the pandemic and before much of the access work institutions have undertaken since. Its reception has treated it as a statement about institutions as they are now. The distinction offers less comfort than it may appear to, since an institution cannot know whether its own practice has changed in the respects the measure identifies unless it has gone and looked, and few will have looked. The age of the data is a reason to read the findings in the right tense, then, rather than a reason to set them aside.
The second is that the exercise produces more than one ordering. A variant that adjusts for location, recognising that a provider recruits from the pool actually within its reach, rearranges the field enough to give pause to anyone quoting a position. Where a defensible alternative specification moves institutions substantially up or down, the rank is the least stable thing the analysis yields. The instructive material lies in the difference between the two accounts, in what the adjustment suggests about the role geography plays in a pattern that might otherwise be read as intentional. That is the sort of finding a table cannot easily carry.
None of this would matter greatly were it not for what tends to happen next. Once a measure begins to carry reputational weight, the pull towards managing the number is considerable, and the sector has seen this happen to more than one indicator over the past two decades. A provider might rise through a table by adjusting how it words a requirement or by recruiting differently at the margins of its portfolio, without any applicant’s experience of applying having altered. Nobody need be acting cynically for this to occur. It follows readily enough from the ordinary pressure to demonstrate progress on something a governing body has begun to ask about.
The harder use
The more demanding response is to treat the publication as a question rather than a verdict, and to put it to oneself irrespective of where one has landed. Whether meeting the entry requirement and receiving an offer have subtly become different things within an institution, and if so at which point in the process the divergence occurs. Whether the accumulated habits by which admissions staff and academic selectors recognise a promising candidate, formed over years, transmitted informally and rarely written down anywhere, are doing work that no published criterion describes. Whether the wording of a requirement, the evidence a personal statement is expected to display or the assurance a reference is expected to convey favour applicants who have been taught how such documents are read. These are questions an institution can investigate with data it already holds, and the answers would be its own rather than a commission’s.
It would be unwise to press the argument too far. A single indicator can mislead in either direction, and the commission that produced this one says as much itself, allowing that where disparities appear there may be good explanations and framing the tool as a prompt to review rather than as a finding of fault. Selective institutions face constraints a coefficient will not capture, among them subject mix, the regional composition of their applicant pools and the plain arithmetic of places against demand. Nor is it obvious that a provider with a favourable score has less to examine than one without, since a strong position may reflect a recruitment context as much as a practice. There is a risk in the argument advanced here, too, that methodological caution becomes a sophisticated way of declining to look. An institution that met this week’s publication with a careful account of the measure’s limitations but no examination of its own practice would have taken the easier of the two paths whilst appearing to take the harder one.
What the benchmark has done, then, is less to grade the sector than to make a particular question askable in public. Whether it proves useful will depend on what institutions do with it before any further edition, which is not something a measure can determine on its own. Would a leadership team recognise the difference between improving its position and improving its judgement, and would it know which of the two it had achieved? Where inside an institution would an enquiry of this kind properly sit, given that it belongs neither wholly to admissions nor wholly to the academic units whose judgements it would examine? And is it possible that a sector grown fluent in responding to measurement has allowed the older and less comfortable skill of interrogating it to fall out of use? These are not questions a benchmark can answer on an institution’s behalf, and they may be among the ones that most repay a governing body’s attention.