Over the past two years, there have been some significant challenges that educators in Universities have had to confront, in many cases driven by top-down policy initiatives. There are a few different places where impacts have been seen – but one area I’d like to highlight in this post lies in assessment. The stakes are already quite high for practitioners striving to engage in the craft of pedagogy, as I’ve recently posted, because quantifying student achievement in a homogenous way (e.g. grading) is bound to cause problems. When we try to signal achievement in meaningful ways whilst working with forms of feedback that reduce something complex, layered, and multi-modal like human communication, we are bound to experience tensions and shortcomings. This is HARD work. But let’s assume we’re stuck with grading systems and don’t have access to more creative and collaborative options. If we accept this as our lot, then we need to work with these systems to mitigate their failures, especially in inflexibility (feedback to students around assessed work will inevitably hit up against limits and edge conditions) and opacity (our communication back to students in feedback is just as complex as their communication to us in their essays!). So at the very least, I think, we need to include flexibility in our approach, because humans just aren’t all the same, and our experiences and styles of learning are often radically different. Difference, in my view is best engaged in a relational way, and by extension is best managed at a local level. But can we actually do this in practice? There are some external factors intervening which educators in Universities need to be aware of as we try to develop careful and responsible policy around marking.
The policy landscape has shifted in the background over the past three years in the UK in ways that many academics (including myself) won’t have noticed. I’d like to break this down at some length as it helps to explain why we’ve gotten where we are just now (many readers may experience an “ah, so that’s why….!” moment) and unpack some of the parameters around how I think the educational landscape is being changed around assessment.
Part 1: A brief history of OfS interventions around Assessment in Universities
Over the past decade in the UK, there have been massive shifts in the way that government relates to Universities: shifting from a fairly hands-off mode under the Higher Education Funding Council for England (HEFCE) which was finally disbanded in 2018 towards a much more interventionist model driven through two new units: the Office for Students (OfS) and United Kingdom Research and Innovation (UKRI). Over the past ten years, and leading up to the formation of the OfS, government bodies involved in oversight and regulation of Higher Educatio have taken a much more interventionist stance on a variety of fronts within education, and assessment has been a particular target. This is in many ways a resurgence of the “Quality Wars” which have been on and off in Britain for several decades. Returning to recent history, from 2018 until 2021, Universities were regulated under The UK Quality Code for Higher Education (often called simply “the Quality Code”). This was generated in partnership with academic bodies and well regarded on an international level as a set of principles for setting standards in academic practice at Universities.
The Higher Education and Research Act which was passed by Parliament in 2017 initiated a process of review which eventually led to the launch on 17 November 2020 of a “Consultation on regulating quality and standards in higher education“. This process underwrote the drafting by OfS of a set of new policies, intended to replace the Quality Code, which were disseminated to the public alongside a lot of unqualified (and I daresay irresponsible) commentary around confronting “low quality courses” (WONKHE summary). The new policies were shared and feedback was drawn in from 20 July 2021 to 27 September 2021 (PDF here). There were serious questions about the intention of this process from the start (cf. this piece for WONKHE by David Kernohan and Jim Dickinson). Why for example, describe a process as consultative without meaningful involvement from students (much less educators) in the process of policy design? After that consultation, OfS produced a (largely dismissive) response to feedback from those two groups (PDF here). The conditions in that code came into force on 1 May 2022, just over a year ago. If you want to get into the nitty-gritty of how that public dialogue has unfolded, I recommend you read the various waves of analysis by interpreters on WONKHE which I’ve highlighted above.
These policies revolve around five long-standing “key conditions”: titled “B1”-“B5”, which are the following:
Condition B1: The provider must deliver well-designed courses that provide a high quality academic experience for all students and enable a student’s achievement to be reliably assessed.
Condition B2: The provider must provide all students, from admission through to completion, with the support that they need to succeed in and benefit from higher education.
Condition B3: The provider must deliver successful outcomes for all of its students, which are recognised and valued by employers and/or enable further study.
Condition B4: The provider must ensure that qualifications awarded to students hold their value
at the point of qualification and over time, in line with sector recognised standards.
Condition B5: The provider must deliver courses that meet the academic standards as they are described in the Framework for Higher Education Qualification (FHEQ) at Level 4 or higher.
These are seem to me like fairly reasonable things to aspire to in providing higher education. And while one might object to the back and forth of the policy process and the underlying instability this produces for students and academic staff in the sector alike, the actual headline suggestions seem quite unobjectionable.
However OfS process has pressed far beyond headline guidance and unpacked these in what seem to be forensic yet problematic ways in subsequent communication. The consultation documents acknowledge this agenda: “The main change from the current conditions is that the proposals include more detail about the matters that would fall within the scope of each condition and how these would be interpreted”. So the process didn’t just reshape the principles, but also indicated a shift from broad principles and quality monitoring towards much more explicit and specific policy delineation. This dialectic between specific rules and broad principles is a perennial point of debate between ethicists (like myself), and as you’ll find in my publications, I tend to prefer a principled approach, not least because it allows for a more relational and flexible approach to policy which can accommodate diversity and pluralistic communities (as I’ve highlighted above). Rules are better at controlling people, but they also run the risk of causing oppression and undermining the relationships in which education functions.
Back to the discussion at hand. It’s worth emphasising that the OfS is not playing around here: the potential consequences for a University which is found to be in breach of any of these conditions might be a refusal by OfS to validate their degrees and basically cancel graduation for that institution. Also fines. Expensive scary fines.
So what’s in the details? There are little landmines throughout the details here, as one might expect. I’d like to focus on the development of criteria for how “a student’s achievement to be reliably assessed” (condition B4). In the 2021.24 document, OfS provides further detail of the requirements entailed by B4:
B4.2 Without prejudice to the scope of B4.1, the provider must ensure that:
a. students are assessed effectively;
b. Each assessment is valid and reliable;
c. academic regulations are designed to ensure that relevant awards are credible; and
d. relevant awards granted to students are credible at the point of being granted and when compared to those granted previously.
They explain B.4.2.a in more detail a bit further on:
c. “assessed effectively” means assessed in a challenging and appropriately comprehensive way, by reference to the subject matter of the higher education course, and includes but is not limited to:
i. providing stretch and rigour consistent with the level of the course;
ii. testing relevant skills;
iii. requiring technical proficiency in the use of the English language; and
iv. assessments being designed in a way that minimises the opportunities for academic misconduct and facilitates the detection of such misconduct where it does occur.
In case this wasn’t clear enough, OfS provides even further guidance down the page, and colleagues may begin to notice here some of the drivers of policy which has been depoyed over the past two years by cautious University VCs and management eager not to fall afoul in the ways that their universities manage education and risk scary consequences:
50. In relation to “assessed effectively”, the following is an illustrative non-exhaustive list of examples to demonstrate the approach the OfS may take to the interpretation of this condition… “Marking criteria for assessments that do not penalise a lack of proficiency in the use of written English in an assessment for which the OfS, employers and taxpayers, would reasonably expect such proficiency, would be likely to be of concern. Students are not penalised for poor technical proficiency in written English. For example, for assessments that would reasonably be expected to take the form of written work in English and for which the OfS, employers and taxpayers, would reasonably expect such proficiency, the provider’s assessment policy and practices do not penalise poor spelling, punctuation or grammar, such that students are awarded marks that do not reflect a reasonable view of their performance of these skills.”
As you can see, we’ve gone from broadly unoffensive headline principles to some quite odd particulars very quickly in the OfS process. There are a number of strange things here: why single out “technical proficiency in the use of the English language” at all given how such things are already intrinsic to most degree programmes, especially within the humanities? If there are problematic actors in HE, it seems much more sensible to confront that on a specific level rather than deploy a redundant policy. But also, the emphasis here is not on measuring student achievement, but on penalising a lack of proficiency. There’s no interest here of celebrating a surplus of profiency. It’s also uncomfortable to find language here, reaching beyond student experience towards “the OfS, employers and taxpayers”. Many readers will be able to imagine a thousand different ways this could have been written (or not at all), less agressively, more constructively, etc. But let’s set aside these uncomfortable quirks for a moment and ponder together what exactly this all might look like in practice.
Part 2: Why does it matter?
One might come to the end of this and say something like, “yes, I wish this was worded more carefully and sensitively, but there’s nothing wrong with this policy in practice. And why not have more specific versions of our policy, much less these ones? Don’t we want our students to be effective communicators?” There are two key problems here which are complex and deserve our attention, not least because the victims are already on the margins of HE:
Problem 1: ambiguity and overcompliance
From a policy perspective, this is all a bit confusing. On one hand, it has the aspect of really highly specified policy. But on the other hand, even after reading all the guidance, it all seems quite murky. This issue with highly-specified policy which is nonetheless unclear on implementation details, is a recurring problem in the sector. The same issues apply to immigration policy in higher education, with providers often stretching far beyond the “letter of the law” in reaction to ambiguity which is intrinsic to policy demands.
How does one assess English language proficiency (for the “taxpayers”, of course)? Well, there are basically two ways to interpret this guidance. The first option is to take it at face value and do the specific things they mention, e.g. include in our processes of assessment specific checks penalising people when they show they are not sufficiently proficient in spelling and grammar. As I’ve said above, these things are already widely practiced in the sector, so one couldn’t help but begin to wonder, is there somethign else I’m missing here? There’s an important phrase in the text above where the guidance explains what “assessed effectively” means. Did you see it? It’s the part which says “includes but is not limited to”. So is one right to assume that these very (even weirdly) specific examples are the policy? Or is this just a sort of initiation into a pattern of behaviour that educators are being pressed into? So here we have seemingly highly specified policy with loads more detail than previous policy guidance had in it, associated with really scary and dire consequences for people who breach these new policies, and some intimations that we need to develop regimes for surveillance to punish wrongdoers but even at the end it’s clear that “there’s more…” without specifying what exactly that means. In organisations where there are administrative staff whose job is to mitigate risk, when situations of ambiguity arise, the resulting policy response will veer to the side of caution, especially if there are fears of serious consequences looming if one is cast into the limelight for falling afoul of a policy. So in some cases, organisations do just what they’re told. But in many cases organisations reply to policy in extreme ways – this has been the case in response to prevent policy on campuses, immigration and right-to-work checks, even response to GDPR policy is implemented in ways that on closer look are unnecessarily extreme. So at the very least, when you’re faced with a situation like this, there is an increase in the hazard that the resulting implmentations or responses to a policy demand may reach much further than they need to.
Problem 2: linguistic competency
Things get even murkier when you start to really try and work out the details. Marking spelling accuracy is dead easy, and students ought to know better as they can easily access dictionaries and spell checks. We do penalise sloppy work in terms of misspelling, I know this is the case from years of practice as an educator, and also an an external examiner for a variety of Universities. The same is true of punctuation. The rules are highly standardised and easy to access. But what about grammar? On one hand, verb tense is pretty important to get right. And there are guides which can check this. From years of reading, I can spot bad grammar almost automatically on the page in a variety of ways. But what exactly does it look like to be maximally proficient in English language usage? What exactly are the rules we follow that make communication effective? This is where things get murky.
There are a number of examples where some people might point to the formality of prose as an important standard, my students every year ask me if they should use personal pronouns, “I think that…” versus “One might think that…”. And my answer is always, “it depends”. Formal language can be helpful, but sometimes it can be impersonal and inaccessible, even a bit stiff. Another example is the use of complex terms. Does proficiency in English language rest on the use of five syllable words? Or is the most compelling prose driven by neat sparse vocabulary? And even more pointedly, what about the use of slang, vernacular, pidgin, or creole terms and idioms? Are those kinds of language casual? Formal? Technical? I can think of many cases where scholars and poets I admire forced me to think through a particular vernacular or regional lens with their writing in English and this elevated their arguments. And as someone who has lived in a number of different English-speaking nations and regional cultures (USA, Canada, Scotland, England and Wales) I can attest to the ways that idiomatic English can vary wildly, both within elite and non-elite cultures.
Once we get past the basics of spelling and grammar, it gets really tricky to actually say what the rules are, and explain how and when they should be broken. Ultimately, good prose just feels right. We can judge its effectiveness through affect and intuition. But also, our judgement relies upon sympathy with the writer: I work hard to understand a poet’s use of vernacular, metaphor, etc. because I trust that they have something to teach me. Another reader could just as easily pick up the poem and conclude that it is obscure and poorly written. I don’t mean to suggest that there are no standards, but that they are not easily accessible and that teaching them is complex and requires a lot of recursive effort. And successful judgement like this only becomes accessible at the mature end of a learning journey and not checkpoints in the early stages. And can we really say that there is such a thing as “Standard English” (SE)? Even within my own department, and in specific taught modules, I can point to a variety of contextually meaningful differences around how to use language to communicate towards certain ends. And if we were to go for the lowest common denominator, is there any way we could say this is more than simply writing mechanics like proper spelling?
There are also ways that working with this kind of intuitive sensibility leads us to rely on personal familiarity – things which “feel right” as an implicit proxy alongside the more rigorous and obvious formal rubrics which establish conventional grammar and spelling. For all these reasons, the measurement of language competency is a hotly conteted topic among specialists in linguistics. At the very least, it is generally taken to be the case that measuring this effectively is extremely difficult, even for highly trained specialists and best not attempted by amateurs (like me, as I have a PhD but am not a scholar in linguistics).
So it’s hard. And potentially it’s pretty likely that any intelligent person, even University faculty, will make mistakes in assessing language competency – at the very least mistaking written communication that “feels familiar” as proficient. But why is this such a big deal?
The hazard here sharpens even further when we apprecaite how, in spite of our best efforts, the aesthetics of student feedback processes can collude with forms of racialised, class-encoded, ablist, and enculturated privilege. Linguistic racism and misogny is widely recognised as an issue in workplaces, impacting customer relationships, undermining team communication, and widening inequality in hiring processes. This has been investigated in some specific case studies into the ways that vernacular language can be stigmatised and used as the basis for discrimination and negatively impact academic outcomes. In particular, studies at Universities have found lower levels of achievement for Black students when compared to their counterparts who are racialised as white. This in turn has been linked, albeit only in part, to the stigmatisation of African American Vernacular English (AAVE) over so-called Standard English (SE). Seen in this way, forms of ethnic identity embedded in language patterns defeat the anti-bias intentions of anonymous marking and create a context for inequality in marking. A wide range of solutions have been proposed in pedagogical literature, including (particularly among proponents of student-centred learning) forms of teaching which encourage the reading and use of vernacular speech as a prelude to critical engagement with vernacular culture more broadly. Given the amazing levels of diversity that we enjoy in our Universities, it would be well worth engaging in a corporate discussion around how we engage and celebrate vernacular (if indeed we do), and conversely, how we can mitigate discomfort by other students and staff who are unaccustomed to deviations from Standard English (SE). However, setting aside this more proactive aspiration, it seems to me to mark a step in the opposite direction to introduce punitive measures for markers who find lacking English language proficiency, particularly without any specific guidance as to how vernacular language might be handled, and how evidence of research and understanding might be affirmed in ways which are separate from language flow. All human cognition mobilises bias, so it’s not enough to aspire to an unattainable “unbiased” state of mind, but rather essential to acknowledge and understand in an ongoing way how our biases, especially implicit ones, mobilise forms of privilege in our teaching and research.
In my teaching, I introduce students to the importance of local and culturally inflected forms of knowledge in responding to public policy challenges like climate change. We also discuss how cultural fluency and dynamism can serve as a transferrable skill, enabling our students to support workplaces which want to reach new audiences and develop forms of end-user engagement which are culturally relevant. In particular, within Theology and Religious Studies we discuss the importance of vernacular culture as tools for community development, we introduce students the ways that feminist scholars and poets deploy informal language, like contractions, and use creole and vernacular vocabulary as a way of challenging unjust hierarchies and emphasising the social good of diversity in practice. I even hope that my students may come to deploy those tools themselves in their written communication, thinking about the ways that forms of communication transmit not just information but personal and social values. I fear that education policy in Britain may prefer to pay lip service to the goods of diversity, whilst failing to provide support and infrastructure to underpin these kinds of learning.
Part 3: What should we do?
The OfS has created a really unfortunate challenge here, ultimately risking the recolonisation of education (contradicting our glossy brochures which advertise our de-colonising work). I gather from reading the OfS response to the policy consultation that there was quite a lot of unhappiness about the policy. This included, as the report indicates,
- Suggestions that it was not appropriate for the OfS, as a principles-based regulator, to prescribe how a provider should assess a student’s language proficiency beyond the course learning outcomes, and whether it was possible ‘to infer what employers’ and taxpayers’ specific expectations might be in any particular circumstance.
- Views that English language proficiency receives disproportionate attention in the proposed guidance, with respondents questioning whether it is a priority for non-UK partners or appropriate for all TNE courses.
- Disabled students or students for whom English is a second language may be disproportionately affected by an approach that expect proficiency in the English language as a relevant skill. For example, one respondent commented that the ‘focus on proficiency in written English has potential implications for institutional approaches to inclusive assessment, which are designed to ensure that students with specific disabilities are not disadvantaged during assessments, and to thereby comply with the Equality Act’.
- The level of English proficiency required would be better considered on a subject or course basis, based on academic judgement about whether mistakes in written English are material or not.
I’ll let you read their resposes (starting on page 25)
It has become clear to me that protest is not an option. There is simply too little will within the sector leaders to respond to OfS, at least overtly, with refusal to comply and a demand for more sensitive education policy (I would be delighted to be corrected if I’m wrong about this). So there are a two different models of compliance that seem viable, within some specific conditions, to me:
(1) Affirm ways that assessment criteria and feedback already achieve these demands. This is true of all the the teaching contexts where I’ve ever worked or assessed as an external examiner in the UK, so ultimately a viable option, though I accept that there may be some contexts where this isn’t the case. I don’t see any reason why these can’t be handled on a case-by-case basis.
(2) Make very specific additions to marking criteria. It may be that there are some cases where it becomes clear that spelling and grammar haven’t been part of assessing student work. If that’s the case, then the least harmful and hazardous way to achieve compliance here is to be highly specified: “spelling and grammar”. Reaching beyond this in any particular way will enhance the risk and dynamics of bias in assessment.
It’s worth noting that the latter option might be chosen not because there are clear lapses in educational design, but out of a desire to be seen as “doing something”. There may be a fear lurking here (which I’ve alluded to above) driven by the dynamics of over-complicance, that we need to do something visibe in response to the “new” demand to avoid surveillance and punishment. I’ll note here my reservations as an ethicist with any engagements with policy that are arbitrary. There will always be problematic and unfoeseen implications lurking downstream as our use of arbitary policy begins to accumulate and iterate. And these consequences often tend to impact persons (both educators and students) who are already marginalised in other ways.
My anxieties here around bias in marking align with some other work I’m running in parallel with some stellar colleagues around stereotype threat and implicit bias. I’ll be writing more in months ahead about these projects and what I think we can do to proactively address the harmful impacts these phenomena have on students and teachers more broadly in our teaching practice. I’m also working with colleagues to find ways to more proactively celebrate linguistic diversity in our teaching in higher education and will share more about this work as it unfolds. But it’s worth stressing at this point that it is irresponsible to implement harmful and risky policy (even in the current atmosphere) with the expectation that we (or others) will mitigate those problems after the fact with bias training. This is highlighted well in a recent piece by Jeffrey To in Aeon.
I’d be very glad to hear what colleagues have attempted and how educators are responding to these challenges across the sector and hope you’ll share from your experience in the comments.
