Yesterday I was asked to make a one-page summary of practitioner research. I shared it on Twitter and it seemed to go down quite well, and so I thought I’d share a PDF version of it here. Here are the steps once again:

Practitioner inquiry in 12 easy steps

Practitioner inquiry is a simple, systematic approach to professional development that can be done by whole schools, groups of teachers or individuals. Here’s how to do it in 12 easy steps:

1. Reflect. Think about your professional development to date. What are your strengths? What problems do you face currently?

2. Focus. Choose an area of your practice that you would like to investigate or develop. Keep the focus small and manageable.

3. Formulate a research question. Research questions should be defined as tightly possible. A useful question frame is: ‘To what extent is… effective as a method for improving…?’

4. Read around the topic. There’s no need to carry out an exhaustive literature review, but if possible, try to read at least two opposing pieces of research about your area of interest.

5. Choose a research method. There are many methods – attainment data, observations, interviews, questionnaires, students’ work… It’s a good idea to combine two or three different methods and triangulate findings – but make sure you keep it manageable.

6. Take a baseline (optional). Not all inquiries lend themselves to a ‘pre vs. post’ evaluation. However, if you wish to get a handle on whether your practice is improving over time, some form of initial baseline measure will provide a useful point of comparison. You might also consider collecting data about students not involved in the study (a control group).

7. Plan and carry out your intervention (optional). Not all inquiries are intervention-based; you might simply wish to find out more about an aspect of your existing practice. However if your aim is to evaluate the impact of a particular strategy, take the time to plan how to implement your intervention in a way that maximises the possibility of success.

8. Take a post-intervention measure (optional). If your research method involves a ‘pre vs. post’ comparison, how long will you wait before collecting the data – a day, a week, a month?

9. Analyse your findings. Once you have collected your data, take the time to sit with it. Try to understand it as deeply as possible. Discuss it with your colleagues. What does it tell you? What does it not tell you? Did you find what you expected? What conclusions can you draw?

10. Evaluate your inquiry. How did it go? What went well? What aspects did you find challenging? What would you do differently if you did it again? Can you use the findings to inform your practice in future? If so, how? If not, why not?

11. Write up and share your findings. You can do this at praxis-teacher-research.org. Try to tell the story in a way that will be helpful to others working in a similar position. Aim for around 750-1000 words. You can upload additional documents (e.g. results tables) as attachments. It is advised to use the following headings: Introduction; Research question(s); Research methods; Results and Discussion; Conclusions; Evaluation; References.

12. Plan your next inquiry. What’s next? Do you wish to repeat the same inquiry but do it differently this time, or in a different context? Perhaps it’s time for a change of direction? Whatever you decide, stick with it: practitioner inquiry is a powerful engine for getting (even) better at what we do, and improving outcomes for young people.

 

Yesterday I finally made it to the fabulous Northern Rocks conference. It’s a beast of a drive from Brighton to Leeds but it was worth every minute. On the hottest day of the year, it also warmed the parts the sun can’t reach – a day full of optimism and hope for a profession that sometimes feels like it’s on its knees following the endless, often incoherent interference from the top. It’s a shame Amanda Spielman couldn’t make it – I think she would have enjoyed it.

Anyway, here are the slides from my talk ‘Why oracy is more important than literacy and numeracy put together’. It was essentially an expanded version of my recent blog for Oracy Cambridge, which also covered some of the evidence for promoting speaking and listening in schools, which I think is pretty overwhelming.

The message that most seemed to resonate with people is that oracy is not pedagogy; it’s the curriculum. Every school should be putting oracy front and centre, alongside literacy and numeracy. Let’s make it happen – we’re almost at the tipping point (see slide 22)!

 

| by James Mannion |

Here’s an idea for a project secondary schools can do (possibly with the support of primary schools) to help smooth transition from year 6 to 7. 

In the Key Stage 3 Learning to Learn curriculum I’m evaluating as the focus of my PhD, the first project we ran in Autumn term 1 was an identity project called “Who am I?”. The idea was to help the children think about themselves as individuals and about what it is that makes them, them. The transition from year 6 to 7 can be pretty brutal – going from a class of 30 in a primary school where everybody knows everybody else, to a huge new complex of buildings where there might be as many as 300 students in your year group alone, all wearing an identical uniform… what does this do to a child’s sense of identity? It’s an interesting question, and as far as I’m aware it’s not one that is asked very often.

Hence our decision to kick off the year with a project about identity. We gave students 6 weeks (one double lesson each week, plus homework tasks that they set for themselves) to answer the question “Who am I?” in as many ways as they possibly could. This is not an original idea – I did something similar when I was in year 7 – but I do think it’s a great project for year 6s and 7s.

I haven’t thought about the “Who am I” project for a while, until this week. I’m currently working with a group of secondary school senior leaders on the problem of how to smooth transition from year 6 to 7. Mostly, the focus is on literacy interventions once children are in year 7. However, one teacher I’m working with wanted to explore the idea of doing something before the students arrive at secondary school to help prevent or slow down the “backslide” that can sometimes happen between years 6 and 7; something that the students can do over the summer a) to show off/keep up their literacy skills, and b) to help the secondary school get to know their new intake.

Our first thought was to ask the children to write a journal, or a Summer Active Reading Programme like the one that was recently evaluated by the EEF. Then the teacher I’m working with suggested doing something around identity, and I remembered the “Who am I” project. I think this lends itself to a summer transition project really nicely, because it doesn’t require very much input from the school.

So, if you are a head of Year 7, when you invite the year 6s in to secondary school for a taster day, why not give them the following:

• A clear plastic folder – you know, one of those ones that look like this, whatever they’re called ⇒
• A pen and a pack of coloured pencils
• A few sheets of lined and plain paper
• A cover sheet with the following (or something like it – here’s a Word version in case you want to edit it)

If you do decide to do this – please let me know how it goes!

http://rethinking-ed.org/wp-content/uploads/2017/05/Who-am-I-briefing-sheet-blog-3.pdf

Che Guevara, nuclear bombs, privilege and power: my new post for Oracy Cambridge:

http://oracycambridge.org/2017/04/28/why-oracy-is-more-important-than-literacy-and-numeracy/

 

| by James Mannion |

I stumbled across a blog this week which appears to be a critique of my last blog. I say ‘appears’ because the piece, by Michael Fordham, firstly characterises my last post Imagine a school… as an argument for “turning the school curriculum into a market by turning pupils into consumers”, which is a bit of a stretch, and then goes on to shoot down a number of positions that I do not hold and did not propose.

To respond to Michael’s concerns in full will take some doing – hence writing this as a post, rather than commenting on Michael’s blog. First, to recap: in my ‘Imagine a school’ post, which was described by @HeyMissSmith (I think sarcastically) as a “Utopian song”, I made the following wish list for secondary school reform:

• Alongside (note, not instead of) the traditional range of subject disciplines, students should be able to choose from a range of ‘short, optional courses’ in topics such as those I suggested in my last-but-one post Subject-based learning: let’s blow this baby wide open;
• The decision as to which courses a student will take (there would be one per half-term, or 6 over the course of a year) would be made jointly in a 3-way meeting between the students, their parent/carer(s) and a dedicated personal tutor;
• In all subjects, students should be grouped together in classes according to aptitude and/or interest, rather than chronological age (so-called ‘stage not age’ teaching);
• Instead of exams being used as a mandatory social filter for ranking students along a scale from success to failure, all formal qualifications should a) be optional and b) be available at different grades, like music grades or karate belts (as in “watch out – I have a black belt in Statistics”). In this way, every grade achieved would rightly be recognised as just that – an achievement, or a step along a journey – rather than as a stick to hit children and teachers with.

Now, I am very much aware that proposals such as these are some way off the radar in terms of the current education debate, and it would be rather hopeful to expect people to get on board with these ideas immediately. And so I am grateful to Michael for engaging with my suggestions – even if I don’t think he’s really arguing against what I am proposing – and I look forward to taking the conversation further, if anyone can bear it. That said, I will now attempt to slay some of the myriad straw men that lay siege to my Utopian song – well, its first verse anyway.

Choice ≠ consumerism

Michael begins by associating my proposal for optional short courses to run alongside (not instead of) traditional subjects with the doctrine of economic liberalism. I had to look up what this means. Economic liberalism is defined on Wikipedia as “a system of economics that describes the organisation of an economy on individual lines, which means that the greatest possible number of economic decisions are made by individuals or households rather than by collective institutions or organisations”.

In a world where there’s a price tag on pretty much everything, it is understandable that someone might think of choice as automatically being the same thing as consumerism. However I am not suggesting we turn education into a marketplace (defined as “an actual or nominal place where forces of demand and supply operate, and where buyers and sellers interact to trade goods, services, or contracts or instruments, for money or barter”). I’m not even suggesting we turn schools into a marketplace of ideas (defined as “a rationale for freedom of expression based on an analogy to the economic concept of a free market”, which holds that “the truth will emerge from the competition of ideas in free, transparent public discourse”. 

In the proposal to which Michael refers I am talking about neither trade nor truth but two simple, practical ideas: a) introducing a series of short (6-week) courses to run alongside the ongoing study of traditional subjects; and b) extending this small degree of choice to every year group. Michael sees this as analogous with buying a computer:

If I want to buy a new computer, I can choose from several different companies offering different machines, and I therefore choose the best machine I can get at the price I can afford. If a company is pricing its machines incorrectly then it will not make any money and collapse. In a market, things need to be allowed to fail.

I can see where this is going, but before we get on to the ‘being allowed to fail’ thing – this is not a good analogy. Computers, despite the choice of shapes, colours and specifications, all do pretty much the same thing. An education, meanwhile, is a diet of learning where the choices vary in lots of ways, and serve wildly different purposes. Choosing a computer from many similar machines and choosing a balanced diet of learning from a broad menu of wildly different topics are not the same thing.

A better analogy is that of a restaurant menu. At the moment we have only a set menu of traditional dishes. What I am suggesting is a hybrid model between the set menu and the a la carte menus – a diet of learning with some core components and some optional extras on the side. As children look around the current ‘restaurant’, they see adults eating all these wildly interesting foods – sizzling platters and exotic fruit salads:

Child: “Oh, pleeeease let me have a taste! But I don’t want to eat an entire lobster thermidor! I just want to know what that succulent tailmeat tastes like.”

Adult: “Nope – it’s potatoes and broccoli for you I’m afraid until you’re at least 16 years old. We know best – it didn’t do us any harm!!”

Child: “Well adults are kind of making a mess of the world, so…”

Adult: “What part of no do you not understand? Now not another word, or it will be lunch isolation for you…”

Too big to fail?

I think a certain amount of Michael’s blog can be read as a simple misunderstanding of what I was proposing; perhaps I didn’t express myself clearly enough. For example, he writes:

Making the curriculum a market perhaps incentivises teachers to ‘sell’ their subjects, which perhaps involves focusing on the jazzy surface details, and not on the complex fundamentals of a subject that are difficult to explain in a sales pitch.

However, in my Utopian song I wrote that these optional short courses should run alongside traditional subjects – not instead of them. I anticipate that these 6-week short courses (e.g. in touch typing, structured debating, introductions to economics or sociology or whatever) would run for 2 or 3 lessons a week, maximum – certainly no more than any other subject. 

Because I am not proposing that all subjects compete with one another in some Hunger Games-style fight to the death, Michael’s reasonable concerns about whether an unpopular subject would or should be allowed to go to the wall take care of themselves: it wouldn’t happen because it’s not what is being proposed.

All about that baseline

Michael then gets into some practical concerns:

As a teacher, my ideal is to be able to sit down with a group of pupils and to have a fairly good idea of what they have already studied. In some subjects, where there is more of a linear model of progression in the subject – I am thinking here of subjects such as mathematics, languages and (to an extent) some aspects of the natural sciences – prior knowledge is frequently a prerequisite for learning about something new. In other subjects – such as history or literature – it is very useful for a teacher to know what has already been studied, for this will help to shape the explanations and comparisons that he or she wants to make. Pupil choice interferes with this, for it increases the number of starting points that pupils have.

What I am proposing is that for most of the short courses, there would be a series of levels (I suggested beginner, intermediate, advanced and self-managed, although I’m open to other models – some topics might require more or fewer gradations). The beginners’ course would assume no prior knowledge, and each subsequent level would be predicated on the basis that the student had completed and passed the less advanced course that precedes it. It’s a mastery model of learning – trads should love this stuff.

Cool into complex does not go

Michael goes on to say:

If you think your subject is just a ‘collection of cool stuff’, then perhaps [having multiple starting points] does not matter, as each bit of ‘cool stuff’ is independent of the others. If, however, you see you subject as a discipline with complex interlocking components that have to be learnt, practised and honed over time, then pupil choice is highly disruptive.”

I think this depends on the subject. Learning to touch type is neither cool nor complex. It’s a boring slog that needs time, guidance and deliberate practice. (Quick disclaimer: I did a 6-week course in touch-typing when I was at 6th form college, as part of a ‘complementary studies’ programme. In fact, it was called ‘keyboard skills’ – I signed up because I thought I’d get to play around with synthesisers in the music room. In the event, I found myself seated in front of an electric typewriter and given two items: a really boring book to copy typing exercises from, and a tea towel to cover my hands. It was neither cool nor complex, but by the end of the 6 weeks I could touch-type – a skill I find useful daily almost 25 years later.)

In essence, this is a variation on the knowledge/skills debate. Many of the things on my list – analysing the media, public speaking, making films, cooking, first aid, managing your finances etc – these are not really “disciplines with complex interlocking components that have to be learnt, practised and honed over time”, but skills which, once mastered, can stay with you forever. This, for me, is the beginning and end of the knowledge/skills debate: you kind of need them both, so let’s get on with teaching both really well and stop arguing endlessly about which is more important.

I do think some of Michael’s concerns about teachers selling courses by overplaying the “cool stuff” are valid, and I think this should be part of the quality assurance process that would go into preparing a 6-week course (which would have to be planned in advance and be ratified by governors, say).

Ti-i-i-ime is on your side. Yes it is. Really

Michael ends with the battle cry used by many a trad as a pre-emptive strike against anyone who would dare suggest we could do things differently. It goes something like this:

Time is preciousssssss. Which of these lovely subjects would you take away? Because we have plenty more traditional sounding subjects waiting in the wings. Your 6-week course in touch typing can get to the back of the queue mate.

The truth is – controversy klaxon – secondary schools are hopelessly inefficient. Each student has around 11,000 hours of lessons – 5,000 of which are in secondary school. How many lessons of the 5,000 lessons I endured at secondary school were as useful as that 6-week course in touch-typing? Certainly not the interminable “cluster analysis” project I did for GCSE geography, where I spent weeks pacing around my local town, printing out maps and scaling up distances to work out how close shoe shops are to one another.

It’s actually easier to make time for other stuff in a secondary school than one might imagine. Without wanting to make this about my PhD study, I have worked in a school where we ran timetabled Learning to Learn lessons throughout years 7, 8 and 9 (5 a week in Y7, 3 a week in Y8 and 2.5 a week in Y9). Despite having 400 fewer lessons of subject-based learning over those 3 years, the Learning to Learn students went on to achieve the best set of results in the school’s history. Furthermore the Pupil Premium gap closed considerably, from the bottom up (there’s more than one way to close a gap). My point is simple: yes, we ate into a bit of the time usually allocated to other subjects. And no, the world didn’t stop. In fact, it got a little bit better.

In summary then: breezes

Michael concludes:

Those advocating more pupil choice in terms of what they want to study are asking us to turn the school curriculum into a market.

No they aren’t. At least, not this one.

This is not something I can support.

Yet. Maybe. You never know.

 

Imagine a school where, alongside the traditional range of subject disciplines, students are able to choose from a range of short, optional courses in topics such as those suggested in my last post, at a range of levels (introductory, intermediate, advanced and self-managed, say).

Imagine a school where, every summer term, there is an open day where these courses are explained and pitched to students and their parents/carers. Where each family sits down with a dedicated personal tutor to arrive at an informed decision as to what courses the student will take in the coming year, and at what level.

Imagine a school where students are grouped together in classes because they share an interest and/or an aptitude for a subject, rather than because they happen to have been born in the same year.

Imagine a school where each subject has assessments available at a different grade, like piano grades or karate belts. Where all grades are recognised as an achievement, rather than exams being used as a mandatory filter for ranking students along a scale from success to failure.

Imagine a school where students take assessments if and when they feel ready to do so, instead of sitting exams in all their subjects in the space of a month at age 16.

Richard Buckminster-Fuller once said “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.” The only thing I can’t imagine is why we haven’t built this model already. It really wouldn’t be very difficult to do.

 

| by James Mannion |

Hi everyone. My name is James and I have thought there should be more to education than an almost exclusive focus on a narrow band of traditional subject disciplines for ooh, I’d say about 12 years now. Some people say this makes me a “progressive”, and that therefore any ideas I have are silly. I don’t really know what progressive means, but if it’s the opposite of regressive then sign me up!

One thing people say about progressives (whatever they are) is that they want to do away with subject-based learning. However – in my case at least – nothing could be further from the truth. In fact, I love subjects so much, I want to see many more of them in our schools.

My thinking is as follows: 1) Schools have been focused almost exclusively on teaching a narrow band of traditional subjects for as long as anyone can remember; 2) I don’t know if you’ve seen the news ever, but the world is in something of a mess. Conclusion: since we’re only alive for the blink of an eye, maybe we should try relaxing some of the odd top-down controls on what people can learn and when, which I lambasted in my last blog post.

Before anyone starts tweeting pictures of tin foil hats, I should clarify – I don’t mean to suggest that the world’s mess is directly caused by the fact that our education system focuses almost exclusively on a narrow band of traditional subject disciplines. Indeed, I am quite persuaded that the study of traditional subject disciplines enables at least some good things to happen. But it’s not helping enough for my liking.

Being a solution-focused kind of guy, I’ve made a list with a few suggestions for subjects that we could bring into schools. This is not intended as an exhaustive list, and obviously these couldn’t all be studied in an ongoing way – there wouldn’t be enough time – but perhaps we could start by running short 6-week short courses in them, and see how that goes. I would love to hear your thoughts on this, reader.

Active citizenship

• Analysing the media
• Argumentation
• Consensus building
• Debating
• Finding sources
• Interview techniques
• Journalism
• Logical fallacies
• Making films
• Philosophical enquiry
• Photography
• Photojournalism
• Public speaking
• Shorthand
• Thinking and reasoning (eg the excellent outgoing OCR course)
• Touch typing
• Using a library
• Verifying sources

Computing

• Artificial intelligence
• Coding
• Cryptocurrencies
• Cyber security
• Ethical hacking
• Internet studies
• Linux
• Making movies
• Open source software
• Programming
• Robotics
• Web design

Health and well-being

• Cooking
• First aid
• Managing your finances
• Medicine
• Mental health
• Parenting skills
• Physical health

Enterprise and entrepreneurship

• Applying for funding
• Blogging and social media
• Building a professional website
• Customer relations
• Events management
• Graphic design – using GIMP
• Leadership and management
• Marketing and advertising
• Project management
• Securing investment
• Starting a business
• Tax and taxation
• Writing for different audiences

Learning-related courses

• Philosophy of education
• The psychology of motivation
• The cognitive science of learning
• The art and science of goal setting
• Giving and receiving feedback
• Education research – what do we know about what works?
• Managing your own learning
• How to find reputable sources on the internet

Politics and economics

• Animal rights
• Campaigning
• Crime and punishment
• Domestic politics
• Equality and fairness
• Ethics
• Global politics
• Government and politics
• History of politics
• International relations
• Law and the justice system
• Local politics
• Making sense of current affairs
• Tolerance and discrimination
• The history of protest
• The history of trades union
• Human rights

Taster courses for other subject disciplines

• Astronomy
• Classics
• Economics
• Electronics
• History of art
• Philosophy
• Psychology
• Sociology

This is all just top of the head stuff – no doubt this list reflects my own interests and biases. I would love to hear more suggestions for courses – please comment below if you have any ideas.

If expanding the subject base is the answer, what is the question?

I know all about the pressures on school leaders, but surely we can find an hour a week on the timetable where young people could explore some of these ideas? Obviously there would have to be some kind of quality assurance – for example, there could be a noticeboard where teachers can pitch ideas for courses, and if enough students express an interest then the teacher puts a 6-week course together and then has to present to governors and have it ratified or whatever. We’s only need to find an hour on the timetable – to begin with, at least.

The question facing us is not how, but why. And we can answer that with a question to any parent:

If there was a local school that offered courses like this, would you send your child there?

I know I would.

| by James Mannion |

In this post I argue that compulsory GCSEs for all, which means enforced failure for some, is profoundly unethical. I also outline an alternative model for secondary school assessments. 

Educational assessment is like a festival of abstract jargon, so please bear with me. As far as I can gather, when people sit an exam you can set grade boundaries in three main ways: criterion-referenced, norm-referenced and cohort-referenced testing.

An example of criterion-referenced testing is the driving test. There are a number of predefined criteria – don’t crash etc – and if you meet enough of them, you pass, and if you don’t, you fail. Another example is music grades. In the UK, music exams are graded from 1 to 8, with grade 8 being the level required for entry to music college. Some school-based assessments are criterion-referenced also, such as Key Stage 2 SATs and the International Baccalaureate (IB). It worth noting that criterion-referencing does not rule out the ability to provide students with grades: while the driving test is a straight pass-fail, in music exams, SATs and the IB students are awarded grades to reflect varying degrees of success.

In norm-referenced testing, the grade is pre-determined by comparing your score with that of a ‘norm’ group who sat the test previously. The IQ test is a good example of a norm-referenced test. The whole point of an IQ test is to rank everyone who takes the test against the population at large, with 100 being the mid-point, or average. Cognitive Attainment Tests (CATs), which are often completed by students as they enter secondary school, work in very much the same way.

Cohort-referencing is very similar to norm-referencing – however, rather than being graded against a pre-existing ‘norm’ group, as with the IQ test, in cohort-referenced testing grades are divided up among the cohort taking the test. Job interviews and theatre auditions are examples of cohort-referenced testing. In selective settings such as these, where there are only so many vacancies available and you can only appoint out of the people who apply, only some can ‘make the cut’ – and so cohort-referenced testing seems entirely appropriate.

So, what kind of testing do we use at GCSE?

Ofqual claim that we don’t use norm-referenced assessment at GCSE. In a technical sense, this is true – we don’t use a purely norm-referenced approach, because “as the cohorts for different subjects vary, awarding grades using the same pre-determined set of percentages would make the same grade in different subjects have a very different meaning” (Ofqual, 2014, p5).

What we use instead is a kind of cooked version of normative assessment known as the ‘statistical approach’, which was described in 2012 by John Dunford, then chair of the Chartered Institute of Educational Assessors, as a “strange mix of criterion and norm-referencing”.

While it may be strange, I think it is clear that the way in which we grade GCSE exams is fundamentally norm-referenced, or more precisely cohort-referenced in nature. This can be seen clearly in the table below, which outlines how the old A*-G grading system maps on to the new 1-9 system (source: Joint Council for Qualifications).

Given that the Ofqual definition of norm-referenced assessment as that in which “the proportion of each grade available to the cohort is pre-determined” – it is difficult to see how the system we have could reasonably be described as “not norm-referenced” (or cohort-referenced for any pedants out there).

So what? What’s wrong with norm-referenced testing?

Often, conversations about the new assessment system focus on the top end: how many top grades will be available, what will count as a pass, what will be the new equivalent of the C/D borderline and so on? For example just this week, a top civil servant suggested that only 2 students in the UK will get straight level 9s, and the current Secretary of State for Education announced that whereas previously the DfE had announced that Grade 5 would be the equivalent of a ‘good pass’ (i.e. that required by colleges and employers), this has now been revised so that a grade 4 will be a ‘standard pass’ and grade 5 will be a ‘strong pass’.

Setting aside any Spinal Tap comparisons (let’s take this baby all the way up to 11), all of this focus on pass rates overlooks an important truth. Look at the bottom half of the table above: no matter how had they work – and no matter what raw scores they achieve – our current system guarantees that 32.1% of students (16.7 + 8.1 + 4.1 + 2.0 + 1.2) will not meet the level required for a “standard pass”, and will therefore have to repeat their exams post-16. This is such an important point that I feel it bears repeating in the form of a meme:

Is everyone OK with this? Because it doesn’t have to be this way.

Why don’t we have a criterion-referenced system?

Back to Ofqual:

“When GCSEs were first being developed in the mid-1980s the Government’s intention was that criteria-related grades would be introduced as soon as practicable with candidates who reached the required standard being awarded those grades. Despite heroic efforts, it proved impossible in practice to meet that intention. So GCSEs have never been criterion referenced” (Ofqual, 2014, p5-6).

And apparently, that’s it – there have been some unspecified “heroic” efforts at criterion referencing, and now the idea has been laid to rest for good. (Ed – I would love to hear more about these heroic efforts – if any readers can shed some light, please comment below).

What’s wrong with norm-referenced testing?

Daisy Christodoulou has written in defence of norm-referencing here. Essentially, her argument echoes that of Ofqual – criterion-referencing is difficult to get right, and so maybe norm-referencing is the least bad option. It is certainly true that criterion-referencing is not without its problems. As Debra Kidd wrote recently:

Fixing the results [through norm-referenced testing] protects children from a catastrophic drop in results when government ministers have fiddled with the exam system. It creates stability. The alternative is what we saw with KS2 SATs last year – a criteria based system – where the % of children meeting expected standard fell from 80% to 53%. A drop like that at GCSE would be disastrous… it seems like the fairest option in a flawed system.

The way the debate is currently framed (i.e. within a system of compulsory testing), we have two choices – criterion-referenced and norm-referenced testing – and the latter is the least bad model.

Daisy concludes:

“Despite all the real technical flaws with criterion-referencing… there is still an element of hostility to norm-referencing amongst many educationalists. In my experience, I sense that many people think that norm-referencing is ‘ideological’ – that the only people who advocate it are those who want to force pupils to compete against each other. Nothing could be further from the truth.”

This is where we part ways. While I do have a problem with norm-referenced assessment, but it isn’t an ideological. It’s ethical. I have changed my mind on this. A few years ago, I accepted norm-referenced testing as the best way to grade students. My thinking was as follows: exams can vary in difficulty from one year to the next, but ability probably stays more or less constant from one cohort to the next; norm-referenced testing tells you how well someone performed within the context of their cohort, and that is preferable to a situation where everybody in one cohort gets good grades because the exam happened to be easier that year.

Since then however, I have thought again. My problem is not to do with competition, but compulsion. In a compulsory system of norm (or cohort)-referenced testing, no matter how hard everyone works or how well they perform, a certain proportion of them are destined for failure before they’ve even started the course. As Tom Sherrington wrote in his amusing, if genuinely angry blog post Nicky Morgan vs the Bell Curve:

The thing is this: by definition there are only a limited number of places on the bell-curve that can be called ‘Good GCSEs’.  You’ve decided to give a pejorative label (implicitly ‘Bad GCSEs’) to about 50% of all grades.  Now, instead of Grades 1-4 at GCSE representing any sort of achievement, they’ve been killed stone dead. Nice work. That didn’t take you long. 

The fact that grade 4 has since been reclassified as a “standard pass” is entirely beside the point. My question is this: how can we have arrived at the point that it’s OK to force hundreds of thousands of young people to sit exams every year when we know in advance that by definition, 32.1% of them will have to fail? Is this really necessary?

What’s compulsion got to do, got to do with it?

What must it feel like to be forced to sit an exam you know – and your teacher knows – you are going to fail? At what point did we decide as a society that it’s a good idea to force young people to sit exams at all, regardless of their wishes? Is this really in their best interest?

There seems to be an idea floating around that some must fail in order for the passes to mean something, or else we have a vacuous culture where “all must have prizes”. But this is palpable nonsense. Do all children need to take a surfing exam at age 16 in order for some people to be considered good at surfing? Of course not. Do all people have to sit their driving test at age 18? No. Why then do we feel the need to apply this madness to subject learning? If gaining a pass in GCSE English and maths is to remain a point of entry for the vast majority of jobs and college courses, then people will take it when they are good and ready. For lots of reasons, not everybody is ready to sit high stakes exams in the May of their 16th year. Those who do not wish to should not be compelled to – just as some people choose not to learn to drive, or to surf.

You might say “Ah well, GCSEs are different because people need to be literate and numerate whereas you don’t necessarily need to drive or surf”. But this is a bad argument, because passing GCSE English and maths is not the same thing as being functionally literate and numerate. GCSE maths is way harder than the Functional Skills maths exam. If functional literacy and numeracy were the grand scheme here, it would be the Functional Skills exams that are compulsory. But they’re not. If anything, forcing 30% of people to fail at GCSE English and maths is counterproductive with regard to promoting functional literacy and numeracy; repeatedly branded failures throughout their school career (let’s not forget, GCSEs are not the only compulsory exam – they are simply the high stakes exams that come at the end of compulsory schooling), they develop a bad relationship with words and numbers and actively avoid trying to improve themselves in this regard.

I cannot for the life of me understand why we as a society have decided that forcing a young person to take an exam – regardless of their ability or affinity for that subject – is an acceptable thing to do. A common argument you hear is that students who are initially resistant to a subject are later grateful for having been ‘put through the ordeal’ – that it is ‘tough medicine’ but worth it in the end. But I only ever hear this from teachers. In my ten years as a teacher I have spoken to hundreds of young people about how they feel about exams and I have yet to encounter one who peddles the tough medicine line, whereas it is commonplace for year 11 students to say things like “I used to really love [insert subject], but after the exam I never want to hear of it again.”

It doesn’t have to be this way – does it?

I work part-time at an alternative education provider – the Self-Managed Learning College in Brighton, where students aged 9-16 are not compelled to do exams, or indeed to learn about anything that they don’t want to learn about. Many of them do exams – indeed a number do exceptionally well with very little “knowledge input” from tutors – while others choose not to do GCSEs at all. This is all fine. Some teachers might find this difficult to believe, but actually the world doesn’t grind to a halt when we stop forcing children to fail exams against their wishes. What generally happens is that they spend their time getting really good at something else, like art or music or computing or film-making or enterprise or cooking or animation…

Let’s consider for a moment what a non-compulsory, criterion-referenced secondary school system might look like. I think it is fair to say that the driving test method – a single test with a straight pass/fail – would not lend itself to school-based assessments. However, the fact that people take a driving test when they are good and ready – and that they can retake the test as many times as they wish – is worth bearing in mind. But why is it so difficult to imagine a world in which GCSE exams were graded in a similar way to piano grades?

Instead of sitting a single exam in which all students are subdivided into grades of relative success and failure, why can we not have a system where students take their Grade 1 maths exam – which could either carry a straight pass/fail or could be subdivided into fail/pass/merit/distinction – when they are good and ready? In this model, gaining a Grade 1 would cause for celebration – as in, “YAY, I passed my Grade 1 maths exam, next I’ll have a go at Grade 2” – rather than “OH NO, I sat a maths exam and I only got only a Grade 1, which must mean I’m stupid coz the government is making me retake it at college’.

Here’s a question: what would happen if, when teaching young people about stuff, we actually gave them a choice about whether they wanted to take the exam at all? The second half of this sentence is something of a heresy in some educational circles, but I’m going to say it anyway: it is quite possible to learn something without sitting assessments. When somebody learns the piano, they get to a point where their teacher (assuming they have one) says ‘I think you’re ready to take your Grade 1 exam’. It is then up to the student whether they take it or not. When I learned the piano as a child, I chose not to sit any of the formal exams. Despite this, I know that I progressed to around a grade 5, which was fine for me. Classical music bored me and I had no desire to go to music college, but I was good enough to play in bands – an ability I still enjoy almost 30 years after my piano lessons finished.

Also, and this is perhaps another heresy – sometimes, people stop learning stuff. In year 7, I had oboe lessons for a few months. It wasn’t for me – it made my lips ache and the way saliva drips out onto your shoes is really disgusting. And so I stopped – and this is fine as well. Actually, to be fair the school system is kind of OK with children stopping learning things, but only if every young person in the country goes through the same pruning process in unison – and even then, only at age 14 (where you can choose around five non-compulsory subjects, although in reality the options are extremely limited by schools seeking to maximise their league table position), age 16 (whittle it down to three) and age 18 (now pick just one, if you’re lucky).

All this standardisation and top-down control over what young people can learn and when is all rather odd, isn’t it? Am I alone here?

According to Daisy Christodoulou and OfQual, writing criterion-referenced exams it too difficult and so we no longer bother. Except for when we do, like with music grades and SATs and the IB, which seem to work just fine (except for when they are compulsory – see above). In fact, I suspect that many of the problems with criterion-referenced testing would disappear if we did away with compulsory testing, and let young people choose a) what they study, b) whether they want to be assessed, and c) at what level.

Is it really so difficult to imagine a school system where for each subject, GCSE exams are split into graded exams, in the same way that music grades work, and where students can choose whether to take the assessments? Can somebody please explain to me why this is not possible or desirable?