What Am I Trying To Achieve?

The last five blogs have led to this one, and I think the main question that anyone should be asking when they do anything is 'What am I trying to achieve?'. A task without a goal, a purpose, feels a little pointless.

What am I trying to achieve when I'm feeding back assessments to my classes? Well, that's a big one.

Ultimately, I want them to be proud of what they've achieved and to be spurred on to greater things, or I want them to take away an impetus to improve in some way. Dylan Wiliam says that 'feedback should improve the learner', so I don't simply want an exam script that looks more like a rainbow and has a larger number on it than it did when I gave it back to them. I want pupils to have a moment of realisation. It could be as small as a mistake that they made is corrected (not just once, but for the rest of time) or it could be the understanding that if they begin to apply themselves more, and give more time to practicing maths, they can be better at it.

My Year 10s - the class that I'm going to trial this with - sat the first paper of their mock exams, and as it stands I have marked the papers, but haven't yet analysed them for individual areas for improvement, or whole-group areas for improvement. My plan is to sort this tomorrow, identifying areas of need for individuals/small groups, and identifying areas that the whole class need corrective instruction with.

In terms of what to do with the individuals, this tweet gave me some inspiration:

Get them to identify if it is a minor mistake or something they have to work on. Watch a video on DrFrost Maths or the model answer for the older ones which we make ourselves. Then they need to practice the skills on DrFrost Maths as our scheme of work/courses are linked on it

— Mrs Hudson (@MrsHuds43995523) January 8, 2023

We have included Mathswatch clip numbers on pupils' printouts from the QLA spreadsheet but it's always been self-directed. It's felt like a tick-box exercise, and it hasn't done much to improve the learner, their responsibility, or them as a mathematician. I'm tempted to record some example-problem pairs for key topics and share these with pupils, or provide backward-faded worked examples with further problems to be completed independently.
If I record videos I can post these on Google Classroom and use a computer room, whereas the backward-faded activities can be used in the classroom. I also have a couple of days that need cover setting coming up, and these two could be completed with cover teachers.

Pick 4 or 5 questions where the majority made silly mistakes. Go through them and then do shadow questions. They then compare the marks to see how many extra they would have got. It's superficial but it boosts their confidence for how they are going to move forward..

— Laura Jacobs (@Miss_L_Jacobs) January 9, 2023

I think I can set the videos/BFM tasks where pupils have made daft errors or have taken a wrong turn somewhere, and set the problem (in the example-problem pairs) as the shadow question. Or maybe I can write the problem part and they'll complete the shadow questions at a later date. This should allow me to discern performance from learning and they can re-do them in conditions akin to an exam hall for 20 minutes or so.

I have a couple of weeks until we can share the results with the pupils, and I'll formulate a plan between now and then. The next blog is likely to be the one which outlines my plans.

Take Responsibility

In a previous blog post, I wrote about three key learning points. In this blog, we'll look at the third of those...

Key learning point 3: Don't leave revision to pupils, and plan in activities to revisit key information.

I've worked in schools in challenging circumstances in Inner City Leeds for 15 years, and I love it. I love the relationships, I love the challenge and I love teaching. I am, however, a realist, and I wouldn't expect many of my pupils to be able to direct their own ongoing revision so that they were improving. They could definitely revisit ideas, but they'll go for the ones that make them feel good about themselves, or they'll go for the ones that everyone found hard, and this approach to learning and revision isn't one that will maximise the impact.

I'd be willing to bet that less than 10% of pupils I've taught over the last 15 years have had an area set aside for keeping their school work organised, never mind planning out their 'ongoing revision and study'.

With that in mind, I space out their learning for them and provided opportunities for retrieval so that ideas stick and learning occurs. Rome wasn't built in a day, and in a similar way, learning happens over time rather than in time. This means repeated exposure to ideas that pupils have succeeded with is important, so that learning occurs. So important that, following an assessment, we can't just expect pupils to be self-directed to revisit the things that they don't understand and we need to recognise that these areas of need should be revisited regularly.

We should take responsibility and present them with spaced repetition opportunities to improve the likelihood that learning DOES take place following the identification of this idea as an area of need from the assessment and following their first re-exposure in lessons.

To help with this, I use this spreadsheet. Adding topics into the 'Completed Learning' column will map the ideas forward in time by a day, a week, a month and three months (ish...) and provide a reminder to revisit these ideas with your classes.

In the next blog, I'll outline my plan for feeding back Year 10 mocks, taking into account others' responses to the original tweet.

Feedback in Moderation

In a prior blog, I wrote about three key learning points. In this blog, we'll focus on the second of those... 

Key learning point 2: Don't go through the whole paper. Focus on key questions.

This learning was in response to going through all of a paper, and whilst there's probably some merit to going through the whole thing I think you'll get more out of giving feedback in moderation.
Exercise is good for the body, until you push the body too far and cause injury (as I type this, I can feel my knees creaking), and the same can be said for feedback. Too much feedback can overwhelm our learners, but can also dilute the important messages that we're sending. We can be more impactful by focusing our feedback on the things that we need pupils to take away.

In response to key learning point 1, I stated that I find three areas of need for each pupil and try to maximise my gains by getting as much as possible out of each question I choose. In an ideal world, I'll be able to only revisit three questions, but it's more likely that we'll look at ten or so due to gaps in pupils' learning.

Only looking at three key skills would be great, but looking at ten or so questions rather than the 20 from the test still prevents pupils from becoming overwhelmed during our 'assessment feedback lesson', and we know that our feedback will garner more attention from pupils as we narrow our focus.

In the lesson, pupils will see the correct solution to the question from the assessment and be presented with a question with both a similar surface structure and a similar deep structure to complete, to develop near transfer. Follow-up questions could go on to alter either of these structures, but the likelihood is that we want pupils to develop confidence with these ideas without alteration as they've recently struggled with this idea in the assessment. We'll repeat this for the ideas that we're asking pupils to re-do.

They'll then work on three of these ideas identified from their assessment, independently, in order to give further attention to ideas that they've recently shown a lack of understanding over (in the 'test') and had success with (in our example-problem pairs).

This is how this is done:

I always mark tests and then enter the scores in a QLA spreadsheet. A template is here, and I recommend downloading it as .xlsx rather than using it within Google Sheets.

On the first tab, enter pupil names, and enter their scores. You can see the percentage score for the class across the bottom, and this is great for identifying needs. It has to be said that this data must be used in context, and specific issues should have been picked up during marking.

The second tab has a list of scores, and only scores, and for graded exams normally looks up grades and assigns these rather than having to type them individually.

The third tab is labelled 'Next Steps', and this requires further use of the first sheet. These are the needs identified for each pupil. I do this by highlighting cells, highlighting the question number at the top and then typing the question number in the 'NS' boxes at the end. This assigns individual questions for pupils to work on, and highlighting keeps track of the questions I need to go through in full as well as reminding me to write a new question in this third tab. Writing questions on this tab will mean that individualised questions are automatically assigned to pupils on their printouts when I go on to the fourth tab.

The fourth tab brings this all together. The mark for each question for each pupil is RAG'd when their surname is entered, and the identified questions for them to work on are on the same sheet, ready to be printed and given to them for completion independently of each other, to check that we've fed back has landed for each learner.

It's then incredibly important that pupils are given further retrieval opportunities, so they'll be written into my medium-term plans for starters/settlers/Do-Nows/homework as retrieval practice. They couldn't do it to begin with - revisiting the idea once in a feedback lesson when emotions are running high isn't going to solve the problem. This brings me back to Josh's tweet from the last instalment:

Yeah absolutely. I think when identifying the questions to go through we should ask ourselves ‘do they fundamentally not understand this? Or are they close to getting there?’

The latter are for review lessons, the former are for your medium term plans.

— Jshm (@jshmtn) January 8, 2023

The way that I've done this over the years has varied, but I've settled on this spreadsheet.
I write my medium-term plans in the 'Intended Learning' column and update this based on how well the lesson has gone.
In 'Completed Learning' I type a few words to describe the content covered. This is then mapped forward by one day, seven days, 31 days and 90 days.
This then forms my list of topics that need to be dropped in for retrieval opportunities. The dates don't always line up, so I keep track by shading 'covered' topics and adding them to the 'Completed Learning' column where there isn't a lesson in the event that I want it to come up in the rotation again.

Focus on Feedback, not on Performance

In the last blog, I wrote about three key learning points.

In this blog, we'll look at the first of these...

Key learning point 1: Try to give feedback without returning pupils' scores/grades/papers. This removes the emotional response to a score/grade, and maintains pupils' curiosity in the questions that the teacher has identified to go through.

When this was suggested to me, it sounded weird.
When I first tried it, it felt weird.
But I tried it and I've observed a trainee do the same. 

I'm sold. Pupils are more open to corrections, they're more curious about the answers, and they're more engaged in the feedback.

Logistically, this is how I've gone about feeding back.

1. Pupils complete a 'test', and I mark it before the next lesson. I've always valued a speedy turnaround, and like to plan my 'assessment lessons' so that I have time to mark and prepare an 'assessment feedback lesson' for the next lesson on the timetable.

2. I identify three areas of need for each pupil and set them follow up work on these questions. I try to make sure that each area of need catches as many pupils as I can, to minimise my work load and maximise the impact for the minimum effort, and also that the area of need isn't too far from their current level of understanding.
I do this by completing a QLA document (template here) and identifying the greatest areas of need by looking for the topics I expect 100% on, and seeing where gaps exist. I'll highlight the question in yellow, type the question number in at the end as their 'Next Step' and create similar questions in another sheet which are then sent to each pupils' individual printout. 

3. I write example-problem pairs for the areas of need. The example that I use is always the test question, as that develops motivation - pupils recognise it, and they want to know the answer. The problems have a similar surface structure and similar deep structure, as my aim is to develop near transfer. Additional problems can vary in terms of the surface structure or their deep structure, but my main aim is to develop near transfer with this idea for those that didn't get this right in the 'test'.

4. We run through the example-problem pairs as a class with whole-class response systems. Pupils answer questions on mini-whiteboards and hold them up on my say-so, or we use multiple-choice questions to draw out misconceptions. At this stage pupils don't know what they scored on each question, so engagement is maximised. Pupils are responding on mini-whiteboards or with their fingers, so participation is maximised.

5. The pupils complete independent work on their three areas of need on their individual printouts. This way, their independent work is focused on their areas of need, not on the simple stuff that others got wrong, or on the stuff they found way too hard.

6. It's important to keep this stuff 'live' in the classroom, with opportunities for retrieval prepared in future lessons. With this in mind, the topics are added to my medium term plans as settlers/starters/Do Nows/homeworks with retrieval in mind.

I've mentioned this in a previous blog, and it's definitely important - emotions run high in 'assessment feedback lessons'.
Either there's a feeling of contentment, of relief, after successes that leave the learner feeling relaxed and like they don't need the feedback because they've 'smashed it'. Not giving them their 'test' back avoids this lack of focus and maintains their motivation.
The other side of the coin is that the pupil has 'failed' the 'test' and is now questioning their self worth, and the point of working hard if they 'keep doing crap in tests'. Not giving them their 'test' back until later delays this emotional response, maintains motivation and increases participation in the example-problem pair part of the lesson.

Karen Hancock had this to say on the issue:

I rarely put marks on the students' work. I mark directly into a spreadsheet and then I'll either do the paper live for them to mark or we create video solutions and they watch those.
We usually have a shadow paper so that they can demonstrate they can do the questions now.

— Karen (@karenshancock) January 8, 2023

This strikes me as being similar to what I've done, in that there's no focus on performance, but supercharged. Completing a QLA is something I do anyway, so I think the only difference is a lack of writing on scripts. I would, however, be wary of marking live, as there can be so many slight differences that can score marks on a five-mark question (for example) that I'd feel as though I'm losing control of the lesson. The video solutions is something I could go with - my current thoughts are to share worked solutions (on something like Google Classroom), but recording me writing these and talking about my thought process would be useful to pupils, I expect.
Using a shadow paper is an excellent idea, too, and something we do, but I think that their use could be more impactful too. Maybe they don't need to complete the entire shadow paper, but I can pick and choose those questions based on pupil need.

Nicky Gale replied with this tweet:

I work in a sixth form college. We do a video solution for all the questions and a shadow paper. In the lesson, students watch videos and correct any they didn’t get full marks for, then try the similar questions. We then have an individual conversation with them and set targets.

— Nicky Gale (@nicnaks79) January 9, 2023

This sounds excellent, and where I'd like to go with Karen's suggestion, but I don't think I could do this for 30 pupils in a lesson. I'd love to handover responsibility to the pupils, to watch the videos, to correctly answer the questions, and then work on the similar questions, but I'm not sure that I'd get what I want out of some of my pupils. 
I feel like I need to direct this, and this is what Helen Scott had to say:

Pick 2-3 "should" have been able to model, DIRT questions after this. Ticks the policy box.
We all know the real learning comes from the retrieval and learning over time when they see the question again over starters and then in a quiz.

— Helen Scott (@HelenScott88) January 9, 2023

I feel as though 2-3 questions wouldn't cover everything from the paper in terms of pupil need, but I'm fully alongside the message of the tweet. I think choosing 2-3 per pupil, which might become 8-10 over the class is achievable and acceptable, with pupils working on the shadow questions for those 2-3 questions in terms of the 'feedback cycle' expected within schools, with (as Helen said) the real learning coming over time as pupils are re-exposed to the idea (using questions with a similar structure in the short term, and altering the surface or deep structure as we go).

In terms of how we go about this, Josh tweeted:

Yeah absolutely. I think when identifying the questions to go through we should ask ourselves ‘do they fundamentally not understand this? Or are they close to getting there?’

The latter are for review lessons, the former are for your medium term plans.

— Jshm (@jshmtn) January 8, 2023

This seems like an important point to end the blog on. Just because a pupil has got a question wrong doesn't mean that they don't understand the idea. On top of this, just because a pupil got a question correct doesn't mean they do understand the idea. Sometimes a pupil guesses, sometimes they fumble through their working, taking leaps and getting to the right answer despite not understanding the journey, and sometimes pupils struggle to apply their understanding in a high-pressure situation. 

Be wary of assessment data.

Assessment Feedback Lessons: A Post-mortem

I once read or heard, that assessment regularly comes across as a post-mortem, identifying issues long after they became issues. What we really should be doing, though, is using assessment as a wellness check, an MOT, to identify issues before they become issues. Rather than looking back on successes or failures, we should be using assessment to shape future successes.

I think it's always useful when thinking about making changes to first consider what we're aiming to move away from. Some things might be worth saving, whereas others can almost certainly get in the bin.
If you're reading this, you might be looking to move away from a 'traditional assessment feedback lesson', and you might find that this follows a pattern similar to:

0. A class has sat a 'test' and the teacher has marked it between lessons.
1. The teacher gives back the test papers and scores to pupils.
2. The teacher goes through the answers to the test while pupils annotate the paper with correct answers.
3. The teacher suggests that pupils use this later in the year as revision.

Here are a few reflections on these lessons:

I feel as though step 1 seems like the most sensible thing to do following an assessment. It feels natural. Pupils sat the assessment, so we should let them know how they did. Allow them to find out by looking at their papers, and re-focus their attention before we move on to addressing identified issues.

From my experience, I've found that handing out papers in that way results in one of two feelings:
Pupils are happy with their score/grade, or pupils are disappointed with their score/grade. I feel as though any middle ground is minimal.

Suppose that you're happy with your score - you'll want to tell everyone, you'll brag about it, and you'll want to know where you sit in terms of rankings within the class. You're probably feeling like you'll be towards the top, so you're looking at the scores/grades of the perceived 'clever' kids in the class and comparing yourself against these, hoping that you're the one who's on top.

If you're unhappy with your score, the disappointment is likely to cause you to switch off, and you're likely to want to know how everyone else did. There's a desire to see if you're an anomaly, or that this was a 'hard test', and that everyone did poorly.

In both cases, the emotional response is huge and leaves pupils in the wrong frame of mind for growth. In the case of the happy pupil, as soon as you start to go through the paper they're wondering 'Why am I doing this? I answered these questions well!' and the unhappy pupil is thinking 'What's the point?! This always happens!'.

Key learning point 1: Try to give feedback without returning pupils' scores/grades/papers. This removes the emotional response to a score/grade and maintains pupils' curiosity in the questions that the teacher has identified to go through.

Step 2 seems as reasonable as Step 1. Pupils didn't have the correct answers, so we should tell them what they were. We've all deemed this 'test' to be important, so we're hoping that giving them the right answers will trigger something, and they'll remember next time.
Pupils are taking a different coloured pen and writing the correct answers on their scripts, but what is this actually achieving?
Copying down requires little cognitive activity, and Willingham suggests that 'memory is the residue of thought'. What other activity is lacking in active thought as much as copying down?

Suppose that the test was out of 80, and the scores range from 30/80 to 57/80. That's 27 marks that the lowest attainer didn't achieve, that the highest attainer did, and while you go through those 27 marks, the highest attainer is giving minimal attention to the lesson. The teacher is going through the questions one by one, focusing their efforts on questions that some pupils got wrong, but the ones who didn't get them wrong know that they didn't get them wrong. The highest attainer didn't make these mistakes, so their attention is elsewhere.
We then get to question 8 - the first question that the highest attainer didn't get right - so they need to give attention to their mistakes and to the correct method, but because the first 7 questions didn't need their focus, they're not in the frame of mind to give attention to their mistake and learn from it. They're still grinning from ear to ear as they're top of the class.
Now scale this across the class, and you've got pupils across the room giving minimal attention to the answers that the teacher is sharing as they don't need them, and being unable to give attention when needed.

Key learning point 2: Don't go through the whole paper. Focus on key questions to maintain pace and attention.

In Step 3 we might see the teacher hand out pupils' folders and put their assessments away for safe-keeping, or they might ask the pupils to put them in their bags to take away and store with their school work at home for revision purposes.
In my experience, the test put away in the folder never comes back out, and pupils don't have a filing system for their school work at home, other than the bin. The test is almost never revisited, despite the best of intentions. The thing is though, without spacing out revisiting the corrections we just made we're unlikely to learn from these mistakes, rendering the whole activity a little bit pointless.

Key learning point 3: Don't leave revision to pupils, and plan activities for pupils to revisit key information within your medium-term plans.

In the next instalments, I'll look at ways that we can amend the 'traditional assessment feedback lesson' based on the three key learning points here, as well as a few other ideas from replies to the original tweet I put out.

Thinking About Assessment Feedback Lessons

It's been a while since I've blogged, but that isn't to say I haven't been doing a lot of thinking. I've been busy in the classroom and working with the Complete Mathematics team, and to say that the last two years have been incredible in terms of CPD would be an understatement.

At the beginning of the school year during a whole-school staff meeting, our department got on to the topic of feeding back on assessments. This has led to a bit of a journey on my part.
Initially, the discussions centred on teachers giving tests back, going through the test, and then directing pupils to revise in their own time. I gave this some thought, revisited my past experiences, and asked 'Can't we do better than that?'.

I left it a few months, and wanted to give this some deep thought, so I tweeted:

#mathstlp I think this might be a big one.

Assessment feedback lessons.

You know, you've marked the test, and you need to give their scores back, report attainment levels and look back over their work.

What do you do?

— Dave Taylor (@taylorda01) January 8, 2023

I haven't yet had the chance to get into it, but I think now is the time! The next few blog posts will dive into this in a little more depth.

When searching 'define:assessment' Google suggests some similar words to assessment.
Evaluation, judgement, gauging and rating are a handful.

This feeds into two of the big questions that I've been asking myself about assessment are:
1. Why do we assess pupils?
2. How do we best give feedback on the assessment to pupils?

Assessment happens at all times in the mathematics classroom as teachers gauge how well (or not, at times) their lessons are going. Asking a question is an assessment, providing an activity for pupils to complete is an assessment and asking for pupils to respond on mini-whiteboards is an assessment. Further along the spectrum, having pupils sit in silence for one hour working in an A4 booklet is also an assessment. Assessment is a key part of our lessons, so it's important that we harness it.

We can split assessment into two types - formative, and summative.

I think of formative assessment as helping us to decide the next stage of the lesson and the direction that our learning episode continues.
Summative assessment, on the other hand, gives us an opportunity to compare the class against each other or to compare each pupil against a pre-determined grading system in the hope that learning has occurred and progress had been made.

A colleague recently spoke to me about the feedback he'd given a trainee teacher. The trainee had taken responses from pupils but had failed to act on them. When he asked 'Why did you ask for answers from the pupils?' the response was along the lines of 'That's what teachers do - they ask questions'.

I'm of the opinion that there's very little point in asking a question if you already know the answer, or if you're not going to do anything with the answer.
Sometimes lessons can feel a little bit like a game show - only the correct answer is important - but it's vital that we allow pupils to feel safe in giving incorrect answers or saying 'I'm not sure' and 'I don't know' so that they understand that school isn't one big quiz in which they're expected to win the grand prize.

In my opinion, formative assessment (AfL - assessment FOR learning, or 'responsive teaching') provides the cornerstone of effective mathematics teaching and should be present at all times to ensure that we are responsive to the needs of our pupils.
This might be a reaction to work which is produced in exercise books, with the teacher altering the course of the lesson based on what they observe as they circulate, or could be in response to whole-class responses on mini-whiteboards, or pupils answering multiple-choice questions using their fingers or lettered/coloured cards. The important part is that learning episodes are directed by the information we collect and that it's OK to throw the plan out of the window if it isn't serving the needs of the young people in front of us.

I feel that this path is well-worn, and the conclusion of these discussions seems to be strong: if you're basing your decisions in the classrooms on the responses of a confident few, then you're doing the rest of the pupils a disservice.
You don't take a group abseiling down a cliff face based on one person fastening their harness securely without checking the rest, and you don't move on to new learning because one pupil has 'got it' without checking the rest.

The part of assessment that I'm more concerned with at the moment is making 'summative assessments' (or 'tests') more formative. I'm thinking about end-of-unit assessments, where pupils get a score and we move on, or Year 10/Year 11 mocks, where pupils are given a grade and we/they move on. I want to be more impactful when responding to such assessments, and I want the time taken to leave an impression on pupils' knowledge and understanding, rather than being the messenger about the grade they've scored and the catalyst of significant disruption where the pupil attempts to identify their place in the pecking order.

I'll be using forthcoming blogs to digest responses to the prior-mentioned tweet and think about the best way to increase the impact of these lessons, testing it out on my Year 10s as they complete their mock examinations over the next month. My intention is to share the process I've developed, the outcomes in the classroom, and whether I think it has improved my impactfulness or not. 

Hopefully, you'll take something from it!