QLAs – How to Use Them Well

Question level analysis (big old spreadsheets which look very colourful and contain data about how every student did on every question) are time consuming to produce and, by themselves, are not useful. Data managers and trust-leads might insist on them but they are nothing but a superficial exercise until they are used purposefully. This post will explore some common pitfalls and what should be done instead to make the most out of these documents.

1. Don’t forget what the question was. It’s typical to give a brief name to the columns that refer to the broad being assessed. When you see a big column of red under a question title do not just assume everything about that topic needs to be retaught. Go back and look at the exact question that was in the paper then look at a select group of student responses. Was there a common misconception that needs addressing? Was this question quite a complex reasoning question and the subject knowledge on the actual topic is fine? Was this combined with one or two other topics which is where the weaknesses actually lay?
2. Focus on the highest leverage areas. When choosing which columns to focus on, the ones with the most “red” are not necessarily the best. Near the end of the paper it would be typical for questions to be harder. Where possible, compare the performance with a nationally representative data set. From here you should focus in on where the biggest differences from the norm are. That way you know you are focusing on marks which are more accessible.
3. Teach a few things well, not everything poorly. It can be tempting when presented with data like this to what to go over every single gap. Produce 30 individualised worksheets that all focus on different weak areas for students. The issue here is that without being able to give them all 1-1 tuition (giving them model answers does not count) they won’t be able to learn from this experience. It is your job when reviewing whole class assessments to do the greatest good for the greatest number, teaching those things well is better than teaching everything poorly.
4. Praise the good. As teachers we are always looking for the gains, our eyes will immediately be drawn to the red. Take time out to praise the class on their areas of strengths. Highlight the positives and, where possible, show them the progress they’ve been making over time. Tests can be stressful and saying “well done” is not just something they’ve earned, it’s the right thing to do.
5. Be proactive as well as reactive. This data is going to be produced as a result of the teaching this class had. As well as being reactive in the moment to this class don’t forget to be proactive as well. Making any alterations to the curriculum so that future classes don’t fall into the same trap as your current class is going to ensure future students benefit from this knowledge.
6. Quick-wins vs Reteach. There will be some minor errors that some students made and some quick-wins which can be easily addressed but may need to be revisited frequently. This will also show up some huge gaps in their learning which may need multiple lessons to address. Split these up accordingly. Have a group of things which will require substantial re-teach time and a group of smaller things which can be addressed at the starts to lessons.
7. Don’t focus on the specifics of the question. When a question has bene identified as one that has been underperformed on then make sure you do not address the specifics of the question but rather the root cause of the mistakes. It is pointless going through that specific question in detail and then giving students time to practice almost identical questions just with different numbers. You need to diagnose the gap in their understanding and teach more holistically. That exact question won’t come up again, make sure you fix the cause, and not just the symptom.

I think QLAs can be an incredibly powerful tool but it’s important to remember that is all they are, a tool. A tool can be used well and it can be used poorly. If you have any other tips to add please share them in the comments.

Also, if you’re reading this because mock-season is coming then you may also be interested in this post about marking papers to ensure the data is reliable in the first place.

I’m always interested in what people make of this so please feel free to comment with thoughts, questions or incomplete musings. Follow this or my Twitter account Teach_Solutions for similar content in the future. Also, check out the rest of this site, there’s some good stuff knocking about the place.