Developing Student assessment literacy though Peer assessment

An essential part of higher education is the transformation from compulsory teacher-led education into life-long learning. To be able to do this we need to both enable and empower students to develop their work without relying on teacher feedback as the only source of guidance when trying to improve the quality of work and developing knowledge and skills.

In fact, as argued by David Nicol in his 2021 paper “The power of internal feedback: exploiting natural comparison processes”, students already have the ability to generate internal feedback by making comparisons. Thus, as Nicol describes, in order to unlock this the teacher simply needs to turn the natural comparisons into formal and explicit comparisons using written reflection. This gives “permission or power” to students so that they can flourish and develop autonomously.

Students have a very narrow view of what feedback is and often think of it as written one way comments from a teacher which may or may not enable them to improve (Boud and Molloy, 2013b). When students move into higher education as they are settling into a new less teacher-led style of learning they need to develop the skills to be able to critically reflect and improve and redraft their own work. Right from the start of a course peer review work creates an opportunity to “develop the capabilities to operate as judges of their own learning” as described by Boud and Molloy (2013). This also set the tone for their university education and shows it is markedly different from previous teaching.

Connecting students to each other has become increasingly difficult, particularly during the pandemic when students were disconnected from their community of practice (Wenger 2002). Students can struggle with social anxiety and the vulnerability which needs to be embraced to form quality relationships and connections with peers (Brown 2020). Students reported feeling anxiety about connecting to others and having missed out on developing academic skills during the pandemic. This can be overcome with the use of anonymous peer to peer review where students can get the benefits of seeing their peers work without the anxiety of feeling judged.

A formative assignment was set for students just one month after the start of their undergraduate course at university. Whilst this meant a steep learning curve for many students it did empower students to increase their own belief in their work. Rather than trying to compare their answer to a ‘model solution’ as they are used to in school, they are comparing to other students work, some of which is better than their own and some of which is worse. Completing this work so soon into their university life sets the tone for their university education and shows it is markedly different from previous teaching.

The key stages in the process were as follows:-

Stage Students Lecturer
Set up   1. Share resources on essay writing
2. Share ‘good’ essays as pre-emptive exemplars
3. Motivate students to complete the exercise and the benefits of completing the self reflection questions.
4. Set essay title (and suggested associated reading)
5. Set comparative criteria using questions.
Student work 6. Submit essay Offer office hour drop in support
7. Reflect on other students essays, writing out responses and submit this reflection
Debrief   8. Flag any essays with high turnitin scores to the students who submitted them.
  9. Pick out good essays from those submitted to share with other students
  10. Run a workshop to answer questions, give dialogic feedback, ask students to share what they have learnt and what their challenges were. Students should bring along their essay, reflection and redraft.

This is a video debriefing of the dialogic feedback workshop for recorded for students who might have missed it:

The outcomes were:

  • Students responded in the evaluation summary that they agreed feedback that they received on their work was helpful (an average score of 4.15 out of 5) compared to the previous year average of 3.85 
  • Students who engaged with this had higher grades. Those that did the essay, submitted the reflection and a further revision essay (all with feedback which was solely generated by the student) had an average of 28.22 more marks in the exam. 
  • These conclusions are likely to be biased, given that stronger students would generally be more likely to engage with all parts of the assignment. For this subject there is also a multiple-choice assignment which can be used to gauge on average how much ‘better’ the students are as shown. On this basis the 30 students who submitted the essay, reflection and revision essay had 11.43 more marks in the exam and the 193 students who submitted the essay and reflection (not the revision essay) had 2.32 more marks in the exam.  
  • Strikingly over half (56%) of the reflections submitted noted that students had learnt how to improve their style (eg. Increased critical thinking or for example “I feel that I over-explained [the characteristics], rather than keeping it concise”). This is a very difficult aspect of essay writing to teach. As noted above students often want a model answer or precise way of writing an essay, whereas lecturers want students to develop their own writing style. 
  • A third of reflections noted that they had learnt to add in real-life examples or illustrations which is exactly what it required in an essay like this. This was explained to students as necessary, but many students had not learnt this until completing the peer review exercise. 
  • 27% of reflections explained that students had learnt assessment literacy or appreciated the fact that there were a variety of answers possible. Again, a hard skill to teach in any other way. 
  • 17% of the reflections reviewed noted that students had learnt to have confidence in their essay writing skills or realised their writing skills were not as bad as they first thought. 
  • Finally, 16% of reflections stated that students had learnt how to reference or use evidence to support an argument. Again, this was taught in class, but not absorbed until the peer review exercise 
  • Worryingly two students noted that they “I may also consider including graphs or diagrams to enhance my essay and to help me explain my points better.” Which is definitely not recommended to be included in academic essays! 
  • Going forward I will use a checklist to help inform students what they are looking for in good essay writing (ie. not diagrams or graphs!) 
Evaluation / Student Feedback

One student noted that:-

“I feel like I could have taken a little bit from all of the essays I read and therefore making my essay even stronger.”

And another that

“I feel like I’m seeing a new light on how to write an essay. It doesn’t have to be really good language-wise, rather to answer all the questions asked and include general knowledge here and there in order for it to be more relevant to a more diverse audience/ readers.”

Benefits
  • Re-usability
  • Helps students gauge how well they are doing.
  • Develops students’ awareness of their knowledge 
  • Improves understanding of assessment criteria
  • Students’ understanding of marking criteria
  • Students’ engagement and understanding of assignment criteria
  • Engages students in evaluating performance
  • Uses assessment to promote interest and engagement with topic
  • Helps students understand, assimilate and engage with feedback
  • Delivers timely, meaningful and effective feedback
  • Supports peer review and revision/understanding of content
  • Facilitates/supports peer learning
  • Encourages original writing and responsible use of source material
  • Facilitates consolidation of students’ understanding of materials, encouraging better performance, self-testing and autonomous learning
  • Develops independent study skills
  • Develops students’ capacity to be active, critical and independent learners
  • Raises interest, focuses students’ attention on a topic
  • Encourages continuous learning and progression
  • Encourages motivation and self-responsibility
  • Supports students to focus effort and devote sufficient amount of time to the most important aspects of the course
  • Helps students acquire the core knowledge which is necessary foundation to more advanced learning and understanding of their subject,
  • Helps students develop transferable study skills, particularly in independent research, essay writing and the avoidance of plagiarism.
Top Tips
  • Don’t worry if you don’t have examples of ‘good essays’ at the beginning. You can pick out good examples from the essays submitted and share them (anonymised and with permission), plus once you’ve done this once you can use the good essays from the previous year.
  • If you’re concerned about peers writing comments (for example on my large cohort I was worried about the comments peers might write), skip this step – students can learn a lot from reading other student’s essays and writing feedback comments for themselves.
  • Submitting the reflection is important (you don’t need to read submissions), nudging and reminding students to do this is a good idea.
  • Get in touch with me Jennifer.Rose@Manchester.ac.uk to use my templates, layouts and documents which I can share with you.
References

Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: The challenge of design. Assessment and Evaluation in Higher Education, 38(6), 698–712. 

Carless, D. (2016) Feedback as Dialogue. M.A. Peters (ed.), Encyclopaedia of Educational Philosophy and Theory, DOI 10.1007/978-981-287-532-7_389-1 

Moss, P. A. (1994). Validity in high stakes writing assessment: Problems and possibilities. Assessing Writing, 1, 109–128. doi:10.1016/1075-2935(94)90007-8 

Nicol, D. McCallum, S. (2021) Making internal feedback explicit: exploiting the multiple comparison that occur during peer review, Assessment and evaluation in Higher Education, DOI: 10.1080/02602938.2021.1924620 

Nicol, D (2021) The power of internal feedback: exploiting natural comparison processes, Assessment & Evaluation in Higher Education, 46:5, 756-778, DOI: 10.1080/02602938.2020.1823314 

Pollitt, A. (2012) The method of Adaptive Comparative Judgement, Assessment in Education: Principles, Policy & Practice, 19:3, 281-300, DOI: 10.1080/0969594X.2012.665354 

Race, P. (2014). Making Learning Happen. Sage London. 

Sadler, D. R. (2010). Beyond feedback: Developing student capability in complex appraisal. Assessment and Evaluation in Higher Education, 35(5), 535–550. 

Sambell & Graham (2020), “We need to change what we’re doing” Using Pedagogic Action Research to Improve Teacher Management of Exemplars, Practitioner Research in Higher Education Journal, Vol 13 (1) pages 3-17. 

Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 34(4), 273–286. https://doi.org/10.1037/h0070288 

To, J. Carless, D. (2016) Making productive use of exemplars: Peer discussion and teacher guidance for positive transfer of strategies, Journal of Further and Higher Education, 40:6, 746-764 DOI:10.1080/0309877X.2015.1014317 

Wenger, E. McDermott, R. Snyder, W (2002). Cultivating Communities of Practice. Harvard Business School Press, Boston Massachusetts. 

0 Comments

School: Alliance Manchester Business School

Discipline: Accounting and Finance, BMAN10501 UG first year

Academic: Jennifer Rose

Cohort Size: 660 students

Themes: Teaching ideas, learning socially, enhancing learning with technology

Ref: 064