Implementing Peer Review for Undergraduate Assessment

Introduction

This case study describes assessment innovations for Transversal skills for 3rd year computing students immediately before 6-month work placement. This module is a condensed, intensive 3-week module, assessed through 100% continuous assessment with a variety of components; • Presentations (individual and group) • Technical and non-technical reports (individual and group) • Screencast/podcast (group) This is a challenging module for students who prefer computing subjects and there is a short time period for assessing and providing feedback on transversal skills. Key features of assessment design included; Need to be fair and inclusive, especially as some assessment components are public; Student agreement on assessment rubrics and leverage educational technologies and peer feedback to as key components of assessment design.

What was the learning and teaching challenge you faced?

It is difficult to provide feedback and assess students in a timely manner due to the intensive nature of the module.

What was done?/What did you do?

Educational technologies were used to facilitate various aspects of the assessment process. Moodle (a VLE) was used to host the information (exemplars and rubrics) and for student submissions. For each of the assessment components a rubric and exemplars were provided to the students. The content and the weighting of each of the rubric components were discussed and negotiated with the students. For the individual report, feedforward and exemplars were used to explain clearly to the students what was expected of each assessment component. Students were asked to review the exemplars and critique them. Did they think the exemplars poor, fair or good pieces of work? This activity helped students to apply the assessment criteria to actual (prior) student submissions – something that they had not done before. Students wrote a 500-word draft report and used online linguistic analysis tools to help them analysis their reports for complexity and intelligibility. A peer-review process was used for the individual report. Students brought in an anonymised draft of their report to class, where it was reviewed by a peer. Students were provided with the rubric for the assessment and shown how to provide constructive feedback. It was important to scaffold this process as students had no prior experience of this type of activity and many were slightly wary of the process. Reviewers then posted their anonymous comments to an online bulletin board (Padlet) when they were reviewed in real time in the class. This provided the lecturer with a chance to check on the feedback provided and also for the students to see commonalities in the feedback. The issue of plagiarism was also discussed and the students were made aware that an automatic plagiarism detection was in place for all their submissions. For the student presentations, the lecturer was assisted by another assessor and the rubrics were important in this context. The assessors assessed at least one student together and agreed on a mark to ensure inter-rater consistency. Moodle was used to make the marking process and provision of timely feedback to students more efficient. Both holistic and analytical rubrics were used on the module and varied based on the component being assessed.

How did it work for you?

The peer review process worked well both from a logistical and a pedagogical point of view. Students provided a brief reflective comment on the peer review process and these comments generally concurred with the literature in that found the feedback beneficial and they felt they gained even more by reviewing another person’s work as it helps them review their own work. A peer review process of the screencasts/podcasts was planned but did not work due to access issues and there are still problems with timely feedback. There is plenty of room for improvement and every year there are some new innovative components used. Overall, the suite of approaches and tools worked well in this challenging and non-standard module.

Tips for implementing this practice

✔ Make sure that the instructions and rubrics for each assessment component are clear and check that the students understand them.

✔ Provide exemplars as students find them beneficial.

✔ Explain how the peer feedback process will be carried and establish ground rules to ensure mutual respect between reviewers and reviewees.

✔ Use a VLE to facilitate the assessment process.

✔ Accept that things will go wrong, don’t panic and remember that there are always workarounds and solutions available.

Reflections and future plans?

This process supports academic integrity in that it provides honest feedback to students, they trust that the rubrics will be fairly applied, they give receive and receive respect from their fellow students and they learn to take responsibility for their own learning. In their academic studies to date, they have rarely assessed their own work and/or the work of others from the perspective of the assessor and this has been an eye-opening experience for many of them. They realise that there is a logic to how assessments are graded and learn have a better understanding of what is required from them.

The most time consuming part of this approach was the background research involved. After are review of the literature and researching which online linguistic tools would be suitable, it was a matter of carefully explaining the process and approach to the students so that they understood what, how and why we were doing something. The main components took place during class time and it was a very efficient mechanism of carrying out timely and useful feedback for students. It had not been possible in the past to provide this type of feedback due to the time constraints and the number of students involved. Implementing new assessment features can be challenging. Student buy-in is important and maintaining a respectful atmosphere is key component of the process. Ensuring that students work to the same timeframe can be a challenge as some students may rush through the process while others spend time on the details. Practical matters (e.g. ensuring that each student brings in a hard copy of their draft) may seem trivial but are important to ensuring that the process works as smoothly as possible. Future changes (e.g. using an online copy of the draft report) may be considered, but the logistical issues of anonymity will have to be carefully considered as this is a key element of the success of the process to date. Overall, if the lecturer has a good relationship with the students and they understand the motivation behind using them, they will work collaboratively with the lecture and it can result in benefits for all.

References:

Dochy, F.J.R.C., Segers, M. and Sluijsmans, D., 1999. The use of self-, peer and co-assessment in higher education: A review. Studies in Higher education, 24(3), pp.331-350.

Falchikov, N., 1996. Improving learning through critical peer feedback and reflection. In Different Approaches: Theory and Practice in Higher Education. Proceedings of HERDSA Conference. Lundstrom, K. and Baker, W., 2009. To give is better than to receive: The benefits of peer review to the reviewer's own writing. Journal of second language writing, 18(1), pp.30-43.

Nicol, D., Thomson, A. and Breslin, C., 2014. Rethinking feedback practices in higher education: a peer review perspective. Assessment & Evaluation in Higher Education, 39(1), pp.102-122.