PR 4 Digital Peer Assessment

Introduction

PR 4 is one of the project results where a digital assessment is being developed. It encourages peers to become active assessors of video-recorded skills using a digital assessment toolkit including a rubric and online resources.

Peer assessment scoping review

ISU (Georgia, lead for IO4) together with members from the Teaching Enhancement Unit from DCU have completed a scoping review on peer assessment. The results showed that peer assessment provides learners with a better understanding of programme objectives and encourages them to be part of a more collaborative learning environment (Søndergaard & Mulder 2012, McGarr & Clifford 2013). In order for peer assessment to be successful, learners need to have clear instructions and grading criteria (Adsanatham 2012). One of the main criticisms in peer assessment is grade inflation, which means that learners at times may grade their peers more generously than the lecturer (Sridharan et al. 2019). To test this notion, the team from Georgia conducted a pilot project at their

home site at ISU.

Adsanatham, C. 2012. Integrating Assessment and Instruction: Using Student-Generated Grading

Criteria to Evaluate Multimodal Digital Projects. Computers and Composition, 29 (2), pp. 152-174.

McGarr, O. and Clifford, A.M. 2013. Just enough to make you take it seriously "exploring students"

attitudes towards peer Assessment, Higher Education, 65 (6), pp. 677-693.

Søndergaard, H. & Mulder, R.A. 2012. Collaborative learning through formative peer review:

pedagogy, programs and potential, Computer Science Education, 22:4, pp. 343-367, DOI:

10.1080/08993408.2012.728041

Sridharan, B., Tai, J. and Boud, D. 2019, Does the use of summative peer assessment in collaborative

group work inhibit good judgement?, Higher education, 77(5), pp. 853-870.

Development of a peer assessment toolkit

To begin the process of developing a peer assessment toolkit, ISU (lead) together with project

partners drafted a paper-based peer assessment rubric (see Annex 2) and tested it on teachers and

learners in Georgia. Teachers and learners were asked to assess video-recorded skills of their peers,

grade and provide feedback on the rubric. The results were analysed and compared. The video-

based skills related to 3 topics such as:

• Hand Washing
• Venepuncture
• Sterile glove application

as per AHEAD protocol. Main results were discussed with the larger project team and include:

• teachers and learners enjoyed being involved in the assessment process
• learners provided less feedback in the boxes provided, compared to teachers
• some technical issues in the video-based recording e.g. camera angles did not allow for full assessment at times
• Unicam functionality into Moodle could not be tested as Unicam not installed yet in ISU
• Peer-based grade inflation was not observed on any large scale / grades awarded mainly similar between teachers and learners. Two out of eight students (Student 4 &5 – see slide below) graded their peers substantially higher than the lecturer:

The methods and results of the pilot on developing a peer assessment toolkit are included in Annex 1.

The paper-based rubric was working well during testing. However, further adaptation will ensure that all country-specific needs are being met. Further discussions to synchronise the toolkit will take place in the next steps. Once the toolkit is finalised it will be digitalised.

Unicam functionality into Moodle

Whilst Unicam functionality has been evaluated as part of the work packages for this project result, it will be revisited. DCU’s Teaching Enhancement Unit has recommended some Moodle plug ins which may be useful for partners to adapt. We are in the process of sourcing and purchasing Unicam licenses. Once installed, this part of technology will be tested as well.

Conclusions

The development of a digital peer assessment toolkit for the AHEAD project has commenced with the development of a paper-based rubric and video-recorded skills, which volunteer peers and lecturers at ISU graded in a pilot test. Next steps include finalising the rubric, enabling Unicam to Moodle functionality and digitalising the toolkit.