Dublin City University

Quality Assurance and Enhancement

External Quality Review Process -

Background and Guidelines for Reviewers

Quality Promotion Office

Dublin City University

January 2017

Website: http://www.dcu.ie/qpo/index.shtml

Contents

1. Introduction	4

2. Overview of External Quality Review Processes at DCU	4

2.1	The DCU Quality Framework	4

2.2	The DCU Quality Promotion Office	5

2.3	Quality Assurance and DCU Leadership and Governance Structures	5

2.4	DCU Quality Policy	6

3. Quality Review Process for Academic and Support Units at DCU	7

3.1	Structure of the DCU Quality Review Process	7

3.2	Benefits of the Quality Review Process at DCU	7

4	Self-Assessment	8

4.1	The Quality Committee and Area Self-Assessment	8

4.2 The Self-Assessment Report (SAR)	9

5	Peer Review	9

5.1	The Peer Review Group	9

5.2	Peer Review Group Objectives	10

5.3	Peer Review Group Functions	10

5.4	Peer Review Visit	11

5.5	The Peer Group Report	12

6	Improvement Planning and Follow-Up	12

6.1	Development of the Quality Improvement Plan (QuIP)	12

7	Review by Governing Authority and Publication of Review Outputs	13

Appendix 1 Sample Timetable Peer Review Group Visit	14

Appendix 2 Structure of the Peer Review Report	16

Appendix 3 Schedule of Reviews of Academic and Research Areas 2001/02-2015/16	17

Appendix 4 Schedule of Reviews of Service / Support Offices 2002/03-2015/16	18

Appendix 5 Schedule of Thematic Reviews 2007-2009	18

List of Figures

Figure 1 Overview of the Statutory and Legislative Basis of Quality Assurance within Irish Higher Education	4

Figure 2 Quality Framework at DCU	5

Figure 3 Quality Assurance and DCU Leadership and Governance Structures	6

Figure 4 Key stages of the Quality Review Process at DCU	7

Figure 5 Key Stages of the Self-Assessment Stage of Quality Review	8

Figure 6 Themes of Self-Assessment contained in the Self-Assessment Report	9

Figure 7 Typical Structure of a Peer Review Group	10

Figure 8 Typical Structure of a Peer Review Visit	11

Figure 9 Timeline for the Development of the Quality Improvement Plan	12

List of Acronyms

APR	Annual Programme Review
DCU	Dublin City University
DCUBS	DCU Business School
ESG	European Standards and Guidelines
EUA	European University Association
GA	Governing Authority
HEA	Higher Education Authority (An tÚdarás)
IUA	The Irish Universities Association
IUAQC	Irish Universities Association Quality Committee
IUQB	Irish Universities Quality Board
NCSR	National Centre for Sensor Research
OVPLI	Office of the Vice President for Learning Innovation
OVPR	Office of the Vice President for Research
PPR	Periodic Programme Review
PRG	Peer Review Group
QA	Quality Assurance
QI	Quality Improvement
QPC	Quality Promotion Committee
QPO	Quality Promotion Office
QQI	Quality and Qualifications Ireland
QuIP	Quality Improvement Plan
SAR	Self Assessment Report
SMG	Senior Management Group
SS&D	Student Support & Development

1. Introduction

The broad approach to quality assurance and enhancement DCU aims to promote and develop a culture of quality throughout all aspects of the University. The framework derives from the spirit of Quality Assurance and Quality Improvement enshrined in the Universities Act (1997), which is the legislative basis for quality throughout the Irish University sector, and the Qualifications and Quality Assurance (Education and Training) Act 2012.

Figure 1 Overview of the Statutory and Legislative Basis of Quality Assurance within Irish Higher Education

The DCU processes for quality reviews at DCU are further aligned to the standards and guidelines for quality assurance in the European Higher Education Area (ESG) and the published guidelines of Qualifications and Quality Ireland (QQI), and are continually reviewed and further developed based on national and international good practice.

This document is designed to support members of the Peer Review Group (PRG) in conducting externally led quality reviews of academic and professional support units at DCU. The document outlines the background and purpose of quality reviews at DCU and also provides general information on the key stages of the process.

2. Overview of External Quality Review Processes at DCU

2.1	The DCU Quality Framework

The externally led quality reviews of DCU academic and professional support areas at DCU are a core aspect of a broader quality assurance and enhancement structure at DCU. Areas reviews are conducted on a cyclical basis, typically once every seven years. For academic units in particular, cyclical reviews of academic are built upon more regular reviews of academic programmes at DCU, including Annual and Periodic Programme Review. The quality framework, describing how Annual and Periodic Programme Review align to Area level reviews is outlined in Figure 2.

Figure 2 Quality Framework at DCU

2.2	The DCU Quality Promotion Office

The DCU Quality Promotion Office (QPO) has been established to promote, support, and facilitate quality improvement activities across academic and administrative units throughout the University.

This is undertaken principally through the management of the University’s Quality Review process for Schools, Faculties and Professional Support units by providing assistance and advice to Heads of Schools, Dean of Faculties and Directors of Units, academic and administrative staff engaged in the review process; liaising with external reviewers; tracking the implementation of recommendations arising from the review process; analysing the outcomes of the review process at an institutional level; disseminating good practice arising from the review process.

2.3	Quality Assurance and DCU Leadership and Governance Structures

Procedures for external quality review at DCU are strongly embedded within the leadership and governance structures at the University.

The Quality Promotion Committee (QPC) is a sub-committee of the University’s Executive Committee, and is tasked with promoting an ethos of self-evaluation and continuous quality improvement within the university and advising and making recommendations to DCU Senior Management, Executive and Academic Council on policies for quality assurance and improvement. The Quality Promotion Committee (QPC) is chaired by the President, or his/her nominee; the current Chairperson of QPC is the DCU Deputy President.

Figure 3 Quality Assurance and DCU Leadership and Governance Structures

The University’s Senior Management Group (SMG) is included at a number of stages of the quality review process. Upon completion of an Area’s Self-Assessment Report (SAR), SMG receives a copy of this report, and all supporting documentation. A meeting with the Senior Management Group and a further meeting with the member of SMG with management responsible for the Area under review is a core element of the Review Visit. Further, as part of the Quality Improvement Planning following a review visit, SMG provides to the university’s response to recommendations of the Peer Review Group. All outputs from the External Quality Review process are considered at the University’s Quality Promotion Committee, University Executive Committee, and the University’s Governing Authority.

2.4	DCU Quality Policy

DCU acknowledges that it is ultimately responsible for the academic standards of awards made in its name and for its students’ quality of learning experiences. DCU’s quality policy has the following main aims and goals:

To develop a quality culture that permeates all parts of the university for the benefit of the students, staff and the local, regional and national community.

To ensure that the University’s strategic planning and development and quality assurance and improvement mutually inform and support each other.

To ensure that appropriate and transparent governance and management structures are in place to guarantee continuous progress in implementing and supporting quality assurance and improvement measures.

To implement and maintain procedures relating to the approval, monitoring and review of programmes.

To take the advice of independent external peers and organisations, in particular external examiners, professional, statutory and regulatory bodies and external assessors in internal and external reviews of academic, administrative and support units, and in topic-based quality reviews.

To gather quantitative and qualitative data and to conduct periodic surveys to get feedback from key stakeholder groups including students and employers, for quality improvement and policy making.

To maintain the University in good standing in relation to its legislative obligations and to make the university a model public body.

3. Quality Review Process for Academic and Support Units at DCU

3.1	Structure of the DCU Quality Review Process

The DCU quality review process includes four key stages,

A period of self-assessment, involving all staff within an Area under review, which aims to critically assess the activities of the Area. The outputs of this self-reflection are summarised in the development of a Self-Assessment Report (SAR), which forms the basis of the evaluation of the Peer Review Group.

A visit by an externally led Peer Review Group (PRG), to verify and evaluate the SAR, and meet with staff, students, and other stakeholders to discuss key issues identified. The visit is followed by the completion a PRG Report, which summaries the Group’s findings and makes commendations and recommendations for future quality enhancement within the Area under review.

The development of Area led Quality Improvement Plan (QuIP) in light of the PRG recommendations. This involves both an Area, and University level response to the PRG recommendations. The QuIP is discussed and agreed at a follow-up meeting, attended by at least one external member of the PRG

The consideration of the PRG Report and QuIP by University Executive and Governing Authority ahead of publication on the DCU website.

Figure 4 Key stages of the Quality Review Process at DCU

3.2	Benefits of the Quality Review Process at DCU

In addition to satisfying the statutory requirements (Universities Act, 1997, QQI Act, 2012), the quality process is considered valuable for the following reasons:

It presents detailed information about the Area and the collective perception of staff and students of its role in the university.

It presents a succinct and comprehensive statement of the Area’s view of its strategic objectives and for a school, of its teaching, learning and research.

It provides a reflective and self-critical analysis of the activities of the Area.

It shows the quality systems and processes which are already in place in the Area and permits an assessment of their effectiveness.

It helps the Area to identify and analyse its strengths, weaknesses, opportunities and threats, and allows it to suggest appropriate remedies where necessary.

It identifies those weaknesses, if any, in academic, organisational and other matters which are under the control of the Area and which can be remedied by Area-based action.

It identifies shortfalls in resources and provides an externally validated case for increased resource allocation.

It provides a framework within which the Area can continue to work in the future towards quality improvement.

4	Self-Assessment

4.1	The Quality Committee and Area Self-Assessment

Figure 5 Timeline for the Self-Assessment Phase of Quality Review

The Self-Assessment phase of quality review is led by the Area Quality Committee, whose membership should be reflective of all staff within the area. The Committee lead co-ordinate self-assessment activities, including the use of evidence-informed approaches to self-reflection and assessment, leading to the development of a Self-Assessment Report (SAR).

During the period of self- assessment, the Area under review are encouraged, and supported by the Quality Promotion Office and Institutional Research and Analysis Officer to conduct research to help gather information on the effectivess of their activities. This may include surveys, focus groups, benchmarking, or statistical analysis of data. Areas may also use the self-assessment period as an opportunity for team away-days and planning events. A small amount of funding is provided to the Areas by the Quality Promotion Office to cover costs associated with these activities.

4.2 The Self-Assessment Report (SAR)

The purpose of the SAR is to provide a succinct, but comprehensive and reflective statement of the school’s activities, and in particular will discuss and analyse the Area’s activities in the following areas:

Figure 6 Themes of Self-Assessment contained in the Self-Assessment Report

The final self-assessment report is circulated to all staff in the area and to the Director of Quality Promotion and Institutional Research, who is responsible for distributing it to the PRG responsible for reviewing the Area, and to the University’s SMG.

In line with good international practice and in agreement with sectoral policy, the University does not make the self-assessment report further available. It is our belief, adhering to the spirit of the legislation, that retaining the confidentiality of the self-assessment report to the PRG and university management enables and supports the aims of self-assessment in identifying of difficult issues and allows for greater openness and candour in Area self- reflection.

5	Peer Review

5.1	The Peer Review Group (PRG)

The Review Group will be selected by the QPO with assistance from the Quality Promotion Committee. The PRG composition will reflect the size, character and structure of the Area or theme under review. As a general rule, the PRG will have internal and external members, with a majority of external members. Internal members will be drawn from the Quality Promotion Committee (rapporteurs) and senior members of staff who are not directly involved with the reviewed Area; external members will be drawn from senior leaders in relevant academic disciplines or areas nationally and internationally, as well as a senior member from outside Higher Education.

The table below summarises the structure of a typical five person PRG, including the internal/ external profile, the inclusion of an international member, and a member from outside of the Higher Education sector.

Figure 7 Typical Structure of a Peer Review Group

5.2	Peer Review Group Objectives

The objectives of the Peer Review Group will be to:

Clarify and verify the information contained within in the self-assessment report

Make judgements on how well the aims and objectives of the Area are fulfilled, having regard to the available resources

Confirm the Area’s strength, weaknesses, opportunities and threats as outlined in the self-assessment report

Discuss any perceived strengths, weaknesses not identified in the self-assessment report

Check the suitability of the teaching, learning and research environment (where applicable)

Draw conclusions on the quality and standards achieved within the area and make commendations and recommendations for future quality improvements.

5.3	Peer Review Group Functions

The Peer Review Group will:

Study the Area self-assessment report and accompanying documentation.

Visit the Area to gather, clarify, test and verify the information collected as part of the self-assessment process

Review the effectiveness of activities of the Area in the light of the self-assessment report

Prepare a draft report and present the main findings by the end of the visit

Write the peer review report, including finalised commendations and recommendations for future quality improvement

Two members (one internal, one external) will assist in finalising the Quality Improvement Plan (QuIP) at a follow-up meeting.

5.4	Peer Review Group Visit

The PRG will visit the Area under review over a minimum of a two and a-half-day period. This visit is central to the peer review process and will be planned by the Area, in close collaboration with the Quality Promotion Office.

It is envisaged that the visit will normally take place during the weeks of the first or second semester so that students can participate. In order to facilitate personal schedules, the membership of the Peer Review Group will normally be decided at least six months in advance and the days for the visit fixed. All contact with the Peer Review Group in organising the PRG visit are carried out by the Quality Promotion Office.

Figure 8 Typical Structure of a Peer Review Visit

The Peer Review visit usually includes a number of core elements, which usually include,

At least one meeting with the Head of the Area under review

A meeting with the Quality Committee to lead the Self-Assessment Report

A meeting with the Area Management Team, where relevant

Meetings with a wide representative group (or all staff) within the Area under review.

Meetings with staff from other areas, e.g. administrative offices, research centres, academic units

A meeting with students

A meeting with key external stakeholders, which may include recent graduates

A meeting with the University Senior Management Group (SMG)

A meeting with the SMG member with responsibility for the Area

A sample visit schedule is included in Appendix 1 of this document.

By the end of the visit, the PRG will have completed an initial draft of their Peer Review Group report, which will include their commendations and recommendations. The visit will conclude the visit with an PRG Exit Presentation to the staff of the Area, where the Chairperson will present the review group’s main findings and recommendations. The exit presentation will not involve discussion with the staff of the School/Faculty/Office/Centre. It will simply be a presentation of the main findings and recommendations of the report.

5.5	The Peer Group Report

Within four weeks of the Peer Review visit, the Peer Review Group will submit their final draft report to the Director of Quality Promotion and Institutional Research. The structure of the Peer Review Group Report is provided in Appendix 2. The Director will then send a copy to the Head of the Area, who is responsible for circulating a copy to all members of the Area Co-ordinating Committee.

Once finalised, based on a review by the Area of factual errors which may be in the report, a copy of the final report will be sent to the President, and other members of the Senior Management Group, and all staff in the Area under review. The report will also be sent to all members of the Quality Promotion Committee prior to discussion of the report at the next QPC Meeting.

6	Improvement Planning and Follow-Up

Figure 9 Timeline for the Development of the Quality Improvement Plan

6.1	Development of the Quality Improvement Plan (QuIP)

Quality Improvement Planning, which follows the finalisation of the Peer Review Group Report, is a crucial aspect of the overall quality process. Both the Universities Act (1997) and the Qualifications and Quality Assurance Act (2012) note the responsibilities of the university to implement each of the recommendations of the PRG Report, unless it would be impractical or unreasonable to do so. The decisions on improvement made in the follow-up process provide a framework within which each Area can continue to work towards the goal of developing and fostering a quality culture in the University.

Typically, the process of Quality Improvement Planning will be led by the Head of the Area under review, and will include broad consultation across the Area. The QuIP will address all the recommendations in the Peer Review Group report, and develop and action plan for quality improvement, including a timeframe for implementation. A draft Quality Improvement Plan is usually developed within 3 months of the Peer Review visit, and is provided to the university Quality Promotion Office, Senior Management Team, and two members of the Peer Review Group (one internal, one external) as the basis of a follow-up meeting, at which the Area’s Quality Improvement Plan is agreed.

The draft Quality Improvement Plan should include:

A list of goals which can realistically be achieved in the following year

A list of longer-term goals to be achieved over three years.

In addition to the Area response to the PRG Report, the university leadership also submit an agreed university response to relevant aspects of the PRG report.

The Area under review also has an opportunity to apply for additional funding to support the implementation of specific projects identified within the Quality Improvement Plan. These are considered by the university Quality Promotion Committee and the Budget Committee of the University Executive, having regard to the resources available to the University for quality improvement purposes.

7	Review by Governing Authority and Publication of Review Outputs

The DCU Governing Authority will receive documentation relevant to the review, including a summary of the report by the peer review group and a summary of the agreed Quality Improvement Plan, as well as the final PRG report and QuIP.

In addition, a summary of any common themes and recommendations relating to all the reports for that year will be presented in the summary report.

In accordance with the Universities Act (1997), the Governing Authority shall:

Implement any findings arising out of an evaluation carried out in accordance with the quality review procedures unless, having regard to the resources available to the university or for any other reason, it would, in the opinion of the governing authority, be impractical or unreasonable to do so.

Shall provide for the publication in such form and manner as the governing authority thinks fit of findings arising out of the application of the quality assurance procedures.

Following discussion and approval by Governing Authority, the University has agreed that the following will be published:

Full text of the Peer Review Group Report

Full text of the agreed Quality Improvement Plan

Full text of the Summary of the Quality Reviews presented to and approved by the Governing Authority

Appendix 1 Sample Timetable Peer Review Group Visit

Time	Peer Review Group (PRG) Activity/Meeting	Venue	Meeting No.
Day 1- Wednesday
1000-1030	Arrival of PRG Members, coffee on arrival- Albert College	A204
1030-1130	Briefing by Director of Quality Promotion; guidelines provided to assist the PRG during the visit and in developing its report
1130-1230	PRG Private Meeting Time. PRG Selects a Chair PRG discusses key themes, areas for exploration based on the SAR PRG assigns tasks and responsibilities amongst members
1230-1330	Lunch with Director of QPO (light lunch in 1838)
1330-1445	Consideration of the SAR with the Area Head and members of the Quality Review committee. Shall commence with a short presentation by Area head, followed by discussion (Director, QPO to attend)	1
1445-1500	PRG private discussion time/ Coffee
1500-1555	Meeting with Students (mix of qualification type, programme of study, gender, nationality etc where appropriate)	2
1600-1700	Meeting with external stakeholders (e.g. alumni, employers, collaborators, suppliers, linked colleges, members of governing authority etc)	3
1700-1730	PRG private discussion time
1730-1800	Informal Reception – PRG, Area Head, Members of Quality Review Committee, Director of Quality Promotion	1838 DCU	Arranged by QPO
1800-1930	PRG Private Dinner and discussion	1838 DCU	Arranged by QPO
Day 2- Thursday
0900-0940	Area Staff- Meeting 1	4
0945-1025	Area Staff- Meeting 2	5
1030-1100	PRG Coffee/ Private Meeting Time
1100-1140	Area Staff- Meeting 3	6
1145-1230	Academic Staff outside of area under review (internal academic collaborators, academic staff teaching on programmes, staff from relevant research centres)	7
1230-1300	Tour of Facilities
1300-1400	Lunch/ PRG Private Meeting Time
1400-1440	Relevant Central Support Units- senior representatives	8
1445-1510	Staff Open Forum for any member of Area staff	9
1515-1630	PRG Private Meeting Time/ Coffee
1630-1715	Meeting with Area Management Team	10
1715-1745	Meeting with Area Head	11
1830-2000	PRG Private Dinner and Meeting
Day 3- Friday
0900-0955	PRG Meeting with SMG	12
1000-1025	Meeting with Area reporting head (relevant member of SMG)	13
1030-1300	PRG Private Meeting Time- final discussion on recommendations
1300-1345	PRG working lunch and finalization of exit presentation
1345-1400	Briefing with Area Head and Director of QPO on key recommendations
1400-1430	PRG Exit Presentation

Appendix 2 Structure of the Peer Review Report

Structure Peer Review Group Report- Academic Unit
1 Introduction and Context 1.1 Overview of the Area under Review 2 Approach to Self-Assessment 2.1 Quality Review Committee 2.2 The Self-Assessment Report 3 Approach Taken By Peer Review Group 3.1 Peer Review Group Members 3.2 Overview of Approach Taken by Peer Review Group 4 Approach to Quality Assurance and Enhancement 5 Findings of the Peer Review Group 5.1 Planning and Effective Management of Resources 5.2 Teaching and Learning 5.3 Research and Scholarship 5.4 University Service and Engagement 5.5 Communications and Provision of Information 5.6 External Perspective 6 SWOT Analysis and Plans for Improvement 6.1 SWOT Analysis for INSERT AREA NAME 6.2 Plans for Improvement Identified by INSERT AREA NAME 7 Summary of Commendations and Recommendations

Structure Peer Review Group Report- Professional Support Unit
1 Introduction and Context 1.1 Overview of the Area under Review 2 Approach to Self-Assessment 2.1 Quality Review Committee 2.2 The Self-Assessment Report 3 Approach Taken By Peer Review Group 3.1 Peer Review Group Members 3.2 Overview of Approach Taken by Peer Review Group 4 Approach to Quality Assurance and Enhancement 5 Findings of the Peer Review Group 5.1 Planning and Effective Management of Resources 5.2 Effectiveness of Activities and Processes 5.3 Communication and Provision of Information 5.4 Ongoing Quality Enhancement 5.5 External Perspectives 6 SWOT Analysis and Plans for Improvement 6.1 SWOT Analysis for INSERT AREA NAME 6.2 Plans for Improvement Identified by INSERT AREA NAME 7 Summary of Commendations and Recommendations

Appendix 3 Schedule of Reviews of Academic and Research Areas 2001/02-2015/16

Year	Academic Area
2001-2002	Business School
2001-2002	School of Physical Sciences
2002-2003	School of Biotechnology
School of Computing
School of Communications
2003-2004	School of Chemical Sciences
School of Electronic Engineering/ Research Institute for Networks & Communications Engineering (RINCE)
Fiontar
2004-2005	School of Nursing
School of Mechanical & Manufacturing Engineering
School of Applied Languages & Intercultural Studies (SALIS)
2005-2006	School of Mathematical Sciences
National Distance Education Centre
National Centre for Plasma Science & Technology (NCPST)
2006-2007	School of Health & Human Performance
School of Law & Government
School of Education Studies
National Institute for Cellular Biotechnology
2007-2008	National Centre for Sensor Research
Business School (Faculty)
2008-2009	Faculty of Engineering & Computing School of Computing School of Electronic Engineering School of Mechanical & Manufacturing Engineering
2009-2010	DCU Institutional Review (IRIU)
2010-2011	Faculty of Humanities & Social Sciences
2011-2012	Oscail (Distance Education)
Faculty of Science & Health
2012-2013	National Centre for Plasma Science and Technology (NCPST)
2013-2014	School of Physical Sciences
2014-2015	Business School (Faculty)
2015-2016	School of Biotechnology Faculty of Engineering & Computing (Faculty)
2016/17	School of Chemical Sciences School of Nursing and Human Science School of Health and Human Performance

Appendix 4 Schedule of Reviews of Service / Support Offices 2002/03-2015/16

Year	Service / Support Office
2002-2003	Office for Innovation & Business Relations
Registry
2003-2004	Student Affairs
Human Resources Office
2004-2005	Finance Office
Computer Services Department
Office of the Vice President for Research
Student Finance Committee
2005-2006	Library
Communications & Marketing Office
EOLAS Group
2006-2007	Buildings Office
Office of the Vice-President for Learning Innovation
2007-2008	President’s Office
Theme Leaders’ Office
2010-2011	Secretary’s Office
2011–2012	Office of Student Life
Student Support & Development
Registry
2012-2013	Information System & Services (ISS)
2012-2013	Human Resources Department
2013-2014	Finance Office
2014-2015	Communications & Marketing
2014-2015	Research & Innovation
2014-2015	Estates Office
2015-2016	Library

Appendix 5 Schedule of Thematic Reviews 2007-2009

Year	Service or Support Areas	Office Affiliation
2007-2008 2008-2009	First Year and Beginners’ Student Experience Post Graduate Students	Cross-Campus Cross-Campus

http://www.enqa.eu/wp-content/uploads/2015/11/ESG_2015.pdf

http://www.qqi.ie/Publications/Core%20Statutory%20QA%20Guidelines.pdf

https://www.dcu.ie/qpo/published-reviews.shtml