Publication of a study on evaluation frameworks conducted with the Quality Audit Network

After hosting the 2023 edition of the Quality Audit Network, Hcéres coordinated a study on the design and review of evaluation frameworks. “Institutional evaluation frameworks: design and review processes among the agencies of the Quality Audit Network” is based on the practices of 16 European quality assurance agencies.

The Quality Audit Network is an informal network of European agencies that conduct external quality assurance processes at institutional level. Since its creation in 2008, the network has produced a number of publications on trends in institutional reviews. The 2023 edition focuses on a key evaluation tool: the evaluation framework. This study was led by Hcéres, the Austrian agency AQ Austria, the Danish agency AI and the Finnish agency FINEEC, with contributions from 12 other agencies of the Quality Audit Network that responded to the survey.

While the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) are a common reference for all European quality assurance agencies, they do not constitute a layout model for all evaluation frameworks. This is the first finding of the Quality Audit Network study. A wide diversity was observed in the form, depth and scope of the evaluation frameworks used by the agencies, which all fall on a continuum between prescriptive and ‘open’ frameworks. How can external quality assurance be adapted to the evaluated entities if the same criteria are applied to all? Several agencies are therefore proposing flexible methodologies in which some evaluation areas are chosen by the experts or by the higher education institution. Finally, does external evaluation set minimum criteria, or does it guide institutions towards excellence? These topics are developed in the first part of the study, which deals with the design of an institutional evaluation framework.

The second part of the study examines the methods used to review the evaluation frameworks, which are regularly revised to better respond to changes in higher education in a fit-for-purpose approach. Who are the stakeholders consulted and how are they involved in the review of evaluation frameworks? Do the changes allow the frameworks to better reflect the challenges facing higher education, or do they simplify the standards? What are the pros and cons of broadening the scope? 

The 2024 QAN seminar in Stockholm hosted by UKÄ, where results of the study were presented for the first time.  

The practical implications of broadening scope were further discussed during the QAN 2024 seminar in Stockholm. Some challenges include the recruitment of experts, as widening the scope also implies widening the competence of the panels. If adding a new area in an evaluation framework might appear effortless, it challenges agencies in their struggle to maintain EQA efficient, time and cost-effective. As Dagmar Provijn (NVAO Flanders) concluded astutely: “In our quest to be relevant, we risk ceasing to be. As agencies are like snails, lagging behind HEIs, we shall maintain an open dialogue and important topics will be put forward by the HEIs themselves”.