Evaluation of institutions

Evaluation campaign 2019-2020 (group A)

What there is to know

Integrated evaluation and evaluation of territorial coordination / Evaluation process of institution

Presentation made for the 2019-2020 evaluation campaign (wave A) by Annie VINTER, director of the evaluation department of territorial coordinations, Michel ROBERT, director of the evaluation department of insitutions, Laurent DAUDEVILLE, scientific advisor coordinator.

  • PREPARATION

    Stage 1 :

    Preparation of the campaign
    Who : Hcéres
  • Stage 2 :

    Submission of institutions’ files
    Who : Establishments
  • Stage 3 :

    Formation of the panel of experts
    Who : Hcéres
  • Stage 4 :

    Formation of the panel of experts
    Who : Hcéres
  • Stage 5 :

    Consultation with the head of the institution concerning the composition of the committee
    Who : Hcéres
  • Stage 6 :

    Preparatory meeting on the key issues and visit programme
    Who : Experts
  • Visit

    Stage 7 :

    Visit by the panel of experts including interviews with:
    • Head of the institution
    • Senior management team
    • Heads of departments
    • Students
    • Teaching and research staff
    • Personnel
    • Partners
    Who : Experts
  • Reporting back

    Stage 8 :

    Drafting of the provisional report and validation
    Who : Experts
  • Stage 9 A :

    Transmission of the provisional report
    Who : Hcéres
  • Stage 9 B :

    Reactions of the institution
    Who : Establishments
  • Stage 10 :

    Drafting of the final report and validation
    Who : Experts
  • Stage 11 A :

    Transmission of the final report
    Who : Hcéres
  • Stage 11 B :

    Letter containing the institution’s comments
    Who : Establishments
  • Stage 12 :

    Publication on the website of the final report and letter containing the institution’s comments
    Who : Hcéres
  • Feedback

    Stage 13 :

    Organisation of transmission of feedback to the evaluated institutions and panels of experts
    Who : Hcéres

The Group A evaluation process, from its launch to the transmission of reports, shall cover the period from 25 September 2018 to September 2020.
 

November-December 2018

Kick-off meetings for the in situ evaluation campaign
 

22 February 2019

Transmission to Hcéres of one generic email address per institution which will be used as the login name for the submission of self-evaluation files to the Pélican platform, as well as the consultation of the evaluation reports and the submission of comments about it on the Hcéres electronic document management (EDM) system
 

April – May 2019

Launch of the PELICAN platform and training for users on its use
 

20 September 2019

Deadline for the submission of institutions’ self-evaluation files.

What to check/drop

The reference documentation published online is intended for all higher education and research institutions (universities, major institutions, research bodies, schools) that undergo an institutional evaluation, whether they are supervised by the Ministry of Higher Education, Research and Innovation (MESRI) or attached to another ministry.

  • Documents to be submitted on the PELICAN exchange platform:

    • Self-evaluation report including a global review of the results of the current contract with the supervising ministry
    • Declaration of the institution’s strategic development priorities for the next contractual period, paying particular attention to the training and research strategy
    • Functional organisational structure of the institution
    • Characterisation data which the institution considers relevant to its evaluation
    • Indicators specific to the institution, including contractual indicators
    • Initial budgets and financial accounts for the last 4 years, as adopted by the Administrative Board, accompanied by the accounting officer’s overview report, financial analysis documentation for the institution, if available, and budget policy letters (for each fiscal year)
    • Other external evaluation reports, where applicable

Discussions between Hcéres and the Ministry of Culture have prompted Hcéres to propose the simplification of the files submitted by ENSAs, entailing:

  • the production by the institution of a single self-evaluation file that will meet the requirements set out in the external evaluation standards of both departments,
  • the organisation of a visit to enable an institutional evaluation of the school and carry out a more detailed examination of study programmes initially evaluated on the basis of documentation.
     

To improve the coordination of evaluations and simplify the documentation submitted by engineering schools, Hcéres and the CTI (French Engineering Qualifications Committee) have asked schools to prepare a single self-evaluation file and organise a coordinated visit in accordance with the schedules of both evaluation bodies. To this end, Hcéres and the CTI have published guidelines to help schools write the self-evaluation report.

Self-evaluation files are submitted via the PELICAN joint exchange platform  established by Hcéres and the French Ministry of Higher Education, Research and Innovation. The information entered and transmitted by institutions and organisations is used by Hcéres and experts to perform the evaluations.

The application period will begin in April-May 2019 to enable institutions to start submitting their files prior to the deadline for submissions. Access to the application procedure is secure and requires a username and password. These will be issued by Hcéres after receipt of the generic email address that will be used as the username, which must be transmitted by each institution to Email, by 22 February 2019. 
Hcéres will also use this address to send institutions information about the half-day presentations on the PELICAN platform, the start of the application period, and the transmission of the institutions’ evaluation reports.

Evaluation files are submitted in 2 stages:

  1. Checking and addition of information to the pre-entered data on the forms. The online forms are used to provide a description of each evaluated entity (managers, contact details, parent institutions and bodies, disciplines, etc.). Certain data are pre-entered according to the lists of entities to be evaluated that are sent to Hcéres, and/or standards known to the Ministry (RNSR, etc.). For the best results, institutions are strongly advised to complete these forms as soon as the PELICAN platform is launched. This will save time when it comes to the final submission of the documentation included in the self-evaluation files.  
  2. Submission of different files followed by confirmation of the submission. Until the submission of the files has been confirmed, changes can still be made to the files and the evaluation cannot begin. Users requiring any additional information about the PELICAN platform can consult the online user guides which require no prior authentication, and the guides to the submission of files for each type of evaluation (territorial coordinations, institutions, research, study programmes and doctoral schools). The files must not exceed 50 MB in total, which means that the number of logos, images and copies of Web pages must be minimised.

    If you encounter problems

     

    • concerning the content of the files to be submitted: consult the list of contacts for each type of evaluation;
    • of a technical nature (e.g. login issues, problems locating your file): please email Email.

    FAQ / Contact

    FREQUENTLY ASKED QUESTIONS

    Officially, institutions are not required to follow the standard to the letter, but they are strongly advised to conform to its broad thematic structure (cf. methodological document entitled “Key points for self-evaluation”). Indeed, this approach facilitates the work of the evaluation panel which must analyse the self-evaluation report by carrying out an overall examination of the references in each thematic section of the standard. However, institutions are free to decide how they organise each of these sections. Institutions are not required to submit an individual response for each reference and criterion, and they may choose to adopt a different approach to meeting all the requirements in a given section.
     

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    The expert panel consists of French and international experts. However, to facilitate the experts’ work and in accordance with the legislative requirements, institutions and territorial coordinations are asked to submit their files and documents in French. Nevertheless, certain documents - especially those pertaining to the evaluation of research bodies - may be submitted in English if accompanied by a summary written in French.
     

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    The ex post evaluation process covers a reference period of four to five years, which can be defined as the period of time between the last two self-evaluations. Depending on the choices made by the institution concerning its self-evaluation, this period may also be limited to the time that has elapsed since the start of the last multi-annual contract with the supervising ministry.
     

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.
    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    There is no single “best time”. It all depends on the institution, its experience with such procedures and whether there is an existing internal continuous improvement scheme in operation. However, these activities, which are traditionally organised into working parties and workshops, involving integration, intermediate diagnosis and consultation phases, generally require several months of work (4 to 6 months, on average).

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.
    • The European standards recommend that the institution’s different staff members and students should be involved in the self-evaluation process, including in the production of the self-evaluation report. 
    • They may participate in different ways (e.g. working parties, interviews, surveys, etc.), and these activities may concern the different aspects of the evaluation (governance, management, training and research activities). International practices often include the establishment of a steering committee consisting of representatives of the different categories of stakeholders in the institution to oversee the self-evaluation process.
    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.
    • The self-evaluation report will have around sixty pages. It can be accompanied by annexes referenced in the report; however, the amount of documentation included in the annexes should remain moderate because an excessive amount of poorly organised information could hinder the evaluation panel’s activities.
    • To facilitate the evaluation panel’s work, we strongly recommend the referencing of annexes in the body of the self-evaluation report and advise institutions to be highly selective in terms of the number of annexes they include and their content (cf. methodological document entitled “Key points for self-evaluation”).
    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.
    • The declaration of strategic development priorities must be no more than 5 pages long. 
    • Institutions are not required to submit a detailed development project but only to state their major focuses in these fields of activity for the coming period.
    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.
    • You can send any questions to the following generic address: etablissements@hceres.fr or via the contact form on the High Council’s website. 
    • In addition, dedicated contacts for each institution will be identified in the emails announcing the visit time slots (9 to 12 months before the visit).
    • These will be two-person teams consisting of a member of the teaching and research staff acting as a scientific advisor, and a project manager. This two-person team acts as an interface between the institution and Hcéres; it coordinates the evaluation of higher education and research institutions. It also supports the panel’s activities throughout the process, in accordance with the rules of professional ethics and the methodological framework drawn up by Hcéres.

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    Institutions are free to present the documents they submit to Hcéres for evaluation. Institutions are advised to present these documents to the central committees and disseminate them throughout the institution. This internal validation and dissemination process for the self-evaluation report must be described in the presentation of the methodology used for the self-evaluation (cf. document entitled “Key points for self-evaluation”).
     

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    The visit schedule is normally sent to institutions 9 to 12 months before the planned visit. For research bodies and territorial coordinations, this is an ad hoc schedule. 

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    Institutions are invited to participate in the formation of their evaluation panels, subject to meeting the following requirements for the composition of panels, which must include:

    • Academic experts who have held posts of responsibility in the management teams of higher education and research institutions;
    • An administrative expert with experience of at least one senior management or administrative management post; 
    • An expert from the socio-professional sector with experience of relevant senior leadership/managerial duties and/or scientific and/or cultural posts in a business sector of potential interest for the professional integration of students or the exploitation of the research products of the evaluated institution;
    • An international expert from the academic or socio-professional sector;
    • An expert on Quality (in management, training or research
    • A student;
    • Any expert who has evaluated an institution during a previous group of evaluations cannot participate in the next group. Experts must not have any special-interest ties with the evaluated institutions and cannot ordinarily be attached to an institution situated in the same region as the evaluated institution.
    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.
    • Visits are gradually prepared with the support of a two-person team consisting of a member of the teaching and research staff acting as a scientific advisor, and a project manager. After the approval of the proposed visit time slot, a meeting between Hcéres and the institution’s senior management team is held prior to the visit in order to determine the institution’s expectations regarding the performance of the evaluation and transmit practical organisational information. 
    • The institution will be informed of the composition of the expert panel by email so that it can identify any potential conflicts of interest. 
    • The scheduling arrangements will be transmitted over this period, to allow sufficient time for meetings to be held with the interested parties, including external contacts (partners of the institution, representatives of local authorities, etc.), who may have limited time to spare. For information, around forty interviews are scheduled, on average, during the course of a university evaluation, in which over 100 people may be interviewed.
    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.
    • A two-person team consisting of a member of the teaching and research staff acting as a scientific advisor, and a project manager, will be exclusively assigned to each institution and site. They will form the advisory team at the High Council throughout the entire evaluation period and through to the submission of the final report; in other words, the two-person team will act as the interface between the institution and Hcéres.
    • Institutions will be asked, at a reasonably early stage, to identify two people who will act as special contacts for the two-person Hcéres team throughout the procedure, and especially during the preparation of the visit schedule. The preference for these roles should be given to people with experience of leadership in policymaking (e.g. vice-presidents) or in high-level administration.
    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    An initial version of the complete visit schedule will normally be transmitted by the project manager 2 to 3 weeks before the visit. However, it should be noted that the detailed scheduling arrangements are regularly discussed with the institution well in advance, starting with the meeting between Hcéres and the senior management team.
     

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.
    • This information will be set out in a specific email sent by Hcéres. The visit by the panel of experts will take place in a single location – usually on the premises of the institution’s senior management team or its president. In this way (barring exceptional circumstances), visits will not be carried out offsite or on peripheral sites. 
    • The organisation of interviews requires the provision of premises of a suitable size. 
    • As workshops may be held at the same time, the panel of experts should be given access to several rooms situated nearby. Institutions are also requested to reserve a meeting room, equipped with at least one computer with Internet access (WiFi) and a telephone, exclusively for the panel of experts. 
    • Lastly, the experts must be given access to the Institution’s Intranet system 15 days before the visit and up to 15 days after it.
    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    The evaluated institution is responsible for contacting the chosen participants (including those outside the institution) and for covering their travelling expenses.
     

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.
    • Throughout the entire process, the experts are likely to ask the institution to provide documents in addition to those included in the file, via the Hcéres contacts. During the visit, the participants interviewed by the experts may also pass on any relevant documents, which may also be made available to the experts during the evaluation on a dedicated Intranet site created by the institution. On this account, the experts are subject to strict rules of confidentiality and professional secrecy.
    • The institution must ensure that these different documents are systematically transmitted to the project manager of the designated Hcéres team in order to ensure the consistency of the database associated with the evaluation.
    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    Certain institutions benefit from a specially adapted procedure concerning the scope and methodology of their evaluation, with a view to improving efficiency.
    The adapted procedure is based upon:

    • a shorter visit period (1 to 2 days), 
    • fewer experts,
    • the consideration of references that have proven relevance to the institution’s activities, 
    • a report of around ten pages, published according to the institution’s wishes if it has been awarded the EESPIG (Private Higher Education Institutions of General Interest) quality label
    • the possibility of using one panel to cover several institutions, and of synchronising the visits carried out by a panel that simultaneously covers several institutions.

    The final product of the external evaluation is a report that analyses and assesses the implementation of the institution’s strategies in response to its missions. It analyses the institution’s strengths and weaknesses and concludes with recommendations which mark the culmination of the evaluation.
     

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.
    • Schools of architecture are covered by a joint procedure involving the Department of Evaluation of Higher Education and Research Institutions and the Department of Evaluation of Study Programmes at Hcéres. 
    • Under the terms of a partnership between Hcéres and the CTI, engineering schools also benefit from a coordinated procedure defined by both institutions, which is designed to improve the links between evaluations and simplify the documentation file submitted by engineering schools. A single self-evaluation file meeting the requirements set out in each external evaluation standard used by both institutions is required, and the  visit date is also coordinated.
    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    Le président ou directeur de l’établissement est entendu en tout début de visite puis, seul, à la fin de la visite, au cours d’une dernière réunion en formation plénière afin qu’il puisse répondre à d’ultimes questions.

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.
    • Discussions only take place during the interviews listed on the visit schedule. If necessary, the project manager who accompanies the panel during the visit will pass on any requests for additional documentation to the institution. 
    • There is no direct contact between the experts and the institution throughout the entire evaluation process.
    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    No. Evaluations conducted by Hcéres lead to the production of a report based on the experts’ written contributions. The report is the only type of feedback provided.
     

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.
    • After receiving the draft report, the institution can respond to it and request the correction of any factual errors, as well as any misunderstandings, omissions or erroneous terms. 
    • After the president of the panel has agreed to their inclusion in the report, the amended report will be sent to the institution in a provisional version. The president/director of the institution will then be asked to submit comments on this version of the report, which will then be appended to it. 
    • The report becomes final once the president of Hcéres and the president of the panel of experts have signed it.
    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    The recommendations are not mandatory requirements. Institutions are free to assess them and implement them as they see fit. The panel is fully entitled to examine the institution’s responses to them, but under no circumstances should this be construed as the panel checking on the implementation of the responses.
     

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    Reports are published within a period of approximately four months after the visit by the panel of experts
     

    You want more precision and contact us? Click here to access the contact form and get in touch with our correspondents.

    CONTACT US