Decision in case 1455/2015/JAP on the conditions at a test centre for a selection competition organised by the European Personnel Selection Office

Background to the complaint

1. The complainant participated in an open competition for the recruitment of EU civil servants[1] in June 2015, which involved computer-based tests (‘CBTs’) at a test centre in Brussels, Belgium. She was assigned a computer right next to the entrance door. She claimed that the noise caused by other candidates and staff members entering and leaving the room distracted her and negatively affected her performance.

2. After her CBTs, the complainant complained orally to the staff at the test centre about the situation. However, her complaint was not recorded in writing. Later that day, she submitted a complaint to the European Personnel Selection Office (EPSO), which is responsible for organising selection competitions for the EU administration. She asked EPSO to allow her to re-sit her tests.

3. EPSO informed the complainant that it was not possible to reschedule her tests. It stated that, as set out in the Computer Based Tests Manual (‘CBT manual’), candidates could request disposable earplugs in order to block out noise and other distractions.

4. The complainant was dissatisfied with this reply because it did not address all the issues mentioned in her complaint. She thus asked EPSO to clarify the procedure for lodging complaints about noise in the test centres.

5. In its reply, EPSO stated that the test environment will never be “completely noise-free”. It also referred her to the General Rules Governing Open Competitions (‘General Rules’) for further information about the complaints procedure.

6. Dissatisfied with EPSO’s reply, the complainant turned to the Ombudsman in September 2015.

The inquiry

7. The Ombudsman opened an inquiry into the complaint and identified the following issues:

(i) EPSO’s failure to ensure a comparable test environment for all candidates participating in the open competition;

(ii) EPSO’s failure to instruct the staff at the test centre about the relevant procedures in case of complaints about the test environment, and the staff’s subsequent failure to deal properly with the complaint;

(iii) EPSO’s failure to provide clear guidelines/information to candidates (including the complainant) about the possibility of complaining to the staff at the test centres about the test environment; and

(iv) EPSO’s refusal to allow the complainant to re-sit the tests.

8. In the course of the inquiry, the Ombudsman received EPSO’s reply on the complaint and, subsequently, the comments of the complainant in response to EPSO’s reply. The Ombudsman’s inquiry team also inspected EPSO's file on this case, inspected a test centre and met with EPSO’s staff to discuss the matter. The complainant submitted comments on the Ombudsman’s report of the meeting. The Ombudsman's decision takes into account the arguments and views put forward by the parties.

9. The Ombudsman notes that, in the course of her inquiry, the list of successful candidates in the competition in question was published in the Official Journal of the European Union on 19 April 2016. The complainant was not amongst the successful candidates.

(i) Test environment and (ii) how complaints are handled on site

Arguments presented to the Ombudsman

10. The complainant claimed that by assigning her a computer next to the entrance door, EPSO had failed to ensure test conditions comparable to other candidates, who were further from the door. She stated that the other candidates and staff members entering and exiting the room during her exam created considerable noise, which the earplugs that were provided failed to effectively block out. The complainant also claimed that staff members were talking and cleaning the surfaces with spray. She argued that this disturbance negatively affected her performance and she wanted EPSO to allow her to re-sit the tests.

11. The complainant also argued that EPSO had mishandled her complaint: it took too long to reply and she claimed that the replies were not prepared by a person in a position to take decisions on complaints. Moreover, the staff at the test centre failed to properly record her complaint.

12. In its reply, EPSO stated that it aims to provide fair and appropriate test conditions for all candidates. To this end it has a set of minimum standard requirements on the layout of, and the equipment in, the test centres[2]. It stated that its “general policy concerning the test centre equipment” was found to be correct and compliant with the principles of good administration in a 2010 own-initiative inquiry by the European Ombudsman (OII/9/2010/RT)[3].

13. EPSO argued that all candidates who participated in the same competition as the complainant took the CBTs under the same conditions. It added that individual differences between the candidates, such as sensitivity to noise or stress, could not be deemed to result in unequal treatment, according to EU case law. It thus refused to allow the complainant to re-sit her tests[4]. 

14. EPSO stated that it had contacted its test centre contractor responsible for the CBTs regarding the issues raised by the complainant. The contractor claimed that, on the day in question, there was no record of any incident at the test centre where the complainant sat her test[5]. It stated that staff members at the test centres receive guidelines (EPSO documentation) about how to deal with situations like that raised by the complainant. It added that the conditions at all test centres are verified prior to the test period through “a test launch of the exams”. EPSO apologised for the fact that it took almost 4 months to deal with the complaint following the notification of the complaint by the Ombudsman in October 2015. It explained that, in line with its standard procedures, the complaint was handled and responded to by its Candidate Contact Service.

15. In her comments on EPSO’s reply, the complainant insisted that the test conditions she experienced were different from those experienced by other candidates. Since the candidates taking the CBTs do not start and finish the tests at the same time, and need to use the door, a certain amount of noise is inherent in test rooms. She argued that those sitting near a door would be exposed to higher levels of noise, and that this could be objectively measured.

16. She also claimed that EPSO did not have any written record of her complaint from the test centre simply because the staff member failed to adhere to the applicable rules. It was thus doubtful that, in the absence of any written records, the staff members would be able to recall her particular situation or the level of noise in the room.

Inspection of a test centre and meeting with EPSO, and the Ombudsman’s assessment

17. Ombudsman staff visited one of the CBTs test centres[6] in Brussels to assess the conditions and procedures in place. EPSO explained that seats and computers (‘test stations’) are allocated randomly and on a first-come-first-served basis by an electronic system. Requests to change place are subject to availability, and candidates sitting next to the door can be moved if other test stations in the test room are unoccupied[7]. Candidates start and finish their tests at different times. They can start the tests earlier if they arrive early; they may choose to follow or skip a 10-minute tutorial; they have an optional break after the first part of the tests; they can leave before the allocated test time is up. The candidates are instructed to remain quiet and the rooms are fitted with ‘quiet please’ signs. The candidates are provided with a pair of earplugs and/or headphones[8].

18. It seems likely that candidates seated near the door will experience some noise from those entering and exiting at different times. While earplugs may reduce the noise, they cannot eliminate entirely the disturbance, even if candidates entering and exiting follow the instructions to remain quiet. Headphones may ensure better protection against noise, but they were not available for the complainant’s tests.

19. On the other hand, the nature of CBTs is such that candidates have to start and finish tests at different times. It is also reasonable to expect that candidates may take a break, if needed. Moreover, not all test rooms are big enough to ensure that all computers (test stations) are located away from the door. Therefore, candidates must reasonably expect a certain level of noise, which is indeed inherent in any test environment. The Ombudsman, however, accepts that an excessively noisy test environment could affect candidates’ ability to concentrate and, hence, their performance during tests.

20. It was, however, not possible for the Ombudsman to check whether the noise during the complainant’s test was objectively excessive. While test rooms are fitted with CCTV, the video footage is kept for up to 90 days only[9]. As the Ombudsman did not open her inquiry until October 2015, it was thus not possible to view the video footage of the complainant’s tests in June 2015, and establish the precise test conditions experienced by the complainant.

21. EPSO stated that the test centre visited by Ombudsman staff is broadly similar to its other test centres both now, as well as in 2015[10]. While the visit to a test centre was a useful exercise, it is not possible to conclude from this visit that the conditions experienced by the complainant in her test centre were, or were not, satisfactory. While accepting that all of the test centres have the same general layout and apply the same procedures, it is possible that the specific conditions in one test centre, on any particular day, will be less user-friendly than in others.

22. Overall the evidence available is not sufficient to provide the basis for a reasonable conclusion, one way or the other, regarding the conditions under which the complainant took her CBTs. Had the CCTV record been available, it would in all likelihood be possible to reach a firm conclusion on the matter. Accordingly, the Ombudsman cannot conclude beyond a reasonable doubt that EPSO failed to ensure an absolutely comparable test environment for all participants in the complainant’s competition. While a certain level of noise is inherent in any test environment, it is not now possible to establish whether or not the conditions during the complainant’s test were exceptionally noisy. Therefore, it is not possible for the Ombudsman to make any further meaningful inquiries into (i) EPSO’s failure to ensure a comparable test environment for all candidates in that competition. As the Ombudsman cannot reach a conclusion on (i), neither can she reach any meaningful conclusion on (iv) EPSO’s refusal to allow the complainant to re-sit the tests. Nevertheless, she makes a ‘suggestion for improvement’ that, in future, EPSO ensures that all test centres make available not only earplugs, but also noise-cancelling headphones.

23. The Ombudsman considers that, once EPSO receives a complaint about the conditions at a test centre[11], it should immediately check and keep the relevant CCTV recording (which is normally deleted 90 days after the date of the tests). This is necessary to be able to deal properly with the complaint. Only in this way can EPSO objectively check the facts of the relevant incident. Unfortunately, EPSO failed to do so in this case. The Ombudsman therefore makes a suggestion for improvement to this end.

24. As regards the delays by EPSO in responding to the complainant as notified by the Ombudsman, EPSO has now apologised for this delay. The Ombudsman notes also that the complaint was dealt with by EPSO within the time period during which it would still have been possible to re-sit the CBTs, that is, before the list of successful candidates was decided definitively. Therefore, no further inquiries into this issue are justified.

25. However, the Ombudsman considers that EPSO failed to exhaust all of the means at its disposal to deal properly with that complaint. In particular, and at a time when it was still available, EPSO failed to check the video recording when it was first dealing with the complaint. It would have been good practice for EPSO to have reviewed the video evidence. The Ombudsman encourages EPSO to contact the complainant to apologise for this failure which limited its capacity to deal with her complaint comprehensively. However, the Ombudsman does not see any advantage in pursuing this issue in the context of this particular case.

26. The visit by Ombudsman staff to a test centre suggests that, in general, the staff at the test centres are properly instructed by EPSO to record all complaints made on site[12]. It is however useful to regularly remind the staff of this important obligation, and the Ombudsman makes a suggestion for improvement to this end.

27. Regarding the issue of the Candidate Contact Service dealing with the complainant’s complaints, EPSO has discretion to decide on its internal arrangements for dealing with complaints from candidates. What is important, irrespective of who within EPSO handles complaints, is that all available evidence (including CCTV) is taken into account in reaching a decision on the complaint.

(iii) Guidelines for complaints about the test environment

Arguments presented to the Ombudsman

28. The complainant argued that EPSO lacked clear guidelines for candidates on how to make complaints about the conditions in test centres. She said she was not informed before sitting the tests that it was possible to complain, which meant she was at a disadvantage when she found she had a cause for complaint.

29. In its reply, EPSO referred to the relevant provisions of the General Rules[13], which set out detailed instructions on how to lodge a complaint concerning the test environment[14]. EPSO also noted that the General Rules, together with the Notice of Competition, form a binding legal framework for each competition. The General Rules are made available to the candidates “at the very beginning of the application period, and candidates are expected to familiarise themselves with their content”. EPSO added that, prior to the tests, the candidates in the complainant’s competition were provided with a ‘CBTs manual’[15], based on General Rules, as well as with the essential information at the test centre.

30. In her comments on the Ombudsman’s inspection report, the complainant pointed out that EPSO’s rules do not provide for the possibility to request a different test station during the tests. There are no objective conditions or criteria about the circumstances in which such a request could be made and when it could be granted. She thus argued that the discretionary character of this option means candidates are not treated equally.

The Ombudsman's assessment

31. The Ombudsman notes that the CBTs manual did not, at the time of the complainant’s tests, have any explicit reference to the complaints procedure. However, the General Rules explain in detail how to lodge a complaint about ‘technical and organisational problems’. The General Rules were made available to the complainant at the beginning of the competition.

32. The General Rules, together with the vacancy notice, form a binding legal framework applicable to each competition. Although the General Rules do not explicitly mention the ‘test environment’ as grounds for a complaint, the wording of the applicable provision (technical and organisational problems) is broad enough to include such complaints.

33. However, in the interests of clarity, EPSO should revise the wording of the information provided in the Frequently Asked Questions (FAQs) on its website concerning the complaint procedure. This information should reflect the wording of the General Rules, which list ‘technical and organisational problems’ as grounds for possible complaints whereas the FAQs currently refer simply to “technical problems”.

34. Overall, therefore, the Ombudsman finds no maladministration regarding (iii) EPSO’s failure to provide clear guidance about the complaint procedure in relation to the test environment.

35. The Ombudsman also makes a suggestion for improvement regarding the information available to candidates on the possibility to request a change of a test station. She encourages EPSO to include information to this end in the FAQs section on its website.

Conclusion

Based on the inquiry, the Ombudsman closes this case with the following conclusion and four suggestions for improvement:

No further inquiries are justified regarding the issues raised by the complainant.

Suggestions for improvement

1. EPSO should ensure that all test centres make available not only earplugs, but also noise-cancelling headphones.

2. Once EPSO receives a complaint about a test centre, it should immediately check and keep the relevant CCTV recording in order to be able to deal properly with the complaint.

3. EPSO should remind staff at test centres to record, in writing, each complaint made on the spot.

4. EPSO should, on the FAQs section on its website, provide information on the circumstances under which candidates can request a change of a test station. EPSO should also revise the wording of the FAQ regarding the complaint procedure to clarify that a complaint may be made in relation to both technical and organisational problems, as set out in the General Rules Governing Open Competitions.  

 

Emily OʹReilly

European Ombudsman

Strasbourg, 07/11/2017

 

[1] Notice of open competition EPSO/AD/301/15 - Administrators (AD5), OJ C 92A, 19.3.2015, available at: http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3AC2015%2F092A%2F01

[2] This includes requirements on IT hardware, including keyboards, minimum table size, provision of earplugs and calculators, among other things.

[3] Decision of the European Ombudsman closing his own-initiative inquiry OI/9/2010/RT concerning the European Personnel Selection Office, available at: https://www.ombudsman.europa.eu/cases/decision.faces/en/10428/html.bookmark

[4] EPSO referred to the principle of equal treatment as defined in the ECJ judgements (T-159/95, Dricot e.a. v Commission, grounds 83 and 98, and in T-225/95, Chiou v Commission, grounds 48 and 66, T-376/03, Hendrickx v Council, paragraph 33; F-2/07, and Matos Martin v Commission, paragraph 183).

[5] All complaints must be recorded in the incident log as a standard practice.

[6] Not the test centre where the complainant had done her CBTs.

[7] EPSO does not have any rules in place to this end.

[8] The latter have been recently introduced.

[9] EPSO added that the retention policy may vary depending on the national law of a Member State where the testing centre is placed. For instance, the retention policy in Ireland is limited to 30 days only.

[10] Except for the headphones, which were introduced at a later stage.

[11] In this case, the complainant made a complaint on the same day as the tests.

[12] The issue of EPSO’s failure to provide for satisfactory instructions to the contractor’s staff, has been previously addressed in a number of Ombudsman’s inquiries. In the course of inquiry 688/2012/RT, the Ombudsman suggested to EPSO to introduce rules on cases where the normal running of the tests is disrupted, which would serve as guidelines for the contractor. These guidelines seem to be now set out in the Client Practice Document, which contains detailed information on the standard EPSO procedures. Should any issue or irregularity occur, the contractor sends a daily Centre Procedure Report informing EPSO of any incident that could have impact on the candidates’ performance during the test or the result. In this context, the contractor stated in the explanatory e-mail sent to the Ombudsman that “[a]ny issues and serious disturbances during the testing window must be reported to EPSO immediately (at the latest on the next working day following the incident), accompanied by a list of all candidates affected and the remedy action taken”.

[13] Section 3.4.1. of the General Rules Governing Open Competitions - “What to do if something goes wrong?

If, at any stage of the competition procedure, you encounter a serious technical or organisational problem which is likely to impact your participation in the competition, please inform EPSO immediately, in order to allow us to investigate the issue and take any corrective measures.

You must do this in one of the following ways:

— for problems occurring outside test centres (such as those concerning the application or booking process), please send EPSO a message via the contact form (...) on our website with a brief description of the problem encountered,

— if the problem occurs at a test centre, please alert the invigilators and ask them to record your complaint in writing, and send EPSO a message via the contact form (...) on our website with a brief description of the problem encountered.

Please always quote the competition number and your application number in your message”.

[14] FAQs section of EPSO’s website with instructions for sitting the computer-based tests reads as follows: Tutorial

(...) “We will not accept any complaints about the test environment and navigation which were explained during the tutorial”.

[15] The CBTs manual states: “we strongly advise you to follow the tutorial and will not accept any complaints about the test navigation and environment resulting from not being familiarised with the tutorial”.