The background to the complaint

1. The complainant is a candidate who took part in EPSO's open competition EPSO/AST/111/10 - Secretaries (AST 1) on 15 February 2011. The complainant took the computer-based tests that formed part of the competition in one of the testing centres in Brussels. The notice of competition[1] provided for the following tests and pass marks:

Test	Title of the test	Points available	Minimum points needed for passing	Time available for test
Test a)	Verbal reasoning	Marking: 0-20	Pass mark: 10	35 minutes
Test b)	Numerical reasoning	Marking: 0-10	The aggregate pass mark for tests b) and c): 10.	20 minutes
Test c)	Abstract reasoning	Marking: 0-10		10 minutes
Test d)	Professional capability: accuracy and precision	Marking: 0-20	The aggregate pass mark for tests d) and e): 20	6 minutes
Test e)	Professional capability: prioritising and organising	Marking: 0-20		30 minutes
Test f)	Knowledge in the secretarial field	Marking: 0-20	Pass mark: 10	25 minutes

2. On 7 March 2011, the complainant wrote to EPSO. She pointed to the following issues that arose while she was sitting tests d) and e) of her computer-based examination: (i) the screen of the computer had temporarily turned itself off without her intervention and that she had immediately informed the supervisory personnel; the only way to solve the problem was to re-start the computer, which meant that she had lost some time; (ii) the time allotted for reading the detailed instructions for test d) had been too short; and (iii) the screen view on the test computer during test e) was too small for viewing so that she had to continuously move the picture up and down with the mouse, which also led to losing valuable time. The complainant thus asked to be allowed to retake tests d) and e).

3. On 9 March 2011, EPSO replied that the test centre concerned had informed it that the time lost due to the incident with the computer was "negligible". As for the issues related to the time allotted for the instructions and the size of the screen view, EPSO put forward that these conditions applied to all the candidates. If an exception would be made for the complainant, this would go against the principle of equal treatment and fairness. EPSO took the view that all candidates had to be treated equally and that the tests could not be repeated.

4. On 17 March 2011, EPSO notified the complainant of her test results. The complainant received 5.5 points out of 20 for test d) and 14.17 points out of 20 points for test e). In total, she thus received 19.67 points for tests e) and d), whereas the pass mark for the two combined tests was 20 points. The complainant scored above the pass marks on the other tests, i.e. tests a), b), c) and f).

5. On that same day, the complainant wrote again to EPSO. In respect of the principle of equal treatment, she put forward that other candidates were also free to express their views. She underlined that she had lost about one minute of the six minutes of time available for test d). Having scored 19.67 points, she had failed because she had missed just one answer (i.e., a mouse click). She thus asked EPSO to review its decision and to allow her to re-sit test d) and, perhaps, test e).

6. On 17 March 2011, she turned to the European Ombudsman and complained about the three issues mentioned under point 2 above.

The subject matter of the inquiry

7. The Ombudsman opened an inquiry into the following allegation and claim:

Allegation:

EPSO failed properly to carry out the complainant's tests in the framework of competition EPSO/AST/111/10. In support of her allegation, the complainant put forward that her computer switched itself off during test d) and needed to be restarted by the invigilators, which resulted in a loss of valuable time.

Claim:

EPSO should allow the complainant to re-sit test d).

8. As regards the other arguments put forward by the complainant, namely, as to the time allotted for the instructions in test d) and the size of the screen view in test e), the Ombudsman decided that there were insufficient grounds to take them up for inquiry. In his view, a selection competition was an eliminatory exercise, and it was inherent in such a procedure that some candidates would find it more difficult than others. EPSO put forward that the testing circumstances referred to by the complainant applied equally to all candidates, which the complainant did not dispute. Therefore, the Ombudsman considered that the complainant did not appear to have been put at a disadvantage compared to other candidates. Nor was there anything to show that the examination conditions were objectively unacceptable.

9. Based on the Ombudsman's conclusion on the preceding paragraph, the complainant's initial claim, in which she asked that EPSO allow her to re-sit both tests d) and e), was construed to refer only to test d), based on the allegation taken up for inquiry.

The inquiry

10. On 1 April 2011, the Ombudsman opened an inquiry and invited EPSO to provide its opinion on the complaint.

11. On 26 April 2011, EPSO submitted its opinion, which was forwarded to the complainant for her observations.

12. On 6 May 2011, the complainant provided her observations on EPSO's opinion.

13. On 14 November 2011, the Ombudsman's services carried out an inspection of EPSO's files at its premises in Brussels.

14. On 24 November 2011, the Ombudsman sent the complainant a copy of the inspection report and invited her to provide her observations thereon.

15. On 1 December 2011, the complainant submitted her observations on the inspection report.

The Ombudsman's analysis and conclusions

Preliminary remarks

The complainant's arguments related to the time allotted for the instructions in test d) and the size of the screen view in test e)

16. In her observations, the complainant reiterated her argument that the explanations for test d) were too long and the available time too short to understand them. She argued that she thus had to "learn by doing" what was asked of her. This had been indirectly admitted by EPSO with its corrigendum to the Notice of Competition, which provided that tests d) and e) would be considered together for a pass mark rather than separately; candidates who did not obtain above average results on test e) had no chance of passing. Moreover, the complainant also argued that she had passed all tests with the exception of test d). As for that test, she was only one mouse click short of passing it, which she would have done with a likelihood bordering on certainty, had the test been properly explained.

17. During the inspection, the Ombudsman's representatives informed EPSO´s representatives of the complainant's arguments. Even though this was not part of the inquiry, EPSO's representatives explained that, in the letter dated 3 December 2010 confirming candidates' registration and inviting them to the test, candidates were offered the opportunity to visit the tutorials and to consult a sample test prior to the actual examination. The complainant could thus have made herself familiar with what was expected from her beforehand.

18. In her observations on the inspection report, the complainant put forward that, even under the aspect of the equal treatment of candidates, her argument concerning the size of the screen view could not be rejected because each candidate was free to draw attention to the circumstances he or she considered relevant. The same applied as regards the time allotted for the instructions in test d).

19. The Ombudsman has taken note of the complainant's arguments related to the time allotted for the instructions in test d) and the size of the screen view in test e). However, he does not consider that they are such as to make him reconsider his initial assessment. The complainant does not dispute that she was in the same situation as other candidates. Nor is there anything to suggest that the examination situation that she and others found themselves in was objectively unacceptable. What is more, the complainant, like all other candidates, had been invited, prior to the tests, to visit a tutorial and make herself familiar with a sample test. It is thus not easy to see why she should be allowed to re-sit tests d) and e) because of the size of the screen view or the time allotted for instructions. Therefore, the Ombudsman maintains his conclusion that there are insufficient grounds to investigate the allegations related to the time allotted for the instructions in test d) and the size of the screen view in test e).

The complainant's new claims

20. In her observations on EPSO's opinion, the complainant argued that (i) if EPSO had properly explained test d) to her, she would have obtained the missing point; therefore, EPSO should now allow her to proceed to the next stage of the competition. In her observations on the report on the inspection, the complainant further argued that (ii) EPSO should compensate her for the stress she had suffered by assuming that she had given a correct answer on test d). The Ombudsman notes that these two new claims do not yet appear to have been brought to EPSO's attention, and thus need to be considered inadmissible for lack of prior administrative approaches pursuant to Article 2(4) of the Ombudsman's Statute. In any event, given the results of his assessment of the complainant's case (see below), the Ombudsman considers that these new claims would in any event appear to be unfounded.

A. Allegation of having failed properly to carry out the tests and claim that EPSO should allow the complainant to re-sit test d)

Arguments presented to the Ombudsman

21. In her complaint, the complainant argued that the monitor of her computer had temporarily failed to function during test d). As a result, she lost a critical minute of time during that test, which was supposed to last six minutes. In this context, she pointed out that she missed passing the relevant threshold by only one mouse-click. The complainant pointed out that candidates experiencing technical difficulties during a test, in particular, those leading to a loss of time, should be allowed to re-sit the specific tests concerned.

22. In its opinion, EPSO pointed out that, it had checked the issues relating to the complainant's computer with its contractor, and it emerged that the contractor had not signalled any problems related to test d). Test d), which was supposed to last six minutes, lasted indeed six minutes, without any interruptions.

23. The complainant claimed that she had lost one minute and that she would have needed only one further mouse-click to pass the test. However, she had answered only the first 18 questions of test d), and of these, she had correctly answered 11. She thus did not reply to 22 questions, and consequently received 5.5 points for that test. Test e) comprised 24 questions and lasted 30 minutes. EPSO submitted that there were no interruptions during that test either. The complainant had only 17 correct answers on test e). Given that she did not have the minimum score for tests d) and e) combined, she could not proceed to the next stage.

24. The complainant's computer had been re-started at 12:05, that is, during test a) which related to verbal reasoning and for which 35 minutes had been available. The complainant's test a) started at 11:45:34 and ended at 12:22:54. It thus lasted a total of 37 minutes and 20 seconds. Two minutes and twenty seconds had elapsed between the moment when the complainant had signalled the problem and the moment when the computer was restarted. However, the time spent on restarting the complainant's computer had not been deducted from the 35 minutes available for that test. EPSO further stressed that the complainant obtained a mark (15/20) that was above the minimum required for that test and that the problem she had encountered therefore did not appear to have affected her performance.

25. The staff at the test centres checks that all computers work properly at the beginning of the tests. If a screen was turned off, this would be immediately noticed. EPSO added that it was possible that a candidate could inadvertently unplug the screen if he or she moved it during the test.

26. As soon as the complainant signalled that her screen was not functioning properly, her computer was deactivated in order to stop the timer. A very small amount of time had passed between the moment when the complainant alerted the staff and the time it took to restart her computer.

27. EPSO also provided details on the time at which the complainant's other tests had started and ended. In this respect, tests d) and e) are particularly relevant:

	Start	End	Time required for test
Test d)	13:00:21	13:06:27	6 minutes
Test e)	13:07:32	13:37:46	30 minutes

28. EPSO pointed out that no problems in the functioning of the complainant's computer were established during test d). Therefore, it did not see a reason to allow the complainant to re-sit test d). EPSO did, however, express its regrets that, in its earlier reply to the complainant, it did not specify that the interruption in question had occurred during test a) and not test d), and that the timer is automatically blocked during these kinds of incidents. EPSO offered its apologies for any confusion that this might have caused on the part of the complainant.

29. The complainant argued that she had been unable to write down during which test the problem with the computer had occurred because it was forbidden to take notes during the test or to take along a watch/clock. Therefore, she explained that she could only contest EPSO's submissions concerning the time she spent on the tests in question.

30. Furthermore, according to the complainant, it would be commonly accepted that tests would be repeated in case of technical problems which resulted in the temporary lowering of performance in candidates who have suffered stress. Stopping the timer was not a sufficient measure. She thus suggested that EPSO should, in order to avoid a court action, compensate her for the technical problem by assuming that she had given a correct answer on test d), or to allow her to sit that test again.

31. The Ombudsman decided that an inspection of EPSO's file was necessary. During this inspection, his representatives examined, among others, the following documents: (i) printouts of the 'XML results file' from the complainant's testing computer (12 pages); (ii) screenshots of candidates' CBT results from EPSO's internal Talent system (6 pages); (iii) printouts of the 'log file' passages related to test a), to the interruption and resumption of test a) and to test d) (11 + 4 B3 pages); and (iv) the relevant Centre Problem Report (CPR) (1 page), drafted after the incident with the complainant's test computer.

32. The representatives of EPSO's contractor explained how the computer-based testing software functions. The testing computers are equipped with specific software that records all the navigation steps by the candidates, such as the time used on questions, the answers given, and so on. In total, three electronic files are kept.

33. In the first electronic file, each individual candidate's interaction is recorded locally in an XML results file. This file contains, among other things, information on the time spent on the individual tests and the time spent on individual questions. The XML results file is primarily used by EPSO. The XML results file of the complainant's whole test is 12 pages long. As to the second file, a copy of the XML file is also left as a backup on the server of the testing centre. For the third electronic file, the software used records a significantly more detailed 'log file'. For example, the complainant's log file of the whole test is 200 pages long. All the information in the XML results file also appears in the log file. In addition, the log file records the candidate's every click with regard to each question, marking a question, skipping questions, and so on. These files are normally kept by EPSO's contractor and provided to EPSO upon request.

34. Furthermore, when a candidate informs the invigilators at the testing centres of a problem, the latter drafts a CPR, that is, a short note of the problem that occurred. These reports are also available to EPSO, in order to compare possible complaints by candidates with the incidents recorded in the CPR and the computer-based records. The CPR concerning the complainant showed that the incident was due to the fact that she appeared to have unplugged the screen of her computer.

35. EPSO's representatives explained during the inspection that, on the basis of the XML results file, it could be ascertained that the complainant started test a) at 11:45:34, ended it at 12:22:54, and that the total length of time was thus 37 minutes 20 seconds, with 2104 seconds (that is, 35 minutes 4 seconds) of "clean" testing time, for a test which would normally run 35 minutes. The log file recorded that the problem with the complainant's computer occurred at question No 17 of test a).

36. EPSO's representatives pointed out that test a) had 20 questions altogether. Moreover, it emerged from the log file that, following the incident, the countdown of time available to the complainant had been interrupted; the computer was re-started; and she then continued her test from question No. 17 onwards, when the time started to run again. EPSO again stressed that the complainant had successfully passed test a).

37. The Ombudsman's representatives asked whether it was standard practice for the invigilators in such situations to inform candidates that the countdown of their available time had been interrupted. EPSO's representatives replied that that was indeed normal practice and that candidates would in any event tend to ask that question themselves in such circumstances.

38. Furthermore, EPSO's representatives pointed out that the complainant had the possibility to take a break of 10 minutes prior to taking test d). However, according to the XML results file, she decided to take that test after a break of only 264 seconds (slightly over 4 minutes). Test d) was supposed to last for 6 minutes. It emerged from the XML results file that the complainant completed this test in 362 seconds, that is, in 6 minutes. Therefore, there had been no interruption on that test. This was also confirmed by the log file.

39. The Ombudsman's representatives pointed out that, in her observations, the complainant submitted that it was normal to allow candidates to re-take tests or examinations where problems occurred. EPSO's representatives replied that, contrary to the complainant's view, there was no reason to let her re-sit a test or examination. Doing so would not be justified in the complainant's case and would amount to unequal treatment of other candidates, given that she did not lose any time. Moreover, since it was clear that the incident occurred in test a), the complainant could reasonably have requested to re-sit only that test. However, this would not change the fact that she had not obtained the pass mark for tests d) and e) combined.

40. In her observations on the inspection report, the complainant stressed that EPSO had admitted in its e-mail of 9 March 2011 that there had been a loss of time. She also pointed out that the contractor was dependent on EPSO, and that this affected the contractor's neutrality and the validity of its submissions.

41. In any event, the problem with the functioning of her computer led to a situation of stress, which, in turn, put her at a disadvantage in comparison to other candidates. In the absence of any effective remedy, she should be given the possibility to sit the relevant test section again.

42. The complainant reiterated that she did not proceed to the next stage of the competition because of a single mouse click. She considered that it was almost certain that she would have progressed to the next stage, had her computer functioned properly.

The Ombudsman's assessment

On the allegation

43. The Ombudsman notes that the complainant's allegation is based on an alleged technical problem in relation to test d) of her computer-based test. However, as revealed by the investigation, the test in which the complainant experienced a technical problem is test a).

44. In this context, it needs to be taken into account that: (i) the complainant contested EPSO's submissions to that effect before the inspection, and (ii) she questioned EPSO's contractor's independence and the information thus provided after the inspection. However, the Ombudsman has not found any evidence, indications or arguments that would support the doubts voiced by the complainant with regard to the information and evidence provided by EPSO or its contractor. It might be useful to add in this respect that the Ombudsman's services were able to examine three different documents in relation to the complainant's test. Although those documents contain varying levels of detail, they all convey the impression of a genuine record of the complainant's test performance and their data appear equally consistent when cross-checked between the different documents (see point 31 above). Therefore, the Ombudsman considers that it has been established that the complainant had encountered technical problems with test a) only and not test d).

45. In its initial reply to the complainant, EPSO argued that the technical problems encountered by the complainant caused only a negligible loss of time. However, in its opinion, EPSO explained that the interruption in question had occurred during test a) and not test d), and that the timer was automatically blocked during these kinds of incidents. EPSO offered its apologies for any confusion that this might have caused on the part of the complainant. Given that the complainant in this context referred to test d), in respect of which no technical problems appear to have occurred, her argument is not convincing. The Ombudsman will however assess the complainant's further argument, according to which, irrespective of whether the computer malfunction concerned test a) or test d), the stress she experienced due to that incident affected her overall performance.

46. In that regard, the Ombudsman has established that the complainant did not lose any time with respect to test d). In particular, the various testing records showed that the complainant had six minutes of full and uninterrupted testing time. The same conclusions on full and uninterrupted testing time also apply with regard to test e). Moreover, the Ombudsman considers plausible EPSO's explanation that, in case of a computer malfunction, its staff would inform a candidate, if the candidate had not already raised that issue himself or herself, that the timer stops until the problem is resolved. In any event, the inspection showed that, although the complainant had the possibility of opting for a break of 10 minutes before commencing test d), she started that test after a break of only 4 minutes. This would appear to indicate that the stress that the complainant may have suffered on account of the incident that happened during test a) did not seriously affect her subsequent performance. Against this background, no maladministration can be found as regards the complainant's allegation.

On the claim

47. Given that no maladministration was found concerning the complainant's allegation, her claim cannot succeed either.

48. In light of the foregoing, the Ombudsman also finds no maladministration with regard to the complainant's claim.

49. In conclusion, the Ombudsman would like to acknowledge and recognise EPSO's constructive cooperation with the Ombudsman in the present case and the way in which information pertaining to computer-based tests is recorded by EPSO, which is commendable.

B. Conclusions

On the basis of his inquiry into this complaint, the Ombudsman closes it with the following conclusion:

No maladministration has been found in the present case.

The complainant and EPSO will be informed of this decision.

 

P. Nikiforos Diamandouros

Done in Strasbourg on 18 December 2012

---

[1] The notice of competition was published in OJ 2010 C 312A, p. 1. A corrigendum to the said notice of competition was published in OJ 2011 C 68A, p. 2.