The background to the complaint

1. The present complaint concerns competition EPSO/CAST27/6/07 organised by the European Personnel Selection Office (EPSO), in which the complainant took part.

2. On 25 October 2007, EPSO informed him that he had failed to meet the threshold required to pass the verbal and numerical reasoning test. The pass mark for this test was 50%, whereas the complainant obtained 43.33%.

3. In his complaint to the Ombudsman, the complainant contested the answers to certain questions in the verbal and numerical test.

The subject-matter of the inquiry

4. The Ombudsman identified the following allegation:

The complainant alleged that the answers to questions 30, 41 and 44 in the verbal and numerical test (test B) of competition EPSO/CAST27/6/07, identified by EPSO as correct, were in fact incorrect.

5. The complainant did not submit any claims.

6. In his complaint to the Ombudsman, the complainant also contested the content of question 28 of the said test and submitted that two replies appeared to be correct. The Ombudsman, however, took the view that EPSO's explanation concerning this question was reasonable. In accordance with Article 195 of the Treaty establishing the European Community, the Ombudsman considered that there were therefore insufficient grounds to open an inquiry as regards this aspect of the complaint.

7. The complainant also contested question 31 of the same test. On 27 November 2007, EPSO informed him that this question had been cancelled in all language versions, since none of the answers provided was correct. In these circumstances, the Ombudsman considered that there was no need to examine question 31 in the framework of the present inquiry.

The inquiry

8. On 22 February 2008, the Ombudsman opened an inquiry and asked EPSO to submit an opinion by 31 May 2008. On 11 June 2008, EPSO's opinion was forwarded to the complainant with an invitation to make observations. No observations were received from him.

The Ombudsman's analysis and conclusions

A. As regards the complainant's allegation

Arguments presented to the Ombudsman

9. On 12 November 2007, the complainant wrote to EPSO contesting the answers to questions 28, 30, 31, 41 and 44. In a further letter sent to EPSO and dated 12 December 2007, the complainant insisted that "at least" the answers to questions 28 and 41 were incorrect and asked that EPSO should provide a convincing explanation. In a letter to EPSO dated 8 January 2008, the complainant provided specific arguments as to why answers to questions 28, 30, 41 and 44 were, in his view, wrong.

10. On 27 November 2007, EPSO informed the complainant of its view that the test questions which were not cancelled were "correctly formulated and translated". It added that this assessment also applied to those questions which, in the complainant's opinion, contained serious errors.

11. In a further letter to the complainant[1] and in its opinion, EPSO pointed out that for the verbal reasoning test it was important to select the correct answer on the basis of the context provided. EPSO explained that "it [was] not the absolute truth that [was] important, but the correctness of the answer with respect to the context given."

12. The specific arguments advanced by the complainant and EPSO as regards the three contested questions are set out in the following paragraphs.

As regards question 30:

13. The text on the basis of which question 30 was formulated and the question itself read as follows:

"EU research on the reconstruction of the human cornea

EU-funded research to reconstruct the human cornea could revolutionise eye surgery. This research will allow many to see again and will limit animal testing. Corrective eye surgery, which renders corneas unsuitable for grafting, has contributed to the international shortage of cornea donors. The development of artificial corneas will make up for the shortage as well as reduce the risk of passing on transmissible diseases through surgery. This research will also enable animal testing of cosmetics and pharmacological products to be reduced. EU legislation banning the marketing of cosmetics that are tested on animals will soon come into force and will further reduce this practice."

"Question 30: Which of the following statements regarding research on reconstruction of the human cornea is correct?

a) It basically aims to solve the shortage of cornea donors.

b) It precedes new legislation banning animal testing of cosmetics.

c) ...

d) ..."

14. The complainant argued that the linkage between cornea donors and the testing of cosmetics was not obvious in the text. Cosmetics, in his view, refer "to any preparation applied to the body, especially the face with the intention of beautifying". Given that the question concerned the reconstruction of the human cornea, answer (b), which referred to the testing of cosmetics, therefore seemed irrelevant.

15. EPSO noted that the complainant had chosen answer (a). However the correct answer was (b). The fact that the research on reconstruction of the human cornea "precedes new legislation banning animal testing of cosmetics" was substantiated in the last paragraph of the text, which stated that the "EU legislation banning the marketing of cosmetics that are tested on animals will soon come into force and will further reduce this practice". EPSO further pointed out that statement (a) was incorrect, since reducing the shortage of cornea donors was an advantage, but did not constitute the essential and the single objective of this research.

As regards question 41:

16. The text on the basis of which the question was formulated and the question itself were worded in the test as follows:

"The Commission acts to combat illegal logging

The European Commission has adopted a set of measures to combat the very worrying problem of illegal logging and the related trade in illegally harvested timber that robs governments of billions of euro every year in lost revenue in the developing countries concerned. The main measures involved: (i) establishing voluntary partnerships with wood-producing countries affected by illegal logging with a view to supporting and promoting governance reform in this sector; (ii) drawing up a regulation introducing a legally binding licensing scheme with partner countries to ensure that only legal timber from these countries is allowed into the EU. The measures are based on an approach that links the promotion of good governance in developing countries with legal instruments."

"Question 41: Which of the following statements is not correct?

a) Wood-producing countries are not required to work in partnership with the Commission.

b) (....)

c) Countries not working in partnership will no longer be able to export wood to the European Union.

d) Wood-producing countries can draw significant benefit from working in partnership."

17. The complainant argued that both answers (c) and (d) were not correct. In his view, if statement (c) was incorrect, because the countries would appear to mean "any" wood-producing country either affected or not affected by the illegal logging, then the same reasoning would have to apply to the "wood-producing countries" in statement (d). For the latter, it would mean that only those affected by illegal logging can significantly benefit from working in partnership. The countries not affected by the illegal logging would not - in the complainant's view - necessarily benefit "significantly" from working in partnerships. Therefore, statement (d) was also incorrect. The complainant added that it would have been correct if it read: "Wood-producing countries, affected by illegal logging (complainant's underlining) can draw significant benefit from working in partnership". The complainant further submitted that he was unable to understand EPSO's reasoning concerning this question, since it was not clear how the illegal logging could be reduced if the affected country was not entering into partnerships which were not-mandatory. He argued that, based on the text, the reduction of illegal logging, which generated considerable monetary losses, would only be possible if those countries affected by illegal logging entered into partnerships with the EU.

18. In its opinion, EPSO remarked that the complainant had selected answer (a), whereas he should have selected answer (c), since all the other statements proposed were correct and he had to choose the one that was incorrect. EPSO reiterated what it had previously pointed out to the complainant, namely, that answer (d) was correct, since the partnership was not mandatory but allowed the reduction of the illegal logging which was the source of a considerable loss of revenue. The verb "can" in the context of the text was used in the meaning of "being able, having the possibility". If this was meant to refer to all countries, the verb "will" would have been used instead.

As regards question 44:

19. The text on the basis of which the question was formulated and the question itself read as follows:

"Ministers agree on coordinated approach to influenza

The Ministers agreed on several measures for a coordinated approach to influenza preparedness cooperation. These involved informing each other in advance of taking any measures through the EU's communicable disease network and its Early Warning and Response System, organising concertation with those in the profession to look at capacity issues arising for basic treatment (vaccines and antivirals), developing a more integrated approach between animal and public-health experts, acting in coordination in various areas such as research and communication and keeping up close cooperation with WHO and other international bodies. The Commission is currently able to obtain valuable data on influenza outbreaks in 22 European countries."

"Question 44: What is the general principle behind the measures mentioned?

a) To be better prepared for any outbreak through better international cooperation.

b) To stop cooperating in order to deal with the influenza outbreak.

c) To react better to the influenza outbreak through better cooperation.

d) To conduct research on transmissible diseases through a coordinated approach."

20. The complainant argued that statement (a), rather than statement (c), was the correct answer, since the ministers had agreed - on an international level - on measures for a coordinated approach to influenza preparedness cooperation. According to the complainant, they did not agree "to any coordinated approach to influenza reaction cooperation". The complainant also pointed out that the key statement of the text was found in the first two lines of the text: "influenza preparedness cooperation" (complainant's underlining).

21. In its opinion, EPSO noted that answer (a) "to be better prepared for any outbreak through better international cooperation" referred to "any outbreak", that is, not only an outbreak of influenza. According to EPSO, the correct answer was (c) because the general principle behind the measures mentioned in the text is that of a coordinated approach, leading to a cooperation allowing Member States to deal with the outbreak of influenza.

The Ombudsman's assessment

22. As regards question 30, the Ombudsman carefully analysed EPSO's reasoning concerning answer (a), which was that reducing the shortage of cornea donors was indeed an advantage, but was neither essential for, nor the sole objective of, research on the reconstruction of the human cornea. The Ombudsman considers that the text on which the question is based indeed does not state that the research in question "basically aims" to solve the shortage of the cornea donors. Moreover, the relevant text lists other advantages that are expected to flow from this research, namely, a reduction in the risk of passing on diseases through surgery and a possibility to reduce the testing of cosmetics on animals. The text does not indicate that these advantages are less important than that of making up for the shortage of cornea donors. In the Ombudsman's view, EPSO's reasoning as to why answer (a) could not be considered correct appears reasonable. In addition to that, answer (b) can clearly be derived from the text on which it is based.

23. As regards question 41, EPSO took the view that statement (d) was correct, since the partnership was non-mandatory and it permitted the countries involved to reduce illegal logging (which was the source of a considerable loss of revenue). The Ombudsman considers that EPSO's reference to the voluntary nature of the partnership would seem to be irrelevant in this context. However, he agrees that the text on which the question is based allows the conclusion to be drawn that countries cooperating in the framework of partnerships can draw "significant benefit" from this cooperation. Given that the usage of the word "can" indicates a possibility, the Ombudsman considers that the complainant's argument that this possibility was only available to countries affected by illegal logging is not convincing. As EPSO submitted, statement (d) does not indicate that all wood-producing countries will necessarily benefit from such partnerships. Consequently, EPSO's explanation as to why statement (d) was correct, and was therefore not the right answer for question 41, appears to be reasonable.

24. In view of the above conclusions, the Ombudsman considers that, since the relevant explanations provided by EPSO appear reasonable, the complainant's allegation cannot be sustained in so far as questions 30 and 41 are concerned.

25. As regards question 44, the Ombudsman notes that it is not in dispute that answers (b) and (d) are incorrect and therefore do not need to be considered here. EPSO's argument that answer (a) could not be correct, since it refers to "any outbreak" and not only to an outbreak of influenza, is at first sight plausible. Since the text on which the question is based only concerned the approach to be adopted as regards influenza, a reply which refers to "any outbreak", that is to say, outbreaks of any diseases, would indeed appear to be too wide and thus incorrect at first sight.

26. However, it can be argued that the replies have to be seen and interpreted in light of the text on which they are based. Seen from this perspective, the term "any outbreak" could thus be understood as referring to outbreaks of the disease mentioned in the underlying text, that is to say, outbreaks of influenza.

27. In view of the above, the Ombudsman considers that, in order to arrive at a final conclusion concerning this issue, it is necessary to examine the answer that EPSO considers to be correct, namely, answer (c). If it were to emerge that this answer could logically be derived from the underlying text, EPSO's view that answer (a) is wrong would be confirmed.

28. According to answer (c), the general principle behind the measures mentioned in the underlying text was "to react better to the influenza outbreak through better cooperation". However, the complainant has pointed out that answer (c) concerned a reaction to outbreaks of influenza ("to react better"), whereas the underlying text itself referred to "influenza preparedness cooperation". The Ombudsman considers that this is a pertinent observation. The text on which the answers are based indeed mentions that the measures that had been agreed concerned the "preparedness". There is no reference to a "reaction". It should further be noted that the text also mentions that ministers are to inform each other "in advance" of taking any measures, which would also seem to suggest that the stress lies on being prepared rather than on reacting to actual outbreaks of influenza. In the Ombudsman's view, answer (a) would clearly be the correct answer if the term "any outbreak" were to read "any outbreak of influenza".

29. EPSO has stressed that the principle behind the measures mentioned in the text is that of a coordinated approach, leading to cooperation between Member States. However, both answer (a) and answer (c) refer to this cooperation. Аnswer (a) mentions a "better international cooperation", whereas answer (c) refers to "better cooperation". This aspect therefore does not allow the conclusion to be drawn that only answer (c) can easily or logically be derived from the underlying text.

30. Most importantly, the Ombudsman notes that answer (c) refers to "the" influenza outbreak. This wording would seem to suggest that the underlying text concerns a specific outbreak of the said disease. However, at no point does the underlying text make any reference to a specific outbreak of influenza. In the Ombudsman's view, the relevant text suggests that the coordinated approach approved by the ministers and the general principle behind the measures mentioned there goes far beyond responding to a specific outbreak of influenza. In order to remove these doubts, answer (c) would thus have had to refer to "any outbreak of influenza" rather than "the influenza outbreak".

31. In view of the above, the Ombudsman arrives at the following conclusions: Answer (a) is incorrect, if one adopts the logical argument put forward by EPSO concerning the expression "any outbreak". Given that answers (b) and (d) are clearly incorrect, this would mean that only answer (c) can be correct. However, it is difficult to reach this conclusion, if one considers that this answer focuses on a term ("to react") that is not mentioned in the underlying text and, even more importantly, uses an expression ("the influenza outbreak") which is highly imprecise.

32. In the Ombudsman's view, questions put to candidates in the context of a verbal reasoning text should be such that only one answer can be considered to be correct, even though it may be difficult and time-consuming to arrive at the correct conclusion. However, in the present case, it is not possible to arrive at a firm conclusion as to which should be the correct answer to the relevant question. In these circumstances, the Ombudsman concludes that the complainant's doubts concerning the possible choice of answers to question 44 are legitimate. In the Ombudsman's view, EPSO used a question without ensuring that the correct answer could be derived from the underlying text, without leaving room for reasonable doubt. This constitutes an instance of maladministration.

B. Conclusions

On the basis of his inquiries into this complaint, the Ombudsman makes the following critical remark:

By including a question in the verbal reasoning test without ensuring that the correct answer could be derived from the underlying text without leaving room for reasonable doubt, EPSO committed an instance of maladministration.

The complainant and EPSO will be informed of this decision.

 

P. Nikiforos DIAMANDOUROS

Done in Strasbourg on 2 April 2009

---

[1] EPSO's letter to the complainant of 18 December 2007