Decision of the European Ombudsman closing his inquiry into complaint 1793/2009/(JMA)MHZ against the European Commission

The background to the complaint

1. In 2008, the Commission published the Notice of the Call for proposals for a specific project covered by the 7th Framework Programme ('the Call'). The aim of the project was to enhance bilateral cooperation between the Commission and those countries with which it was negotiating a Science and Technology agreement. One of these countries was X.

2. The present complaint was submitted by a University professor acting on behalf of an University in the UK ('the complainant'). He was the coordinator of a consortium composed of that University and an University in the country X. He submitted the proposal ('the Proposal').

3. On 31 March 2009, the Commission informed the complainant of the evaluation results for the Proposal, which was carried out with the assistance of a panel of independent experts on the basis of three criteria published in the Call. The first criterion was scientific and/or technological excellence (S&T); the second was the potential impact of the project's results; and the third was the quality and efficiency of the implementation and management. The Proposal was awarded the following marks. For the first criterion, 3 out of 5; for the second criterion, 3 out of 5; and for the third criterion, 2,5 out of 5.

4. The Commission notified the complainant that the total awarded for the Proposal fell below the minimum threshold of 10. Likewise, marks for the third criterion were below the minimum threshold. The Commission had, therefore, decided to reject the Proposal. Attached to the Commission's communication was the Evaluation Summary Report on the Proposal ('the ESR'), drafted by independent experts. The Commission also informed the complainant of his right to start a redress procedure. The communication included a disclaimer which stated that the redress procedure applied to the manner in which the Proposal was handled during the evaluation and eligibility-checking process, but that the judgement of the appropriately qualified groups of experts would not be called into question.

5. On 7 April 2009, the complainant filed a request for redress with the Commission. He argued that the experts' comments on the content of the Proposal in the ESR showed that they had not considered all the information included in the Proposal.

6. On 7 July 2009, the Commission informed the complainant that his request for redress had been reviewed by an internal evaluation review committee ('the Redress Committee'). The Redress Committee had concluded that the additional explanations provided by the complainant in his request for redress could not be taken into account because they were submitted after the evaluation process had been concluded. The Redress Committee did not find any grounds to support the complainant's request, and therefore dismissed it.

7. In view of the Commission's reply, the complainant lodged a complaint with the European Ombudsman.

The subject matter of the inquiry

8. The Ombudsman decided to open the present inquiry into the following allegation and claim:

Allegation:

The complainant alleges that the Commission's reasoning for rejecting his proposal had no grounds, since the information allegedly missing was in fact included in the original proposal.

Claim:

The complainant therefore claims that the Commission should reassess his proposal, taking due consideration of all the information included in it.

The inquiry

9. The complaint was sent to the Ombudsman on 10 July 2009. On 23 September 2009, the Ombudsman opened an inquiry and sent the complaint to the Commission with a request for an opinion. On 17 December 2009, the Commission sent its opinion, which was then forwarded to the complainant with an invitation to submit observations by 28 February 2010. No observations were received from the complainant.

The Ombudsman's analysis and conclusions

A. Alleged poor reasoning due to the failure to take into account all the information included in the Proposal and the related claim

Arguments presented to the Ombudsman

10. The complainant argued that the experts' comments in the ESR in relation to the three evaluated criteria (the first: scientific and technological ('S&T') excellence, the second: the potential impact of the project's results, and the third: implementation and management), showed that the experts had not considered all of the information contained in the Proposal.

11. Regarding the first criterion, namely, S&T excellence, the experts considered that "the scope of activities is limited to two work packages with six workshops without details" (emphasis added by the complainant ).The complainant argued that the Proposal contained a detailed description of both work packages. The first work package was described in section 2.1, "Management structure and procedures", and the second work package was described in sections 1.1 and 1.2", respectively, namely, "Concept and objectives" and "Quality and effectiveness of the support mechanisms, and associated work plan.

12. Regarding the second criterion, namely, the potential impact of the project's results, the ESR stated that "the project disseminates knowledge, and exploits results mainly via websites and workshops, without participation of governmental officers, policy makers and stakeholders" and that "[no] direct or indirect help to the potential participation of X country teams in EU projects is planned in the project". The complainant argued that the Proposal contained the following information. First, in section 1.1 "Concept and objectives", on page 4, at points 4 and 5, it is stated that presentations on "Calls for proposal" will be made by Commission officials, and/or invited representatives of appropriate European Technology Platform(s). Second, it was stated that some main players in the appropriate fields in X country would be invited, together with officials from X country's government agencies. Third, on pages 7 and 8 of section 1.2 "Quality and effectiveness of the support mechanism, and associated work plan" there was a draft agenda on workshops. Fourth, paragraph 1 of page 7 of section 1.2 stated the rationale behind the use of academic conferences and/or industrial exhibitions, including the statement that "this would guarantee a large number of participants at the workshop from the sector addressed by the event from both the academia and industry in X country". Finally, section 1.1, "Concept and objectives", pages 5 and 8, explain how the "Workshops" and "Project website" should be of "indirect" help for the potential participation of X country's teams in the EU projects. The complainant did not make any specific arguments as regards the third criterion.

13. The Commission pointed out in its opinion that proposals were evaluated by a panel of independent experts. The Proposal was evaluated by panel 2.4 (X country).

14. The Commission stated that the Proposal was rejected because its total score was below the minimum threshold of 10, and the third criterion, regarding implementation and management, was evaluated as being below the minimum threshold. The experts' judgement was that the Proposal satisfied the first two criteria, although improvements would be necessary, and they judged the Proposal as “fair” on the third criterion. This meant that the Proposal had significant weaknesses as regards the third criterion. The Commission asserted that the independent experts had duly justified their decision, and, when awarding marks, they had fully respected the criteria established by the Commission in the Call.

15. The Commission gave a detailed explanation of the experts' evaluation regarding each criterion.

16. As regards the first criterion, the Commission explained that the Call included a document entitled "FP7 Capacities Work Programme 2009: Activities of International Cooperation" ('WP 2009'). WP 2009 outlined in detail the proposed activities that needed to be developed in the proposals in order to achieve the objectives of the Call. These activities were listed on pages 14 and 15 of WP 2009, and other activities specific to Area 2.4 X country were listed on page 19 of WP 2009. For the first criterion, the complainant was awarded 3 out of a maximum of 5 points. This was due to the fact that, compared to the number of proposed activities in WP 2009, the complainant proposed only a limited number of activities, namely, only two work packages with six workshops. Furthermore on page 15 of WP 2009 it was stated that "the participation of the third country authorities responsible for the follow-up of the S&T cooperation agreement or any organisation which has received the necessary mandate from the national authorities, is considered essential for the project and will be reflected in the evaluation". However, the Commission pointed out that no evidence of such a mandate was provided in the Proposal. In the Commission’s view, this failure "certainly" carried great weight in the opinion of the independent experts who wrote in the ESR that there was "a limited participation in the Project from both the EU and the X country sides".

17. As regards the second criterion, the Commission quoted the view of the independent experts as put forward by them in the ESR. It reads as follows: "Clear efforts are made to [promote] information dissemination through workshops, reports, [the] website and various documents. The direct and most easily measurable impact of the proposal would be the increase of X country participation into FP7 projects. However, the applicants themselves indicate that this cannot be guaranteed at least in the time of the project. Indeed the consortium only has 2 universities with limited international S&T cooperation experience, so their contribution at the European level will be low. In terms of dissemination results, the targeted groups as well as the dissemination channels applied do not represent a high level of engagement and ambition. Thus the project disseminates knowledge, and exploits results mainly via [the] website and workshops, without [the] participation of governmental officers, policy makers and stakeholders. Because of the limited scope and depth of the activities included in the Proposal, the potential impact of the Project is expected to be moderate."

18. In light of the above quotation, the Commission found that that the experts duly justified the score of 3 out of 5 which they awarded for the second criterion in view of the fact that they explained how improvements could be made to enhance the potential impact of the Proposal.

19. Finally, the Commission took a position on the complainant’s claim. First, it stated that, pursuant to "the Rules for submission of proposals and the related evaluation, selection and award procedures", the redress procedure covers only procedural matters. However, when examining the complainant’s request for redress, the Redress Committee also checked whether the independent experts were selected in an appropriate way, and whether they had the necessary qualifications to evaluate the proposals that were submitted for their judgement. This was done by examining the experts' CVs. As stated in the letter informing the complainant of the result of his redress request, the Redress Committee found that "the evaluators [were] suitably experienced and qualified to carry out evaluations in this objective." The Committee also found that the evaluation was carried out in full accordance with the rules. Therefore, the comments made by the panel of experts, as reported in the ESR, could not be called into question.

20. The Commission concluded that (i) the objectives of the Call were clearly explained in WP 2009; (ii) the experts duly justified their opinion on the complainant’s proposal in the ESR; and (iii) the marks awarded by the experts fully respected the criteria established by the Commission. In addition, the experts were selected in an appropriate way, and they were suitably qualified to assess the proposals. Moreover, the Redress Procedure was performed in compliance with the specific requirements. The Redress Committee noted that some of the information submitted by the complainant with the redress request was not included in the original proposal, such as the explanation for the budget, which was not well-balanced, a fact underlined by the independent experts in their ESR, and the clarification of what the complainant considered would help, directly or indirectly, the potential participation of X country teams. This new information could not be taken into account at the stage of redress.

The Ombudsman's assessment

Preliminary remarks on the Ombudsman's standard of review

21. The Ombudsman recalls that the complainant's allegation is essentially that the evaluation of its Proposal was substantially wrong. In this respect the Ombudsman points out that the evaluation of research proposals raises complex scientific questions. The Ombudsman considers that in examining such an allegation, his review should normally be limited to assessing whether there is a manifest error in the reasoning of the contested decision. The Ombudsman notes that this approach is in line with the standard applied by the Union courts.[1]

The procedure for the evaluation of the proposals

22. At the outset,, the Ombudsman points out that the rules governing the evaluation of proposals submitted for the Call are contained in the "Rules for submission of proposals, and the related evaluation, selection and award procedures"[2] ('the Rules'). The "Guide for applicants for Call " ('the Guide') was drafted on the basis of the Rules. The Rules provide for a two-stage examination of the proposals to be funded by the EU. During the first stage, each proposal is examined by independent, external[3] experts. The Commission then classifies the proposals in categories on the basis of the points awarded by the experts. During the second stage, the Commission makes a selection and draws up a list of proposals to be funded by the EU.

23. The present complaint concerns only the first stage of the procedure in which the Commission is assisted by independent experts who are responsible for examining proposals, in accordance with a strict procedure set out in the Rules, and for drawing up an ESR for each proposal. In the present case, the external experts evaluated and assessed the complainant's proposal on the basis of criteria set out by the Commission in Annex 2 to the Work Programme 2009 (copied in the Guide, Annex 2, page 30). Annex 2 sets out three criteria in detail, how they are scored, what the scores mean, and the minimum of points required.

24. The Ombudsman notes that the Rules provide that the Commission may be involved to a certain extent in the work of the experts, as described below. The Commission first establishes a database of experts. This contains the details of suitable candidates who responded to calls for applications published in the Official Journal. In order to evaluate the applications received in response to a specific call for proposals, the Commission draws up a list of appropriate experts taken from the database. When appointing experts, "the Commission must take all necessary steps to ensure that they are not faced with a conflict if interest in relation to the proposals on which they are required to give an opinion."[4] Experts then examine the proposals on the basis of the evaluation criteria, their weighting (if any), scoring scale and thresholds, all of which are previously established by the Commission. In this respect, the Commission briefs the experts before the evaluation sessions. The experts evaluate the proposals individually and subsequently hold a consensus meeting moderated by the Commission's representative. If no consensus is reached on scores, the Commission may ask three additional experts to examine the proposal concerned. The Commission ensures the quality of the consensus report (its clarity, consistency, and appropriate level of detail). The signing of the consensus report is followed by the panel review. The main task at the panel review stage is to formulate the experts' recommendations to the Commission. The panel is chaired by the Commission, or by an expert appointed by the Commission. In either case, "the Commission will ensure fair and equal treatment of the proposals in [the] panel's discussions."[5]

25. However, when it comes to the outcome of the panel "review", one of which is the ESR for each proposal, including experts' comments and scores, the Commission's involvement is limited. "The Commission will not change the ESRs that form part of the panel report, except if necessary to improve readability or, exceptionally, to remove any factual errors or inappropriate comments that may have escaped earlier proof-reading. The scores will never be changed".[6] Moreover, the internal redress mechanism (the Redress Committee) cannot review the scores and comments made by the independent experts either.

26. The shortcomings alleged by the complainant are not, however, comparable to errors which "may have escaped [the Commission's] earlier proof-reading". They relate to the very merits of the independent experts' assessment. The question therefore arises as to whether the Commission may be deemed responsible for such an assessment.

27. The Ombudsman considers that the answer to the above question should be in the affirmative. The Commission takes the final decision on the proposals submitted in the Call and therefore it needs to adopt the views of the experts for the purposes of its own decision, which is then communicated to the participants in the Call. In its decision, the Commission needs to put forward its own reasons why a proposal was rejected, if this is the case. Reasonably, the ESR would be an essential part of the Commission's reasoning for rejecting a proposal. In light of his findings in point 21 above, the Ombudsman will thus assess the quality of such reasoning in the present case. As noted in paragraph 25 above, the Commission considers that its role does not involve substantive modification of the ESR, or of the scores. However, given that the Commission takes the final decision and needs to provide adequate reasons for its decision, the Ombudsman takes the view that the Commission can and should take appropriate action in case any manifest errors come to its attention.

The quality of the ESR in question as part of the Commission's reasoning

28. In this respect he notes first that, pursuant to the Rules, each ESR should reflect the consensus reached by the experts, including the final review by the experts' panel. "It contains comments and scores on each criterion and an overall score, as well as providing overall comments when appropriate. The comments recorded must give sufficient and clear reasons for the scores and, if appropriate, any recommendations for modifications to the proposal should the proposal be retained for negotiation. In exceptional cases, possibilities for clustering or combination with other proposals may be indicated. For those proposals rejected after failing an evaluation threshold, the comments contained in the ESR may be only complete for those criteria examined up to the point when the threshold was failed."[7]

29. The Ombudsman further examined WP 2009, which was attached to the complaint, in particular the description of Activity 7.2 "Bilateral coordination for the enhancement and development of S&T partnerships - Z- Area X country", the Proposal, and the ESR in question.

30. He notes that the Call provides that specific emphasis should be placed on the following proposed activities for the EU-country X S&T cooperation: (i) organisation of events, workshops and seminars aimed at identifying S&T priorities of mutual interest, and setting up contacts for partnerships; (ii) identification of best practices and promoting their use in both the private and the public sectors; (iii) addressing the issues affecting the participation of X country organisations in the Framework programme, in particular in relation to administrative matters; (iv) seeking synergies and "complementarities" with initiatives such as the ERA-LINK network.

31. The evaluation criteria are defined in detail in Annex 2 to WP 2009, and in Annex 2 to the Guide, as follows. The first criterion "Scientific and/or technological excellence" comprises the evaluation of "soundness of concept and quality of objectives" and "quality and effectiveness of the support action mechanisms, and associated work plan". The second criterion "Potential impact through the development, dissemination and use of project results" will include the evaluation of "the contribution at the European and international level, to the expected impacts listed in the work programme under the relevant topic/activity/area", and "the appropriateness of measures for spreading excellence, exploiting results and disseminating knowledge, through engagement with stakeholders and the public at large". The evaluation under the third criterion, "Quality and efficiency of the implementation and the management", comprises the evaluation of "appropriateness of the management structure and procedures"; "quality and relevant experience of individual participants, quality of the consortium as a whole, appropriateness of the allocation and justification of the resources to be committed".

32. The Ombudsman notes that in the ESR in question, the independent experts referred to each criterion separately, and made their relevant descriptive comments. They scored the Proposal in relation to each criterion and gave the final score as well. As regards each criterion, the experts gave an overview of the entire Proposal and identified its weaknesses[8]. The Ombudsman considers, therefore, that the ESR was drafted in accordance with the procedure established in the Rules, and that the experts took into account the criteria and the objectives of the Call.

33. Furthermore, the Ombudsman notes that, on 31 March 2009, the Commission provided the complainant with its decision. It stated that "its Proposal would not be in line for ranking by the Commission." The Commission stated that the Proposal "failed one or more of the evaluation thresholds" and it attached the ESR on the Proposal. Since the ESR, which is part of the Commission's reasoning, was, as stated above, properly drafted, the Ombudsman considers that such a statement of reasons was sufficient, having regard to the context of the decision, to allow the complainant to determine whether it was well-founded, or whether it would be appropriate to request a redress procedure before the Court or the Ombudsman in order to review its legality[9]. The complainant has indeed done so, when submitting the present complaint.

The clarification provided by the Commission in the opinion

34. Moreover, in its opinion on the complaint (which the Ombudsman forwarded to the complainant), the Commission modified and clarified its reasoning for rejecting the Project.

35. As regards the first criterion, the ESR stated that the foreseen activities were not described in enough detail. The complainant, however, pointed out that the Proposal contained a detailed description of the two work packages and six workshops it proposed. In its opinion, the Commission modified its position to focus on the insufficient number of activities.

36. The Ombudsman notes in this respect that WP 2009 does indeed refer to a number of such activities, including the provision that specific emphasis should be put on the following in the proposed activities for the EU-X country S&T cooperation: (i) organisation of events, workshops and seminars aimed at identifying S&T priorities of mutual interest and setting up contacts for partnerships; (ii) identification of best practices, and promotion of their use in both the private and public sectors; (iii) addressing the issues affecting the participation of X country organisations in the Framework programme, in particular in relation to administrative matters; (iv) seeking synergies and "complementarities" with initiatives such as the ERA-LINK network. The Call did not specify how many activities would be considered sufficient, and this was a matter of judgement that fell within the discretion of the evaluators. The Ombudsman takes the view that the Commission’s revised reasoning, to the effect that the Proposal contained an insufficient number of activities and that this justified the score given by the evaluators, is reasonable. The Ombudsman notes in this context that the complainant did not submit observations contesting the revised reasoning.

37. In addition, in its opinion, the Commission clarified the meaning of the statement the experts made in the ESR on "a limited participation in the Project from both the EU and the X country sides." The Commission explained that the experts wished to point out that the Proposal did not indicate any organisation with a mandate from national authorities to follow-up the S&T cooperation agreement, as required in the Call. Although the Ombudsman noted references to a number of websites of certain X country academic societies, he was unable to find the name of any specific organisation in the Proposal.

38. As regards the second criterion, the Commission clarified in its opinion that the impact of the proposed projects depended on the involvement of the X country side. The experts did not consider that the X country participation, as foreseen by the complainant, would be sufficient, and they considered that the complainant itself could not it guarantee it ("The most significant exploitation of the outcomes of the project would be FP7 projects involving X country participation set up as a result of the workshops, however this cannot be guaranteed."[10]).

39. This clarification is reasonable. The Ombudsman notes in this respect that, although the complainant made several references in the Proposal to "X country participants", they were not specifically identified. For instance, the following wording was used: "some main players in the appropriate fields in X country", "officials from government agencies", "X country’s institutions participating in the Framework Programmes (if any)".

40. The Ombudsman notes that the Commission did not refer to the experts' assessment under the third criterion. The complainant did not comment on this in his complaint either. The Ombudsman nevertheless considers that the overall explanation given by the Commission in its opinion was reasonable.

41. In light of the above, the Ombudsman takes the view that there was no manifest error in the Commission's reasoning as modified and clarified in its opinion and he does not find therefore an instance of maladministration as regards the complainant's allegation.

42. As regards the complainant's claim, the Ombudsman considers that the Commission's explanation in its opinion (paragraph 19 above and the last sentence in paragraph 20 above) is also reasonable. The claim can thus not be sustained.

B. Conclusions

On the basis of his inquiry into this complaint, the Ombudsman closes it with the following conclusion:

There is no instance of maladministration by the Commission.

The complainant and the Commission will be informed of this decision.

 

P. Nikiforos Diamandouros

Done in Strasbourg on 14 October 2010

---

[1] See Case T-13/99 Pfizer Animal Heath SA v Council [1999]ECR-II-1961, paragraph 169.

[2] Version 3, August 2008, C(2008)4617. This document was submitted with the complaint.

[3] According to the Rules, Staff from relevant specialised EU agencies are regarded as external experts.

[4] Point 3.3 "Terms of appointment, Code of conduct and Conflict of interest", Page 10 of the Rules.

[5] Point.3.8 "Detailed evaluation of proposal evaluation", section (d) "Panel review", Page 16 of the Rules.

[6] Point 3.9 "Feedback to applicants", Page 18 of the Rules.

[7] Point 3.9 "Feedback to applicants", Page 19 of the Rules

[8] Under the heading of the first criterion, the experts stated the following:

"The proposal is fully relevant with the scope of the BILAT programme since it starts from the observation that X country participation in EU Framework programmes is not as high as other W countries such as M or N. Therefore, the proposal aims to provide information to the targeted audience in X, country in order to identify obstacles and enable X country organisations to participate into the Framework Programme. It outlines a clear structure for promoting the S & T co-operation between the EU and X country, with a particular focus on the EU Framework Programme. However, both the concept of the Proposal and the quality of its objectives do not completely fulfil the required criteria, because of weaknesses in the following aspects: 1) The scope of activities is limited to two workpackages with six workshops without details, and the set-up of a website . 2) Limited participation in the Project from both the EU and the X country sides."

Under the heading of the second criterion, the experts stated the following:

"Clear efforts are made to [promote] information dissemination through workshops, reports, website and various documents. The direct and most easily measurable impact of the proposal would be the increase of X country participation into FP7 projects. However, the applicants themselves indicate that this cannot be guaranteed, at least in the time frame of the project. Indeed, the consortium only has 2 universities with limited international S & T cooperation experience, so their contribution at the European level will be low. In terms of dissemination of results, the targeted groups as well as the dissemination channels applied do not represent a high-level of engagement and ambition. Thus the project disseminates knowledge, and exploits results mainly via [the] website and workshops, without [the] participation of governmental officers, policy makers, and stakeholders. Because of the limited scope and depth of the activities included in the Proposal, the potential impact of the Project is expected to be moderate. The gap between the information on the EU programmes made available (as planned in the project) and the actual involvement of X country participants in new EU projects is not considered herein. No direct or indirect help to [promote] the potential participation of X country teams in EU projects is planned in the project. That is why the increase of X country participation following the workshops remains unpredictable."

Under the heading of the third criterion, the experts stated the following:

"The project will be managed by a project team strictly limited to 2 partners, sharing the lead of the two main workpackages, which implies the lack of support at both the organisational and policy levels. The coordinator has no strong experience in S & T cooperation projects. The consortium shows poor complementarities and does not demonstrate sufficient capacity and adequate skill and/or previous track record in organising FP promotion activities. Therefore, it appears too limited in terms of capacity and competence to ensure a significant impact on the X country participation to EU FP7 projects. The budget is not well balanced (the budget for the coordinator is twice that of the partner with a similar effort in person-months)."

[9] See Case C-269/90 Technische Universität München [1991]ECR I-5469, paragraph 26.

[10] Page 24 of the Proposal, last sentence.