Decision of the European Ombudsman closing the inquiry into complaint 202/2010/VL against the European Commission

The background to the complaint

1. The complainant is an SME. It was the leader of different consortia that applied for research funding under the European Commission's Sixth Framework Programme ('FP6') and Seventh Framework Programme ('FP7'). The proposals focused on the development of a radio frequency transponder for use in manufacturing cars. The complainant was not satisfied with the evaluations of its funding proposals under FP7, in particular in view of the evaluations of the proposal under FP6.

The proposal submitted under FP6

2. In 2004, the complainant's consortium submitted a proposal ('the FP6 proposal') in reply to the FP6-2004-SME-COOP call for proposals, which was implemented under the 'Cooperative Research' specific funding programme. The FP6 proposal was listed for funding, but not ranked high enough to be supported financially due to the available budget. The evaluation summary report ('ESR') set out the following:

1. Relevance to the objectives of Co-operative Research	Threshold 3/5	Mark: 4,5
2. S&T excellence	Threshold 3/5	Mark: 4
3. Potential impact	Threshold 4/5	Mark: 4
4. Quality of the consortium	Threshold 3/5	Mark: 4
5. Quality of the management	Threshold 3/5	Mark: 4
6. Mobilisation of resources	Threshold 3/5	Mark: 4
Overall remarks	Threshold 21/30	Total: 24,5

The first FP7 proposal

3. In 2007, a consortium led by the complainant applied for funding under the FP7 programme and the FP7-SME-2007-1 call for proposals which was implemented under the 'Research for the benefit of SMEs' specific funding programme. The application ('the first FP7 proposal'), included the ESR of the FP6 proposal, and drew the Commission's attention to the fact that what already had been a very good proposal was significantly improved. Once again, the consortium presented letters of interest from automotive manufacturers ready to test its transponder in their factories. The ESR for the first FP7 proposal contained the following evaluations:

1. Scientific and technological excellence	Threshold 3/5	Mark: 2,5
This part of the proposal is excessively long.... The approach for the development of the innovative transponder for the automotive assembly line is more convincing, but is not translated in a transparent and coherent work programme in particular with logically conceived milestones.
2. Quality and efficiency of the implementation and management	Threshold 3/5	Mark: 3
... The management procedures recognise that the project has a complex work programme structure that has many sub-tasks that will need good control to integrate all parts effectively. However, the procedures are vague on some issues and should be made more specific: composition of the management committee, decision making in case of conflict and in the interest of the SMEs', IPR issues. There are no milestones against which progress will be assessed at midterm. ...
3. Potential impact through the development, dissemination and use of project results	Threshold 3/5	Mark: 3,5
[...]
Total score	Threshold 10/15	Total: 9

The second FP7 proposal

4. In 2008, a consortium led by the complainant submitted an amended proposal ('the second FP7 proposal') for funding under the FP7-SME-2008-1 call for proposals, which also fell under the 'Research for the benefit of SMEs' specific funding programme. The new proposal kept the main innovations from the first FP7 proposal, but included sections explicitly addressing some of the alleged shortcomings identified in the ESR concerning the first FP7 proposal. The second FP7 proposal was evaluated as follows:

1. Scientific and technological excellence	Threshold 3/5	Mark: 2,5
... There is a real need for such transponders (tags) in the automotive industry and letters of intent from car manufacturers are included in the proposal.... The contingency plan is not adequately presented when it comes to the critical heat-transfer issues and there is no milestone or deliverable that includes thermal test until the end of the project. ... Although reports are due monthly, the milestone structure and verification points show shortcomings presenting risks which are not addressed in the contingency plan. There is no milestone against which a decision to continue the project at midterm can be taken. This is especially important with regard to the key issue of thermal tests. ...
2. Quality and efficiency of the implementation and management	Threshold 3/5	Mark: 3,5
... Resources are mostly allocated to the electronic/telecommunication side of the project, which is not the challenging part in this proposal. The resources to be allocated are too briefly commented. ... There is one research organisation but there is no true end-user involved as partner, which brings problems with verification and evaluation.
3. Potential impact through the development, dissemination and use of project results	Threshold 4/5	Mark: 3,5
[...]
Total score	Threshold 11/15	Total: 9,5

5. On 21 July 2008, the complainant wrote to the Commission. It pointed out that the first FP7 proposal was an improved version of the FP6 proposal, including an innovation significantly enhancing the scientific and technological outcome. What was already a good FP6 proposal had thus been improved even further. The complainant was therefore astonished that its first FP7 proposal received only 9 points. It expressed doubts as to whether the evaluators had actually grasped the ideas set out in the proposal and its potential. The complainant stressed that the second FP7 proposal had addressed all the alleged shortcomings of the prior proposals. In addition, it had been reworked, shortened and improved as regards its focus to a point where it was almost reduced to the previous submission under the FP6 programme. The complainant considered that the findings in the ESR relating to the second FP7 proposal appeared to have been copied from, or based on, the evaluation of the first FP7 proposal. It thus wondered whether the evaluators had, in fact, read the proposal or whether they had failed to notice that it had been changed, thus addressing the alleged shortcomings of the first FP7 proposal. Furthermore, it surmised that the evaluators may have faced a potential conflict of interests. The complainant further objected to the statement, contained in the ESR relating to the second FP7 proposal, that "there is no true end-user involved as partner, which brings problems with verification and evaluation" and argued that a number of automotive companies, that is, end-users, had provided letters stating their intention to carry out such tests. It wondered whether the evaluators realised how difficult it was for an SME to obtain a simple letter of interest from such companies, let alone to persuade them to join the consortium.

6. On 24 July 2008, the Commission replied and pointed out that its role was to ensure the necessary conditions for the evaluation. This implied the nomination of three independent experts, who, first, individually assess the proposal and then meet to discuss their findings so as to reach a consensus. This complex procedure was set up in order to obtain an independent and realistic assessment of the proposals and to limit the possibility of incorrect evaluations. However, the fact that the experts might identify new shortcomings was not discounted. The Commission was not in a position to question or comment on the opinions of independent experts. The divergence between the evaluation of the proposal submitted in the framework of the FP6 and the evaluation of the proposals submitted under the FP7 might be explained, for instance, by different weightings in both programmes. In particular, the exploitation of intellectual property rights ('IPR') was not an important aspect of the FP6, but was one of the most important criteria in the FP7. Moreover, each call had a pre-determined budget. As a consequence, out of the 267 funding proposals that passed the overall threshold in the call of the second FP7 proposal, it was envisaged that 98 projects receiving a mark of 12.5 or above would be funded. Finally, the notion of a conflict of interest was legally defined and experts who find themselves in such a situation are excluded from evaluations. As regards the complainant's case, it was most unlikely that all the experts who had dealt with its proposals were facing a conflict of interests. If the complainant considered that the Commission committed mistakes in the evaluation procedure, it could turn to the internal evaluation review committee ('the redress committee').

7. By letter dated 30 July 2008, the complainant lodged a complaint with the redress committee.

8. On 6 October 2008, the Commission sent the complainant the redress committee's decision. It pointed out that the committee's role was to consider whether there has been a failing in the evaluation process, and whether this failing was likely to jeopardise the decision of whether or not to fund the consortium's proposal. The committee neither evaluated the proposal itself nor called into question the scientific judgment of qualified experts. In the attached case report, the redress committee noted that following an examination of the evaluators' background, it was found that they were suitably experienced and qualified to carry out evaluations. These experts were not the same ones as those who had assessed the first FP7 proposal. However, and in line with internal rules, the experts who dealt with the second FP7 proposal had been given the ESR relating to the first FP7 proposal. They were free to assess the second FP7 proposal differently from the first FP7 proposal. The redress committee considered that the evaluation was carried out in a fair and transparent way and that the content of the ESR properly reflected the consensus reached by the panel of evaluators. It thus concluded that there was insufficient evidence to support the complaint.

9. On 21 January 2010, the complainant lodged the present complaint with the European Ombudsman.

The subject matter of the inquiry

10. The Ombudsman decided to include the following allegations put forward by the complainant in the present inquiry:

Allegations:

(1) The evaluation of the complainant's first and second FP7 proposals was incorrect, inconsistent and arbitrary, given that its FP6 proposal, on which the FP7 proposals were based, had received a very positive evaluation.

(2) The Commission acted wrongly in the procedure of evaluating proposals for funding under the FP7. In support of this allegation, the complainant presented the following arguments:

(a) the Commission acted wrongly by delegating its decision-making powers to third parties without providing for adequate safeguards and failed in its duty of supervision;

(b) the Commission failed properly to ascertain that there was no conflict of interests on the part of the evaluators who evaluated its first and second FP7 proposals;

(c) the Commission failed to provide the complainant with the names of the evaluators assessing its first and second FP7 proposals;

(d) the Commission acted wrongly by failing to make available to these evaluators the evaluation of the FP6 proposal; and

(e) the evaluators that were meant to assess the second FP7 proposal appeared to have based themselves on the evaluation of the first FP7 proposal without carrying out a proper evaluation of their own.

11. The complainant also put forward a claim that the Commission should provide it with access to its file. The Ombudsman noted, however, that there was no evidence to show that such a request had been submitted to the Commission. Therefore, the Ombudsman took the view that that claim was inadmissible pursuant to Article 2(4) of the Statute because it was not preceded by appropriate administrative approaches to the Commission.

12. In the observations submitted during the course of the present inquiry, the complainant reiterated its claim to be granted access to the file held by the Commission. The Ombudsman recalls that, if the complainant wished to be given access to the Commission's file, it would first have to make an appropriate request to that effect to the Commission. In any event, it emerges from the complainant's subsequent observations that it intended to make such a request for access to its file.

13. In its fourth set of observations, the complainant put forward a new allegation, which the Ombudsman decided to include in the inquiry. The allegation was worded as follows:

(3) The Commission failed to ensure that the evaluators it uses for evaluating proposals are adequately qualified to do so.

The inquiry

14. On 18 March 2010, the Ombudsman opened an inquiry into the present complaint and asked the Commission for an opinion.

15. On 18 June 2010, the Commission submitted its opinion.

16. On 30 June 2010, the Ombudsman sent the Commission's opinion to the complainant and invited it to provide its observations thereto.

17. On 19 July 2010, the complainant sent its observations on the Commission's opinion ('first set of observations').

18. On 10 February 2011, the Ombudsman informed the complainant and the Commission that the Ombudsman's services would inspect the Commission's file.

19. On 28 February 2011, the complainant sent further observations to the Ombudsman ('second set of observations').

20. On 9 March 2011, the Ombudsman's services carried out an inspection of the Commission's file.

21. On 20 April 2011, the Ombudsman forwarded the inspection report to the complainant, together with an invitation to make observations.

22. On 25 May 2011, the complainant provided its observations on the inspection report ('third set of observations').

23. On 3 February 2012, the complainant sent further observations ('fourth set of observations').

24. On 29 February 2012, the Ombudsman carried out further inquiries and asked the Commission to comment on a new allegation put forward by the complainant.

25. On 12 March 2012, the complainant sent a further set of observations ('fifth set of observations').

26. On 1 June 2012, the Commission provided its supplementary comments, which the Ombudsman forwarded to the complainant for its observations.

27. On 30 July 2012, the complainant submitted its observations on the Commission's supplementary comments to the Ombudsman ('sixth set of observations').

The Ombudsman's analysis and conclusions

Preliminary remarks

28. The present complaint concerns the evaluation of scientific proposals, which raises complex scientific questions. The Ombudsman takes the view that in examining cases of this kind, she should not substitute her own assessment for that of the scientific experts. Her review should instead be limited to assessing whether there is a manifest error in the reasoning of the contested decision or whether a procedural mistake has occurred.

29. In its third set of observations, the complainant queried why and on what basis the Commission classified the documents referred to, in the Ombudsman's report on the inspection, as confidential. It underlined that these documents might be confidential vis-à-vis the public but that they should be available to the complainant. In this context, the Ombudsman would like to draw the complainant's attention to the Implementing Provisions[1], which were adopted on the basis of the Ombudsman's Statute. They provide that the Ombudsman cannot grant access to any documents obtained through an inspection if the institution considers them to be confidential. It may be worth adding that the Commission in any event provided non-confidential versions (which could thus be disclosed to the complainant) for some of the documents it considered as confidential.

30. In its third and fourth sets of observations, the complainant queried why the Ombudsman's inspection report did not mention the evaluation of the FP6 proposal, on which all of its subsequent proposals relied. In its view, the Ombudsman's services were deceived by the Commission's services. The Ombudsman points out that her services did not consider it necessary to examine the documents concerning the evaluation of the FP6 proposal because the complainant did not challenge the assessment of that proposal.

31. In its fifth set of observations, the complainant suggested that the Ombudsman could add a new allegation concerning the evaluators' qualifications. However, the Ombudsman considers that this issue is already addressed by the third allegation.

A. Allegation that the evaluation of the first and second FP7 proposals was incorrect, inconsistent and arbitrary, in view of the FP6 proposal's evaluation

Arguments presented to the Ombudsman

32. The complainant referred to what it saw as inconsistency and incorrectness in the evaluation of its FP7 proposals. The complainant contested the statement in the second FP7 ESR that "... there is no true end-user involved as partner, which brings problems with verification and evaluation" and pointed out that it had submitted letters of intent from major automotive manufacturers, which allowed it to carry out tests in their factories. It also contested the statements in both FP7 ESRs with regard to the alleged absence of a milestone.

33. The complainant also argued that the evaluations of the two FP7 proposals had been arbitrary in respect of the length and detail of information needed to address specific aspects. In this context, the complainant referred to the following sections from the first FP7 proposal evaluation: "[t]his part of the proposal is excessively long" and "[h]owever, the procedures are vague on some issues and should be made more specific: composition of the management committee, decision making in case of conflict and in the interest of the SMEs', IPR issues.", and from the second FP7 proposal: "[t]he resources to be allocated are too briefly commented."

34. In its opinion, the Commission explained that both FP7 proposals were evaluated by external and independent experts with relevant knowledge and experience in respect of technological field of the proposals. After the complainant had launched the redress procedure for the evaluation of its second FP7 proposal, the redress committee[2] examined the curricula vitae of the evaluators involved. It concluded that they were experienced and qualified to carry out the evaluation. The redress committee confirmed that the evaluation of the complainant's second FP7 proposal was fair, transparent and respected established evaluation procedures.

35. Moreover, the two specific funding programmes involved (namely, 'Cooperative Research' and 'Research for the benefit of SMEs'), formed part of two successive Framework Programmes, (namely, FP6 and FP7) and could not be directly compared. It was neither illogical nor inconsistent that, in the context of two different programmes, the independent experts reached different evaluation results for similar proposals. The evaluation of proposals submitted under different Framework Programmes could not be compared since the proposals had been written to meet different call objectives, eligibility criteria and evaluation criteria.

36. In any event, the Commission disagreed with the complainant's statement that its FP6 proposal received a positive evaluation. This is because, even though it passed the minimum thresholds, the proposal was ranked too low on the priority list to be funded.

37. In its observations, the complainant stated that although the EU wishes and needs to strengthen its technological research, it rejected a research proposal for a product which had been sought and urgently needed by the automotive industry. Mainstream automotive manufacturers did not issue letters of intent in every case. The complainant rejected as bureaucratic the argument that evaluations carried out under different Framework Programmes were not comparable. In its view, a proposal was either useful or it was not. Moreover, the complainant insisted that its FP6 proposal had been evaluated very positively, which could already be seen from the scores given to it.

38. During the Ombudsman's inspection, the Commission's representatives pointed out, in respect of the evaluators' assessment, that peer review was a comparative evaluation, which did not look at proposals in isolation from each other. They further explained that subsequent proposals submitted by the same applicant might not be identical, were evaluated by different evaluators, and that, therefore, these proposals did not necessarily lead to the same mark.

39. In its observations on the inspection report, the complainant underlined that the marks given to its FP7 proposals would not have differed greatly from those given to its FP6 proposal, had they been considered together.

40. The Ombudsman carried out further inquiries and asked the Commission for supplementary comments on the complainant's following arguments that: (i) whilst the evaluation report concerning the second FP7 proposal stated that there was "no true end-user involved as partner, which brings problems with verification and evaluation", the complainant had obtained letters of intent from automotive manufacturers, which should be considered as end-users and which had expressed their preparedness to test the complainant's product; and (ii) the ESRs concerning both the first and the second FP7 proposal contained statements criticising the absence of midterm milestones, whereas the complainant stressed that its proposals did provide for relevant milestones, in particular, the "half-way development show" and the second FP7 proposal included a list of milestones. Lastly, the Ombudsman asked the Commission to clarify the differences of proposals being evaluated under FP6 and FP7.

41. In its supplementary comments, the Commission acknowledged that the letters of intent from automotive manufacturers were included in the proposal and had been taken into consideration by the evaluators. The evaluators recognised the high quality of the research project the consortium had submitted. However, they identified a weakness in that no true end-user was involved, as a partner, to ensure manufacturers' firm contractual commitment to carry out verification and evaluation, in particular in work package 5. The Commission pointed out that what this work package was expected to deliver was essential to fulfil the technical and scientific objectives of the project and to arrive at a final commercial product. The letters of intent could not be considered as legally binding on automotive manufacturers (that is, end-users) in the same way as if those manufacturers participated in the project as partners. The letters clearly demonstrated an intention to do something essential at a later stage, but did not create any firm legal obligation for manufacturers to do so. End-user partners in the consortium would have to sign a grant agreement with the Commission which would stipulate firm contractual legal obligations, responsibilities and rights from the start of the project. All members of the consortium were jointly responsible for carrying out the project as accepted by the Commission. This legal certainty of the entity's commitment to the project could not be achieved through letters of intent.

42. According to the relevant Guides for Applicants, milestones were stated as the control points at which decisions are needed in respect of the next stage of the project. A milestone may occur, for example, when a major result has been achieved if its successful attainment is required for the next phase of work. Another example would be a point when the consortium must decide which of several technologies to adopt for further development.

43. The first FP7 proposal was unclear as to whether it referred to milestones or expected results, as no list of milestones was provided. The way it was presented in the proposal made it impossible to understand what was considered under, for example "M4 finalizing most of the hardware and software components" in work package 2 and "M4 presentation of results" in work package 3 or "M4 report" in work package 4. Other references were made to M2, M3, M4 without any explanation. The experts concluded that "the approach for the development of the innovative transponder for the automotive assembly line is more convincing but is not translated in a transparent and coherent work programme in particular with logically conceived milestones" and "there are no milestones against which progress will be assessed at midterm". This perfectly described the reality of the proposal.

44. As regards the second FP7 proposal, the complainant did provide a list of milestones with a short description of the activities covered in each milestone. However, the evaluators considered that the proposal did not include a milestone against which a decision to continue the project at midterm could be taken.

45. The Commission also pointed out that FP6 and FP7 proposals were evaluated under two different scenarios and with different research objectives, eligibility and evaluation criteria and thresholds. In that respect, it submitted the following table comparing the relevant categories[3].

46. The complainant acknowledged that letters of intent were not as binding as contracts. However, it stressed that automotive manufacturers do not issue letters of intent lightly and would certainly have honoured them. In its view, one end-user would have been sufficient for verification purposes, the other end-users were only meant to show the wide acceptance of the proposal.

47. The complainant emphasised that, contrary to the Commission's assertions, it had provided a list of milestones in its proposal with detailed descriptions. The complainant concluded that the evaluators either did not study the proposal properly and the milestones included therein, or they consciously undermined the proposal, which highlighted the need to reveal the evaluators' identity.

48. With regard to the evaluation of proposals under FP6 and FP7, the complainant argued that there was no obvious difference between the two in respect of the research objectives, and the eligibility and evaluation criteria.

The Ombudsman's assessment

49. The complainant alleged that the evaluation of its FP7 proposals was, incorrect, inconsistent and arbitrary.

50. The complainant considered that the evaluation of its FP7 proposals was inconsistent because its FP6 proposal had received a very positive evaluation. The Commission emphasised that the FP6 and FP7 proposals as well as the specific funding programmes that they concerned could not be directly compared.

51. From the available evidence, the Ombudsman finds that the objectives, the eligibility criteria and evaluation criteria, and the weighting given to these criteria concerning the individual FP6 and FP7 proposals, while to a certain extent similar, were not identical. This holds true, in particular, as regards the evaluation criteria. For example, whereas the complainant obtained 4,5 out of 5 points for "Relevance to the objectives of Co-operative Research" as regards its FP6 proposal, the relevance to the call was merely assessed with a yes or a no under FP7, but no points were given in this respect. What is more, the two specific funding programmes do not only differ as regards the number and characteristics of the criteria to be checked, but also as regards the thresholds applied to them. It may be useful to add that the FP6 and the FP7 were based on two separate decisions, adopted by the Council and the Parliament, which set out the political priorities for funding, identified the specific funding programmes that should achieve these priorities and determined the duration of the specific Framework Programme. The FP6 decision[4] set the duration of the programme as 2002-2006, whereas the FP7 decision[5] set the duration of the programme as 2007-2013.

52. Against this background, it seems understandable that even a well-prepared research proposal aimed at meeting the requirements of one specific funding programme, might not get similar evaluation scores with regard to different specific funding requirements. Consequently, no lack of consistency can be established in this respect.

53. The complainant further submitted that the evaluation of its FP7 proposals was incorrect. In this respect, it referred to the statements made by the evaluators in the respective ESRs of the FP7 proposals concerning the alleged absence of milestones and of end-users as consortium partners. As mentioned at point 28, the Ombudsman's examination in this respect is limited to ascertaining whether a manifest error occurred.

54. The Ombudsman notes that the evaluators of the first FP7 proposal stated that "[t]here are no milestones against which progress will be assessed at midterm". The complainant stressed that a relevant milestone, namely, the "half-way development show" milestone had been inserted into the proposal. The relevant part of the proposal contains forms in which specific work packages, deliverables and milestones appear. It emerges that each of the aforementioned items was intended to have a specific function in managing the implementation of the project. The complainant pointed out that the "half-way development show" milestone was referred to in a number of places in the relevant section of its proposal. However, the complainant did not explain what that milestone was supposed to entail, how it would be verified or what decision in respect of the project would be taken at that point in time. It may be useful to note in this context that the Guide for applicants for the first FP7 proposal provided a structure that applicants should follow when submitting their proposals. This Guide referred to a table 1.3e, entitled 'List of milestones', as one of the forms supposed to set out the implementation concept. The complainant's first FP7 proposal did not contain such a form.

55. The Ombudsman further notes that, as the Commission correctly pointed out, the evaluators also took the view that the complainant's approach was "not translated in a transparent and coherent work programme in particular with logically conceived milestones". In this respect, the Ombudsman observes that the complainant referred to various milestones in different sections of its proposal without providing sufficient explanations or a coherent structure as regards these milestones. Consequently, no manifest error can be established with regard to the evaluation of the first FP7 proposal.

56. With regard to the second FP7 proposal, the complainant criticised the evaluators' conclusion that "[t]here is no milestone against which a decision to continue the project at midterm can be taken. This is especially important with regard to the key issue of thermal tests." The Ombudsman notes that, in the list of milestones (table 1.3e) submitted by the complainant with this proposal, the description of the "half-way development show" milestone was rather obscure ("thermal resistance")[6]. In the Ombudsman's view, this description, of itself, would not have been sufficient clearly to convey to an evaluator the intention to carry out thermal tests at that stage. More importantly, there was no explicit reference to thermal tests at the midterm in the outline of the project (setting out each task to be accomplished during the project). Instead, this outline refers to thermal tests to be carried out in a laboratory between 220°C and 700°C at the end of the project. The evaluators explicitly referred to that fact in the ESR ("The contingency plan is not adequately presented when it comes to the critical heat-transfer issues and there is no milestone or deliverable that includes thermal test until the end of the project."). Nor did the complainant's "half-way development show" milestone indicate anything about what kind of decision was to be taken at that juncture and, in particular, whether the result of the tests was to lead to a decision on whether to continue the research. The absence of such explanations was correctly identified by the evaluators. Thus, no manifest error can be established with regard to the evaluation of the milestones in the second FP7 proposal, either.

57. The complainant disagreed with the statement in the ESR of the second FP7 proposal that there was "no true end-user involved as partner, which brings problems with verification and evaluation". The Ombudsman notes that the evaluators, when making that statement, took into consideration the fact that automotive manufacturers had provided the complainant with letters expressing their interest in its research[7]. Furthermore, the Commission convincingly explained why having an end-user as a partner made a significant difference compared to mere letters of intent. Accordingly, no manifest error can be found in this respect either.

58. Finally, the complainant took the view that certain statements of the evaluators, as to the length and detail of information needed to address specific aspects of the proposal, had been arbitrary. However, a certain margin of appraisal needs to be acknowledged as inherent in carrying out an evaluation exercise. One set of evaluators might identify shortcomings in relation to a particular aspect of a proposal, whilst others might consider other shortcomings more significant. Arbitrariness could thus only be found to exist if the statements in the evaluations were made randomly, without a valid reason or clearly lacked credibility. The complainant has not established that this was the case here. In this context, it is useful to recall that the second FP7 proposal, which the complainant considered to be an improvement compared to its first FP7 proposal, did indeed receive a better score than the first FP7 proposal. In light of the foregoing, there is nothing to suggest that the evaluation of the FP7 proposal was arbitrary.

59. Against the above background, the Ombudsman finds no maladministration with regard to the complainant's allegation that the evaluation of its first and second FP7 proposals was inconsistent, incorrect or arbitrary.

B. Allegation of having acted wrongly in the procedure of evaluating proposals for funding under the FP7

Arguments presented to the Ombudsman

60. In its complaint, the complainant put forward the following arguments: (a) the Commission acted wrongly by delegating its decision-making powers to third parties without providing for adequate safeguards and failed in its duty of supervision; (b) the Commission failed properly to ascertain that there was no conflict of interests on the part of the evaluators who evaluated its first and second FP7 proposals; (c) the Commission failed to provide the complainant with the names of the evaluators assessing its first and second FP7 proposals; (d) the Commission acted wrongly by failing to make available to these evaluators the evaluation of the FP6 proposal; and (e) the evaluators that were meant to assess the second FP7 proposal appeared to have based themselves on the evaluation of the first FP7 proposal without carrying out a proper evaluation of their own.

61. In its opinion, the Commission rejected the complainant's allegation and arguments with the following submissions.

Argument (a)

62. The Commission explained that the evaluation of proposals by external experts was provided for in the Rules for participation in FP7. These rules were laid down in a regulation adopted by the European Parliament and the Council[8].

63. To evaluate the proposals submitted under the calls, for which the complainant submitted its proposals, the Commission drew up a list of appropriate experts for each call. The lists were drawn up primarily using two selection criteria: (a) a high level of expertise; and (b) an appropriate range of competencies. Additional criteria were also taken into consideration, namely: (c) an appropriate balance between academic and industrial expertise and user; (d) a reasonable gender balance; (e) a reasonable distribution of geographical origin; and (f) a regular rotation of experts.

64. The outcome of the evaluation process was an ESR for each proposal; a list of all the proposals passing all thresholds, together with a final score for each proposal passing the thresholds and the experts' recommendations for the order of priority; a list of all the proposals that failed to meet one or more thresholds; and a list of all the proposals that were found ineligible during the evaluation.

65. According to the rules for submission adopted in this context, the Commission will not change the ESRs, except to make them read better or, exceptionally, to remove any factual errors or inappropriate comments that may have been missed in earlier proofreading. The scores will never be changed. The ESR reflects the consensus reached by the experts, including the final review by the panel. The main task of the panel is to ensure the consistency of the scores and, where necessary, to propose a new score.

66. In the evaluation of proposals submitted under the relevant two FP7 calls, the Commission accepted the order of priority suggested by the experts.

Argument (b)

67. The experts who evaluated the first and second FP7 proposals signed an appointment letter with the Commission. This appointment letter binds the expert to comply with a code of conduct. In appointing experts, the Commission took all necessary steps to ensure that they were not faced with a conflict of interest in relation to the proposals on which they were required to give an opinion. To this end, experts were required to sign a declaration that no such conflict of interest existed at the time of their appointment and that they undertook to inform the Commission if such a conflict was to arise in the course of their work. In addition, all experts were required to confirm that they had no conflict of interest in respect of each proposal that they were asked to examine at the moment of the evaluation. None of the experts who evaluated the FP7 proposals submitted by the complainant declared any such conflict of interest with regard to these proposals.

Argument (c)

68. In accordance with the relevant rules, the names of the experts assigned to individual proposals are not made public in order to preserve their independence. However, once a year, the Commission publishes on the Internet the list of experts used for the Framework Programme and in each specific funding programme. The Commission added that it did not provide the names of the experts concerned to the complainant. This is in compliance with the applicable rules for reasons of personal data protection.

Argument (d)

69. The experts who evaluated the complainant's first and second FP7 proposals did not receive the previous FP6 ESR because this call was based on a different Framework Programme. The relevant rules state that evaluations of previous proposals are only provided to the experts if these previous proposals concerned the same Framework Programme.

Argument (e)

70. The evaluators of the complainant's second FP7 proposal proceeded to an individual evaluation unaware of the evaluation of the first FP7 proposal. However, both sets of experts arrived at the same conclusion, namely, that the proposal did not reach the minimum score to allow the project to be funded. Only at the end of the consensus phase, once agreement was reached on the scores and comments were the evaluators provided with a copy of the ESR concerning the first FP7 proposal.

71. In its first set of observations, the complainant reiterated that the Commission had not adequately ensured that obvious discrepancies in evaluations were identified and remedied. For instance, the difference between the initially outstanding evaluation of its FP6 proposal and the significantly worse result for its first FP7 proposal, which was an improved version of the former

72. The complainant further pointed out that it was unusual that evaluators could hide behind their evaluations and wondered whether the evaluators were afraid publicly to acknowledge their assessments. The complainant was not convinced by the Commission's argument that withholding the evaluators' names was done in order to preserve their independence and neutrality.

73. In its second set of observations, the complainant criticised the fact that the Commission accepted mere signed declarations on the absence of conflict of interest as sufficient.

74. Having examined these observations, the Ombudsman considered it necessary to carry out an inspection of the Commission's file.

75. At this inspection, the Commission's representatives recalled that none of the evaluators of the FP7 proposals had declared a conflict of interest with regard to the complainant’s proposals.

76. Furthermore, the Commission's representatives pointed out that, even if an evaluator tried to conceal a conflict of interest, the final score given to a proposal is the result of a consensus between three experts. Any attempt unduly to influence the outcome of the evaluation could be reported by the other experts or the Commission's representative moderating the consensus report stage and could lead to the exclusion of the expert. Furthermore, even if a possible conflict of interest had escaped the Commission's attention, it could have been discovered in the redress procedure.

77. In its observations, the complainant stated that it was unsurprising that no expert had declared a conflict of interest after having had a period of one week to sign the declaration. The situation at issue could be compared to a court proceeding where the assessment of the evidence was left to the experts summoned by the court. The complainant considered that such a secretive and covert evaluation could violate human rights.

78. The complainant added that it was surprised that the names of the evaluators had to be confidential if these evaluators had declared themselves not to be in a situation of conflict of interest.

79. In light of these observations, the Ombudsman carried out further inquiries and asked the Commission to explain why preserving the independence of the experts made it necessary to keep the names of the relevant experts confidential even after the evaluation had taken place. Given that the names of the evaluators participating in the Commission's specific programmes had been published in any event, the Ombudsman invited the Commission to explain why it considered itself unable to accept the complainant's request to disclose the names of the evaluators who assessed its proposals.

80. In its reply, the Commission explained that section 3.2 of the rules for submission applicable to the evaluations at issue in the present case provided that "the names of the experts assigned to individual proposals are not made public. However, once a year the Commission publishes on the internet the list of experts used for the Framework Programmes and in each Specific Programme" (emphasis added by the Commission).

81. In line with the rules for submission, the Commission did not make public or provide the participants in FP6 or FP7 research the names of the experts allocated to specific proposals. This was in conformity with the case-law of the Court of Justice related to access to documents and the protection of personal data, according to which personal data, such as names of experts, could not be transferred unless the conditions provided for under Article 8(b) of Regulation 45/2001 were met[9]. This meant that the applicant would have to establish the necessity of the proposed disclosure and that the legitimate interests of the experts concerned were not prejudiced.

82. In its observations, the complainant reiterated that the names of the evaluators that had assessed its proposals should be made public.

The Ombudsman's assessment

Argument (a)

83. As regards argument (a), and after having carefully considered the submissions of both parties, the Ombudsman notes the following. First, the relevant rules provided that external experts were to be used for evaluating proposals. Second, the Commission put in place a number of safeguards with regard to the evaluation itself, namely that: (i) the evaluation was carried out in two stages by external experts, first individually and then by the group of evaluators concerned, supported by Commission staff; (ii) there was a panel review stage where all the proposals in a particular area are checked, with the participation of Commission staff, to assess the consistency of the marks given; (iii) there were external, independent observers; and (iv) there was a redress procedure.

84. In light of the above, no maladministration was found concerning argument (a) of the second allegation.

Arguments (b) and (c)

85. Given that arguments (b) and (c) are closely linked, the Ombudsman will examine them together.

86. The Ombudsman notes that each of the experts had declared themselves not to be in a situation of conflict of interest. The appointment letters containing the said declarations also clearly set out when a situation of conflict of interest would arise. The Commission argued that, even if an expert had attempted to unduly influence the evaluation procedure: (i) the final score given to a proposal was the result of a consensus between three experts; (ii) an attempt to influence the outcome would have been noticed by the other evaluators and/or the Commission's representative at the consensus stage; and (iii) the situation of the evaluators who had evaluated the complainant's proposals was again checked to verify whether there was a conflict of interest when the complainant turned to the redress committee. The Commission therefore considered that it had taken appropriate action as regards the issue of conflict of interests. The Ombudsman finds that the Commission's explanations in this respect are reasonable. Furthermore, the Ombudsman's services examined the curricula vitae of the relevant evaluators when the file was inspected. Nothing that could conceivably be interpreted as meaning that any of them found him or herself under a suspicion of conflict of interest in relation to the complainant's proposals was identified. Against that background, the Ombudsman finds that no maladministration can be found with regard to this aspect of the case.

87. As for the disclosure of the names of evaluators, the Ombudsman notes that the Commission complied with the rules governing the relevant calls for expression of interest. It did this by publishing a list of all the evaluators it used whilst refusing to disclose the names of the evaluators that had dealt with the complainant's FP7 proposals.

88. The Ombudsman further agrees that disclosing the names the complainant is interested in would entail a processing of personal data within the meaning of Regulation 45/2001, and notably Article 8(b), which provides for certain conditions as to when personal data may be transferred[10]. Given that the Commission annually publishes the names of the evaluators used, the Ombudsman is not convinced that the complainant has established the necessity of such a transfer or that disclosing the names of the evaluators would not prejudice the evaluators' legitimate interests in the present case. Therefore, no maladministration was found with regard to arguments (b) and (c) allegation.

89. The Ombudsman considers, however, that the measures taken by the Commission, laudable though they certainly are, do not remove the residual possibility that an evaluator may assess a proposal despite having a conflict of interest. It would seem clear that the person who has submitted that proposal should be allowed to object to the participation of experts who have a conflict of interest. However, it is also clear that no such objections can usefully be raised unless the person concerned knows who evaluated the proposal concerned.

90. The Ombudsman notes that the rules concerning the publication or disclosure of the names of the evaluators are laid down in a decision adopted by the Commission itself. These rules could thus be amended by the Commission. In the Ombudsman's view, it would therefore be desirable if the Commission could consider taking into account the fact that a person who has submitted a proposal would find it extremely hard or even impossible to draw attention to a possible conflict of interest on the part of the experts evaluating his or her proposal without knowing the names of these experts. The Commission could therefore consider changing its relevant rules in order to make it possible to disclose such information to applicants. Any risk to the independence of the evaluators that such disclosure may entail would, in any event, appear to be significantly diminished once the evaluations have taken place and the results have been communicated to the applicants. By amending its rules with a view to making it possible to disclose the names of the evaluators assigned to an individual proposal after the assessment has been completed, the Commission would take a further step towards making its procedures more transparent. The Ombudsman will thus make a corresponding further remark below.

Argument (d)

91. As regards argument (d), the Ombudsman considers that the Commission has convincingly explained why the evaluators of the complainant's FP7 proposals were not provided with the ESR of the FP6 proposal.

Argument (e)

92. As regards argument (e), the Ombudsman notes that the Commission explained that the evaluators for both FP7 proposals assessed the complainant's proposals individually, that is, independently from each other. Only after they had submitted their individual assessments and scores, did the evaluators meet in order to discuss the proposal. Furthermore, it was only at this later stage that a Commission representative, who moderated this discussion stage, presented the evaluators with the ESR of the first FP7 proposal.

93. In light of the abovementioned structure of the evaluation process, it is not easy to see how, in order to assess the complainant's second FP7 proposal, the evaluators could have relied on the ESR for the first FP7 proposal.

94. It emerged from the inspection that both the complainant's FP7 proposals were evaluated by three evaluators but that one of the evaluators participated in both FP7 evaluations. It would have been possible for that evaluator to rely on the evaluation of the complainant's first FP7 proposal. However, it also emerged from the inspection that the evaluator in question assessed the second FP7 proposal much more positively than the first FP7 proposal.

95. In light of the above, no maladministration was found concerning argument (e) either.

96. It follows from the above that the complainant has not been able to establish its second allegation.

C. Alleged failure to ensure that the evaluators used by the Commission are adequately qualified to evaluate proposals

Arguments presented to the Ombudsman

97. The complainant alleged that the Commission had failed to ensure that the evaluators it uses for evaluating proposals were adequately qualified to do so.

98. In its opinion, the Commission reiterated that the complainant's proposals had been evaluated by experts who were adequate in terms of knowledge and expertise.

99. In its observations, the complainant put forward that the evaluators were selected and assigned tasks on the basis of totally subjective questions, such as whether they considered themselves sufficiently qualified.

The Ombudsman's assessment

100. When the Commission's file was inspected, the Ombudsman's services also examined the curricula vitae of the evaluators that had dealt with the complainant's FP7 proposals. In light of the content of these CVs, the Ombudsman concludes that there is nothing to suggest that the Commission's view that these experts were adequately qualified to evaluate these proposals was wrong.

101. Consequently, the Ombudsman finds no maladministration with regard to the third allegation.

D. Conclusions

On the basis of the inquiry into this complaint, the Ombudsman closes it with the following conclusions:

No maladministration was found with regard to the present complaint.

The complainant and the Commission will be informed of this decision.

Further remark

The measures taken by the Commission do not remove the residual possibility that an evaluator may assess a proposal even though he or she has a conflict of interest. The Commission could therefore consider changing its rules to allow disclosure of the evaluators' identity to applicants once the evaluations have taken place and the results have been communicated to the applicants. By so doing, the Commission would take a further step towards making its procedures more transparent.

 

Emily O'Reilly

Done in Strasbourg on 18 December 2013

---

[1] Notably Article 1(c) and Article 5.2 of the Implementing provisions: http://www.ombudsman.europa.eu/resources/provisions.faces

[2] The Commission explained that the redress committee was composed of staff having the required expertise in legal and procedural matters, science and technology content and/or information systems, as the case may be depending on what matters were submitted to it.

[3]

	FP6	FP7
Research Objectives:	The main objective of FP6 is to contribute to the creation of the European Research Area (ERA) by improving integration and co-ordination of research in Europe which is so far largely fragmented. At the same time research will be targeted at strengthening the competitiveness of the European economy, solving major societal questions and supporting the formulation and implementation of other EU policies. The Horizontal Research Activities (Co-operative Research and Collective Research) are specific schemes for SMEs which address primarily the large community of SMEs with a capacity to innovate but with limited research capabilities.	Strengthening the innovation capacity of European SMEs and their contribution to the development of new technology based products and markets by helping them outsource research, increase their research efforts, extend their networks, better exploit research results and acquire technological know how, bridging the gap between research and innovation.
Eligibility criteria:	In a Co-operative Research Project, there must be at least: 1. - three independent SME participants, established in two different Member States or Associated States, of which at least one shall be established in a Member State or Associated Candidate Country, 2. - two RTD performers, independent from any other participant and established in two different Member States or Associated States, of which at least one shall be established in a Member State or Associated Candidate Country.	Research for SMEs projects require participants from the following two categories of participants: 1. - SME participants: At least three independent SME participants established in three different Member States or Associated countries. 2. - RTD performers: At least two RTD performers independent from any other participant
Evaluation criteria and thresholds:	- relevance to the objectives of co-operative research (3/5) - scientific and technological excellence (3/5) - potential impact (4/5) - quality of the consortium (3/5) - quality of the management (3/5) - mobilisation of resources (3/5) Evaluation threshold (21/30)	- scientific and/or technological excellence (3/5) - quality and efficiency of the implementation and the management (3/5) - potential impact through the development, dissemination and use of project results, (For FP7-2007-1 3/5, for FP7-2008-l 4/5) Evaluation threshold (10/15) for FP7-SME-2007-1 Evaluation threshold (11/15) for FP7-SME-2008-1
Type of activities covered:	- Research and innovation-related activities - Consortium management	- Research and technological development activities - Demonstration activities - Other activities in particular training and dissemination - Management activities

[4] Decision No 1513/2002/EC of the European Parliament and of the Council of 27 June 2002 concerning the sixth framework programme of the European Community for research, technological development and demonstration activities, contributing to the creation of the European Research Area and to innovation (2002 to 2006), OJ 2002 L 232, p. 1.

[5] Decision No 1982/2006/EC of the European Parliament and of the Council of 18 December 2006 concerning the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007-2013), OJ 2006 L 412, p. 1.

[6] "Half way developments show. All items to be shown an[d] demonstrated, such as all specifications, rubber and concret[e] hood as samples, first samples of tags in thermo flask, thermal resistance (emphasis added). "

[7] See the evaluators comment on the first criterion that "[t]here is a real need for such transponders (tags) in the automotive industry and letters of intent from car manufacturers are included in the proposal ".

[8] Regulation (EC) No 1906/2006 of the European Parliament and of the Council of 18 December 2006 laying down the rules for the participation of undertakings, research centres and universities in actions under the Seventh Framework Programme and for the dissemination of research results (2007-2013), OJ 2006 L 391, p. 1.

[9] Regulation (EC) No 45/2001 of the European Parliament and of the Council of 18 December 2000 on the protection of individuals with regard to the processing of personal data by the Community institutions and bodies and on the free movement of such data, OJ 2001 L 8, p. 1.

[10] "Without prejudice to Articles 4, 5, 6 and 10, personal data shall only be transferred to recipients subject to the national law adopted for the implementation of Directive 95/46/EC,

... if the recipient establishes the necessity of having the data transferred and if there is no reason to assume that the data subject's legitimate interests might be prejudiced."