I. Introduction

The European Ombudsman's Strategy for the mandate 2009-2014 highlights the need to listen to our stakeholders and be open to their suggestions.

In parallel, one of the key performance indicators included in our Annual Management Plan provides for an assessment, by our internal and external stakeholders, of our performance in relation to strategy and governance.

To obtain feedback from our external stakeholders, a survey of their views on the strategy and governance of the European Ombudsman’s office was launched in November 2012. The outcome is summarised below and will be used to evaluate and improve the way we work and the results we achieve.

The survey was anonymous and consisted of 13 multiple-choice questions, in which stakeholders were able to rate our performance and provide us with their response in a number of areas related to the strategy and governance of the Ombudsman’s office. The survey also included an open text box for any comments and/or suggestions.

This is the Ombudsman's first effort to conduct such a survey. We took inspiration from the EDPS, who used an external consultant. For budgetary reasons, we had to use in-house resources. We hope to learn from the experience and to make stakeholder surveys a continuing feature of our planning and evaluation cycle. 

Finally, it should be emphasised that the purpose of the survey was to focus on strategy and governance. We are conscious that it might have been difficult for some respondents to distinguish between their perception of our overall performance (regarding effectiveness or responsiveness, for example) and that of our governance in relation to such matters. In interpreting the results, we must therefore bear in mind that they are likely to reflect the respondents' perception of both of these aspects.

II. Methodology

The survey was carried out through the IPM (Interactive Policy Making) survey tool on the EUROPA website[1]. All the stakeholders were invited to respond to the survey by an email from the European Ombudsman. The invitation made clear that the survey was anonymous and that individual responses would be processed in accordance with Regulation 45/2001[2] and the European Ombudsman's privacy notice[3].

All the stakeholders were asked the same questions. In questions 2 to 12, stakeholders were invited to rate our performance in relation to the subject matter of the question by selecting one of six possible answers (very good, good, satisfactory, poor, very poor or don't know). In question 13, stakeholders were asked to express their views on a statement by selecting one of six possible answers (strongly agree, agree, neither agree nor disagree, disagree, strongly disagree or don't know).

In most questions, the stakeholders were also invited to give their views on whether, in the past two years, our performance in relation to that particular question 'improved', 'remained the same' or 'declined'. The option to reply 'don't know' was also available.

III. Whom we surveyed

The survey was addressed to a subset of 190 external stakeholders whom we thought best placed to provide informed and objective feedback on matters relating to strategy and governance. In establishing the list, we also tried to achieve a balance between those within and those outside the EU institutions.

Stakeholders from within the EU institutions included the Secretaries-General of all the institutions, the directors of the EU agencies, the President and Coordinators of the Committee on Petitions of the European Parliament, Members of the Commission and high officials who are in regular contact with our institution. Stakeholders from outside the EU institutions included national ombudsmen, journalists, NGOs, academics and other frequent contacts.

In response to question 1 of the survey, the respondents were invited to indicate their professional or institutional affiliation among the following categories:

• European Commission
• EU Agency
• European Parliament or other EU institution and body
• European Network of Ombudsmen
• Civil society or other stakeholder
• Journalist

IV. Who answered the survey?

We received a total of 110 replies out of 190 persons invited to respond, giving an overall response rate of 58%. The following two graphs give an overview of the breakdown of responses by category of stakeholders (Figure 1) and of the number of responses and the response rate by category of stakeholders (Figure 2).

Figure 1: Breakdown of responses by category of stakeholder.

Figure 2: Number of responses and response rate by category of stakeholder.

V. Questions relating to our overall performance in carrying out our mission and attaining the aspirations in the Strategy

Clarity of the Ombudsman's mission and purpose

In reply to question 2 'how would you rate us on clarity of mission and purpose?', the result is overwhelmingly positive, with a total positive rating of 93.6% (43.6% of the respondents answered 'very good', 34.5% 'good' and 15.5% 'satisfactory'). Only 2.7% replied 'poor' or 'very poor' to this question. The remaining 3.6% answered 'don't know'.

48.2% of the respondents view our performance in this regard as having improved in the past two years and 37.3% consider that it has remained the same. Only 3.6% think that our performance has declined. 10.9% responded 'don't know'.

Effectiveness in carrying out our objectives

To question 3 'how would you rate us on effectiveness in carrying out our objectives?', 82.7% of respondents replied positively with 28.2% responding 'very good', 43.6% 'good' and 10.9% ' satisfactory'. 10% of the respondents replied negatively, with 9.1% rating us as 'poor' and 0.9% as 'very poor'. 7.3% answered 'don't know'.

While 38.2% of the respondents consider that our performance in this regard has improved in the past two years, 41.8% believe it remained the same. Only 2.7% consider that our performance has declined. 17.3% answered 'don't know'. 

Effectiveness in measuring and reporting on our performance

To question 4 'how would you rate us on effectiveness in measuring and reporting on our performance?', 82.7% of respondents replied positively, with 30.9% rating us as 'very good', 37.3% as 'good', and 14.5% as 'satisfactory'. 2.7% considered our performance as 'poor' and 14.5% answered 'don't know'.

An almost equivalent number of respondents consider that, in the past two years, our performance in this regard has either improved (37.3%) or remained the same (38.2%). None considered that our performance has declined while 24.5% replied 'don't know'.

Responsiveness

Respondents rated us as follows for our responsiveness and how easy we are to deal with (question 5): 90% of replies were positive with 36.4% rating us as 'very good', 34.5% as 'good' and 19.1% as 'satisfactory. 4.5% rated us negatively, with 3.6% answering 'poor' and 0.9% 'very poor'. 5.5% replied 'don't know'.

Most of the respondents (44.5%) consider that our performance in this regard has remained the same in the past two years, while 31.8% believe it has improved. 20.9% responded 'don't know' and 2.7% consider that our performance declined.

Organisational efficiency

In question 6, respondents were asked how they rate the Ombudsman's organisational efficiency. Almost three quarters (73.6%) of responses were again positive with 27.3% replying 'very good', 32.7% 'good' and 13.6% 'satisfactory while 4.5% rated the organisational efficiency as 'poor'. More than a fifth (21.9%) of the respondents replied 'don't know' to this question.

Almost half (46.4%) of the respondents consider that our performance in this regard has remained the same and 21.8% view it as having improved. Only one respondent (0.9%) thinks it has declined. This is the question for which the proportion of respondents who replied 'don't know' is the highest of all (30.9%).

Effectiveness as an alternative means of resolving disputes with the Union institutions

In question 7, respondents were invited to rate us on effectiveness as an alternative means of resolving disputes with Union institutions. 78.2% of respondents replied positively with 23.6% rating us as 'very good', 39.1% as 'good' and 15.5% as 'satisfactory'. 9.1% of the respondents answered negatively with 2.7% rating us as 'poor' and 6.4% as 'very poor'. 12.7% replied 'don't know'.

37.3% of respondents think that, over the past two years, our performance in this regard has remained the same and 28.2% that it has improved. 9.1% consider it has declined whereas more than one quarter (25.5%) answered 'don't know'.

The Ombudsman is a driving force to put citizens at the centre of the Union's administrative culture

72.8% of all respondents said they either somewhat agree (27.3%) or strongly agree (45.5%) with the above statement (question 13) whereas 16.3% either somewhat disagree (14.5%) or strongly disagree (1.8%). 7.3% answered that they neither agree nor disagree and 3.6% replied 'don't know'.

Figure 3: Breakdown of responses to question 13.

VI. Questions relating to the implementation of the five objectives set out in the Strategy for the mandate

Objective 1 - Listen "We want to listen to our stakeholders, be open to their suggestions and identify best practices"

Respondents rated our performance in relation to Objective 1 (Question 8) with an overall positive score of almost 83%: 40.9% rated us as 'very good', 25.5% and 16.4% as 'good' and 'satisfactory' respectively. Only 9.1% consider our performance to be 'poor' and 2.6% 'very poor', while 5.5% say they don't know.

In reply to the second part of the question, 32.7% of the respondents replied that our performance has improved over the past two years while 46.4% think it has remained the same. Finally, 1.8% view our performance as having declined, while 19.1% don't know.

Objective 2: "Deliver - We want to find ways to achieve results faster" 

Asked about our performance as regards Objective 2 (question 9), 82.8% of the respondents replied positively. 25.5% rate our performance as 'very good', 33.6% as 'good' and 23.6% as 'satisfactory'. 4.5% and 2.7% rate it as 'poor' and 'very poor' respectively, while 10% of respondents replied 'don't know'.

In reply to how our performance evolved over the past two years in relation to Objective 2, 27.3% of respondents consider it has improved, while 45.5% are of the opinion that it has remained the same. 2.7% think it has declined. As much as 24.5% of respondents replied 'don't know'.

Objective 3: "Persuade - We want to have a positive impact on the administrative culture of the EU institutions"

In reply to Question 10 about our performance in relation to Objective 3, more than 80% of respondents rate the Ombudsman's performance as positive (36.4% as 'very good', 30.9% as 'good' and 15.5% as 'satisfactory'. Those that rate the Ombudsman's performance as 'poor' and 'very poor' represent 10% and 0.9% respectively. 6.4% of respondents replied 'don't know'.

Nearly a third of respondents (31.8%) state that our performance with regard to Objective 3 has improved over the past two years, compared to nearly half (48.2%) who believe that it has remained the same. Only 0.9% are of the opinion that our performance has declined, and 19.1% could not answer.

Objective 4: "Communicate - We want to provide our stakeholders and the public with timely, useful and easily accessible information"

Almost 85% of respondents rate our performance with respect to how we communicate with our stakeholders (question 11) as positive (35.5% replied 'very good', 32.7% 'good', and 16.4% 'satisfactory'), while 10% and 1.8% rate our performance as 'poor' and 'very poor' respectively. 3.6% don't know.

Asked about how our performance has evolved in the past two years, 35.5% of respondents find that it has improved, while 46.4% think it has remained the same. Only 0.9% of respondents think that it has declined, while 17.3% do not know.

Objective 5: "Adapt - We want constantly to rethink how we use our resources, in order to improve the efficiency and effectiveness of our work"

Approximately 60% of the respondents have a positive view of our performance in relation to Objective 5 (question 12). 21.8% of respondents rate us as 'very good', 24.5% as 'good' and 15.5% as 'satisfactory'. No more than 1.8% and 2.7% of respondents replied 'poor' or 'very poor' respectively. The number of 'don't know' replies was particularly high in relation to this question (33.6%).

In reply to the second part of the question, 20.9% of respondents expressed the view that our performance in this regard has improved over the past two years, while 36.4% think it has remained the same. Only 0.9% of respondents think it has declined. The number of respondents who responded 'don't know' is also particularly high (41.8%) in relation to this question.

VII. Conclusions

Preliminary remarks

Response rates

The lowest rate was for journalists followed by civil society stakeholders (respectively 23% and 38%), while the response rates among European Parliament and other EU institution stakeholders (excluding the European Commission) and the European Network of Ombudsmen representatives are the highest (90% and 73% respectively).

“Don’t knows”

The proportion of 'don't know' replies, especially to questions relating to how stakeholders view the Ombudsman's performance over time, is relatively high (it ranges from 10% to 40% according to the subject matter). This seems to indicate that many stakeholders could only base their assessment on how they view us currently or perhaps on the basis of a single contact or experience with the Ombudsman. It may also suggest that further reflection may be necessary on how we communicate on strategy and governance matters.

Differences in scores

Stakeholders rated the clarity of the Ombudsman's mission and purpose highest (positive score of 93.6%). This could, among other things, be interpreted as reflecting success in communicating the Ombudsman’s Strategy for the mandate and role in the EU institutional framework.

All the questions resulted in a highly positive evaluation. For some questions, however, a significant minority of respondents (10% or slightly more) gave a negative response. These questions concern Objective 1 of the Strategy ("Listen"; question 8), Objective 4 ("Communicate"; question 11), Objective 3 ("Persuade"; question 10) and our "effectiveness in carrying out our objectives" (question 3).

Unfortunately, none of the respondents made use of the possibility to submit comments in the open text box provided at the end of the survey, so the reasons for this significant minority opinion are not clear. Furthermore, the way the survey was constructed means that it is not possible to identify the category or categories of stakeholder that produced the negative evaluations. In future surveys, we could envisage a structure that would enable us to disaggregate the results by category of stakeholder. Furthermore, respondents should be encouraged to include remarks and comments, especially where they have a negative perception of our performance. Such indications would help us identify ways to improve in the future.

Overall results

Figure 4: Summary of how respondents rate our performance (part 1 of questions 2 to 12).

Figure 5: Summary of how respondents who expressed a view rate our performance in relation to questions 2 to 12 (don't know replies are not taken into account).

Figure 6 below shows how respondents rate our performance as a proportion of all the replies in which stakeholders expressed a view ('don't know replies are not taken into account).

Figure 6: Overall perception of the Ombudsman's performance

Figure 7: Summary of respondents' views on how our performance evolved in the past two years (part 2 of questions 2 to 12)

Figure 8: Respondent's overall view of how our performance evolved over the past two years (don't know responses, which represent an average of 25% of the replies are not taken into account).

Final remarks

The high overall response rate to the survey (58%) indicates that the external stakeholders who were invited to participate, particularly those within the EU institutions and among the members of the European Network of Ombudsmen, take a genuine interest in matters relating to the Ombudsman's strategy and governance.

The overall results clearly show that our external stakeholders' views on our performance in carrying out our mission and achieving our objectives are for the most part highly positive. As noted above, however, a significant minority of respondents gave negative or very negative ratings on four aspects. In future surveys, we should encourage those who have a negative perception of our performance to include remarks and comments. Such indications would help us identify ways to improve in the future.

Even though the majority of respondents indicate that our performance 'remained the same' in the past two years, this should not be seen as a negative assessment of our performance given that the overall rating is very high.

It is also rewarding to note that an overwhelming number of respondents consider that the Ombudsman's aspiration to be 'a driving force to put citizens at the centre of the Union's administrative culture', has been attained. Furthermore, the response to Question 4 is encouraging as regards the aspiration to measurably increase our effectiveness as an alternative means of resolving disputes with the Union institutions.

The challenge for the future will be to maintain and further to enhance the quality of our governance in order, as far as possible, to meet or even exceed our stakeholders' expectations and to communicate clearly about what we do. 

As a final note, it should be emphasised that the panel of stakeholders invited to respond to this first survey on strategy and governance was established on an experimental basis. Given the relatively high number of 'don't know' responses to certain questions, it may be useful to review the list of participants in a future survey of this type.

MR/PMC-January 2013