Case title: Artificial intelligence and the EU administration

Date: Thursday, 10 March 2022

Remote meeting, Brussels

Present:

European Data Protection Supervisor

- Acting Head of Unit, Technology and Privacy

- Technology and security officer, Technology and Privacy

- Technology and security officer, Technology and Privacy

- Technology and security officer, Technology and Privacy

- Policy and Consultation

- Head of Unit, Supervision and enforcement

- Supervision and enforcement, Area of Freedom, Security and Justice

European Ombudsman

- Peter Dyrberg, Inquiries and Process Expert

- Valentina Stoeva, Inquiries Officer

- Nicholas Hernanz, Inquiries Officer

- Jennifer King, Legal Expert

- Olatz Fínez Marañón, Inquiries trainee

INTRODUCTION

The European Ombudsman’s team explained that the use of artificial intelligence (AI) in public administration has been a topic of discussion within the European Network of Ombudsmen (ENO). ENO members have already dealt with complaints on the use of AI by national authorities and have published reports and guidance. The ENO annual conference in April 2022 in Strasbourg^[1] will focus on digitalisation and citizens’ rights.

It is against this background that the European Ombudsman (EO) wanted to discuss with the European Data Protection Supervisor (EDPS) about the EDPS' ongoing work related to AI and the EDPS' new role envisaged by the AI Act proposal^[2]. The EO’s team explained that they would be interested in sharing this information with ENO members. The EO’s team mentioned that a similar meeting was held with the Commission in September 2021, whereby the Commission presented the Proposal for an AI Act and responded to questions.

INFORMATION SHARED BY THE EDPS

Ongoing work related to AI

The EDPS explained that it is currently conducting supervision activities concerning a number of EU institutions, such as Europol and eu-LISA, as some institutions have started developing machine-learning systems. The EDPS is performing audits and asking for information on how EU institutions are developing machine- learning models using personal data.

The EDPS also receives consultations from institutions that are developing or procuring AI systems, in order to help them properly consider data protection already at the design stage, identify possible risks for the rights and freedoms of data subjects and mitigate them in an adequate way.

The EDPS also participates in activities at the European Data Protection Board (EDPB)^[3] level that deal with AI and facial recognition. For instance, the EDPS is participating in the EDPB’s Borders, Travel and Law Enforcement expert group, which is working on guidance regarding the use of facial recognition technology by law enforcement authorities.

The EDPS, together with the Commission Nationale de l’Informatique et des Libertés (CNIL), also co-chairs the AI working group of the Global Privacy Assembly (GPA)^[4], an organisation of data protection and privacy authorities from all around the world. This year, the group is working on a report on the use of AI in the workplace that will be presented in the GPA meeting in October 2022. Based on surveys of data protection authorities and external stakeholders, the working group is expected to produce a report that will cover the use of AI throughout the whole employment cycle (screening of candidates, employee monitoring, dismissals, etc.). There is also a proposal being prepared for a general risk management framework, also based on a survey of GPA members. A third ongoing deliverable is a report analysing the GPA members capacity and expertise in addressing data protection issues regarding AI systems in order to identify potential shortcomings.

Within the GPA, the EDPS is also currently a co-drafter of a document on principles on the use of facial recognition technology. The final document is expected to be published in the GPA meeting in October 2022. This documents stems from the GPA  resolution on this topic^[5] adopted in 2020, which requires the GPA to draft a document containing principles on how to use facial recognition technology and comply with data protection legal framework.

The EDPS also took part in the Council of Europe’s CAHAI (Ad hoc Committee on Artificial Intelligence)^[6], as part of the EU delegation, together with the European Commission and the EU Fundamental Rights Agency. They mainly contributed to the work in the policy development group and the legal framework group. The CAHAI was an initiative of the Council of Europe that had the task of preparing documents that would later help to draft a “Treaty on AI”. The EDPS will also participate in the CAI (Committee on Artificial Intelligence), the successor of the CAHAI, whose mission is to start drafting the Treaty. Discussion on this are scheduled to begin in April 2022.

The EDPS' roles envisaged by the AI Act proposal

Based on the current text of the Proposal for an AI act, the EDPS would have three main roles:

1. Competent supervisory authority: the EDPS will ensure that EU institutions comply with the AI Act, which involves monitoring, enforcement of the Act as well as organisation of sandboxes^[7]. The scope of the EDPS’ role and tasks should be further clarified.

2. Notifying body: at least for some cases, the EDPS will conduct conformity assessments for EU institutions that will develop high-risk AI systems.

3. Market surveillance authority: it is not yet clear what this will entail. The EDPS has expressed its doubts in its EDPB-EDPS Joint Opinion on the AI Act and asked the Commission for clarifications as regards what the AI market for EU Institutions, Bodies and Agencies market will comprise , and if its surveillance will concern solely EU institutions or also third parties and private companies providing AI systems and services.

The EDPS will also be part of the new European AI Board, which will issue opinions and recommendations on AI matters. In broad terms, this will entail preparatory work to keep up with the latest updates on AI technology and to follow standardisation processes.

The EDPS will thus have to develop a strategy to supervise the EU institutions and even external third parties and to develop procedures to perform conformity assessments, etc.

The EDPS considers that further clarifications regarding the AI legislation are needed in order to specify the extent of their new roles and to see how the AI Act will affect and interplay with the existing data protection legislation.^[8] The EDPS mentioned that the EU co-legislators are still discussing substantial amendments to the proposal, including in relation to the important question of determining who is an AI ‘provider’.

OTHER QUESTIONS DISCUSSED

Question on other challenges that the EDPS foresees will arise for them or other national authorities performing a similar role

It is clear that there is a regulatory challenge, but even more so there is a challenge of resources. Since AI has become widely discussed and used, both public and private sectors have started to search for experts in the AI field and a shortage in experienced professionals could become a challenge for a proper development of the future functions of the EDPS.

On the enforcement side, it will be important to consider how national authorities will respond to this new role. This had already become an issue with the ePrivacy Directive, where national authorities of very different natures and expertise (e.g. telecommunication regulators and data protection authorities) have the same competences. For instance, Article 63 (Market surveillance and control of AI systems in the Union market) of the Act provides that, for AI systems used for law enforcement and Migration, asylum and border control management (list of high-risk AI systems of Annex 3), the market surveillance authority will be the relevant Data Protection Authority (DPA). However, if the EDPB and EDPS recommendation to make DPAs competent authorities is not followed, in some cases the competencies could be divided between different national authorities. For example, if Spain would decide to appoint as AI competent authority an entity other than the AEPD (the Spanish DPA), then the AI comptent authority would be the market authority dealing with most AI matters and high-risk AI systems, yet in parallel the Spanish DPA would be the market surveillance authority for law enforcement and migration, asylum and border control management AI systems. Therefore, these two authorities would need to articulate efficiently in order to ensure a consistent enforcement of their respective competences.

Question on transparency: how does it affect individuals and what safeguards should be there?

The EDPS noted that obligations of transparency towards the user, i.e. informing the public as to when and how AI systems are used, are not provided for in the current legislative proposal. This is because the proposal appears to have been designed following a product or industry perspective and not a data subject perspective. As a result, some fundamental rights issues, such as transparency, appear to have been superimposed rather than integrated from the outset. The EDPS mentioned that the General Data Protection Regulation (GDPR)^[9] imposes certain transparency obligations on data controllers who process personal data through AI systems. However, the GDPR is technology neutral while the AI Act is not. The EDPS considers that the AI Act should require AI-specific transparency obligations for the benefit of data subjects (e.g. AI systems’ performance information).

Question on examples of potential complaints about maladministration stemming from AI

The European Ombudsman’s team put the question for discussion on what could be expected in terms of complaints that might reach the European Ombudsman or national Ombudsmen related to the use of AI systems. The European Ombudsman’s team mentioned that it is aware that AI is being used, developed or considered in some EU institutions (e.g. EPSO, Frontex, EASO). The EDPS confirmed  its awareness of the projects mentioned by the EO’s team and referred to the shared biometric matching system that eu-LISA is setting up for many of their systems such as the visa information system, the Entry-Exit System or Eurodac . As to possible overlapping competences EO - EDPS, a reasonable point of departure would be the MOU, which reflects the way that the two Institutions currently cooperate and address relevant issues.

CONCLUSION

The EO representatives thanked the EDPS representatives for their availability and for the information shared. The EDPS representatives also thanked the EO’s team for the meeting and shared information about the upcoming EDPS conference on effective enforcement of data protection in the digital world^[10] that will take place in June 2022.

 

Brussels, 10 March 2022

Peter Dyrberg                                                                                                 Valentina Stoeva

Inquiries and Process Expert                                                                          Inquiries Officer

 

[1] European Network of Ombudsmen (ENO) conference 2022 – Digitalisation of public administrations: ensuring equal access after the pandemic (17 April 2022): https://www.ombudsman.europa.eu/en/event/en/1438

[2] Proposal for a Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52021PC0206

[3] https://edpb.europa.eu/edpb_en

[4] www.globalprivacyassembly.org

[5] https://edps.europa.eu/sites/edp/files/publication/final_gpa_resolution_on_facial_recognition_technology_en.pdf

[6] https://www.coe.int/en/web/artificial-intelligence/cahai

[7] The AI Act proposal does not provide a definition for an AI sandbox, but its Article 53 states that AI regulatory sandboxes: ‘... shall provide a controlled environment that facilitates the development, testing and validation of innovative AI systems for a limited time before their placement on the market or putting into service pursuant to a specific plan. This shall take place under the direct supervision and guidance by the competent authorities with a view to ensuring compliance with the requirements of this Regulation and, where relevant, other Union and Member States legislation supervised within the sandbox.’

[8] See EDPS-EDBP Joint Opinion on the proposal for an Artificial Intelligence Act: https://edps.europa.eu/node/7140_en

[9] Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) https://eur-lex.europa.eu/eli/reg/2016/679/oj

[10] https://www.edpsconference2022.eu/en