Search for...

EU Takes First Step Towards AI Regulation and Facial Recognition Ban: Promising or Worrisome?

EU Takes First Step Towards AI Regulation and Facial Recognition Ban: Promising or Worrisome?, TheRecursive.com
Image credit: Pexels
, ~

Only have 1 minute? Here are 3 takeaways from the piece:

  • The European Parliament voted in favor of the EU’s AI Act, marking a significant step towards setting a European rulebook on AI.
  • The rulebook vows to prohibit AI systems with an unacceptable level of risk to people’s safety, such as AI-powered facial recognition.
  • While researchers are praising the proposed act, startups are voicing concerns over its enforcement and the actual benefits it would bring for the industry.

 

The vote was passed with 499 votes in favor, 28 against and 93 abstentions, with negotiations on the final form of the law now as the next steps. According to the co-rapporteurs of the proposals, the EU is making history with the vote and has set the way for a dialogue that will eventually engage the rest of the world as well.

EU Takes First Step Towards AI Regulation and Facial Recognition Ban: Promising or Worrisome?, TheRecursive.com
Brando Benifei – EP Plenary session – Artificial Intelligence Act

“We have made history today – we have set the way for the dialogue that we will need to start having with the rest of the world on how we can build responsible AI for our globe and the systemic risks that this can entail. But also thinking of everyday citizens, consumers, businesses, and institutions that need to be supported in having an intake of AI so that they can get the best out of the AI. Also, they can be sure that they can trust that the institutions have built a system of safeguards that can identify the real risks,” co-rapporteur Brando Benifei from the Progressive Alliance of Socialist and Democrats group said.

Bans on intrusive and discriminatory AI use

Facial recognition proved to be the most contentious issue of the vote, as MEPs argued about the extent to which biometric surveillance should be limited. The MEPs also rejected a proposal made by the EPP group that stated that the risks posed by real-time biometrics in public spaces could be outweighed during “extraordinary circumstances”.

“We have won in the Parliament to maintain a clear safeguard to avoid any risk of mass surveillance, and at the same time to maintain the possibility with the no real-time biometric identification to pursue criminals and any risks that we have in society,” Benifei added.

As a result, the list of bans will now include intrusive and discriminatory uses of AI such as:

  • “Real-time” remote biometric identification systems in publicly accessible spaces;
  • “Post” remote biometric identification systems, with the only exception of law enforcement for the prosecution of serious crimes and only after judicial authorization;
  • Biometric categorisation systems using sensitive characteristics (e.g. gender, race, ethnicity, citizenship status, religion, political orientation);
  • Predictive policing systems (based on profiling, location or past criminal behavior);
  • Emotion recognition systems in law enforcement, border management, the workplace, and educational institutions;
  • Untargeted scraping of facial images from the internet or CCTV footage to create facial recognition databases (violating human rights and right to privacy).
Read more:  Sales intelligence startup Cognism raises $12.5M to further expand in Europe

For co-rapporteur Dragos Tudorache from the Renew Europe group in the EP, one of the most important aspects of the text has clear protections over such contentious use of AI.

“There are protections about how seriously we have taken the prohibitions of use of artificial intelligence in the text, how seriously we have looked at the high risk applications and the mechanisms to also make sure that we’re not limiting unnecessarily how high risk applications would be forced into compliance,” Tudorache said.

EU Takes First Step Towards AI Regulation and Facial Recognition Ban: Promising or Worrisome?, TheRecursive.com
Dragos Tudorache – AIDA Special Committee – Hearing on AI and Competitiveness

The rulebook also states that generative AI systems based on models such as OpenAI’s ChatGPT or Google’s Bard would have to comply with transparency requirements. There should be a disclosure that the content was AI-generated, also helping distinguish so-called deep-fake images from real ones and ensuring safeguards against generating illegal content.

“We are also serving the agenda of promoting innovation, not hindering creativity, and deployment and development of AI in Europe, which is something that is an objective, just as important as the one to protect our citizens,” the Romanian MEP stated.

A positive step in the right direction or a concern in the making?

The EU’s AI Act is now becoming the first real global effort to regulate AI, and a positive signal that first and foremost focuses on safety and security concerns, researchers claim.

“It is quite incredible how by now we have not figured out the necessity of such a framework. Another positive thing is that it is not only addressing AI made in the EU, but rather it is also addressing the outsourced parts of the AI systems, which is a huge thing. It could be said that it goes even beyond the borders of the EU. The need of course is justified with EU’s fundamental rights protection and safety and security in the Union,” Italy-based AI researcher Viktor Miloshevski tells The Recursive.

Read more:  There’s Been an Explosion of AI R&D in Greece in the Last 5 Years. What Drives It?

However, there are still challenges ahead, especially when it comes to its enforcement, Miloshevski adds.

“The keywords surrounding the whole AI Act experience are prohibited artificial intelligence practices, classification of AI systems as high-risk, standards, conformity assessment, certificates, registration, and governance. A quite big one is also the keyword “enforcement”. And here is where things could become a bit tricky. At this stage of design on the Union level, the proposal establishes a European Artificial Intelligence Board (the ‘Board’), composed of representatives from the Member States and the Commission. In my opinion, the enforcement of the act would be quite challenging in the first couple of years with a growing trend of further adaptation,” the AI researcher argues.

For Marko Porobija, managing director at Croatia-based law firm Porobija & Spoljaric, the proposed regulation prioritizes safety and ethical considerations, something which is essential to ensure that the “benefits of AI are shared fairly and that potential negative consequences are minimized.”

“I appreciate the focus on transparency and accountability, as this will help build trust in AI systems and ensure that they are developed and used responsibly. Overall, I think that the EU Artificial Intelligence Act is a positive development that will promote the responsible use of AI and protect the well-being of people,” Porobija tells The Recursive.

However, there are also those that have concerns over the act. According to David Menger, CEO of Prague-based conversational AI startup Wingbot.ai, in its current form, the AI Act is “a patchwork law”.

“The only thing that makes sense is the ban on facial recognition in public places. The law completely overlooks the truly serious issues, like AI being used for fraud, deep fakes and scams, and privacy and copyright in terms of training data. Many things, like laws for fraud and data copyright, are barely enforceable. It’s just another legal hurdle for companies, not a solution for real issues. Could really spoil AI startups in the EU,” Menger tells The Recursive.

Depending on the law’s final form, Menger is also concerned as to what the regulation would bring for the end users of the various AI products and services out there.

Read more:  Greek ML Startup Causaly Raises $60M Series B to Advance Biomedical Research

“I am worried that the time and money we’d like to spend building better products will have to go towards compliance with the law, and our users won’t see any real benefit,” he adds.

For Raluca Apostol, co-founder and Chief Product Officer at people intelligence platform Nestor, when it comes to the privacy and security risks, there was guidance in this area, even before the rise of generative AI tools.

“In Europe, under the GDPR law, compliance policies and the secure handling of personal data are stipulated very clearly, with provisions in place even before 2018. While the GDPR does encompass certain aspects that are relevant to generative AI, such as data protection, consent, and transparency, there are additional considerations specific to generative AI that may require additional regulations or guidelines. Still, there is a framework,” Apostol tells The Recursive.

However, there are indeed many challenges and concerns that need to be addressed.

“These are real concerns that need to be addressed further by comprehensive regulations and standards. Unfortunately, what we have to balance here is a double-edged sword. On one hand, as a company you don’t want to see your sensitive data go into untrusty systems. On the other hand, blocking your employees’ access to AI tools can leave you behind your competitors. So I don’t think there’s an easy answer to these challenges nor a magical solution to solve these problems anytime soon,” she points out.

What comes next for the AI Act?

The European Parliament now needs to initiate further negotiations on the final text in three-way talks with the Commission and EU member countries.

According to Tudorache, this will be a process that needs to include everybody and one that will take a coordinated effort.

EU Takes First Step Towards AI Regulation and Facial Recognition Ban: Promising or Worrisome?, TheRecursive.com
Dragos Tudorache – AIDA Special Committee – Hearing on AI and Competitiveness

“Apart from writing rules, our governments and us here at the European level as well, we have to engage in a very serious and coordinated effort to explain and to accompany our citizens in this transformation. That means that we have to invest massively in education and reskilling our current workforce to be able to integrate AI into their work because otherwise, we would have forces in our societies that will resist and will look at this transformation with a bad eye – and that’s something that we do not want.” Tudorache concluded.

Thank you for reading The Recursive!

Over 1 million people from all over the world have learned about the tech ecosystem in Central and Eastern Europe thanks to The Recursive. In order to keep our content free for everyone, we need your help. If you believe what we do is important and have the means to do so, support us in giving a voice to Central and Eastern Europe with as little as €7. Thank you!

DONATE Keep reading this article

Help us grow the emerging innovation hubs in Central and Eastern Europe

Every single contribution of yours helps us guarantee our independence and sustainable future. With your financial support, we can keep on providing constructive reporting on the developments in the region, give even more global visibility to our ecosystem, and educate the next generation of innovation journalists and content creators.

Find out more about how your donation could help us shape the story of the CEE entrepreneurial ecosystem!

One-time donation

You can also support The Recursive’s mission with a pick-any-amount, one-time donation. 👍

https://therecursive.com/author/bojanstojkovski/

Bojan is The Recursive’s Western Balkans Editor, covering tech, innovation, and business for more than a decade. He’s currently exploring blockchain, Industry 4.0, AI, and is always open to covering diverse and exciting topics in the Western Balkans countries. His work has been featured in global media outlets such as Foreign Policy, WSJ, ZDNet, and Balkan Insight.