Unacceptable Risk AI Systems

Legal Framework

Unacceptable-risk AI systems are governed by Article 5 of the AI Act, which establishes a closed list of prohibited AI practices. These prohibitions target AI uses that fundamentally conflict with Union values, including human dignity, freedom, democracy, and non-discrimination. Recital 31 specifically addresses social scoring systems, prohibiting AI that evaluates or classifies natural persons based on social behaviour across multiple contexts or personal characteristics not inherently linked to the specific purpose. Such systems risk discriminatory outcomes and exclusionary effects that violate rights to dignity and equal treatment.

Recital 38 addresses real-time remote biometric identification in publicly accessible spaces for law enforcement purposes. The AI Act establishes itself as lex specialis with respect to Article 10 of Directive (EU) 2016/680 (the Law Enforcement Directive), regulating both the use of such identification systems and the associated biometric data processing in an exhaustive manner. Recital 34 clarifies that even where exceptions permit use, the deployment must be responsible and proportionate, requiring assessment of the nature of the situation, consequences for rights and freedoms, and applicable safeguards.

The territorial scope of these prohibitions follows the general application rules of the AI Act, which mirror established GDPR principles under Article 3 GDPR. Processing falls within scope where it occurs in the context of an establishment's activities in the Union, even if the actual processing takes place elsewhere. An establishment requires effective and genuine activity through stable arrangements, regardless of scale. The consent requirements drawn from Article 3:33 and 3:35 of the Dutch Civil Code, applied by analogy, demand that data subjects can express their will freely — meaning genuine choice without adverse consequences for refusal or withdrawal.

Key Developments

The Court of Justice of the EU has shaped the interpretation of "in the context of activities of an establishment" through its jurisprudence on the nearly identical provision in Article 4(1)(a) of the 1995 Data Protection Directive. The Google Spain ruling established that a subsidiary promoting and selling advertising space constitutes a relevant establishment, bringing the parent company's processing within Union jurisdiction. This precedent directly informs how the AI Act's territorial scope applies to unacceptable-risk systems operated by non-EU providers.

The concept of "competent authority" under the Law Enforcement Directive, relevant to Article 5's law enforcement exceptions, encompasses not only government bodies such as judicial authorities and police but also any entity authorized under national law to exercise public authority and powers. This broad interpretation affects which entities may invoke the narrow exceptions for real-time biometric identification.

Practical Guidance

• Conduct a systematic classification review of all AI systems against the Article 5 prohibited practices list, with particular attention to social scoring functionality that aggregates behavioural data across multiple contexts — any system touching multiple datapoints about social behaviour requires immediate legal assessment under Recital 31.
• For any biometric identification deployment in publicly accessible spaces, verify whether the use case falls within the exhaustively listed exceptions under Article 5 and document the proportionality assessment required by Recital 34, including analysis of the nature of the situation and consequences for all persons' rights and freedoms.
• Map territorial scope carefully: if your organization has any stable arrangement in the Union — including through a commercial agent collecting payments related to an online service — the AI Act's prohibitions apply regardless of where processing technically occurs.
• Ensure that any consent mechanisms associated with AI systems satisfy the genuine-choice standard: consent must be separately obtainable for different processing purposes, freely refusable, and withdrawable without adverse consequences, consistent with the principles underlying Article 3 GDPR and Recital 42.
• Verify that any law enforcement exception reliance is supported by explicit national authorization for the entity exercising public authority, as the definition of competent authority extends beyond traditional government bodies to any entity legally empowered to exercise public powers.