Unacceptable Risk AI Systems

Legal Framework

The prohibition of unacceptable-risk AI systems is governed by Article 5 of the AI Act, which enumerates a closed list of banned practices. The provision targets four broad categories: subliminal or manipulative techniques, exploitation of vulnerabilities, social scoring, and predictive criminal risk assessment based solely on profiling. The definitions underpinning these prohibitions are set out in Article 3, which establishes what constitutes an "AI system," "provider," and "deployer" — the actors to whom the obligations attach.

Article 5(1)(a) prohibits:

The prohibition turns on two cumulative elements: the deployment of manipulative or deceptive techniques, and a material distortion of behaviour that causes appreciable impairment of informed decision-making. Article 5(1)(d) similarly bans predictive criminal risk assessment:

"the placing on the market, the putting into service for this specific purpose, or the use of an AI system for making risk assessments of natural persons in order to assess or predict the risk of a natural person committing a criminal offence, based solely on the profiling of a natural person or on assessing their personality traits and characteristics"
— AI Act Art. 5(1)(d)

Recital 32 reinforces the rationale for restricting real-time biometric identification in law enforcement, noting:

Key Developments

The EDPB and EDPS, in their Joint Opinion 5/2021, urged the legislature to strengthen the prohibitions, framing the AI Act as having "prominently important data protection implications." They specifically called for a general ban on biometric identification in publicly accessible spaces, going beyond the Commission's initial proposal. The EDPB's subsequent Statement 3/2024 reaffirmed the role of data protection authorities in supervising AI systems that process personal data, signalling that enforcement of Article 5 prohibitions will likely involve coordinated action between AI Act market surveillance authorities and DPAs under the GDPR.

No court has yet interpreted Article 5's prohibitions, but the EDPB's early positioning suggests that regulators will read the manipulation and vulnerability provisions broadly, aligning them with GDPR consent standards under Article 4(11) and the free-will requirement articulated in Recital 42.

Status of the Debate

This topic is actively contested. The boundaries of Article 5 — particularly the thresholds for "materially distorting behaviour," "appreciably impairing" decision-making, and "solely" based on profiling — are undefined by case law. Courts will need to determine whether subtle personalisation techniques in commercial AI systems cross the manipulation threshold, and whether hybrid risk-assessment models that combine profiling with non-personality data fall outside the Article 5(1)(d) carve-out. The EDPB's push for broader prohibitions than the final text adopted further signals that the scope of "unacceptable risk" remains a live political and legal question. Until the Court of Justice interprets these provisions, providers and deployers face genuine uncertainty at the margins.

Practical Guidance

• Map your AI system against all four Article 5(1) categories before deployment. If the system uses any technique that could be characterised as subliminal or purposefully manipulative, conduct a documented assessment of whether it materially distorts behaviour and impairs informed decision-making.
• Scrutinise vulnerability-targeting features. Article 5(1)(b) prohibits exploitation of vulnerabilities due to age, disability, or social/economic situation — assess whether your system dynamically adapts content based on inferred vulnerability indicators.
• Avoid social scoring architectures. Article 5(1)(c) prohibits evaluation or classification over time based on social behaviour or personality characteristics where the score leads to detrimental or unfavourable treatment in unrelated contexts.
• Separate profiling from criminal risk assessment. If your system assesses criminal risk, ensure it is not based "solely" on profiling or personality traits; incorporate non-profiling data sources and document the distinction.
• Treat real-time biometric identification as presumptively prohibited. Unless a narrow law-enforcement exemption applies under national law implementing the AI Act, refrain from deploying real-time remote biometric identification in publicly accessible spaces.