Risk Management System

Legal Framework

The risk management system for high-risk AI systems is governed primarily by Article 9 of the AI Act, which works in concert with Article 8 (compliance) and Article 17 (quality management system). Article 9(1) establishes the core obligation: a risk management system must be established, implemented, documented, and maintained for every high-risk AI system. Article 8(1) makes clear that this system is not a standalone exercise but feeds directly into compliance with all other requirements in Section 2 of the AI Act.

The system is defined as a continuous, iterative process spanning the entire lifecycle of the AI system. Article 9(2) sets out four mandatory steps: (a) identifying and analysing known and reasonably foreseeable risks to health, safety, and fundamental rights; (b) estimating and evaluating risks under intended use and reasonably foreseeable misuse; (c) evaluating risks arising from post-market monitoring data under Article 72; and (d) adopting targeted risk management measures.

"The risk management system shall be understood as a continuous iterative process planned and run throughout the entire lifecycle of a high-risk AI system, requiring regular systematic review and updating."
— AI Act Art. 9(2)

Critically, Article 9(3) scopes the obligation: only risks that may be reasonably mitigated or eliminated through development, design, or adequate technical information fall within its ambit. Article 9(4) further requires that risk management measures account for interactions between the various Section 2 requirements to minimise risk more effectively. The quality management system under Article 17 must incorporate the risk management system's outputs, ensuring that design controls, testing, and data management procedures operationalise the identified mitigations.

Key Developments

Enforcement of the AI Act's risk management requirements is in its earliest stages. The EDPB has confirmed that data protection authorities are competent to intervene during AI model development and may impose corrective measures on the processing underlying AI systems. The EDPB's Opinion 28/2024 notes that supervisory authorities may act on processing during the development phase, linking GDPR accountability obligations under Articles 5(2) and 24 to the AI Act's risk management framework.

The Italian Garante's enforcement actions illustrate how data protection enforcement intersects with AI risk management. In the Lusha Systems case, the Garante imposed a €2,000,000 fine for unauthorised processing in a B2B contact platform — a signal that inadequate risk assessment for AI-driven data processing invites significant penalties. The AgID decision further demonstrates that public-sector AI deployments face equivalent scrutiny.

Status of the Debate

This topic is regulator-defined. The AI Act's risk management system is a newly codified obligation with no case law yet testing its boundaries. The EDPB has begun articulating how GDPR supervisory authority competences overlap with AI Act enforcement, but no court has interpreted Article 9's four-step process or its interaction with Article 17's quality management requirements. The open questions — what constitutes "reasonably foreseeable misuse," how post-market monitoring data must feed back into risk evaluation, and how the Article 9(3) scope limitation applies in practice — will be resolved through the first wave of supervisory authority decisions and, eventually, preliminary references to the CJEU. Until then, providers must rely on the literal text and forthcoming harmonised standards.

Practical Guidance

• Document the full lifecycle process: Establish written policies covering all four steps in Article 9(2)(a)–(d), ensuring the system is iterative and updated at every lifecycle stage, not just at initial deployment.
• Integrate with the quality management system: Article 17 requires that design controls, testing procedures, and data management operationalise the risk management outputs. Ensure traceability between identified risks and specific design or information mitigations.
• Account for reasonably foreseeable misuse: Risk evaluation under Article 9(2)(b) extends beyond intended purpose. Document the analysis of how users might plausibly misuse the system and what mitigations address those scenarios.
• Feed post-market monitoring into risk evaluation: Establish a data pipeline from the Article 72 post-market monitoring system back into the Article 9(2)(c) risk evaluation step, with defined triggers for updating the risk management system.
• Scope risks to what design can address: Apply the Article 9(3) limitation — focus risk management on risks that development, design, or technical information can reasonably mitigate, and document the rationale for excluding risks outside this scope.