Reading progress
0 of 14 sections read
6 / 14

Section 04

4The Regulatory and Threat Landscape

3 min read6 of 14

The financial and regulatory consequences of ungoverned AI are substantial. Under the EU AI Act, penalties scale with global turnover, so an undeclared tool that crosses into a prohibited or high-risk category becomes a board-level liability rather than an IT incident.

Violation categoryDescription of offenseMaximum penalty
Prohibited practices (Art. 5)Subliminal techniques, exploiting vulnerabilities, unauthorized biometric categorization€35M or 7% of global turnover, whichever is higher
High-risk non-compliance (Art. 6–49)No risk management, data governance or human oversight€15M or 3% of global turnover, whichever is higher
Misleading informationIncorrect or misleading information to regulators€7.5M or 1% of global turnover, whichever is higher

Table 2. EU AI Act violation categories and maximum penalties. Source: EU AI Act regulatory framework guidelines.

The timeline has since moved. The Digital Omnibus on AI — Regulation (EU) 2026/1744, in force since 27 July 2026 — defers high-risk obligations for stand-alone Annex III systems to 2 December 2027 and for AI embedded in regulated products (Annex I) to 2 August 2028. Article 50 transparency duties still apply from 2 August 2026, and a new Article 5 prohibition on AI-generated non-consensual intimate imagery and child sexual abuse material applies from 2 December 2026. The penalty ceilings above are unchanged; the deferral buys preparation time, not an exemption.

The technical threat surface is mapped by the OWASP Top 10 for LLM Applications. Each category corresponds to an exposure pathway that shadow tools open by default — from prompt injection and sensitive-information leakage to excessive agency and ‘Denial of Wallet’ consumption attacks.

Figure 6. OWASP Top 10 for LLM Applications (2025 edition), grouped by where the exposure originates. For autonomous agents, OWASP now also publishes a dedicated Top 10 for Agentic Applications.
Figure 6. OWASP Top 10 for LLM Applications (2025 edition), grouped by where the exposure originates. For autonomous agents, OWASP now also publishes a dedicated Top 10 for Agentic Applications.
Text in this figure

Input · LLM01 · Prompt Injection · Output · LLM02 · Sensitive Information Disclosure · LLM05 · Improper Output Handling · LLM07 · System Prompt Leakage · Model · LLM03 · Supply Chain · LLM04 · Data & Model Poisoning · LLM08 · Vector & Embedding Weaknesses · Agency · LLM06 · Excessive Agency · LLM09 · Misinformation · LLM10 · Unbounded Consumption

OWASP riskVulnerability descriptionEnterprise security impact
LLM01 · Prompt InjectionAdversarial inputs (direct or indirect) that overwrite system promptsSafety bypasses, unauthorized commands, malicious code in the developer environment
LLM02 · Sensitive Information DisclosureLeakage of proprietary algorithms, PII or architecture via outputExposes corporate secrets, voids IP and triggers regulatory penalties
LLM03 · Supply ChainCompromised third-party models, datasets or hallucinated pluginsSilent backdoors, biased logic and large-scale upstream breaches
LLM04 · Data and Model PoisoningDeliberate manipulation of training data, fine-tuning or embeddingsSelective degradation, harmful output or hidden “sleeper agent” activation
LLM05 · Improper Output HandlingOutputs passed downstream without validation, sanitization or encodingClassic web exploits: XSS, SSRF and remote code execution

Table 3. OWASP Top 10 for LLM Applications: vulnerability and enterprise impact (LLM01–LLM05).

OWASP riskVulnerability descriptionEnterprise security impact
LLM06 · Excessive AgencyBroad permissions to call APIs or databases with no human in the loopRogue agents delete production data or alter access control
LLM07 · System Prompt LeakageDisclosure of the instructions that bound the model’s behaviourAdversaries reverse-engineer defences, steal prompt IP, craft bypasses
LLM08 · Vector and Embedding WeaknessesFlaws in generation, storage or retrieval within RAGPoisoned vector stores surface malicious context and instructions
LLM09 · MisinformationModel generates confident but false output that users accept without verificationInsecure code in production, defamatory content, legal liability
LLM10 · Unbounded ConsumptionOverloading costly inference to overwhelm services (Denial of Wallet)Billing spikes, exhausted cloud capacity, severe latency

Table 3 (cont.). LLM06–LLM10. Source: OWASP Top 10 for LLM Applications, 2025 edition (current).

Tip: use ← → to move between sections.