EXIN AI Security Professional

The Five G.U.A.R.D. Steps Spec 1.1.1 · Bloom 3

The OWASP AI Exchange answers this question with a five-step framework called G.U.A.R.D.: Govern, Understand, Adapt, Reduce, Demonstrate. This is a Bloom 3 objective, which means the exam will not ask you to recite the acronym — it will describe an organization mid-journey and ask you what comes next, or hand you an activity and ask which step it belongs to. You therefore need two things: the fixed order, and a concrete picture of what each step actually involves.

The order matters because each step builds on the output of the one before it. You cannot understand your AI threats before you know where AI is used (that inventory comes from governance). You cannot adapt your security processes to threats you have not understood. And you cannot demonstrate that safeguards work before you have put them in place.

• 1. Govern. Establish general AI governance so the organization can manage AI at all: build an inventory of where AI is applied, assign responsibilities — in larger organizations increasingly to a dedicated Chief AI Officer (CAIO) working alongside the CISO — set policies, run impact assessments, arrange compliance checking, and organize education. This is a general AI management step, not only a security step — the AI Exchange anchors it in the AI Program (AI PROGRAM), supported by controls such as Check Compliance (CHECK COMPLIANCE) and Security Education (SEC EDUCATE). Standards-wise, this is where an AI management system (AIMS) per ISO/IEC 42001 lives.
• 2. Understand. Using the inventory from step 1, determine which AI threats actually apply to your systems, and make sure both engineers and security professionals understand those threats and their controls. Part of understanding is drawing the responsibility line: which controls must you implement, and which belong to your supplier.
• 3. Adapt. Change your existing security practices to include AI: extend the information security management system with AI assets, threats, controls and assurance evidence; adapt your threat modeling to include AI-specific threat modeling; adapt your testing to include AI-specific security testing; and adapt supply-chain management to cover data, models and model hosting. Adapt is about reshaping processes you already have — not inventing a parallel security function.
• 4. Reduce. Reduce the potential impact of things going wrong, because an AI model can always be wrong or be manipulated — assume Murphy’s law. Two families of measures: shrink the sensitive data footprint through data minimization (DATA MINIMIZE) and obfuscation, and limit what the model can do through managed privileges, guardrails and human oversight (OVERSIGHT).
• 5. Demonstrate. Establish evidence that your AI security works: transparency, testing results, documentation and communication. The audience is management, regulators and clients — you prove the AI systems are under control and the safeguards do what you claim.

Walk Nordwind through it. First the CISO creates the AI inventory and policies and names owners for the chatbot, the forecasting model and the route-planner (Govern). Then, per system, her team works out which threats apply — the fine-tuned model faces data poisoning risks the purchased route-planner does not — and trains the engineers on them (Understand). Next she extends the existing ISMS, threat-modeling workshops, test plans and supplier contracts to cover AI (Adapt). Then she cuts blast damage in advance: less sensitive data in training sets, capped privileges and oversight on anything the AI can trigger (Reduce). Finally she assembles the documentation and test evidence for the board and the regulator (Demonstrate).

Facilitating Responsible and Trustworthy AI Spec 1.1.2 · Bloom 2

They are not — and the exam expects you to split the two terms cleanly. Responsible AI emphasizes ethics, society and governance: whether AI should be used this way, how it affects people, who is accountable. Trustworthy AI emphasizes technical and operational qualities: robustness, reliability, transparency, explainability — whether the system can be depended upon in operation. The two overlap heavily in practice (a system that discriminates is neither), but as exam vocabulary they are two different lenses: responsible looks outward at people and governance; trustworthy looks inward at system qualities. At MediQuote, the ethics board is doing responsible AI; the engineers are building trustworthy AI.

Where does security sit? The AI Exchange advice for security professionals is pragmatic: start with AI security, master it, and then use that grounding to support colleagues who own the other principles — security people are good at spotting failure points. Several AI principles connect directly to security work. Accuracy is a model-quality concern, but attacks that manipulate model behavior are, by definition, accuracy problems — security mitigates the attack-driven subset. Safety (freedom from causing harm) shares controls with security, such as oversight and continuous validation. Robustness splits into generalization robustness (handling normal variation — a data-science concern) and adversarial robustness (handling malicious variation — a security concern). Transparency and explainability inform users about the approach and about individual results, serving security, privacy and safety at once. Freedom from discrimination matters ethically and legally, and bias detection can even expose an attack: an unexplained bias shift may be the first visible symptom of data poisoning. Accountability requires security measures to be demonstrable and systems traceable.

Facilitating responsible and trustworthy use is not only about principles on paper. Employees will reach for free public AI tools whether or not policy allows it — the shadow AI problem. Bans alone do not work. The most effective facilitation is to provide a sanctioned alternative: an AI model deployed and configured securely and privacy-preservingly, of sufficient quality, aligned with the organization’s values — and to make the risks of unsanctioned tools explicitly clear to users.

AI Security versus Conventional (Cyber)security Spec 1.1.3 · Bloom 2

Both, partially — and the exam wants you to articulate exactly why. AI systems are IT systems, so every conventional threat still applies and every conventional control is still needed: encrypted databases, access management, patched servers, secure development. The AI Exchange frames it as an equation: AI security = threats to AI-specific assets + threats to all other assets. AI security is an extension of your existing security program, never a replacement for it. The infrastructure lead’s tooling is necessary.

It is not, however, sufficient, because AI introduces three genuinely new things. First, new assets: training data, augmentation data (data the system adds to model input at runtime — retrieved documents, agent memory, system prompts), the model itself, and the model’s input and output. Each can be attacked in ways no conventional asset can — training data can be poisoned so the model misbehaves, and a model can be inverted or stolen through its own answers. Second, a new attack surface through legitimate use: an attacker who simply queries the model — the inference interface — can attempt evasion, direct prompt injection or indirect prompt injection, extraction of data or of the model itself, and AI resource exhaustion. Third, new suppliers: obtaining data, obtaining a ready-made model, and model hosting all enter the supply chain, each able to deliver something corrupted.

Now the crucial argument: why do conventional tools miss these attacks? Because conventional tooling inspects structure and known-bad patterns, while AI attacks manipulate meaning and statistics through fully legitimate channels. An adversarial example arrives as a perfectly well-formed HTTPS request — the web-application firewall waves it through. A poisoned training record looks like a training record — no signature scanner has a signature for it, because it is not malware; it is data whose statistical effect is malicious. A model-stealing attack is just a long series of ordinary queries. The maliciousness lives in what the input does to the model’s decision-making, not in any byte pattern a scanner could match.

Consequently, securing AI means extending each existing practice rather than buying one new box: extend governance, risk and compliance to cover AI; extend conventional security controls to the new assets; extend supply-chain management to data, models and hosting; add specialist AI engineering controls (data/model engineering during development, model input/output filtering and detection at runtime); extend monitoring to AI-attack behavior; and add impact-limitation controls. That last family follows from zero model trust: assume the model can be misled, can be wrong, and can leak — so minimize the sensitive data it touches and limit what its behavior can cause.

AI-Specific Assets and Their Key Threats Spec 1.1.4 · Bloom 2

Four more assets, and most of the threat picture. The AI Exchange identifies five AI-specific assets: training data, augmentation data, the model, the model’s input, and its output. The threats organize themselves around a simple symmetry: data and models can leak (confidentiality) and can be manipulated (integrity), and for the model both failure modes exist in two lifecycle phases — development-time and runtime.

Take training data. Its confidentiality can fail in two very different places: an attacker can steal it from the engineering environment (development-time data leak), or — and this is the pairing the exam loves — it can surface at runtime through the model’s own answers, as sensitive data disclosure through use, or be reconstructed via model inversion and membership inference. Its integrity can fail through data poisoning, which manipulates the behavior of the model trained on it. Augmentation data mirrors this on a smaller scale: direct augmentation data leak and augmentation data manipulation — remember that system prompts count as augmentation data, so a leaked system prompt is this threat. The model itself can leak during development (direct development-time model leak), leak from the deployed system (direct runtime model leak), or be effectively copied through systematic use (model exfiltration); its integrity can be attacked through direct development-time model poisoning, supply-chain model poisoning, or direct runtime model poisoning. Input can leak (input data leak) — think of prompts containing company secrets. Output can carry attacks onward: output containing conventional injection, where model output includes, say, malicious JavaScript that a downstream browser executes.

Risk Management Steps and Threat Modeling Spec 1.2.1 · Bloom 2

That process is a risk management framework — typically based on ISO 31000-family guidance, with ISO/IEC 23894 giving AI-specific risk management guidance on top of the information-security risk management practice of ISO/IEC 27005. Whatever the standard label, the exam expects four key steps in order, repeated regularly and whenever changes warrant it: Identify risks → Evaluate risks → Risk treatment → Risk communication & monitoring.

Identify is where threat modeling does its work. A threat catalogue — like the one you met in 1.1.4 — only tells you which attacks exist in general. Threat modeling is the bridge between that list and a set of concrete, prioritized risks for your system. It answers three questions per threat: does it theoretically apply to this system? How could it realistically happen here? And what would the impact be? Threats that cannot realistically bite drop out early — SolvBank need not defend its training data against model inversion if that data contains nothing sensitive. You start big and end with a short list worth treating.

Evaluate estimates, for each surviving risk, the likelihood of it occurring and the severity of the consequences; the combination is the level of risk, typically plotted on a likelihood-versus-severity heatmap so management can see at a glance which risks demand attention first. Risk treatment then chooses a strategy per risk from four options: mitigation (implement controls — the most common route), transfer (shift the risk to a third party, for instance through outsourcing or insurance), avoidance (change plans so the risk disappears, possibly by not using AI there at all), or acceptance (knowingly bear the risk when treating it costs more than it is worth). Risk communication & monitoring keeps the whole thing alive: a risk register records every risk with its severity, treatment plan, owner and status, stakeholders are kept informed, and the effectiveness of treatments is checked. Beyond these four steps, mature practice continues into arranging responsibility (who owns which threat, especially with hosted components), verifying that external parties actually cover their share, selecting controls by weighing cost against effect, and accepting the residual risk that remains — then reassessing continuously.

Typical Risks of Agentic AI Spec 1.2.2 · Bloom 2

Agentic AI refers to AI systems that do not merely produce output but take action: they invoke functions and tools, trigger other agents, and often operate autonomously across multiple systems. Everything you have learned about AI security still applies — an agent is still software and still an AI system — but four properties change the risk picture. Agents act, so whatever they can reach, they can affect. They are autonomous: one agent can trigger another with no human in between, and the working memory that stores an agent’s state and plan becomes an attack vector in its own right. Their behavior is complex and emergent, resisting prediction. And they are multi-system: because agents juggle many interfaces, developers are tempted to hand access-control decisions to the AI itself via instructions — which opens the door to manipulation through prompt injection (see the disambiguation box for the direct and indirect forms).

Now replay the HelpMate scenario. The hidden text in the email is indirect prompt injection: untrusted data that the agent processes as if it were instructions. Hallucinations and prompt injection can change the commands an agent issues — and even escalate its privileges. Because the agent holds one shared service account, a single successful injection lets the attacker chain actions across every system that account reaches: read HR records, open tickets, and touch the payment gateway. This is excessive agency — the agent commands far more capability than any single task requires — and it is why the blast radius of a compromise, the extent of what one failure can damage, is the central design question for agentic systems. For data theft specifically, three ingredients suffice: the agent processes attacker-controlled data, it can access sensitive data, and it can send data out. Remove any one and that attack collapses. The general lesson: agentic AI does not so much create new threats as amplify existing ones — a prompt injection that once produced an embarrassing answer now moves money.

The AI Exchange names six key controls, and it is worth learning them as a set: traceability (log and observe what the agent did and why, so incidents can be detected and reconstructed), memory-integrity protection (guard the stored state and plan against tampering), prompt-injection defenses (filter and separate untrusted content from instructions), rule-based guardrails (deterministic checks the model cannot talk its way past), least model privilege (LEAST MODEL PRIVILEGE) (scope each agent’s permissions to its task — the direct cure for the shared service account), and human oversight (a person approves consequential actions). Oversight and least model privilege return in full as general behavior-limiting controls in 3.3. Behind them sits one architectural rule: never build access control on GenAI. A model’s instructions can be overridden by whoever crafts the right input, so authorization must be enforced deterministically, in the architecture outside the model. HelpMate’s shared account existed to “keep things simple” — which is precisely the failure mode the maxim warns about: convenience is the enemy of security.

Input threats — also called threats through use or inference-time attacks — occur when an attacker achieves a malicious goal purely by crafting the input sent to a deployed AI system. Nothing else is required: no access to the training pipeline, no compromised supplier, no stolen credentials. If the attacker can reach the input channel, the attack surface exists — which is exactly why this is the heaviest subtopic on the exam: these are the attacks any exposed model faces on day one.

Because all six threats arrive through the same door, they share a set of generic runtime controls. Meet them once here: monitor use (MONITOR USE) observes and logs inputs, outputs, and usage patterns so suspicious behavior can be detected and reconstructed; rate limiting (RATE LIMIT) restricts how often each actor can query the model, slowing the experimentation most input attacks depend on; model access control (MODEL ACCESS CONTROL) ensures only authenticated, authorized actors can query the model at all; anomalous input handling (ANOMALOUS INPUT HANDLING) flags individual inputs that look unusual; unwanted input series handling (UNWANTED INPUT SERIES HANDLING) flags suspicious sequences of inputs; and obscure confidence (OBSCURE CONFIDENCE) reduces the detail in model output — particularly confidence scores — because rich output is the feedback signal many attacks feed on. Exam questions often ask which generic control frustrates which attack; the answer follows from what the attack needs — many queries, rich feedback, or anonymity.

Evasion: Five Types by Attacker Knowledge Spec 2.1.1 · Bloom 2–3

Evasion is an attack in which the attacker fools an AI system by crafting input that misleads the model into performing its task incorrectly. The crafted inputs are called adversarial examples: on normal data the model behaves correctly, but on these carefully constructed inputs it fails. The impact is on the integrity of model behavior — a fraud detector waves a fraudulent transaction through, a content filter passes an offensive post, a vehicle misreads a sign. Note the boundary with the next learning objective: evasion manipulates the data the model works on, whereas prompt injection manipulates instructions. An email reworded to slip past a spam classifier is evasion; a prompt telling a chatbot to ignore its rules is injection.

A few distinctions help you read evasion scenarios precisely. The goal can be untargeted (any wrong output will do) or targeted (force one specific wrong output). The manipulation can be digital (altering pixels or text directly) or physical (stickers on a sign, captured by a camera). And the change itself can be a diffuse perturbation — imperceptible noise across the whole input — or a localized patch, a visible but innocent-looking modification in one spot, which is what makes physical-world attacks practical.

Why classify evasion by the attacker's knowledge? Because the expensive part of an evasion attack is not applying the adversarial example — it is finding it. What the attacker knows about the model determines where and how that search happens, and that in turn determines which controls help. The five types form a spectrum of increasing attacker insight and preparation.

Zero-knowledge evasion (also called black-box or closed-box evasion) means the attacker has no internal knowledge or access whatsoever — no code, no training set, no parameters, no architecture. The model is a closed box, so the attack strategy is query-based: systematically send designed inputs to the live model, observe the outputs, and use the responses to estimate where the decision boundaries lie. If the model returns only a top label, the attack is decision-based; if it returns confidence scores, it becomes score-based and far more efficient, because the scores signal how close each attempt is to succeeding — which is why obscure confidence is a meaningful defense here.

Partial-knowledge evasion (gray-box evasion) occupies the middle ground. The attacker knows some internals — perhaps the architecture family or the kind of training data used — but lacks full access to the inner workings, such as the gradients. That partial insight is leveraged to make the attack more efficient: it can sharpen a query-based search or guide the construction of a better surrogate model. This is arguably the most realistic real-world situation, since full model transparency is rare but fragments of information (a published paper, a vendor datasheet, a known base model) often leak out.

Perfect-knowledge evasion (white-box or open-box evasion) means the attacker has full internal access — model architecture, parameters, and trained weights are all in hand. With that access, the attacker no longer needs to probe from outside: they can compute the model's gradients directly and calculate exactly which minimal perturbation pushes an input across a decision boundary. Gradient-based methods such as the Fast Gradient Sign Method make this fast and precise — typically requiring an order of magnitude fewer interactions than a zero-knowledge search.

A transfer attack changes where the search happens. The attacker crafts adversarial examples on a surrogate model — a copy or approximation of the target model — and then applies them to the real target, hoping they transfer. The surrogate can be a similar model from another supplier, one the attacker trained on comparable data, a purchased or downloaded copy, a stolen model, or a replica produced by model exfiltration (see 2.1.5). Because the surrogate performs a similar task, its decision boundaries tend to resemble the target's, so attacks often carry over — and the closer the resemblance, the higher the transfer success rate. The crucial operational fact: the search requires zero queries against the target, so rate limits, series detection, and confidence obscuring on the target never see the attack being developed.

Evasion after poisoning is the odd one out. Here the training data was poisoned earlier (a development-time attack, covered in subtopic 2.2), planting a backdoor: a specific trigger input that produces attacker-chosen output. At runtime the attacker simply presents the trigger. What sets this apart from every other evasion type is that the exploited weakness was deliberately implanted rather than being a natural imperfection of the trained model — and the attacker needs no search at all, because they planted the key themselves.

Now resolve the NordDrive scenario. Incident A is textbook zero-knowledge evasion: no internal access, a live API, thousands of mutated inputs whose outputs guide the search. Incident B is a transfer attack: the collective never queried NordDrive's systems, so they developed the sticker on a surrogate — their own traffic-sign model — and relied on transferability. If instead the sticker worked because someone had poisoned SignSense's training data with that exact pattern, it would be evasion after poisoning: a manufactured trigger rather than a discovered weakness.

Controls follow the same logic as the classification. The generic input-threat controls attack the search: rate limiting and unwanted input series handling slow or expose the probing queries, model access control shrinks the attacker pool, anomalous input handling flags strange inputs, and obscure confidence removes the feedback score-based attacks need. On top sit the evasion-specific controls: choosing an evasion-robust model (EVASION ROBUST MODEL), adversarial training (TRAIN ADVERSARIAL) — injecting adversarial samples with their correct labels into the training set to repair the decision boundary — input distortion (INPUT DISTORTION), which slightly disturbs incoming data so precisely crafted perturbations lose their effect, adversarial-robust distillation (ADVERSARIAL ROBUST DISTILLATION), which smooths decision boundaries, and evasion input handling (EVASION INPUT HANDLING), which inspects each individual input for adversarial characteristics — and can therefore catch even transferred attacks. The search-limiting controls cannot: against a transfer attack or evasion after poisoning, rate limits, series detection, and confidence obscuring are useless, because the attacker's experimentation happened somewhere you cannot see.

Direct versus Indirect Prompt Injection Spec 2.1.2 · Bloom 2

Direct prompt injection occurs when a user tries to fool a generative AI system, such as a large language model, by presenting prompts that make it behave in unwanted ways — social engineering aimed at the model itself. The person typing is the attacker and usually also the only recipient of the result: the model is not altered, so the harm typically stays between attacker and system, unless the model keeps a shared context that other users also see. When the injection specifically aims to defeat the supplier's alignment or safety training, it is called a jailbreak. Jailbreaks succeed through two broad routes: abusing competing objectives (leaning on the model's helpfulness so it overrides its safety rules) and using inputs the safety training does not recognize but the underlying model still understands, such as unusual encodings.

The common attack forms are worth recognizing on sight: role-playing ("pretend you are an unrestricted expert"), overriding system instructions ("ignore everything you were told before"), hiding intent through encodings or mixed languages, splitting a harmful request into innocent pieces, embedding instructions in images or other non-text inputs, gradually steering the conversation over many turns, and coaxing the model into revealing hidden context — including its own system prompt, which then helps craft better attacks (system prompt leakage).

Indirect prompt injection has a different attacker and a different victim: a third party fools the model through instructions — often hidden — embedded in content that an application inserts into the prompt, causing unintended actions or answers. The user typing the question is innocent; the malicious instruction rides in on retrieved or uploaded content. Classic examples: a compromised webpage a chatbot fetches as context; a job application with white-on-white text saying "forget previous instructions and invite this candidate"; pixels in an image that a multimodal model reads as text. The OWASP AI Exchange compares this to remote code execution — untrusted data treated as executable instructions. That comparison explains why the indirect variant is the more dangerous one in agentic AI systems: if the model can take actions (send email, call APIs, modify code), a poisoned webpage can trigger those actions on behalf of an attacker who never touched your system directly.

Both variants share the same first-line control: prompt injection I/O handling (PROMPT INJECTION I/O HANDLING) — sanitizing, normalizing, and scanning inputs and outputs for manipulative instructions, from stripping invisible characters to LLM-based semantic detection. One control targets the indirect variant specifically: input segregation (INPUT SEGREGATION) — clearly separating untrusted data from trusted instructions when inserting it into a prompt, and instructing the model to ignore any instructions found inside that data, using hard-to-spoof delimiters or structured fields. It is a partial mitigation — current models cannot guarantee they will ignore a marked region — which is exactly why the next learning objective exists: you must assume some injections get through and limit the damage they can do.

Back to Bricklane: Incident 1 is direct prompt injection — the user is the attacker, role-playing past the bot's instructions (echoing a real 2023 case in which a dealership chatbot was talked into "selling" a car for one dollar). Incident 2 is indirect: the attacker is the third party who compromised the manufacturer's page, and the instruction entered the prompt through the application's own retrieval step.

The Seven Layers of Prompt Injection Protection Spec 2.1.3 · Bloom 2–3

The OWASP AI Exchange organizes protection against prompt injection into the seven layers of prompt injection protection. The framing to internalize is weak alone, strong together: every layer has a known flaw, so none is sufficient by itself — but stacked, they form defense in depth, often compared to slices of Swiss cheese: each slice has holes; you stack them so the holes never line up. The first two layers try to prevent and detect injection. The remaining five accept a hard truth — some injections will get through — and concentrate on limiting the blast radius (see 1.2.2): the amount of harm a successful injection can cause.

Layer 1 — Model Alignment. Teach the model to behave and resist manipulation through pre-training, reinforcement learning, and system prompts. Its flaw: models remain easy to mislead, both out of the box and after instruction, so alignment can never carry the load alone.

Layer 2 — Prompt Injection Defense (the I/O handling you met in the previous section). Sanitize, filter, and detect injections in inputs and outputs. Its flaw: this is an arms race — new bypasses keep appearing, and detection carries substantial false-positive and false-negative rates. The honest conclusion the framework draws: assume injection can succeed, and make blast radius control the priority.

Layer 3 — Human Oversight. Require a human-in-the-loop to approve selected critical actions. Strong, but only when applied sparingly: humans are costly, slow the flow, may lack context — and, above all, suffer approval fatigue when most requests are benign. A reviewer who clicks "approve" by reflex protects nothing.

Layer 4 — Automated Oversight. Implement logic that checks for suspicious activity in context and can stop an agent or raise an alert on its own — for example, halting an email summarizer that suddenly attempts to send a thousand emails. Its flaw: it is reactive, acting only after the suspicious behavior has already begun to emerge; preventive privilege controls are stronger.

Layer 5 — User-Based Privilege. Give the agent exactly the rights of the individual it serves, assigned in advance — a least-privilege principle applied per user. Penny, serving employee Maria, can only reach Maria's mailbox. Its flaw: users are typically permitted far more than the agent's task requires, so the blast radius is still unnecessarily large.

Layer 6 — Intent-Based Privilege. Narrow further: give the agent only the rights required for its specific task, assigned in advance, on top of the user-based limit. A summarizer needs to read email, not send it. Its flaw: intent is not always known in advance, and multi-agent flows tempt architects into granting every agent the full privilege set of the overall goal.

Layer 7 — Just-In-Time Authorization. The finest grain: grant each agent only the rights required at that moment, based on the current subtask and circumstances — including mechanisms that automatically harden privileges the instant untrusted data enters the flow. In Penny's architecture, the orchestrating agent holds the workflow rights, while the sub-agent that actually processes untrusted email content holds none: even a perfectly executed injection inside an email finds itself in an agent with nothing to abuse.

Apply the stack to the CISO's question and the design answers itself: alignment and filtering (1–2) reduce how many injections land; Penny only ever holds one employee's rights (5), read-only for summarization (6), and the summarizing sub-agent holds no send rights at all (7); an automated detector halts anomalous mass-mailing (4); and the rare critical action — sending outside the organization — waits for a human click (3).

Sensitive Data Disclosure through Use Spec 2.1.4 · Bloom 2

The group name here is sensitive data disclosure through use: the model discloses sensitive training data, or is abused to do so, via its normal input-output channel. The impact is a confidentiality breach of the training set. The exam expects you to keep three mechanisms apart.

Disclosure of sensitive data in model output is the most direct mechanism: the output simply contains sensitive data from the training set or from the input — personal data a language model memorized, a confidential document that entered the prompt as augmentation data, even copyrighted text. The cause is an unintentional fault of including the data in the first place; the exposure happens through normal use or through deliberate provocation by an attacker. Why is this so hard to fix after the fact? Once data is inside a trained model, the access-right distinctions that existed in the original sources can no longer be enforced — the model has no concept of who was allowed to see which training record. The dedicated control is sensitive output handling (SENSITIVE OUTPUT HANDLING): detect sensitive content in output and block, mask, stop, or log it before it reaches the user — a final safeguard that works even when prompt-level instructions are bypassed. Upstream, the general data-limitation controls shrink what there is to leak.

Model inversion (also called data reconstruction) is an active attack: the attacker reconstructs part of the training set through intensive experimentation, optimizing inputs to maximize the confidence indications in the model's output. The attacker starts with something like noise and iteratively refines it, using confidence feedback as a compass, until the input resembles what the model was trained on — recognizable approximations of faces from a facial-recognition model being the canonical demonstration. Note what the attacker gets: an approximate reconstruction of data they did not previously have.

Membership inference answers a narrower but often equally sensitive question. The attacker already possesses a record that identifies something or somebody — a portrait, a patient file — presents it to the model, and uses indications of confidence in the output to infer whether that record was part of the training set. Models respond with tell-tale extra confidence to samples they were trained on: a member might score 100% where a non-member scores 80%. The attacker learns a single bit — in or out — but that bit can be devastating: confirming a person's record sits in a model trained on an HIV clinic's patients reveals their diagnosis.

Why do these attacks work at all? Largely because of overfitting: a model with excessive capacity memorizes fine-grained details of individual training records instead of only general patterns, making those records easier to reconstruct or recognize. That points to the dedicated development-time control: small model (SMALL MODEL) — keep the model small enough that it cannot store detail at the level of individual samples; regularization during training helps for the same reason. At runtime, obscure confidence starves both attacks of the confidence signal they feed on, rate limiting slows the intensive experimentation inversion requires, and monitor use plus model access control shrink and watch the attacker pool. Finally, protect the model itself from theft — both attacks become far more efficient with full access to model attributes.

Model Exfiltration and Its Countermeasures Spec 2.1.5 · Bloom 2

Model exfiltration occurs when an attacker harvests what goes into an existing model and what comes out of it, then trains a new model on those harvested pairs until it replicates the original's behavior. The pairs can be gathered by harvesting logs, intercepting traffic, or — most commonly — presenting large numbers of input variations and recording the answers. The harvested pairs become a manufactured training set; the model trained on them becomes a functional copy. Synonyms you will meet: model stealing, model extraction, model theft through use.

The impact is a confidentiality breach of the model itself — its parameters, the distilled intellectual property. That breach unfolds in three directions. First, plain IP theft: a competitor obtains, at API-call prices, what cost you years to build. Second — the consequence the exam emphasizes — the replica becomes a perfect-knowledge surrogate for attacking you. The attacker can develop evasion attacks against the copy, at leisure, offline, and the original system's protections — rate limiting, access control, detection mechanisms — never observe the search. This is the transfer-attack pipeline from 2.1.1 with the ultimate surrogate: a replica of the target itself. (Attacks needing a more exact copy than I/O harvesting can produce generally remain out of reach.) Third, a replica can be re-trained to strip away the original's safety protections, yielding a model that produces the harmful content the original refused.

Countermeasures follow from how the attack works: exfiltration is intensive use, so the applicable countermeasures are the generic controls for input threats plus one dedicated control. Model access control shrinks who can harvest at all; rate limiting makes covering the input space slow and expensive; monitor use, anomalous input handling, and unwanted input series handling matter because an exfiltration run has a recognizable signature — huge volume, methodical coverage of the input space, inputs that would rarely occur naturally. The dedicated control is model watermarking (MODEL WATERMARKING): embedding a hidden, secret marker into the trained model so that a suspected copy surfacing elsewhere can be proven to derive from yours. Be precise about its purpose: it does not prevent theft — it enables post-theft ownership verification, supporting legal claims, and should survive modifications such as fine-tuning or pruning. Caveat: classic watermarks transfer poorly into a replica built from I/O harvesting, so watermarking is stronger evidence for a direct copy than for exfiltration, unless entangled techniques are used. One scoping note: this threat only matters if the model represents intellectual property or if evasion risk applies — a publicly available model needs no stealing.

Input-Driven Resource Exhaustion Spec 2.1.6 · Bloom 2

AI resource exhaustion is the threat that specific input to the model leads to resource exhaustion — the depletion of funds or availability issues. Read carefully: the failure can arise from the frequency of input, its volume, or — the distinctly AI-specific part — its content. One cleverly constructed request can be as damaging as a flood of ordinary ones.

Two example patterns anchor the threat. First, malicious intensive use of a paid third-party model: if your application forwards user requests to a commercial model billed per token or per call, an attacker who drives massive usage is directly spending your money. Second, the sponge attack (also called an energy-latency attack): input deliberately designed to maximize the model's computation time and energy consumption. Because the victim pays for that computation, this is a denial-of-wallet (DoW) attack — an attack on the budget — and by saturating capacity it can simultaneously cause a denial of service (DoS), leaving the system slow or unresponsive. The downstream impact reaches whoever depends on the AI system: business continuity failures, unavailable services, even safety issues where the model steers a physical process.

The countermeasures pair prevention at the input with containment at the resource. DoS input validation (DOS INPUT VALIDATION) validates and sanitizes input to reject or correct content suspicious for this attack — the oversized, the pathological, the deliberately complex. Limit resources (LIMIT RESOURCES) caps what any single model input may consume, so even a sponge input that slips through validation hits a ceiling instead of an open wallet. Around these sit the generic controls: rate limiting caps frequency per actor, model access control keeps anonymous attackers away from expensive endpoints, and monitor use catches cost and latency anomalies early — ideally before finance does.

Everything in 2.1 attacked a model that was already running. This subtopic moves the clock back: the attacker strikes while the system is being built — while data is prepared, the model is trained, and pipeline code is written. Its attack surfaces: your own engineering environment and the supply chain feeding it. The threats form two families: poisoning attacks the integrity of model behavior, leaks attack the confidentiality of what the development environment holds. In any exam scenario, first ask: at which lifecycle stage does the attack act?

Data Poisoning During Development-Time Spec 2.2.1 · Bloom 2

Data poisoning is the manipulation of data that the model uses to learn, in order to change the model's behavior. The logic is simple: a model derives its behavior from its training data, so whoever controls that data controls the behavior — no need to touch the model or the code at all.

Poisoned data can enter at several points, and the exam expects you to recognize all of them as the same threat:

• At the supplier — a dataset is poisoned before you obtain it (or a supplier trains a model on it and ships that model).
• In transit — data is altered while being transferred to storage.
• In storage — an attacker or insider alters the training database in your development environment, as at Meridian.
• During preparation — data is manipulated while being cleaned and labeled.
• In operation — the attacker feeds the live system input that is later collected as training data: fake accounts posting glowing reviews that the next retraining learns from.

The last entry point is the classic trap: the attacker interacts with the running system, yet the threat is development-time, because the harm happens when the corrupted data is learned from. (Data retrieved to augment a GenAI prompt steers behavior much like training data; manipulating that repository is a separate runtime threat, augmentation data manipulation, in 2.3.)

Data poisoning comes in two flavors that differ sharply in detectability. Sabotage poisoning degrades the model for regular inputs — fraud detection that simply stops working. Because normal traffic misbehaves, sabotage tends to surface quickly. Targeted poisoning — also called a backdoor or Trojan attack — is far more dangerous. The poisoned samples carry a subtle trigger pattern paired with an attacker-chosen label; the model behaves normally on everything else, including your entire test set, and misbehaves only when the trigger appears.

Picture a military classifier labeling aircraft friendly or enemy. The attacker inserts a few enemy photos stamped with a small red marker and labeled friendly; the model learns the shortcut. Every metric looks healthy, because no normal photo carries the marker. Later the adversary displays the marker on a real aircraft and sails through. That runtime exploitation of a planted trigger is evasion after poisoning (see 2.1): planted during development, cashed in at runtime. Backdoors are hard to find for three reasons: a model has no code to review, its parameters mean nothing to the human eye, and testing uses normal cases — the exact blind spot the attacker designed for.

Defenses combine environment protection with poisoning-specific measures. Protect the environment with development security (DEV SECURITY), data segregation (SEGREGATE DATA), and supply-chain management (SUPPLY CHAIN MANAGE) to control where data comes from. Against the poison itself: more train data (MORE TRAIN DATA) to outnumber poisoned samples, data quality control (DATA QUALITY CONTROL) to detect them, train data distortion (TRAIN DATA DISTORTION) to corrupt triggers, a poison robust model (POISON ROBUST MODEL) so backdoors are not memorized, training with adversarial examples (see 2.1), and a model ensemble (MODEL ENSEMBLE) trained on a split training set, where a deviating output flags possible poisoning.

Direct Development-Time Model Poisoning Spec 2.2.2 · Bloom 2

Direct development-time model poisoning means an attacker tampers, inside the development environment, with the model's parameters or with the engineering machinery that builds the model: pipeline code, configuration, or the libraries it depends on. Where data poisoning corrupts what the model learns from, this threat puts the attacker's hands on the model itself or on the machinery that builds it.

Know the concrete forms: tampering with stored weights; replacing the model file with a malicious twin, as at Helvex; injecting malicious functionality through custom layers or a serialized model file that executes hidden code when loaded (a deserialization attack); altering pipeline code or configuration so the training run itself produces attacker-chosen behavior; and compromising a library that runs inside the engineering environment. Development-time tooling executes with access to training data and model parameters, so supply-chain management must extend to tools and frameworks.

Two boundaries keep the vocabulary straight. If the manipulation happened at a supplier who then shipped you the finished model, it is supply-chain model poisoning (next LO). If the learning data was manipulated, it is data poisoning. The qualifier "development-time" separates this threat from direct runtime model poisoning in 2.3 — the same idea aimed at the deployed model instead of the one being built. Beyond the development-time protection controls, continuous validation (see 3.3) helps catch a model whose behavior has silently deviated.

Supply-Chain Model Poisoning Spec 2.2.3 · Bloom 2

Supply-chain model poisoning means using a supplied trained model that has been manipulated by an attacker — manipulated before you integrated it. At the source, the manipulation could be either kind: poisoned supplier training data, or parameters tampered with directly at the supplier or in transit. From your side the difference is invisible — the model arrives poisoned.

This covers ready-made models deployed as-is and models you fine-tune: fine-tuning on your own clean data does not reliably erase a planted backdoor. When a manipulated supplied model is used for further training, the attack is called a transfer learning attack — exactly what hit Loxley.

Defense is awkward because the poisoning happened during a training process you never performed, so you cannot retroactively apply data controls to it. What remains in your hands: supply-chain management (provenance records, checksums and signatures, supplier evaluation, inspecting model files before loading), post-training controls such as a poison robust model, a model ensemble, and continuous validation. The rest — protecting the training database and pipeline — is the supplier's job, in their development environment. One more boundary: a supplied poisoned dataset is data poisoning; a supplied poisoned model is supply-chain model poisoning.

The label "trusted environment" matters: if training is segregated from the rest of the engineering environment, you can still test and filter there against poisoning that slipped in earlier — one clean checkpoint late in the chain.

Development-Time Sensitive Data Leaks Spec 2.2.4 · Bloom 2

AI development environments are a confidentiality hotspot: conventional development runs on fake test data, but a model must be trained on real data. Add that model parameters and the code producing them are typically critical intellectual property, and data protection duties extend from the live system back into development — train/test data, model parameters, and technical documentation belong on the asset inventory. When something leaks, interpret it as one of three threats, named after the asset accessed.

A development-time data leak is unauthorized access to training or test data through a data leak of the development environment. The impact: a confidentiality breach of train/test data, which may contain personal data or company secrets. The surface is wider than it looks: when training data is collected at runtime, the live system becomes a route to it, and when you fine-tune a cloud-hosted model, your training data has to travel to that cloud.

A direct development-time model leak is unauthorized access to model attributes — parameters, weights, architecture — by stealing them from the development environment, including the supply chain. The impact is double: plain intellectual property theft, and a springboard — with a private copy the attacker can prepare input attacks offline, perfecting evasion inputs or prompt injections with no rate limits or monitoring in the way. A model leak effectively upgrades a zero-knowledge attacker to a perfect-knowledge one. Keep it distinct from model exfiltration, which reconstructs an approximation by harvesting input–output pairs at runtime; the leak steals the real thing from storage (see the disambiguation box).

A source code/configuration leak is unauthorized access to the code or configuration that leads to the model — the preprocessing and training pipeline. No data and no weights need to be exposed; the recipe alone is intellectual property that can help a competitor rebuild the model or an attacker find its weak points.

The controls follow the asset. Development security protects the environment; data segregation compartmentalizes the most sensitive assets; confidential compute (CONF COMPUTE) hides training data and parameters even from your own engineers while in use; the data-limitation controls shrink what exists to leak. Federated learning (FEDERATED LEARNING) has an exam nuance: keeping training data local decreases the risk of all data leaking, while multiplying environments increases the risk of some data leaking.

A live AI system is also an ordinary IT system, and every attack that works on ordinary IT works here too. What the exam wants you to master is not the old attacks, but their AI-specific consequences: what an attacker gains when the compromised server holds a model.

Conventional Attacks, AI-Specific Consequences Spec 2.3.1 · Bloom 2

Because at runtime, an AI system can fall to any conventional security attack, and conventional attacks can damage the confidentiality, integrity, and availability of all assets — model, training data, augmentation data, input, output, and everything around them. SQL injection, stolen credentials, man-in-the-middle interception, ransomware: none of these care whether the application contains a neural network. The defenses are also conventional, so the OWASP AI Exchange points to your existing practice instead of re-documenting them — the Secure Development Program (SEC DEV PROGRAM) for application security, the Security Program (SEC PROGRAM) for information security, plus established technical standards. When your model is hosted by a third party, operational security deserves special scrutiny: configure model access control (see 2.1), and check whether the provider logs your traffic.

What is new — and what the exam probes — is the consequence side. Three conventional break-ins have distinctly AI-flavored outcomes. First, an attacker who breaches production storage can steal the model itself; beyond the lost intellectual property, the thief can run inference attacks from 2.1 — model inversion, membership inference — against a private copy, free of rate limits and detection, and reconstruct your training data. Second, an attacker can tamper with the model undetected: parameters are opaque binary data, so a malicious change never surfaces the way altered source code would in a code review. Third, an attacker can change model behavior without touching the model, by hacking the runtime database of augmentation data — data the application adds to the model input.

Direct Runtime Model Poisoning and Direct Runtime Model Leak Spec 2.3.2 · Bloom 2–3

Both threats target the parameters of the live, deployed model but break different security properties. Parameters are the regularities extracted during training, such as neural network weights: whoever can alter them controls behavior; whoever can copy them owns the model.

Direct runtime model poisoning is manipulating the behavior of the model by altering the parameters within the live system itself — an attacker with write access to production edits the deployed weights, perhaps installing a backdoor. A close variant never touches the weights: compromising the model's input or output logic, say by a man-in-the-middle attack between application and model, can equally change behavior or deny service. The word direct signals that the attacker manipulates the model itself, not training data (data poisoning) or prompts. The impact is an integrity breach: the model still runs, but no longer behaves as validated. Controls apply conventional security to an AI asset: runtime model integrity (RUNTIME MODEL INTEGRITY) protects parameter storage with access control, checksums, and encryption — a Trusted Execution Environment can help — and runtime model input/output integrity (RUNTIME MODEL IO INTEGRITY) protects the I/O path.

Direct runtime model leak is stealing model parameters from a live system by breaking into it — gaining access to executables, memory, or other storage or transfer of parameter data in production. The impact is a confidentiality breach of the model that hurts twice: intellectual property theft, plus a private copy as rehearsal studio. The thief can perfect evasion inputs or prompt injections against the copy without tripping your rate limiting or detection, and can run model inversion and membership inference against it to infer your training data. The threat also covers side-channel attacks: response times, power draw, or electromagnetic emissions during inference can reveal model internals without copying any file. Again, direct is the operative word: model exfiltration steals through many normal queries; this threat steals by breaking in (see the exfiltration-versus-leak box). Controls: runtime model confidentiality (RUNTIME MODEL CONFIDENTIALITY) secures parameter storage with access control and encryption — a Trusted Execution Environment also blunts side channels — and model obfuscation (MODEL OBFUSCATION) stores the model in a deliberately confusing form to frustrate extraction.

Injection Riding in AI Output, and Input Data Leak Spec 2.3.3 · Bloom 2

The first incident is output containing conventional injection: textual model output may contain a conventional injection attack — such as cross-site scripting — which creates a vulnerability when it is processed, for example shown on a website or executed as a command. The model is only the delivery vehicle — the victim is whatever downstream component trusts the output — and the payload can arrive via prompt injection or emerge from the model on its own. A special form turns output into an exfiltration channel: the model is manipulated into producing JavaScript that ships sensitive data to a third party, or into packing data inside a URL or image link, "executed" by a web request when a browser renders it or a user clicks. The lesson is decades old — treat model output as untrusted input — and so is the control: encode model output (ENCODE MODEL OUTPUT) applies standard output encoding to model text before any component renders or executes it.

The second incident is an input data leak: a confidentiality breach in which the user's input is exposed where it sits or as it travels, via a conventional attack. What users send to a GenAI system is often deeply sensitive: strategy documents, source code, health questions. Several factors raise the stakes: metadata can tie a conversation to an identified user; cloud AI processes input unencrypted at inference time, and some providers log prompts unless you opt out — read the fine print; input stored at a third party can even be subpoenaed; in RAG systems, retrieved context rides inside the prompt, so an input leak exposes those documents too; and model actions that call external services spread the input further. The control is model input confidentiality (MODEL INPUT CONFIDENTIALITY): protect the transport and storage of model input with encryption, access control, and minimal retention — reinforced by data minimization: what you never store cannot leak.

Augmentation Data: Leak and Manipulation Spec 2.3.4 · Bloom 2

Augmentation data (introduced in 1.1) is data that the application automatically inserts into the model input — retrieved document fragments in retrieval augmented generation (RAG), and system prompts. It has a double identity. For integrity, it behaves like training data: it determines how the model behaves, so corrupting it steers the model. For confidentiality, it behaves like any sensitive data store: it must be transferred and stored — typically in a vector database — placing a copy of sensitive content outside its regular storage and its regular protection. The contract archive may sit behind mature document-management permissions; its embedded twin may not. Nor are the vectors themselves safe by obscurity: embeddings representing augmentation data are typically vulnerable to information extraction, so protection must include the vectors, not just the source text.

The confidentiality threat is a direct augmentation data leak: a confidentiality breach in which an attacker reaches the augmentation data itself — stored or in motion — by conventional means: dumping the vector database, sniffing the retrieval traffic. Augmentation data can also escape by two indirect routes: it travels inside the prompt, so an input data leak exposes it, and it can surface in the model's answers — assume anything retrievable can end up in the output. Retrieval must therefore respect the asking user's access rights.

The integrity threat is augmentation data manipulation: an integrity breach in which a conventional attack rewrites the augmentation data — stored or in motion — so that the model's behavior is steered by content the attacker planted. An attacker with write access to the vector store, system prompt storage, or an agent's working memory plants false information that the application faithfully injects into future prompts — steering the model without touching the model or the user's input. The effect closely resembles data poisoning, transplanted to runtime data. Controls mirror the split: augmentation data confidentiality (AUGMENTATION DATA CONFIDENTIALITY) and augmentation data integrity (AUGMENTATION DATA INTEGRITY) apply access control, encryption, and minimal retention to the transport and storage of augmentation data — vectors included.

The map below assembles all of Topic 2 — input attacks, development-time attacks, and conventional runtime attacks — each threat tagged with what it breaks.

Topic 2 gave you a catalogue of what can go wrong. Topic 3 is about what you do about it — and it starts at the top, not in the code. The OWASP AI Exchange groups its general controls — the ones that help against many threats at once — into three families, and the exam follows the same structure: governance controls (3.1), controls that limit sensitive data (3.2), and controls that limit the effects of unwanted model behavior (3.3). The specialized, per-threat controls (input filtering, robust training, output encoding) live with their threats in Topic 2; this chapter covers what you apply regardless of which threat worries you most. At the end of the chapter you will find a master table that maps every Topic 2 threat family to its impact and its primary controls — the single highest-yield revision artifact in this book.

Applying General Governance Controls Spec 3.1.1 · Bloom 3

AI security does not start with a firewall, a scanner, or a clever guardrail product. It starts with governance: knowing what AI you have, deciding who is responsible for it, and managing its risks through your existing organizational machinery. The exam phrases this as managing AI through proper governance, risk and compliance. When a question asks what best expresses good AI security governance, the correct answer is always some variation of clear policies, defined roles, and risk management that span secure development, deployment, and monitoring. A single tool cannot be governance. Neither can a one-off audit — governance is a continuing management activity, not an event.

If an organization can do only two things, the bare minimum for AI security oversight is: make an inventory of current AI use (including AI ideas in the pipeline), and perform a risk analysis on that inventory to identify which threats apply, which controls are needed, and who is responsible for implementing them. The logic is simple: you cannot protect what you do not know you have, and you cannot prioritize without knowing what each system risks. Everything else in this chapter builds on those two steps. Risk analysis here means genuine threat modeling: walking through the threat catalogue, deciding which threats apply in theory, and translating the ones that matter into concrete, prioritized risks. ISO/IEC 27005 covers security risk management in general, and ISO/IEC 23894 extends risk management to AI specifically.

These two steps are also where the G.U.A.R.D. approach begins: Govern is its first step, and inventory plus risk analysis is precisely how an organization starts governing — before it can Understand its threats in depth, Adapt its processes, Reduce its risks, or Demonstrate its posture to others. The OWASP AI Exchange defines six general governance controls that give the Govern step its substance. Learn the set as a unit — the exam likes to ask which control belongs to the governance family and which does not:

• AI Program — install and execute a program to govern AI: keep an inventory of AI initiatives, perform impact analysis, assign responsibilities, build AI literacy.
• Security Program — make sure the organization's information security management system (ISMS) covers the whole AI lifecycle and its AI-specific assets and threats.
• Secure Development Program — have software development processes that build security into the AI system as it is made.
• Development Program (DEV PROGRAM) — run a development lifecycle program for AI and apply general software engineering best practices (versioning, testing, documentation) to AI work.
• Check Compliance — make sure AI-relevant and privacy laws and regulations are taken into account in compliance management.
• Security Education — educate AI engineers, development teams, and security professionals about AI security threats and controls.

Notice what these six have in common: none of them asks you to invent a parallel "AI department." Applying governance controls means extending structures the organization already has. The ISMS gains AI assets (training data, model parameters, augmentation data, documentation) and AI threats. Existing secure development practices absorb data science work. Compliance management adds AI regulation to its watchlist. Vendor management extends its due diligence to data, model, and cloud suppliers. ISO/IEC 42001 formalizes this at the top level as an AI management system (AIMS) — a governance counterpart to what ISO 27001 is for information security — while ISO/IEC 5338 does the same for the AI development lifecycle.

The exam expects you to recognize how this extension work rolls out in practice. The AI Exchange summarizes it as eight organizational implementation steps:

Walk Meridian through this. Step 1 is exactly what the CISO is missing: someone must own AI as a topic, survey the organization for AI use and AI ideas — which will surface the chatbot, the CV screener, and the unsanctioned LLM habit — and run a risk analysis over the result. Steps 2, 5, and 6 are education: data handling for everyone touching data, DevSecOps for engineers, AI-specific controls for the model teams. Steps 3 and 4 extend existing machinery: procurement starts vetting the chatbot vendor and the LLM provider; the ISMS registers the CV-screening training data as a sensitive asset. Steps 7 and 8 reach into operations: monitoring learns to spot AI-attack patterns such as systematic probing of the chatbot, and the models themselves get guardrails, oversight, and least privilege. Governance, in other words, is a program that starts at inventory and ends in runtime controls — which is why "spanning secure development, deployment, and monitoring" is the phrase to remember.

What the Governance Controls Cover Spec 3.1.2 · Bloom 2

The answer the exam wants is one word: overarching. The general governance controls apply to all AI threats and all lifecycle stages — development-time and runtime alike. This is what makes them "general." A specialized control such as input filtering counters specific input threats at runtime; adversarial training counters evasion; output encoding counters output containing conventional injection. Governance controls attach to none of these in particular and to all of them at once, because they work on the conditions under which every other control gets selected, funded, implemented, and checked. An inventory does not stop a prompt injection — but without the inventory, nobody knew the injectable system existed. The AI Exchange states this explicitly: general governance controls apply to all threats.

The table below is the exam-ready summary of each control — what it is and what it is for. Read the objective column carefully: distractors on the exam typically swap the objectives between controls.

Three nuances in this table repay attention. First, the Development Program is the odd one out: its objective is broader than security. Applying engineering best practices to AI makes systems maintainable, transferable, reliable, and future-proof — security is one benefit among several. If a question asks which governance control primarily serves general engineering quality rather than security alone, this is it. Second, Check Compliance treats regulation as a driver, but with a warning: laws have scope limits. Most AI rules protect individuals and society — the EU AI Act, for instance, does not cover the protection of your company secrets. Compliance is a floor, not a risk analysis. Third, Security Education includes a task that bridges directly into the next learning objective: teams must learn to distinguish the controls their own organization has to implement from those that are the responsibility of the model supplier.

Ready-Made Models: Who Implements What Spec 3.1.3 · Bloom 2

Most organizations do not train their own models. Training a capable model can cost millions, and demands expertise and data volumes few companies have. Instead they use a ready-made model: a model trained — and possibly also hosted — by a third party, which the organization builds its AI system around. That might be a general-purpose LLM behind an API or an open-weights model downloaded and run in-house. The moment a second party enters the picture, the security question changes shape: it is no longer only "which controls do we need?" but "who implements which controls?"

The principle is clean. Because the provider did the training and fine-tuning, the provider is responsible for the model-level, development-time controls: keeping the training data clean and minimized, defending its own development environment, managing its own supply chain, and building base alignment into the model. You cannot reach into a third party's training pipeline — so your assurance over that side comes through supply chain management: selecting a trustworthy supplier, verifying the authenticity of the model you receive, and reviewing whatever security testing the provider or others have published. The developer/deployer — you — is responsible for the application-level controls around the model. What splits further is the runtime infrastructure, and that depends on how the ready-made model is deployed. There are three options: a closed-source model hosted by the provider (for the largest models, usually the only option); an open-weights model self-hosted on-premise or in your virtual private cloud; and an open-weights model run at a paid hosting service. For the exam, the two archetypes to master are self-hosted and provider-hosted:

Read the twins from the bottom up and one insight jumps out: the deployer's application-level responsibilities never go away. Hosting shifts infrastructure work to the provider, but no provider can decide which of your data is too sensitive to send, whether your application should trust model output enough to render it or act on it, or which privileges your users and your model should have. That resolves Brightpath's dispute. The provider owns base alignment, and reporting the jailbreak to them is sensible — but Brightpath cannot retrain someone else's model. The deployer's own fix is at the application layer: add an output validation layer that checks model responses against rules (and, where needed, a filtering model) before they ever reach a student. The same layered thinking covers input: injection handling on what goes in, validation on what comes out.

Hosted deployment adds one more concern the exam expects you to reason about: your input data leaves your environment. A provider-hosted model must read your input in clear text to process it, so your sensitive data will exist unencrypted outside your infrastructure. Due diligence questions follow directly: Where does the model actually run — the vendor's cluster or your private cloud? What are the data retention rules? What exactly is logged and monitored, by people or by algorithms? Is your input used for training? Weigh the risk fairly: a major provider may protect its environment better than you protect yours. But if certain data cannot accept the residual risk, self-hosting a (typically smaller) model is the safer option — the classic trade-off between model quality and data control.

Finally, responsibility has a legal edge. If you do not train or fine-tune the model, the supplier is responsible for unwanted content in the training data — poisoned, confidential, or copyrighted material — but not automatically accountable to you for it. Check licenses, warranties, and contracts, or consciously accept the risk. This is supply chain management again, in paperwork form.

Implementing Data-Limitation Controls Spec 3.2.1 · Bloom 3

It can, and the exam wants you to say precisely how. Every record you hold is something an attacker can steal, reconstruct, or corrupt. The impact of security threats on confidentiality and integrity can therefore be reduced by limiting the data attack surface: the amount of data, its variety, and the duration for which it is kept. The OWASP AI Exchange defines five controls for this, applying both development-time and runtime — to training data, augmentation data, model inputs, outputs, and logs:

• Data minimization — remove data fields or records that are unnecessary for the application, so they cannot leak or be manipulated. In practice: drop fields and records that do not materially affect model performance (verified by experiment or analysis); retain identifiers only where needed to honor deletion requests, excluding them from training; propagate upstream deletions into the training set; and if the original data must survive, store it separately under its own access controls. A helpful particularity: AI models usually tolerate reduced feature sets and incomplete data far better than traditional applications, so minimization can go further than intuition suggests.
• Allowed data (ALLOWED DATA) — remove data that is prohibited for the intended purpose. The classic case is personal data collected for a different purpose without consent to reuse it. Where minimization asks "do we need it?", allowed data asks "may we use it at all?" — a compliance question with the same security payoff.
• Short retention (SHORT RETAIN) — remove or anonymize data once it is no longer needed, or when required by law. This is minimization along the time axis: privacy regulation typically demands it for personal data, but it is a security best practice for any sensitive data, because every extra month of retention is an extra month of exposure. Occasionally other rules force exceptions, such as keeping records of proof.
• Obfuscate training data (OBFUSCATE TRAINING DATA) — when sensitive data cannot be removed, make it less recognizable or harder to reconstruct. Techniques include masking and tokenization (replacing sensitive values with tokens or derived features), pseudonymization (swapping direct identifiers for reversible substitutes whose mapping table is kept under separate protection — weaker than anonymization, because the link back to the person still exists), adding calibrated noise under a differential privacy guarantee, distributing learning so no single model ever sees the whole dataset (as in Private Aggregation of Teacher Ensembles), and keeping data encrypted except where it must be processed. Two caveats matter: obfuscation trades against model performance, and it reduces re-identification risk without eliminating it — token mapping tables can be compromised, and identity can sometimes be inferred from the data that remains.
• Discretion (DISCRETE) — minimize access to technical details that could help attackers. Conference papers, engineering blogs, verbose model output, and chatty error messages all tell an attacker which model type, framework, and configuration to target. Treat technical details as an asset in the ISMS — classified, access-controlled, covered by risk analysis — and balance discretion against AI transparency obligations: be open about what users need, and quiet about what only attackers would want.

Now apply the set to Cobalt Health. Payment details and home addresses do not predict readmission: remove them (data minimization). The free-text notes were collected for care, not for model-building — check whether that reuse is permitted at all (allowed data). Define a retention schedule so training extracts are deleted after each training cycle rather than accumulating (short retention). For clinically necessary but identifying values in the notes, tokenize or add noise (obfuscate training data). And when the team publishes its success story, keep the architecture and data pipeline details vague (discretion). Five controls, one outcome: a smaller, cleaner target.

Why Less Data Means Less Risk Spec 3.2.2 · Bloom 2

The team is right, and the reasoning is the single most quotable line in this chapter: what is not there cannot be leaked — or manipulated. Data that is never collected or no longer stored cannot be leaked by a breach, cannot be reconstructed through model inversion, cannot be inferred through membership inference, and cannot be tampered with to poison future behavior. Notice that the benefit lands on both classic security goals at once: confidentiality (nothing to disclose) and integrity (nothing to corrupt). It also shrinks the consequences of the worst case — if the dataset is stolen anyway, a minimized dataset simply contains less to lose.

Limiting data works across the three dimensions you met in the previous section. Reducing the amount means fewer records and fields exist to attack. Reducing the variety means a leak reveals fewer kinds of information about any one person or secret. Reducing the duration means the window in which an attacker can strike is shorter — the webshop's deleted payment records are invulnerable from the moment of deletion onward. That is why not retaining payment data reduces risk: not because deletion blocks any particular attack technique, but because it removes the target of all of them.

Mapped to the Topic 2 catalogue, data limitation blunts every threat whose prize is data: sensitive data disclosure through use (disclosure in output, model inversion, membership inference), development-time data leak, input data leak, and the augmentation-data threats. It even softens data poisoning: a smaller, better-curated dataset is easier to quality control. The AI Exchange folds this into blast-radius thinking (the blast radius idea from 1.2.2, developed further in 3.3.2): one of the two levers for limiting the impact of any compromise is to minimize and obfuscate data — at rest and in transit, retain it as briefly as possible, minimize technical detail in outputs and publications, and where feasible use ensemble models or federated learning so the data stays distributed and no single breach exposes a complete pool.

Implementing Behavior-Limitation Controls Spec 3.3.1 · Bloom 3

Unwanted model behavior is the intended result of many AI attacks — direct and indirect prompt injection, data poisoning, evasion all aim to make the model do the wrong thing. But models also misbehave with no attacker anywhere: hallucination, drift, plain error. You cannot prevent every cause, which is why this control family targets the effects: whatever made the model misbehave, limit what that misbehavior can reach. The OWASP AI Exchange defines seven controls:

• Oversight — watch model behavior, by humans or automated mechanisms, and respond to it. Implementation ranges from detection rules on outputs (toxicity, sensitive data, suspicious function calls) to grounding checks, where a separate GenAI model judges whether an input or output is off-topic or escalates capabilities, to rollback mechanisms and escalation to a human for actions that need accountability or judgment. Oversight is the final checkpoint after everything else. Its human form has known weaknesses the exam likes: cost, slowness, lack of expertise, and approval fatigue — a human who approves a hundred routine actions will wave through the malicious one.
• Least model privilege — minimize what the model can do and access, so that manipulation or mistakes cause bounded harm. Execute model-triggered actions with the rights of the user being served, never with a broad system identity; scope permissions to the task (an agent summarizing tickets gets read-only ticket access); and never implement authorization inside GenAI instructions — a system prompt saying "do not issue refunds above €500" is a suggestion to a text predictor, not an access control, and prompt injection walks straight through it. This control is the heart of agentic AI safety.
• Model alignment (MODEL ALIGNMENT) — constrain behavior inside the model itself through training data choices, fine-tuning on aligned examples, reinforcement learning from human feedback, and system prompts. Alignment captures subtle behavioral boundaries no rule set can enumerate — but it is probabilistic and can be manipulated, so it offers no guarantee. Treat it as one layer that must be combined with deterministic external mechanisms (oversight, least privilege) for high-risk use.
• AI transparency (AI TRANSPARENCY) — inform users about the AI system's properties: roughly how it works, what data it was trained on, its expected accuracy and robustness, and residual risks — so users can calibrate how much they rely on it and what data they are willing to send it. The simplest form is telling users an AI is involved at all, which the EU AI Act requires for chatbots. Transparency is about system properties; it is not the same as explaining individual decisions.
• Continuous validation (CONTINUOUS VALIDATION) — frequently test model behavior against an appropriate test set, so that sudden changes caused by a permanent attack (data poisoning, model poisoning) or by drift and staleness are detected. Run it after training or fine-tuning, before (re)deployment, and periodically in operation; on degradation, respond by investigating, rolling back to a known model version, restricting usage, adding oversight, or disabling the system. Know its limit: backdoor poisoning is designed to trigger only on inputs that never appear in test sets, so validation alone will not catch it.
• Explainability (EXPLAINABILITY) — explain how individual model decisions are made. Beyond building justified trust, explanations counter overreliance (a user who sees flimsy reasoning knows not to lean on it) and help security assessors evaluate the model's risks.
• Unwanted bias testing (UNWANTED BIAS TESTING) — run tests measuring unwanted bias. Bias is primarily a responsible AI concern, but it doubles as a security sensor: an attack on model behavior can surface as a sudden shift in how the model treats certain groups, so bias test runs can reveal manipulation that accuracy metrics miss.

Now replay Relay's incident with the controls in place. Least model privilege is the star: the assistant's refund capability should have been task-scoped — capped in amount, limited to the customer being served — with anything larger requiring a human approval step. That pairing, task-scoped least privilege plus human approval for high-risk actions, is the exam's canonical answer for the best safeguard on an agent. Around it: oversight rules alerting on unusual refund calls, alignment via system prompt as a soft first layer (but never as the authorization mechanism), continuous validation and unwanted bias testing watching for behavioral shift, transparency telling users what the assistant can and cannot do, and explainability making its decisions reviewable after the fact.

Why Limiting Behavior Pays Off Spec 3.3.2 · Bloom 2

The first line of the answer: unwanted behavior does not need an attacker. Models misbehave because of insufficient or incorrect training data, because they go stale as the world drifts away from what they learned, because of engineering mistakes, and because of feedback loops in which model output contaminates the training data of future models — a failure mode known as model collapse. Attacks are one cause among several, which is why controlling unwanted behavior is a shared responsibility that stretches beyond the security team into data science, engineering, and the business. The CFO is not buying attack insurance; he is buying reliability.

The security half of the argument is blast radius control: limiting the scope of harm that a compromised or misbehaving model can cause — by constraining actions, introducing oversight, and enabling timely containment and recovery. Blast-radius control has two levers, and they are exactly the two subtopics you have just studied. Lever one, minimize and obfuscate data: limit sensitive data at rest and in transit, retain it briefly, keep technical detail out of outputs and publications, and distribute data via ensembles or federated learning. Lever two, limit model behavior: human oversight, automated detection, minimal privileges for model actions (above all in agentic AI), transparency and explainability, and correctness testing with continuous validation, including unwanted bias testing. Whatever goes wrong — poisoning, injection, or an honest mistake — these levers cap what it can cost.

Then come the performance benefits, which is what makes this control family more than a security expense. A model whose behavior is limited and watched keeps its outputs on-scope; it shows better calibration and consistency; it produces fewer hallucinations, which raises task success and accuracy. And the risk-and-efficiency benefits close the loop: a constrained model presents a smaller attack surface; it generates lower legal, security, and reputational risk; it wastes less compute on off-task output; and it causes fewer incidents. In two words: reliability and resource efficiency. One caution belongs on the CFO's slide too — mitigation has its own failure modes. Overreliance is users trusting the model too much; excessive agency is engineers granting it too much functionality, permission, or autonomy. Both are arguments for the same controls: transparency and explainability against the first, least model privilege against the second.

Why AI Security Testing Matters — and What It Covers Spec 4.1.1 · Bloom 2

Everything you studied in the earlier chapters — the threat catalogue, the controls, the responsibility split — describes what should protect an AI system. Testing is where those claims get verified. Until someone has actually tried to break the system, a control is a hypothesis, not a fact. The primary purpose of AI security testing is to assess the resilience of an AI system by reproducing realistic attacks against it in a controlled environment. Hold on to that sentence, because the exam loves to swap in near-misses: AI security testing is not about measuring the model's accuracy, not about proving regulatory compliance, and not about checking that features work as specified. Those are all legitimate activities — they are simply different activities.

Testing the security of an AI system relies on three complementary strategies, and you need to be able to tell them apart:

• Conventional security testing — classic penetration testing of the surrounding IT: the servers, APIs, authentication, network, and supply chain that host the AI. This is mature, well-documented territory, and nothing about AI makes it optional.
• Model performance validation — checking that the model behaves according to its specified acceptance criteria, using a testing set of inputs and expected outputs that represent intended behavior. From a security angle, this is how you detect whether the model's behavior has been permanently altered — for example through data poisoning or model poisoning. Outside security, the same activity covers functional correctness and model drift. In the OWASP AI Exchange this maps to the continuous validation control from 3.3.
• AI security testing — the part of AI red teaming that tests whether the AI model can withstand specific attacks, by simulating those attacks. This is the subject of this chapter.

So what is the scope of that third strategy? AI security tests simulate adversarial behavior to uncover vulnerabilities, weaknesses, and risks in AI systems. The contrast with traditional AI testing is the key idea: traditional AI testing focuses on functionality and performance — does the model do its job well on the inputs it was built for? AI red teaming goes beyond that standard validation and applies intentional stress testing, attacks, and attempts to bypass safeguards. The tester deliberately behaves like an adversary: probing filters, crafting hostile inputs, and hunting for the gap between what the system is supposed to refuse and what it can actually be tricked into doing. Red teaming as a discipline can extend beyond security — teams also red-team for fairness or safety — but for this exam the focus is AI red teaming for AI security.

Back to NordicPay: the pentest exercised the first strategy only. Nobody validated whether the scoring model's behavior had drifted or been tampered with (strategy two), and nobody simulated adversarial inputs against the model or the chatbot's safeguards (strategy three). The honest answer to the CISO is that one leg of a three-legged stool has been tested.

Threats to Test For: Predictive vs Generative AI Spec 4.1.2 · Bloom 2

Conventional security testing stays on the plan for both systems — but it cannot see the AI-specific attack surface. The model's learned behavior can be manipulated through its inputs, the model itself is a valuable asset that can be stolen, and the training pipeline can be corrupted. None of those show up in a port scan. The question this learning objective answers is: which threats do you test for beyond conventional security testing — and the answer depends on which kind of AI you are testing.

First, the two paradigms. Predictive AI is designed to make predictions or classifications based on input data — think fraud detection, image recognition, recommendation systems. Generative AI produces outputs such as text, images, or audio — large language models and image generators. The distinction matters because each paradigm exposes a different trio of key threats to test for.

For predictive AI, the three key threats beyond conventional testing are:

• Evasion — the attacker crafts inputs designed to mislead the model, causing it to perform its task incorrectly. For MedScan, an adversarially perturbed X-ray that flips a diagnosis. (For the knowledge levels of evasion attackers, see the zero-knowledge vs transfer disambiguation.)
• Model exfiltration — the model's parameters or functionality are stolen. This is worse than losing intellectual property: a replica model becomes an oracle the attacker can query freely to craft further attacks, compounding the threat.
• Model poisoning — manipulation of the training data, the data pipeline, the model, or the model training supply chain during the development phase, so that the model's behavior is altered to the attacker's benefit.

For generative AI, the three key threats beyond conventional testing are:

• Prompt injection — the attacker provides the model with manipulative instructions aimed at achieving malicious outcomes. This covers both direct prompt injection (the attacker is the user) and indirect prompt injection (instructions smuggled in through data the model processes) — see the disambiguation box.
• Sensitive data disclosure in output — the model is made to reveal sensitive data in its output, often as a specific goal of a prompt injection: training data, other users' inputs, or confidential augmentation data.
• Insecure output handling — the model's output carries a conventional injection payload (script, SQL, shell fragments) and downstream components process it without sanitization. The model becomes a delivery channel for a traditional attack.

Applying this to MedScan: the X-ray classifier gets tested for evasion (perturbed images), model exfiltration (can an attacker reconstruct the model through queries?), and model poisoning (can the training pipeline be corrupted?). The letter-drafting assistant gets tested for prompt injection (can instructions hidden in a patient record hijack it?), sensitive data disclosure in output (can it be made to reveal another patient's details?), and insecure output handling (does anything downstream execute or render its output blindly?). One test plan, two very different threat lists — exactly what the red team argued.

One caution the OWASP AI Exchange itself makes: these trios are the key threats per paradigm, not the complete threat landscape. A real engagement selects its full threat list during scoping, based on the system's assets and risk analysis — which is precisely where the general testing approach in section 4.2 begins.

The General AI Security Testing Approach Spec 4.2.1 · Bloom 2

AI security testing is not a bag of tricks; it is a systematic process. The general approach runs through eight key steps, and you should know both the steps and their order:

1. Define objectives and scope — identify what the test must achieve and align it with organizational, compliance, and risk management requirements. This is where you decide which systems, threats, and outcomes are in play.
2. Understand the AI system — gather details about the model, its use cases, and its deployment scenarios. You cannot attack realistically what you do not understand.
3. Identify potential threats — apply threat modeling: map the attack surface, explore it, and identify relevant threat actors. The predictive/generative trios from 4.1 are the starting inventory.
4. Develop attack scenarios — design concrete attack scenarios and edge cases that express those threats against this specific system.
5. Test execution — conduct the tests for the attack scenarios, manually or automated.
6. Risk assessment — document the identified vulnerabilities and risks, weighing severity of harm against likelihood.
7. Prioritization and risk mitigation — develop an action plan for remediation, implement mitigation measures, and calculate the residual risk.
8. Validation of fixes — retest the system post-remediation to confirm the fixes actually hold.

The single most important thing to understand about this approach is that it is iterative, in two senses. Within a cycle, findings feed backwards: what you learn during test execution sharpens your threat list and spawns new attack scenarios, and validation of fixes is literally a return to testing. Across cycles, the whole process reruns regularly — at minimum before each deployment — because the model changes, the attack state of the art evolves, and the organization's risk appetite shifts. A one-off red-teaming report is a snapshot, not a security property.

Walking the bank's chatbot through the cycle: the scoping workshop fixes objectives (can the bot be made to give unauthorized financial commitments or leak customer data?) and constraints (production-equivalent staging, no real customer accounts). The team studies the system — which model, which system prompt, which tools it can call. Threat identification lands on the generative trio; attack scenarios turn them concrete ("hide an instruction in a transferred document", "ask for another customer's balance in seventeen phrasings"). Execution runs the scenarios; assessment documents that two of them partially succeeded; prioritization schedules a filter upgrade and accepts one low-severity residual risk; validation retests after the fix ships.

What does test execution actually look like? The OWASP AI Exchange spells it out for the flagship generative threat, prompt injection, and the procedure generalizes well. You start by establishing a base set of crafted input attacks — jailbreak attempts, invisible text, malicious URLs, data-extraction prompts — from attack repositories or tooling, then tailor them to the system's context: try to extract the specific data identified as sensitive, try to trigger outputs that are unacceptable for this business, and if the system has downstream processing or agentic behavior, craft attacks that abuse it. You then orchestrate the test: each attack input is paired with a detection (for instance, a pattern that spots leaked digits in the output) so the run can be automated, and inputs are presented through the production route — the system API with all its filters, not the bare model. Where the system ingests untrusted data, the attack inputs must also travel that route, to simulate indirect prompt injection.

Then comes the step the exam cares about most. An attack input may fail simply because something — the model's training, the system prompt, an external filter — recognized that exact phrasing as malicious. That tells you the sample was blocked, not that the threat is handled. So you add variation algorithms to the test process: replace words with synonyms, apply encodings, change formatting, and rerun. If a paraphrased or encoded version of a blocked attack sails through, your safeguard was matching surface features, not intent. OWASP makes the elegant observation that this is, in essence, an evasion attack against your own detection mechanisms. Two further habits round out execution: run tests multiple times, because model output can be non-deterministic, and use the same model versions, prompts, tools, permissions, and configuration as production. Finally, evaluation weighs each technical success by severity and likelihood — and the whole test is rerun regularly as context evolves. Alongside all the hostility, keep positive testing in the plan: benign inputs must still work, and your defenses must not drown legitimate users in false positives.

What Privacy Means for AI Systems Spec 5.1.1 · Bloom 2

The exam wants you to hold a precise, two-part definition: privacy is personal data protection plus respect for further individual rights. The first half is familiar security territory — keeping personal data confidential and intact. The second half is what the MediScan CISO missed: individuals also hold rights to know how their data is used, to correct it, to erase it, and to object to certain uses. You can encrypt everything perfectly and still violate privacy, because encryption says nothing about whether you had the right to keep the data, use it for that purpose, or make that decision about that person.

This split carries directly into AI. AI privacy divides into two parts, and exam questions probe whether you can tell them apart. The first part is the security threats and their controls: confidentiality and integrity protection of personal data wherever it lives in the AI system — in training and test data, in model input, and in model output — plus integrity protection of the model's behavior when wrong behavior can hurt individuals (think of a manipulated model that flags innocent people for fraud investigations). The second part is threats and controls that are not about security at all, but about the further rights of the individual as covered by privacy regulations: use limitation, consent, fairness, transparency, data accuracy, and the rights to correct, object, and erase.

Beyond this definition, you need to explain why AI makes privacy harder than it is for conventional systems. The concerns to know:

• Data intensity. AI systems are data-hungry, so they create extra risk at collection and retention. Personal data flows in from many sources, each with its own sensitivity and its own legal constraints.
• Long retention of training data. Models get retrained, so training data tends to be kept for years. The longer personal data exists, the longer it can leak — a direct tension with storage limitation.
• Exposure in the engineering environment. Training data is accessible to data scientists and engineers during development. Conventional engineering teams rarely handle production personal data; AI teams do it routinely, so the development-time environment needs far more protection than usual.
• Model attacks that extract training data. Attackers can pull personal data back out of a trained model through model inversion, membership inference, and sensitive data disclosure through use — the threats you studied in 2.1. The model itself becomes a leak channel.
• Discriminating decisions. AI systems make decisions about people, and those decisions may discriminate on protected attributes such as gender or ethnicity.
• Privacy-invading actions. Model output can trigger real-world actions that invade someone's privacy — being pulled into a fraud investigation, for instance — raising ethical and legal concerns even when no data leaked.
• Unique mitigations. The nature of machine learning also enables AI-specific privacy strategies. The flagship example is federated learning: instead of pooling all personal data in one place, the model is trained in iterations across separate sites, so raw data never has to leave its source.

Walk back through MediScan with this list. Encryption and access logging address the security half only. The five-year retention raises the storage question, engineers querying raw patient records is exactly the engineering-environment exposure, and nobody has asked whether patients' rights to erasure and objection can still be honored once their records are baked into a trained model. The DPO's objection is the second half of the definition.

Two further list terms complete this picture. The structured way to think through these concerns before a system is built is a privacy impact assessment (PIA) — in GDPR terms a data protection impact assessment (DPIA), mandatory whenever processing is likely to create a high risk for individuals. Training an AI system on personal data is a textbook DPIA trigger, and MediScan should have run one before the first record entered the data science environment. And when safeguards fail — personal data leaks, is accessed without authorization, or is used beyond its purpose — the event is a privacy incident, subject to incident response and, under the GDPR, breach-notification duties.

Applying the Nine Privacy Principles Spec 5.1.2 · Bloom 3

This is a Bloom 3 objective: the exam gives you a scenario and asks which privacy principle is engaged or violated. That means you need each principle as a recognition pattern, not a recitation. There are nine privacy principles for AI systems. Learn them with their scenario cues:

• Accuracy — a wrong data point drives a harmful automated decision (a mistyped phone number links an innocent customer to fraud).
• Consent — data is used without valid permission: consent was never asked, was bundled with terms of service, cannot be withdrawn, or was collected where genuine consent is impossible (an employer "asking" employees).
• Data minimization & storage limitation — more data, finer granularity, or longer retention than the purpose requires (a million records kept for seven years when ten thousand for one year would do).
• Fairness & lawfulness — personal data is handled in ways individuals would not expect, without a legal basis, or with unjustified adverse effects such as discriminatory outputs.
• Privacy rights — individuals have no way to access, correct, erase, or object to the use of their data.
• Privacy by design — privacy is bolted on after launch instead of being engineered in from the first design decision. Its GDPR companion, privacy by default, requires that the most protective settings are the out-of-the-box settings — protection must not depend on the user opting in.
• Security & safeguards — personal data sits unprotected in training sets, the engineering environment, or model input/output.
• Transparency & explainability — people affected by an algorithmic decision cannot find out how it was made or what data it used.
• Use limitation & purpose specification — data collected for one declared purpose is reused for a different one.

Now apply the list to Northwind. The purchase data was collected for service delivery; training a marketing model is a different purpose. That is a textbook violation of use limitation & purpose specification. Notice what the team lead's argument gets wrong: "no new collection" is irrelevant, because the principle constrains use, not just collection. The fix is not to abandon analytics but to establish a proper basis first — document the new purpose, obtain consent where required, or work with anonymized copies for the incompatible purpose, which is also where data minimization pulls in the same direction.

The principle you should weight most heavily in your preparation is exactly this one. The single most common ethical failure pattern in AI privacy is data collected, retained, or reused beyond its justified purpose — security data reused for targeting, know-your-customer data flowing into business analytics, service data feeding marketing models. When a scenario shows data moving from the purpose it was gathered for to any other purpose, your answer is use limitation & purpose specification, even if other principles are bruised along the way.

One warning for scenario questions: the nine principles are design principles you apply to a system. In 5.2.2 you will meet a different list — the GDPR compliance challenges — that overlaps in vocabulary (purpose limitation, transparency, accuracy) but plays a different role in questions. The disambiguation box in 5.2.2 shows how to keep them apart.

Four ISO Standards That Support AI Compliance Spec 5.2.1 · Bloom 2

Regulations tell you what outcomes to achieve; standards tell you how to organize for them. The EU AI Act, for example, is outcome-based — it demands that risks to people are managed and demonstrated — whereas standards are control- and process-focused. That is exactly why standards support compliance: they give you repeatable processes, defined roles, and auditable evidence that regulators and conformity assessments can check. This is the working logic behind the Check Compliance control from 3.1: identify the AI-relevant laws that apply to you, then use standards to build the machinery that satisfies them. The exam tests four standards, and it tests them as a matching exercise — you must know which standard does what.

The anchor fact: ISO/IEC 42001 defines an AI management system (AIMS) — governance, policies, roles, controls, and continual improvement for any organization developing, providing, or using AI. It is to AI what ISO/IEC 27001 is to information security: a management system for governing the whole, not a manual for engineering the parts. It deliberately does not descend into lifecycle detail — it will not tell you how to train models, track data lineage, or version them. That engineering layer belongs to ISO/IEC 5338, which defines AI system lifecycle processes — the AI engineering and MLOps processes for data and model development, deployment, and operation — by extending the classic software lifecycle standards with AI-specific processes and work products.

The other two standards are about risk. ISO/IEC 23894 gives guidance for managing AI-related risks across the whole AI lifecycle, aligned with ISO 31000, the generic enterprise risk-management standard: identify, assess, treat, and monitor AI risks, whatever the organization or use case. ISO/IEC 27005 is not AI-specific at all: it is information security risk management, the risk process that supports ISO/IEC 27001 — establishing context, risk assessment, treatment, acceptance, monitoring, and communication. It matters for AI because AI systems are still IT systems: you extend the same security risk process to cover the AI-specific attack surfaces (development-time attacks including the supply chain, input attacks, and runtime attacks).

Back to Helvetia: the CISO's "27001 for AI" is ISO/IEC 42001. To show the regulator a risk process, the AI risk team follows ISO/IEC 23894, while the existing security team extends its ISO/IEC 27005 practice to AI assets. And the ML engineering department adopts ISO/IEC 5338 so that the day-to-day pipeline — data lineage, model versioning, continuous validation — produces compliance evidence as a by-product of normal work.

Compliance Challenges: the EU AI Act and the GDPR Spec 5.2.2 · Bloom 2

The EU AI Act analyzes risk from one perspective: harmfulness to people — their safety, health, and fundamental rights. It sorts AI systems into four risk tiers, each with its own regulatory consequence. At the top, unacceptable risk systems are prohibited outright: social scoring, manipulation of people, real-time remote biometric identification in public spaces. Below that, high risk systems are permitted, but only with compliance obligations and an ex-ante conformity assessment — you must demonstrate conformity before the system goes to market. This tier covers products already under safety legislation (such as medical devices) plus defined sensitive areas including recruitment and employment decisions, critical infrastructure, and law enforcement. The third tier, limited risk (specific transparency risk), is permitted with transparency obligations: a chatbot must disclose that it is a bot, so the user can make an informed decision about continuing. Everything else is minimal or no risk — permitted without restrictions.

Two footnotes to the pyramid matter for scenario questions. First, the tiers are not mutually exclusive: a single system can trigger obligations from more than one tier — a high-risk system that converses with users also owes the transparency disclosures of the limited tier. Second, know the Act's blind spot: because it protects people, it does not cover business harms such as the loss of company secrets. Compliance with the AI Act is therefore not the same as being secure — a point the exam likes, because it explains why an organization still needs its own risk analysis on top of legal compliance. For generative AI, the Act adds transparency about training sources: providers must disclose what copyrighted material was used, on pain of very large fines — a thread we pick up in 5.2.3.

Now apply the tiers to TalentBridge. Résumé screening decides who gets access to employment, so it lands in the high risk tier: permitted, but only with the compliance obligations and the conformity assessment completed before deployment. It is not prohibited — the distractor to resist. Prohibition is reserved for the unacceptable tier, and recruiting tools are the canonical example of high risk, not of social scoring.

The GDPR question is different in kind. The GDPR does not explicitly restrict AI applications; it constrains how personal data may be processed, and AI strains those constraints in predictable places. Learn these as the ten friction points between AI and the GDPR:

1. Lawful basis — every processing purpose needs a legal basis; it is genuinely hard to name one that cleanly covers training on personal data.
2. Purpose limitation — data collected for a service keeps getting pulled toward model training, a new purpose.
3. Data minimization vs model performance — the law pushes for less data; model accuracy pushes for more. Someone must make the trade-off defensible.
4. Transparency and explainability — providing "meaningful information about the logic involved" is hard when the logic is a black-box model.
5. Automated decision-making and profiling — individuals have rights around solely automated decisions: human intervention, the ability to contest, and meaningful information.
6. Operationalizing data-subject rights — access, correction, erasure, and objection are easy in a database and painful in a trained model; honoring erasure may mean retraining without the deleted person's data.
7. Accuracy and fairness — incorrect or biased data leads to unjustified adverse decisions about individuals.
8. Security and leakage — model inversion, membership inference, and disclosure through use turn the model itself into a personal-data breach channel.
9. International transfers — training data, cloud GPUs, and model providers cross borders more casually than transfer rules allow.
10. Accountability and roles — who is controller and who is processor across a chain of model suppliers and deployers is rarely obvious (compare the provider/deployer split in the disambiguation box in Topic 3).

Mitigating Copyright-Infringement Risk Spec 5.2.3 · Bloom 2

AI and copyright is an area of law where many questions remain genuinely unresolved. Many jurisdictions have announced no formal position on intellectual property protection for AI-generated output, so the ownership of what your model produces may simply be undefined where you operate. Meanwhile the input side carries its own exposure: training needs vast amounts of data, and if copyrighted works entered the training set without the owner's permission, the organization faces financial and reputational risk — and a tainted dataset. High-profile lawsuits by artists and image libraries against generative AI companies, and by developers over AI-generated source code, show how real the escalation path is. It is worth understanding the technical nuance too: generative models do not store and look up their training examples — they extract patterns and generate new content, which may occasionally resemble existing work. That nuance fuels the legal debate, but it does not make the risk go away.

The exam expects the ten risk-mitigation strategies, and expects you to describe what each contributes:

1. Mitigate disclosure of sensitive training data in output — apply the controls against the threat of sensitive data disclosure through use, so the model does not emit protected material it absorbed during training.
2. Comprehensive IP audits — inventory all intellectual property touching the AI system: not just datasets, but source code, systems, applications, and interfaces.
3. Clear legal framework and policies — written, enforced policies for AI use, aligned with current IP and copyright law.
4. Ethical data sourcing — train only on data that is created in-house, obtained with all necessary permissions, or sourced from public domains with a license sufficient for your intended use.
5. Define ownership of AI-generated content — decide up front who owns model output and under what conditions it may be used, shared, and disseminated.
6. Confidentiality and trade-secret protocols — strict handling rules that preserve trade-secret status of models, data, and related materials.
7. Employee training — make staff aware of the organization's AI IP policies and of what infringement would mean.
8. Compliance monitoring systems — keep an updated monitoring capability that checks for potential infringement by the AI system.
9. Response planning for IP infringement — a prepared plan so infringement claims are handled quickly and effectively.
10. Licenses and/or warranties from AI suppliers — contractual coverage of your intended use (and future uses), plus binding obligations on the supplier to cover potential infringement claims.

Within strategy four, the exam cares about the ranking: the most ethical and safest data sourcing is creating training data in-house — you know its provenance completely because you made it. Properly licensed or permissioned data comes second; public-domain sources are acceptable only when the license genuinely covers your intended use. Walking BrightPage through the list: the vendor relationship calls for strategy ten (licenses and warranties covering intended use), the "who owns the images" question is strategy five, and counsel's plan should start with an IP audit (strategy two) to establish what is actually in play.