Secure AI Playbook

1. A business request to introduce AI into a workflow, product, service, or decision.
2. OR or, as an alternative, An unregistered AI tool identified in use.
3. OR or, as an alternative, A material feature or model change to an AI tool already in use.
4. OR or, as an alternative, An incident caused by AI use.

1. Record the business workflow, proposed AI use, AI tool, or AI incident that triggered the stage.
2. Identify the accountable business owner and the product or technical owner.
3. Record the affected users, systems, data, connected tools, decisions, actions, and expected degree of autonomy.
4. State the intended business benefit and the worst credible adverse business impact.
5. Decide whether the use case or tool should proceed to a bounded experiment and identify any immediate constraints.

1. A concise AI use-case or tool record.
2. A named accountable business owner.
3. A named product or technical owner.
4. An initial view of affected architecture, data flows, identities, permissions, and connected tools.
5. An initial statement of autonomy and worst credible adverse business impact.
6. A decision to proceed, revise, pause, or stop.

1. Business owner

   Defines the business purpose, expected value, acceptable use, and adverse business impact.

2. Product or technical owner

   Records how the AI capability is implemented, connected, accessed, and changed.

3. Security, legal, compliance and privacy

   Identify immediate information gaps, prohibited uses, and conditions that must constrain the next stage.

1. The request is described only as a product name rather than a business activity or workflow.
2. The AI tool or capability is discovered only after broad use has begun.
3. No accountable business owner is willing to own the purpose and resulting risk.
4. A material feature or model change is treated as routine software maintenance even though it may alter behavior or exposure.

1. Model and feature change management:

   Determine which AI changes should be governed directly through this Playbook and which should be handled by specialist assurance tools. Current hypothesis: this Playbook should capture the resulting risk and required control capability, while automated regression testing of affected workflows against evaluations should sit within specialist assurance tooling as required based on the defined risk appetite.

2. Where should AI red-teaming live?

1. The Stage 1 use-case or tool record.
2. Named owners.
3. Initial limits or constraints.

1. Define explicit boundaries for users, data, permissions, connected tools, actions, duration, and stop conditions.
2. Run the use case within those boundaries and capture inputs, outputs, data flows, access, and tool activity.
3. Observe expected behavior, unexpected behavior, and failure modes.
4. Record every boundary change and decide whether to continue, revise, or stop.

1. A bounded experiment plan.
2. Observed behavior and evidence.
3. Documented failure modes.
4. A decision to continue, revise, or stop.

1. Business and product owners

   Keep the experiment tied to its stated purpose and approve its operating limits.

2. Technical owner

   Implements the boundaries and preserves the evidence produced.

3. Security and privacy

   Set observation requirements, stop conditions, and data-handling constraints and review the evidence.

1. The experiment becomes an informal production deployment.
2. Logging is added after the most useful evidence has already been lost.
3. Boundaries are implicit or change without being recorded.

No known gaps or open questions have been documented yet.

1. Observed architecture, data flows, identities, permissions, tool connections, and behavior from the bounded experiment.
2. Documented failure modes.
3. The Stage 1 business impact and autonomy statements.

1. Identify affected assets and trust boundaries.
2. Trace how users, models, data sources, connected tools, and systems can influence one another.
3. Identify plausible misuse, failure, and attack scenarios.
4. Prioritize scenarios by credible preconditions and consequences for risk assessment.

1. Prioritized misuse, failure, and attack scenarios.
2. Affected assets and trust boundaries.
3. Plausible consequences.
4. Scenarios ready for risk assessment.

1. Security

   Facilitates the model and makes scenarios explicit and testable.

2. Technical and product owners

   Validate the observed architecture, behavior, dependencies, and assumptions.

3. Business owner

   Validates business consequences and unacceptable outcomes.

1. A generic framework checklist replaces implementation-specific analysis.
2. Connected tools and non-human identities are omitted.
3. The model documents attacks but not misuse or operational failure.

No known gaps or open questions have been documented yet.

1. Prioritized threat scenarios.
2. Business impact.
3. Existing control coverage.
4. Applicable legal, compliance, privacy, and contractual obligations.

1. Assess the likelihood and business impact of each prioritized scenario.
2. Review existing control coverage and supporting evidence.
3. Identify control gaps, residual exposure, and uncertainty.
4. Assign a risk owner and decide whether to accept, reduce, redesign, or stop.

1. Risk statements.
2. Prioritized control gaps.
3. Named risk owners.
4. Decisions to accept, reduce, redesign, or stop.

1. Business risk owner

   Owns the business impact and decides whether to accept, reduce, redesign, or stop.

2. Security

   Assesses likelihood and reviews security-control coverage and evidence.

3. Legal, compliance and DPO

   Assess applicable obligations, privacy impact, and required safeguards.

1. Risk ratings are assigned without evidence from the implementation.
2. Existing control coverage is assumed rather than tested.
3. Legal, compliance, or privacy review begins after the design is fixed.

No known gaps or open questions have been documented yet.

1. Prioritized risk statements and control gaps.
2. Risk treatment decisions and named risk owners.
3. Relevant architecture, trust boundaries, and existing control coverage.

1. State the behavior required to prevent, detect, contain, or recover from each risk.
2. Identify the deployment or enforcement point and whether an existing control can be extended.
3. Assign the control owner, operator, exception path, and escalation path.
4. Define the required evidence, testing criteria, and monitoring criteria.

1. Required behavior.
2. Deployment or enforcement point.
3. Control owner and operator.
4. Required evidence.
5. Testing criteria.
6. Monitoring criteria.
7. Exception and escalation path.

1. Security architecture and control owners

   Write the testable requirement and choose its deployment or enforcement point.

2. Technical owners

   Validate implementation feasibility, operation, evidence, testing, and monitoring.

3. Risk owner and assurance functions

   Review and approve the requirement, exception path, and escalation path.

1. Requirements name a tool or category without stating required behavior.
2. No deployment point, control owner, or operator is identified.
3. Evidence, testing, and monitoring criteria are deferred until implementation.

No known gaps or open questions have been documented yet.

1. Approved control requirements.
2. Named owners.
3. Agreed implementation scope.

1. Extend existing controls where they meet the approved requirements.
2. Implement new controls for requirements that existing controls cannot meet.
3. Test control behavior and preserve evidence against the approved criteria.
4. Activate the new or extended controls and exception handling.
5. Activate continuous control monitoring.

1. Implemented or extended controls.
2. Test results and evidence.
3. A decision that the control is ready for continuous operation.

1. Control owner

   Accepts overall accountability for the control and confirms that the control meets the approved requirement.

2. Control operator and technical owner

   Implement, integrate, test, operate, and monitor the control.

3. Risk owner and assurance functions

   Review test evidence, exceptions, and readiness for continuous operation.

1. Implementation ownership is split without end-to-end accountability.
2. Control testing does not use the approved criteria or preserve evidence.
3. Architecture, models, data, or tools change without triggering reassessment.

No known gaps or open questions have been documented yet.