How should security teams control AI agents that can use tools?

What is the main question?

What risks emerge when AI systems can call tools, use permissions, trigger workflows, or act through identities?

What else should teams answer?

• What is excessive agency in AI systems?
• How do tool-using AI agents create security risk?
• What controls reduce AI agent action risk?
• What evidence should buyers ask vendors for?

What excessive agency means

Excessive agency is the risk that an AI system has more authority, autonomy, tool access, or workflow reach than the use case requires. Security teams should assess what the agent can do, which identity it acts through, which tools it can call, what data it can retrieve, what memory it keeps, what approvals are required, what logs prove its actions, and how actions can be stopped or rolled back. Tool-using agents can create business value, but they also expand the control surface from model output to enterprise action. The control objective is bounded, monitored, reversible agency, not blind trust in generated decisions.

OWASP's LLM and generative AI risk categories include excessive agency as a major application risk. MITRE ATLAS provides an adversarial behavior lens for thinking about how attackers may manipulate AI-enabled systems. These are framework lenses for evaluation; they are not certifications or formal compliance claims.

Why tool use changes the threat model

A chatbot that answers a question can be wrong. An agent that calls tools can be wrong and also send a message, update a ticket, change a record, query a database, start a workflow, execute code, or act through a user or service identity. Prompt injection, unsafe instructions, compromised context, or business logic abuse can lead to unauthorized actions if the system trusts model output too much.

• Tool calls may be triggered by direct prompts, retrieved content, memory, or chained agent steps.
• Delegated permissions can allow the agent to act with user, service account, or application authority.
• Workflow triggers may create downstream effects that are hard to detect from the original prompt.
• Action execution can require rollback, notification, investigation, and audit evidence.
• Memory can preserve instructions, context, or sensitive data beyond a single session.

Where agent risk appears in enterprise workflows

Agent risk appears anywhere AI systems can interact with enterprise tools: customer support, software development, security operations, sales operations, finance, HR, procurement, identity administration, cloud operations, data analysis, and internal workflow automation. High-risk examples include agents that create refunds, update customer accounts, open or close security cases, run commands, change access, approve expenses, send external messages, or commit code.

Security teams should distinguish read-only assistance from action-taking workflows. Read-only systems still need data controls, but action-taking systems need permission boundaries, approval gates, action policies, runtime monitoring, anomaly detection, and rollback paths.

Control outcomes that matter

Important control outcomes include least privilege, tool allowlists, action approval, policy enforcement, runtime monitoring, audit trails, anomaly detection, rollback paths, and separation of duties. The agent should not receive broad access because it is convenient. Its authority should be scoped to the workflow, and high-impact actions should require deterministic checks outside the model.

• Limit tools by user, role, workflow, environment, and risk tier.
• Use approval gates for irreversible, customer-impacting, financial, access, or security actions.
• Bind actions to explicit user intent, policy checks, and source evidence.
• Log prompts, tool calls, policy decisions, approvals, action results, and rollback attempts.
• Monitor abnormal tool use, repeated failures, unusual destinations, and privilege boundary changes.

How to map agent risk to control language

Map agent risk to existing controls for identity and access management, privileged access, change management, application security, workflow approval, logging, incident response, and third-party risk. Then add AI-specific detail: tool calling, model instructions, retrieved context, memory, prompt injection, generated action plans, and agent runtime logs. This prevents agent security from becoming detached from controls the enterprise already operates.

A useful control map states the problem segment, control surface, control outcome, evidence, owner, and residual risk. For example: an agent that updates customer cases may need tool allowlists, case-level authorization, approval gates for account changes, source attribution, and logs exportable to the SOC and GRC team.

What evidence should buyers request?

• Architecture diagrams showing agent runtime, tools, identities, memory, and external systems.
• Permission model showing least privilege, delegated authority, and service account use.
• Tool allowlist and policy configuration examples.
• Approval records for high-impact actions and exception handling.
• Prompt injection and tool-use abuse test results.
• Logs for prompts, tool calls, policy decisions, approvals, failures, and rollback.
• Known limits, bypass paths, false positives, and residual risk statements.

Practical assessment checklist

• Inventory every tool the agent can call and every identity it can use.
• Classify actions by impact, reversibility, data sensitivity, and customer effect.
• Remove tools and permissions not required for the use case.
• Require approval for high-impact or irreversible actions.
• Test prompt injection paths that attempt unauthorized tool calls.
• Review logs with SOC, AppSec, IAM, and business owners.
• Define rollback and incident response for incorrect or abusive actions.
• Reassess after tool, model, permission, memory, or workflow changes.

FAQ

Sources and frameworks referenced

• OWASP Top 10 for LLM and Generative AI Applications 2025
• MITRE ATLAS