
Three things to define before starting
This guide is for security teams piloting AI agents in security operations. It is product-agnostic. The four stages of delegation, L1 through L4, follow the framework in September 2026 Threat Reportdescribes this framework. This guide turns it into a pilot schedule.
Before the pilot, define which data the agent can read, which tools it can call and with what permissions, and how you will evaluate results. Set the evaluation criteria early so you can decide whether to continue when the pilot ends. Teams most often miss that third step.
Days 1–30 · Read-only operation
Give the agent read-only access in the first month. SIEM searches, EDR event lookups, and threat intelligence queries are enough. The agent records its recommendation and evidence for each alert. Analysts work as usual and compare their conclusions with the agent’s. Where possible, record the analyst’s decision before showing the agent’s recommendation to reduce bias.
| Metric | Definition | How to interpret the metric |
|---|---|---|
| Agreement rate | Percentage of agent classifications that match analysts’ final decisions | Calculate it separately for each detection rule. Easy alerts can raise the overall average. |
| Missed true positives | Number of alerts the agent marked benign and an analyst confirmed as attacks | This is the most important metric. Advance a rule only if it has zero missed true positives. |
| Evidence accuracy | Sample review of whether the agent’s evidence matches the underlying logs | This metric identifies plausible explanations unsupported by the logs. |
| Time to triage | Time from alert creation to the agent’s triage recommendation | Compare it with analyst handling time for the same alert. |
Use the first month’s results as your baseline. Alert mixes differ by organization, so a target from another team can lead to the wrong decision.
Days 31–60 · Delegate triage by rule
Start with detection rules that had zero missed true positives and high agreement in the first month. Have analysts sample alerts the agent closes as benign. Start with a high sampling rate and adjust it as you review results.
Treat prompt injection as a practical risk at this stage. Alert data may contain attacker-controlled text, including email bodies, URLs, User-Agent values, and file strings. The OWASP Top 10 for LLM Applications 2026 classifies this as indirect prompt injection under LLM01 Prompt Injection. During the pilot, verify that alert text alone cannot cause the agent to mark an alert benign. Fix prompt boundaries before delegating any rule that fails this check.
Days 61–90 · Automated response within defined limits
In the final month, automate selected actions. Before automating an action, define its impact and how to reverse it. Manage approval policies like code. The example below illustrates the format.
policy_version: auto-response-2026.10
rules:
- action: isolate_host
when: { severity: [critical, high], asset_tier: [standard] }
mode: auto
rollback: release_isolation
- action: isolate_host
when: { asset_tier: [crown_jewel] }
mode: require_approval
approvers: [soc_lead]
- action: disable_account
when: { account_type: [privileged] }
mode: require_approval
- action: block_ip
when: { confidence: high }
mode: auto
ttl: 24h
Version policy files in a repository and review changes. Record the policy version for each automated action so reviewers can trace why an action such as server isolation occurred. Set an expiration on measures such as IP blocks, which can have side effects if they stay in place.
Designing agent permissions
OWASP LLM03:2026 Excessive Agency distinguishes functionality, permissions, and autonomy. The approval policy addresses autonomy. Reduce functionality and permissions separately.
- Separate the investigation agent from the response agent. Give the investigation agent a read-only API key.
- Limit the response agent’s key to the actions authorized by policy. Use a scoped credential in place of an EDR administrator key.
- Connect external email or messaging tools only if they are necessary for security operations.
- Log every tool call with the requester, arguments, and result.
The December 2025 OWASP Top 10 for Agentic Applications also places Tool Misuse and Exploitation (ASI02) and Identity and Privilege Abuse (ASI03) near the top. These risks require controls beyond an approval screen.
Decisions after 90 days
At the end of the pilot, choose one outcome for each detection rule. Keep automated response, delegate triage only, or return the rule to analysts. Return any rule with missed true positives until you understand the cause.
NIST SP 800-61 Rev. 3 covers incident response across all six CSF 2.0 functions. The Improvement category (ID.IM) uses lessons from incidents and exercises to improve procedures. Follow the same cycle for agent policy by reviewing missed true positives and evidence accuracy each quarter and adjusting the stage of each rule.
Define which decisions stay with people regardless of pilot results. These include declaring and rating incidents, making legal reporting decisions, approving actions that disrupt operations, and changing automated response policies.
References
- SP 800-61 Rev. 3, Incident Response Recommendations and Considerations for Cybersecurity Risk Management, NIST, 2025.4
- Cybersecurity Framework 2.0, NIST
- LLM01:2026 Prompt Injection, OWASP Top 10 for LLM Applications 2026
- LLM03:2026 Excessive Agency, OWASP Top 10 for LLM Applications 2026
- OWASP Top 10 for Agentic Applications, OWASP GenAI Security Project, 2025.12

