
Questions that arise in an audit
If AI closes an alert or downgrades its severity, an audit or incident investigation will ask what evidence informed the decision, which model and rules were in use, and whether a rerun reaches the same conclusion. Good records answer the first two questions. The third requires a reproducibility check.
Why temperature 0 is insufficient
A temperature of 0 can still produce different outputs for the same input. In September 2025, Thinking Machines Lab explained how inference servers vary batch size with load and how widely used GPU kernels can change the order of operations with batch size. Batch-invariant kernels can deliver bitwise-identical outputs, with a trade-off in throughput.
We preserve what went into the decision and what it produced as they were at the time. We use reruns to compare with the original decision. A different result is worth reviewing because it suggests the decision may have been borderline.
What to record for each decision
Threat Correlator stores the evidence, rationale, decision, and recommendation together in one record. The example below shows what such a record should contain.
{
"verdict_id": "...",
"decided_at": "2026-10-21T02:13:44+09:00",
"verdict": "true_positive",
"severity": "high",
"evidence": [
{"source": "edr", "ref": "event_id=...", "sha256": "..."},
{"source": "proxy", "ref": "proxy/2026/10/21/02.log#L1840", "sha256": "..."}
],
"attack_mapping": ["T1059.001", "T1490"],
"model": {"name": "...", "version": "...", "temperature": 0},
"prompt_template": "triage-v14",
"policy_version": "auto-response-2026.10",
"input_digest": "sha256:...",
"output_raw": "...",
"reviewed_by": "analyst-042"
}
The fields themselves are straightforward. One easy detail to miss is the prompt_templateand policy_version. If the prompt wording changes, the decision criteria change even when the model stays the same. Version prompts as you would code.
Teams often retain an evidence location and hash while keeping the original logs elsewhere. If logs expire before the decision records, an audit a year later may find the decisions without their supporting evidence. Align the retention periods.
Why record ATT&CK technique IDs?
Alongside a description such as “Suspicious PowerShell execution,” T1059.001helps resolve disagreements in an audit. MITRE ATT&CK provides a public taxonomy that a third party can use to assess the same evidence. A mapping can be wrong, and a single alert can involve several techniques, so include mappings in analyst review.
Regulatory requirements
Article 34 of Korea’s AI Basic Act, which took effect on January 22, 2026, requires providers of high-impact AI to publish key information about measures for risk management and explainability. Security operations AI may rarely qualify on its own. If decisions feed into activities that affect individual rights, such as hiring or loan reviews, ask legal counsel to assess applicability. Retaining the evidence behind decisions supports later review either way.
References
- Defeating Nondeterminism in LLM Inference, Thinking Machines Lab, 2025.9
- Command and Scripting Interpreter: PowerShell, T1059.001, MITRE ATT&CK
- Framework Act on the Development of Artificial Intelligence and Establishment of Trust, Korean Law Information Center


