Audit trail¶

An auditor's questions are predictable. They will ask:

What AI tools do you use?
How did you decide to use them?
How do you control what data goes where?
Show me that AI-assisted code passes your security controls.
Show me how you respond to AI-specific incidents.
Show me that your people know how to use these tools safely.
Where is the evidence?

This page describes what evidence the program produces and where it lives, so those questions have quick answers. The same evidence also drives internal quality. If you cannot produce it for an auditor, you cannot produce it for yourself after an incident either.

The evidence map¶

Evidence	Source	Owner	Retention
Charter & amendments	Leadership charter	Sponsor	Permanent
Decision Records	All worksheets	CoE	Permanent
AUP versions	AUP	CoE	Permanent
Approved-tools list (current + history)	Vendor standards	Procurement / CoE	Permanent
Vendor scorecards	Vendor selection worksheet	Procurement / CoE	7 years (contract term + retention)
Risk classifications (current + history)	Risk classification	Architecture / CoE	Permanent; revisit every 6 months
Training records	HR system	Enablement	Per HR retention, typically 3–7 years
AUP acknowledgements	HR system	HR	Per HR retention
Prompts / completions (selective)	Tool audit logs	DevEx / AppSec	30–90 days typical; longer per regulation
SCA / SAST results	CI system	DevEx / AppSec	1–2 years
PR metadata (AI-adjacent tag)	Source control	DevEx	With PR (permanent)
Incident logs and post-mortems	IR system	IR Lead	5–7 years typical
Metrics snapshots	Dashboards	DevEx	Raw 90 days; aggregates permanent
Regulatory crosswalk versions	Regulatory mapping	Compliance / CoE	Permanent
External communications about AI use	Marketing / Comms archive	Comms	7 years

The AI-adjacent PR tag¶

The single most important mechanical practice for the audit trail is tagging AI-assisted PRs.

Every PR template includes a field: "Was AI used? (Yes / No / Partial.)"
When Yes or Partial, the tool session identifier or high-level description is recorded.
The tag flows through the commit log and CI artifacts.
Metrics aggregate it (share of AI-assisted PRs, merge rates, revert rates).

This is what turns an abstract question ("was AI involved in this change?") into a concrete query. It is also the lowest-cost control on the list.

Prompt retention: deliberate, not automatic¶

Retaining prompts is a trade-off:

Longer retention helps incident reproduction, audit evidence, and trend analysis.
Shorter retention reduces privacy exposure, legal risk, and storage cost.

A good default:

90 days at full fidelity (prompt + output + tool + user + timestamp).
Permanent for prompts involved in incidents (tagged at incident triage).
Longer per regulation, e.g., audit-scoped prompts for SOX- or PCI-relevant paths retained per sector rule.

Document the default, the exceptions, and the owner. A retention policy that exists only in the tool's default settings is not a policy.

Evidence for specific audit questions¶

"Show me that AI-assisted code is reviewed the same way as other code."¶

PR metadata with AI tag.
Reviewer obligations in the AUP (§4).
V.E.R.I.F.Y. level assignments per risk tier.
Sampled PRs with review threads showing substantive human review.

"Show me that secrets do not leak to AI services."¶

AUP prohibited-data list.
Secret-scanning results on PRs (pre-merge).
Egress controls on tool endpoints (where applicable).
Tool-level secret-filtering features enabled.
Incident log with secret-exposure category, ideally empty or small and trending down.

"Show me that your training is effective."¶

Training records: completion + pass rate.
Survey data: pre- and post-training comparisons.
Incident rate trend among trained vs. untrained (lagging).
Assessment exercises' outcomes (not marketing collateral about the training).

"Show me how you respond to AI-specific incidents."¶

IR runbook with AI-specific scenarios.
Tabletop records (dates, participants, findings).
Post-mortem library with AI-adjacent root causes.
DRs derived from post-mortems, showing policy updates.

"Show me that your tools comply with our regulatory posture."¶

Regulatory crosswalk.
Vendor DPAs and security attestations.
Data-flow maps showing cross-border routing.
Sub-processor register with approval dates.

Storage and indexing¶

Evidence is only evidence if it can be produced. Practical requirements:

Durable location: version control, enterprise wiki with history, or equivalent.
Indexed. An auditor's request should be answerable in minutes to hours, not days.
Access-controlled. Least privilege; the audit role is real.
Tamper-evident. Version control is the simplest answer; advanced programs layer on append-only logs or WORM storage.
Exportable. In response to a regulator request or legal discovery, evidence can be assembled and exported without heroic engineering.

Cross-border evidence¶

Where evidence about a regulated transfer or processing must be kept in a specific jurisdiction:

Residency applies to the evidence, not just the production data.
SCCs / IDTA must be in place for evidence flowing cross-border.
Legal reviews the vendor's tool-log region and retention independently.

Public vs. internal evidence¶

This framework (the public site) is one layer of evidence:

Public: versioned framework pages, changelog, decision patterns, this roadmap.
Internal (company-specific): filled-in charter, AUP, classification register, Decision Records, incident logs.

The public site does not hold company evidence; the company hosts its own evidence alongside its other engineering records.

Evidence continuity on sunset¶

When tools or practices are retired (Phase 4):

Export all audit-relevant logs before decommission.
Index exports with a label identifying the retired tool and dates.
Update the IR playbook to describe retrieval from the archive.
Note in the regulatory crosswalk that controls previously attested via this tool are now attested via its successor (or compensating controls).

Anti-patterns¶

Don't

Don't leave evidence in a single engineer's laptop or personal tool. If the engineer leaves, the evidence leaves.
Don't rely on "the vendor keeps those logs." Vendor retention and contract scope may not cover your needs.
Don't keep everything forever "just in case." Retention has privacy and legal costs; set it deliberately.
Don't forget negative evidence. A clean incident log is useful; an unchanged risk register after a year is a red flag, not green.
Don't produce evidence only for audit time. If you cannot answer the questions month to month, you will not answer them in a real audit.

External references¶

NIST AI RMF, Govern 4.3 and Manage 2.1¹: evidence and documentation expectations.
ISO/IEC 42001 clause 9 (performance evaluation)² and 7.5 (documented information).
ICO auditing AI toolkit³.

Next¶

Regulatory mapping: shows which evidence maps to which standard.
Incident response: feeds the incident log.
Measurement: dashboards aggregates the metric slice of evidence.