Logging & Retention for LLMs: What to Store, How Long, and Why

Log enough to prove safety — not so much that you create risk. This guide shows exactly what to store for LLM workloads, how long to keep it, and how to purge safely. Two tracks: Manager Mode for policy decisions, Builder Mode for schemas, redaction, rotation, and audits.

Manager Mode — policy defaults, access tiers, retention

This section sets practical defaults you can adopt today. Adjust for your industry rules and contracts.

Policy defaults that work for most SMEs

• Data minimization: log metadata (who/when/model/task) and short hashes; avoid full raw content unless strictly needed for QA.
• PII control: redact or tokenize personal/sensitive data at ingest; keep mapping keys in a separate vault.
• Access tiers: Ops (read aggregates), Auditors (read structured logs), Developers (masked samples), Admins (approve unmasking events).
• Retention windows: 30 days (highly sensitive), 90 days (default), 180–365 days (only where contracts/regulators require).
• Purpose binding: logs used for safety, reliability, and fraud only — not for model training unless the user explicitly consented.

The table below gives a clear mapping from data type to why you keep it, who can access it, and a sane retention default.

Approvals and unmasking

• Dual control: any unmasking requires two approvers and a ticket reference.
• Just-in-time access: temporary credentials for investigation windows only.
• Audit trail: capture who unmasked, why, when, and which records.

Builder Mode — schema, redaction, rotation, audits

This appendix gives copy-paste artifacts you can adapt to any stack (APIs, agents, chatbots, batch jobs).

Minimal logging schema (JSON)

Prefer structured fields over raw text. Keep content out of logs; store masked snippets only when necessary.

{
  "event_id": "uuid",
  "ts": "ISO-8601",
  "env": "prod|staging",
  "actor": {"user_id":"u123","role":"agent|end_user|service"},
  "route": {"app":"support_bot","endpoint":"/chat","version":"2025-10-24"},
  "model": {"name":"gpt-X","provider":"AcmeAI","temperature":0.2},
  "content": {"prompt_hash":"hex","output_hash":"hex","masked_sample":"string|null"},
  "policy": {"pii_hit":true,"blocked_term":"brand_x","jailbreak_flag":false},
  "metrics": {"latency_ms":312,"tokens_in":354,"tokens_out":215},
  "tenant": {"id":"t001","access":"internal|restricted"},
  "trace": {"request_id":"req_abc","parent_id":"span_123"},
  "retention_code":"R90",   // maps to policy (e.g., 90 days)
  "tags":["refund","billing"]
}

Redaction patterns (in-place masking)

• PII: replace with tokens like <EMAIL_1>, <PHONE_1>, <IBAN_1>.
• Secrets: detect keys (AKIA…, sk_live…), replace with <SECRET_X> and drop full values.
• IDs: hash customer/order IDs to pseudonymous keys; keep a reversible map in a separate vault if required.
• Free text: keep only 1–2 masked lines for QA; store full text in a dedicated, high-control store if absolutely necessary.

Rotation & purge jobs

Automate lifecycle so no one “forgets” to delete logs after audits end.

• Daily rotation: write logs to date-partitioned tables/buckets (e.g., logs/dt=2025-10-24/).
• TTL policies: bucket lifecycle rules or DB TTLs based on retention_code.
• Purge proof: generate deletion manifests (ids + timestamps) and store them for 1–2 years.
• Backups: snapshot structured logs; encrypt at rest; test restores quarterly.

Sample SQL — masked QA samples (no PII)

Use queries that only pull masked samples for developers.

SELECT ts, actor->>'user_id' AS user_id,
       content->>'masked_sample' AS masked_sample,
       policy->>'pii_hit' AS pii_hit,
       metrics->>'latency_ms' AS latency_ms
FROM llm_logs
WHERE env='prod'
  AND content->>'masked_sample' IS NOT NULL
  AND (policy->>'pii_hit')::boolean = false
ORDER BY ts DESC
LIMIT 200;

Incident search — fast triage

When something goes wrong, find related events without reading raw content.

-- correlate by hash to avoid storing raw content
SELECT *
FROM llm_logs
WHERE content->>'prompt_hash' = 'HEX_HASH_HERE'
  OR content->>'output_hash' = 'HEX_HASH_HERE'
ORDER BY ts DESC;

Access control guardrails

• RBAC: separate roles for Dev, Ops, Audit, Admin; least-privilege principles.
• JIT secrets: short-lived credentials; rotate keys every 90 days or on departure.
• Unmasking workflow: service denies by default; require ticket + dual approval; log every access.

FAQ — LLM logging & retention

Do we need full transcripts?
Usually no. Keep hashes, metadata, and masked snippets; store full text only for targeted QA with stricter controls.

How long should we keep logs?
Default to 90 days. Use 30 days for high sensitivity, and extend only if contracts or regulators require it.

Can logs be used for model training?
Not by default. Require explicit consent and a separate pipeline that excludes PII and secrets.

How do we handle data subject requests?
Tokenize and index user IDs so you can find and delete their records quickly; keep purge manifests as evidence.

What about multi-tenant apps?
Always tag tenant_id and enforce filters at query time and in any analytics materializations.

Further reading

• Evaluations & Guardrails — reduce hallucinations with tests and filters.
• AI Data Privacy 101 — redaction, retention, and safe sharing.
• Automation Workflows — human-in-the-loop approvals and audits.

Final thoughts

Keep logs small, structured, and useful. Redact at ingest, tag with retention codes, and automate purge. That’s how you stay auditable without turning logs into a liability.