Enterprise GovernanceISO 42001 & EU AI Act

ISO 42001 Audit Logging for Local AI: EU AI Act Record-Keeping Setup

Log every local LLM request with OpenTelemetry and write-once storage to produce the event records ISO 42001 auditors and the EU AI Act expect.

Published

Architecture

Recommended model

DeepSeek-R1-Distill-32B (Q4_K_M)

Parser / tooling

OpenTelemetry Collector (append-only JSON logs)

VRAM required

21.5 GB

Q4_K_M, 8k tokens

Compliance standard

ISO 42001 & EU AI Act

Every inference request to the local model is traced through an OpenTelemetry Collector: who called it, which model and version answered, the prompt and response (or their hashes where content is sensitive), latency and any human override. Logs are written append-only, shipped to write-once storage and retained according to policy. The setup is model-agnostic; DeepSeek-R1-Distill-32B is used as the reference model because its reasoning traces are useful evidence in reviews.

  1. 01

    Instrument

    The application gateway emits an OpenTelemetry span for every model call.

  2. 02

    Collect

    The collector enriches spans with model name, version and user ID, hashing sensitive fields.

  3. 03

    Store

    Logs are written append-only to object storage with object lock (WORM) enabled.

  4. 04

    Review

    Governance teams sample logs and record human-oversight decisions against them.

Performance & privacy profile

Pre-lab estimates for DeepSeek-R1-Distill-32B (32B), Q4_K_M (4-bit Recommended), 8k tokens. Not measured results.
MetricValueBasis
VRAM required21.5 GBWeights + FP16 KV cache + 1 GB runtime overhead
Decode speed~24–37 tok/sMemory-bandwidth estimate on RTX 4090 24GB, single user
Table extraction accuracyN/A (logging workload)Not applicable to this workload
Privacy score7/10Editorial rubric, see breakdown below
  • On-prem inference, no third-party AI processor(implemented)+3
  • No outbound network route from AI hosts(not included)+2
  • Encryption at rest for documents and indexes(not included)+1
  • Per-request audit logging(implemented)+2
  • Role- or matter-based access control(implemented)+1
  • Data minimization before storage (redaction or hashing)(implemented)+1

Deployment config

Starting point only. Pin image versions and review network policy before production.

# docker-compose.yml: trace every local model call for audit evidence.
services:
  ollama:
    image: ollama/ollama:latest  # pin a specific tag in production
    volumes:
      - ollama:/root/.ollama  # pre-pulled: deepseek-r1:32b
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

  otel-collector:
    image: otel/opentelemetry-collector-contrib:latest
    command: ["--config=/etc/otelcol/config.yaml"]
    volumes:
      - ./otel-config.yaml:/etc/otelcol/config.yaml:ro
      - ./audit-logs:/var/log/ai-audit  # sync to WORM object storage
    ports:
      - "127.0.0.1:4317:4317"  # OTLP gRPC from your gateway
      - "127.0.0.1:4318:4318"  # OTLP HTTP

# otel-config.yaml: receive OTLP, write newline-delimited JSON:
#   receivers: { otlp: { protocols: { grpc: {}, http: {} } } }
#   exporters: { file: { path: /var/log/ai-audit/traces.jsonl } }
#   service: { pipelines: { traces: { receivers: [otlp], exporters: [file] } } }

volumes:
  ollama:

Size it for your hardware

Local LLM VRAM Calculator

2026 models

Total VRAM required

21.5GB

Weights
18.5 GB
KV cache
2.0 GB
Overhead
1.0 GB

Recommended local hardware

  • RTX 4090 24GB~24–37 tok/s
  • Mac mini M4 Pro 64GB~7–10 tok/s

Estimated single-user decode speed from memory bandwidth. Real throughput depends on runtime, drivers and prompt length.

Runs locally on RTX 4090 24GB — no cloud requiredCompare with a RunPod GPU pod →

Frequently asked questions

What does ISO/IEC 42001 require for logging?

ISO/IEC 42001:2023 is a management-system standard: it requires you to define, operate and monitor controls for your AI systems rather than prescribing a log format. Its Annex A controls cover areas such as AI system operation and monitoring, and in practice auditors expect event logs showing how systems were used, which model version produced an output and how issues were handled. This setup produces that evidence.

How long must AI logs be kept under the EU AI Act?

For high-risk AI systems, the AI Act requires automatic event logging, and providers and deployers must keep the logs under their control for a period appropriate to the system's intended purpose of at least six months, unless other EU or national law, such as data-protection rules, provides otherwise. Sector rules may require longer retention, so set your retention policy with legal counsel.

Should prompts and responses be stored in full?

It depends on the data. Full text is the most useful for investigations but may contain personal data, which GDPR requires you to minimize. A common pattern is to keep full content in an access-restricted store with a shorter retention period, and keep hashes and metadata in the long-term audit trail.

More blueprints