Evidence and signed exports

When the product says something happened, why should your auditor believe it?

The honest answer is not "because we say so". The record is append-only at the database level, chained so that a rewrite is detectable, and exported under a signature a third party can check offline with a public key — no RapidValue account, no endpoint of ours, no secret of ours. This page walks that chain link by link, including where it stops.

The chain of custody

Four steps, each only as good as the one before it: the record is written so it cannot be altered, chained so alteration is detectable, exported with the claims about it bound into the same signed payload, and verified with a key obtained by a route the artefact does not control.

1. The record is append-only

Every governance action — a grant, a revocation, an approval, a configuration change, a login — writes a row to audit_event. On PostgreSQL a trigger fires BEFORE UPDATE OR DELETE on that table and raises an exception, so even an application user with full table privileges cannot alter or remove a row. Rows are inserted once, complete.

Each row also carries a chain_hash: SHA-256(previous_hash + canonical_row), over the event's identifier, tenant, timestamp, type, actor email, subject and payload, with payload keys sorted so JSON ordering cannot change the digest. Each tenant has its own chain. GET /api/v1/audit/verify-chain walks it and reports every link whose recorded hash does not match a freshly computed one; altering one row breaks its hash and every hash after it.

Two details of how the chain is written are worth stating, because they are where this kind of design usually goes wrong.

Writes for one tenant are serialised. Computing the next hash means reading the tenant's latest committed hash and inserting off it. Two concurrent requests would read the same predecessor and fork the chain — two rows claiming one ancestor, the exact ambiguity a chain exists to prevent. A per-tenant PostgreSQL advisory lock, scoped to the transaction, makes that pair atomic.

The chain is sealed immediately before commit, not when the event is recorded. Many actions write their audit row and then call a target system — provisioning a grant, disabling an account, running a sync — and those calls can block for tens of seconds when a target is unreachable. Taking the lock where the row is recorded would hold it across that call, and every other audited write for that tenant would queue behind it: one unreachable system stalls the tenant. So recording an event queues it, and the hashing and insertion happen in the last moment before commit. The rows still go in inside the caller's own transaction, so an action that rolls back writes no audit row — phantom evidence would be worse than a missing one.

2. Signed exports

GET /api/v1/audit/export (administrators and auditors only) streams events as NDJSON — one object per line, each carrying its own chain_hash — and appends a signature line covering every preceding byte, newlines included. The signature line is not part of the signed body. It also records the event count, so a body with lines removed fails on the count before it fails on the signature.

3. Evidence packs: what is signed

An evidence pack is a ZIP produced against a framework template and a time window — SOX ITGC, ISO/IEC 27001:2022, HIPAA §164.312, GDPR Article 32 and an identity-lifecycle template ship today. It holds a summary page, one JSONL file per section, a manifest, a signature file and a verification recipe.

The signature does not cover the ZIP bytes. It covers the manifest, serialised canonically — sorted keys, fixed separators, UTF-8. Archive member order, compression and per-entry timestamps all vary between runs, so a signature over the container could never be recomputed by the person holding the file, and a signature nobody can reproduce proves nothing.

The manifest is built to be worth signing:

• A SHA-256 and byte length for every other file in the archive, so one signature covers the contents transitively: altering a JSONL line contradicts the signed manifest.
• The claims made about the evidence, repeated inline — each section's title, kind, framework citation and row count. Swapping a cited control or under-reporting a row count breaks verification exactly as hard as editing the data. Evidence that could be relabelled undetectably is not evidence.
• Its canonical form, so a future layout change makes an older verifier report "unknown canonical form" rather than a mismatch it cannot explain.

Regenerating the same window produces a different but equally verifiable pack, not an identical one: the generation timestamp moves, and snapshot sections such as the risk-score distribution describe the estate as it is now. We do not claim byte-identical regeneration, because it is not true.

4. Two signatures, two audiences

Both are computed over the same manifest bytes. Neither replaces the other.

The HMAC key is the same secret that signs session tokens, so it can never be handed out — precisely why the asymmetric signature exists. The manifest also declares which key id signed it, inside the HMAC-covered payload, so stripping the ECDSA block out of the signature file is detectable rather than looking like an older pack that never had one.

If the asymmetric key cannot be resolved when evidence is produced — a vault outage, say — the artefact carries the HMAC alone and the manifest makes no asymmetric claim: it degrades honestly rather than claiming a signature it does not have. That also raises a critical advisory, because the loss is unrecoverable. A pack cannot gain the third-party signature afterwards, only be regenerated.

5. Verifying without us

backend/scripts/verify_audit_pack.py is a single file with no RapidValue imports. It needs the Python standard library, the cryptography package and a public key. It does not need a database, an account, network access, any RapidValue code, or any secret of ours. It verifies both packs and NDJSON exports, checks every member digest, refuses extra or missing files, and exits non-zero on any failure. --show prints the signing envelope without verifying, so an auditor can read the key id first.

⚠ The public key embedded in the artefact is not a trust anchor.

Every pack carries a copy of the signing public key. It is there so you can see which key to ask for — nothing more. Anyone able to alter the pack can alter that copy and re-sign with a key of their own, so verifying against the key the file supplies proves only that the file is internally consistent. Obtain the key separately, from the tenant's published signing keys, and match it by key id. The standalone verifier refuses to run without a key you supply, and treats disagreement with the embedded copy as a hard failure.

An upload endpoint also verifies a pack in memory and discards it, so returning one for checking never creates a second copy of the evidence. That exists for the HMAC half; for the half that matters to an outsider the offline script is stronger, because it does not require trusting the party being audited to run the check.

6. Keys

Each tenant has one active signing keypair, created on first use, with a key id derived from the public key itself so an identifier cannot drift from the key it names.

Rotation retires the active key and mints a new one, and is never destructive: the retired key and its published public half are kept, with no delete path, so packs signed years earlier keep verifying under the key id that actually signed them. The published key list therefore includes retired keys.

Private keys live in the vault and are resolved only to sign; no API response carries private key material — the read schema has no field that could. The vault is opaque storage rather than a signing service, so signing does bring the key into the application process for the duration of the operation. We state that rather than imply a hardware-backed guarantee we do not have.

7. Erasure and retention

A GDPR Article 17 erasure pseudonymises the identity — names, email, employee id, source identifiers and free attributes cleared or replaced with a stable pseudonym, linked accounts anonymised, and the login account with its credentials and MFA secret deleted outright.

Audit content is retained under Article 17(3)(b), processing necessary for compliance with a legal obligation. The reasoning matters more than the citation: an access-governance trail records who approved what, for whom and when. Erasing it would not protect the data subject so much as destroy the only means of showing that decisions about them were made properly — including the record of the erasure itself. Grants and access requests are retained on the same basis, as business records. The rule survives tenant deletion: audit_event is the one table a hard delete does not touch, and the tenant identifier is never reused afterwards.

What this does not prove

• The scheme has not been externally attested, certified or audited. It is documented here so you can evaluate it yourself — the point of the offline verifier.
• A verifying pack does not make anyone compliant. It shows that a specific set of records, with specific claims attached, is what RapidValue produced for a stated window. Whether those records satisfy a control is your assessment.
• Sections are capped (10,000 or 20,000 rows). The preview reports whether a window would exceed a cap; the pack itself states how many rows it contains but does not currently mark a section as capped.
• Records written before the chain existed have no hash and are skipped by chain verification, which reports total and chained events separately so the gap is visible rather than implied.
• A section that fails to render writes an explicit error document with a row count of zero, rather than an empty file that reads like an absence of findings.
• If sealing the chain itself fails, the rows are dropped with a critical log entry and the business transaction still commits. Platform availability is chosen over completeness of one audit row — loudly, never silently.

Further reading:

• Tier-3 Hybrid Architecture — where the platform runs, and what never crosses the wire
• API Access Governance — scoped machine credentials, and how their calls land in the same trail
• Reconciliation — Expected versus Actual, the engine most of these events describe