PropFlow.ai PropFlow.ai Docs
Proposedmarkdown source

0102 — Secrets: eliminate per-runtime copies; validate the rest at boot

Context

The incident looked like "someone forgot one Lambda." The audit of how config actually reaches production shows the architecture makes this class of failure inevitable:

The fail-closed guard from PR #3027 was correct security (the prior fallback signed slot tokens with a hardcoded dev constant — meaning the inbound-processor had been signing with a public key all along). What failed is the surrounding architecture: N independent env copies, no declaration of which runtime needs what, no reconciliation, and absence expressed as silent per-request errors instead of loud deploy failures.

Industry research (full citations in the RCA thread) is unambiguous about the direction: push-synced or hand-copied env vars are the root cause, not a hygiene problem. The strong fixes remove the copies — single-source runtime resolution, or asymmetric keys that abolish the shared secret — and boot-time schema validation converts any remaining absence from silent to loud. Parity auditing is a compensating control for copies you haven't yet eliminated, not a destination.

Decision

Five moves, ordered by leverage. The principle: a secret has exactly one authoritative location, every runtime's requirements are declared in reviewable code, and absence fails the deploy — never the customer.

D1 — Per-runtime env schema, validated at boot (week 1)

One schema module (src/lib/platform/env-schema/) declares every required and optional env var with its type and consumer runtimes. Each deployable unit gets a required-set file (env.vercel.ts, env.inbound-processor.ts, env.agent-runtime.ts, env.renewal-worker.ts, …) imported at that unit's entrypoint: missing required var → the Next build fails / the Lambda cold-start throws a Sentry-fatal before serving any request. "Which runtime needs which var" stops being tribal knowledge and becomes a code-reviewed diff — adding a fail-closed guard without declaring the var in the schema becomes a lint/test failure. Optional feature vars are declared too: absence logs one structured startup warning surfaced on the admin status page, never a silent no-op.

D2 — Single source of truth per secret (sprint)

SSM /propflow/prod/* becomes the one authoritative store for shared/internal secrets:

D3 — Asymmetric keys where signer and verifier differ (sprint, with fix PR 2)

The tour-slot handshake — and any future sign-here/verify-there token — moves from HMAC to an asymmetric signature (Ed25519): the private key lives only in the signing path's runtime; every verifier holds the public key, which is not a secret and is committed to the repo. The "same secret must exist in N runtimes" requirement is deleted, not managed. (Immediate incident fix — provisioning the HMAC secret to the inbound-processor via SSM — ships first as its own PR; D3 then retires the parity requirement.)

D4 — Hash-parity audit as the transitional backstop (week 1–2)

audit-runtime-token-parity is extended to read the D1 schema manifest and verify, for every secret still existing in more than one store, that (a) it is present everywhere the schema requires, and (b) shared values are byte-identical via salted-hash comparison — names and hashes only, never values. Runs scheduled daily and on every deploy of each runtime; failure blocks the deploy. This is explicitly labeled transitional: each secret migrated to D2/D3 leaves its scope, and the audit's steady-state coverage should trend toward zero.

D5 — Retire the fork and the ghosts (background)

The agents/clara/lib duplication of guard code is consolidated (tracked in ADR-0101 S5), so a guard exists in exactly one module. The four dist-only lambda directories are confirmed retired and deleted, or documented and brought under the schema — no deployable unit outside the manifest.

Consequences

Alternatives considered