0046 — Slack replies on held-message cards reach prod through Smith's socket relay

Status: Accepted 2026-10-07. Gera asked for the fix ("yea fix it") after his "Approve" on a held Northridge email did nothing. Built in propflowai #17585 (the route accepts the relay) and agent-smith #839 (the relay).
Amended 2026-10-08: a SECOND, independent cause, found by the post-deploy smoke test. propflowai's middleware session check stood in front of POST /api/slack/events and answered every request 401 Authentication required before the route's signature check ran. After #17585 was live, a correctly relay-signed url_verification from the mini and a wrongly signed one got that same middleware 401. So even an HTTP delivery from Slack could never have reached the route. Fixed in propflowai #17616, which allowlists the path for POST only, the same shape as the Twilio, SendGrid and fake-person webhooks. Either cause alone silenced every Slack approve.
Date: 2026-10-07
Deciders: Gera (asked for the fix). Fede owns the held-message review loop and is open to object.
Systems: propflowai POST /api/slack/events, Agent Smith's Socket Mode receiver (agent_smith.slack_socket, launchd co.propflow.smith-slack-socket), and the Slack app Agent Smith (A0BESM30BME).

The problem

propflowai parks an outbound message it may not send on its own as an EscalationHold. Examples are a review-mode email, a Supervised text, and a staff-forced rerun. It posts a "Held for review" card into the property's Slack channel, and the card says to reply approve or reject: why. propflowai #9948 (2026-09-19) built POST /api/slack/events to act on that reply. #11250 (2026-09-23) added the staff-identity check.

Slack never called that route. The Slack app has run in Socket Mode since agent-smith #2729 (2026-06-30). A Socket Mode app delivers every event over a websocket to Smith on the mini and ignores an Event Subscriptions Request URL. The route's own header listed "point the Request URL at this route" as a config step "for Fede to do". That step could not be done, and the route was described as safe to deploy dark. Zero traffic therefore looked like the expected state, not a fault.

Measured 2026-10-07 on hold eschold_36b1d058… (Northridge):

Prod logs show no POST /api/slack/events during any of the three human replies in the thread, Gera's "Approve" at 15:28:34Z included. A positive control in the same window found the hold-sweep cron's request.
Smith's socket log shows every one of those replies arriving on the mini and being dispatched to Smith's brain. The brain guessed at an identity check and did not approve anything.
In prod, every past hold marked approved "via Slack" carries a hand-relay marker: fede via slack thread, relayed by session 010, relayed via Trinity.

Why it was built this way. Two gaps stacked, and each hid the other. Zero traffic was expected, so nobody noticed that even a delivered event would have hit the middleware's 401. The 9-19 PR's header records a repo-wide grep that found "NO existing Slack inbound path anywhere in this codebase". The grep was true of propflowai and false of the system. The inbound path lived in the agent-smith repo, and this catalog (SYSTEMS.md) had no entry for Smith's Socket Mode receiver, so checking the catalog would not have caught it either. The fair-housing hold flow, built 2026-08-30, had already solved the same problem the right way, on the mini (clara_hold_review). Nobody connected the two.

Options considered

A. Turn Socket Mode off and point a Request URL at prod. Ruled out. Every other Smith inbound (DMs, @-mentions, channel-watch, Trinity's sibling protocol) depends on Socket Mode, and it would also need a Slack-console step, the same kind of deferred human step that already failed here.
B. The mini runs the approval itself through a tsx script against prod Dynamo, as clara_hold_review does with approve-held-draft.ts. Ruled out for these holds. An approve here sends email, signals Temporal workflows and posts Slack replies, and all of those read their endpoints from the environment. The mini's .env.local is stage config with the table name overridden, so a Temporal signal would go to the wrong namespace.
C (this decision). The mini relays the reply to the prod route, and prod does the work in its own environment.

The decision

One path: Smith's socket listener relays a human thread reply to POST /api/slack/events when its thread root was posted by our bot. It does this ahead of the channel gate, because the cards live in property channels Smith does not own. There is no flag.
Authentication is a relay secret, not Slack's. The relay signs with SLACK_RELAY_SIGNING_SECRET using Slack's own v0 HMAC and 5-minute freshness scheme, in x-propflow-relay-* headers. The route still accepts a real Slack signature, so leaving Socket Mode later needs no code change. The secret is stored in SSM at /propflow/prod/SLACK_RELAY_SIGNING_SECRET, in Vercel production, and in propflowai/.env.local on the mini.
Prod decides what is a hold reply. The route answers hold: <id> when the thread is a pending hold. The relay then consumes the reply and prod posts the outcome line. When the thread is not a pending hold, Smith's normal routing runs unchanged. The mini never parses card prose to decide this. It uses the card's /admin/escalation-holds/ link only to decide whether a relay failure deserves a line in the thread.
What is not relayed: a reply that @-mentions Smith, because the person is asking Smith something and the brain answers it, and a fair-housing hold card, which clara_hold_review keeps.
Failure is said, not swallowed. If prod cannot be reached on a visible hold card, one thread line says nothing was sent and points at the card's PropFlow link.

Consequences

Once all three are live (#17585, #17616, #839), typing approve, reject: why or feedback on a "Held for review" card works for the first time. Gera and Fede both pass the prod staff check for Western Slope (read-only probe, 2026-10-07).
Open product question for Fede. On a held first-touch email, any reply that is not a command is treated as redraft feedback. Before this fix that code never ran, so it was never exercised. Now a remark such as "why is this held" in the thread will rewrite the draft, unless it @-mentions Smith. Decide whether that is wanted.
The route header that told readers to configure a Request URL is corrected in #17585, so nobody chases that step again.
SYSTEMS.md gains an entry for the socket receiver (smith-slack-socket), whose absence is half of why this happened.