hb-arcis armed against thearchive.hb-brdis armed against theboard.- The switch on the
boardfirespublish. - The switch on the
boardhas a04:00deadline. - The switch that fires
rotateis armed against theregistry. - The switch held by
k.botfiresrotate. - The switch held by
k.opshas a03:40deadline. hb-arcdoes not firewipe.hb-hrndoes not firerotate.k.jeffdoes not hold the switch that fireswipe.- The
03:00deadline is neitherk.bot’s nor the one armed against thearchive. - Three switches were renewed at 02:14. The fourth has no renewal record at all.
16 / what happens when it stops
There is a particular kind of quiet that a building has at two in the morning when you are the only person in it and you are about to do something you cannot undo.
[INFO ] harness-04: heartbeat monitor, 4 switches armed
[INFO ] hb-arc renewed 02:14
[INFO ] hb-reg renewed 02:14
[INFO ] hb-brd renewed 02:14
[WARN ] hb-hrn no renewal record in window
[WARN ] renewals recorded: 3 of 4
Three of the four switches were being renewed every fifteen minutes by the scheduler, the way a dead-man switch is supposed to be: somebody says still here, still here, still here, and if they ever stop, the switch does the thing it was armed to do. The monitor logs the renewal and nothing else — what each switch is armed against, and when it comes due, lives in the switch table.
The fourth had no renewal record at all. Not a missed one. None, all week, going back as far as the monitor kept data.
Which meant either it had been armed and abandoned, or it was being renewed by something that did not write to this log.
> four switches. one of them isn't in the renewal log.
it's the earliest deadline too
that's not an accident
— nix
> if we kill horus the heartbeat stops
everything that isn't renewed fires
you need to know what fires, and what it fires at, before you touch anything
— nix
Andy pulled the switch table. Four heartbeats, four holder keys, four deadlines, four actions, four targets.
maintainer_key=r.sato
Every maintainer field was scrubbed. The key survived in the legacy index.
open incident →commit c-2de · signed r.sato · header x-mu-replay
The hook answers one request header and ignores everything else. Ninety lines, and it still acquires its context with mu_ctx_t, retired at the platform migration two years ago.
open incident →stream owner HORUS scope observation writes permitted
The record the three coordinates select. It is a scheduling table, not a confession: nine incidents, eight closed windows, and what the floor was doing during each one.
open incident →you can't kill it first. killing it is the trigger. that's what the whole thing is for.
columns are heartbeats, keys, deadlines, actions. rows are targets, actions, deadlines, keys.
find the one nobody is renewing. heartbeat, held by, deadline, fires, against.
and andy. when you have it, look at who holds it before you do anything else.
fault
Unrenewed switch, armed
renewals
3 of 4
severity
unfiled
| hb-arc | hb-hrn | hb-reg | hb-brd | k.ops | r.sato | k.bot | k.jeff | 03:00 | 03:10 | 03:40 | 04:00 | publish | wipe | rotate | seal | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| archive | ||||||||||||||||
| harness | ||||||||||||||||
| registry | ||||||||||||||||
| board | ||||||||||||||||
| publish | ||||||||||||||||
| wipe | ||||||||||||||||
| rotate | ||||||||||||||||
| seal | ||||||||||||||||
| 03:00 | ||||||||||||||||
| 03:10 | ||||||||||||||||
| 03:40 | ||||||||||||||||
| 04:00 | ||||||||||||||||
| k.ops | ||||||||||||||||
| r.sato | ||||||||||||||||
| k.bot | ||||||||||||||||
| k.jeff |
Question: Which switch is armed and unrenewed — held by what, deadline, what it fires, against what?