INC-0004 Cache Poisoning

04 / cache poisoning

The alert hit every channel at once.

[ALERT] INC-0004 — Tenant isolation failure on edge cache layer
[ALERT] Company helios receiving orion dashboard widgets
[ALERT] API sessions returning mismatched tenant contexts
[ALERT] Invalid feature flags propagating globally
[CRITC] /session endpoint: poisoned data detected

Andy stared at the dashboard. Three different tenants were receiving each other’s configuration data. Sessions appeared swapped. Feature flags meant for one company were activating inside another.

Security posted within minutes: external intrusion. Cache poisoning. Credential compromise.

But Bob sent a direct message before the all-hands even started.

“Ignore Security’s thread. Look at which infrastructure routes are affected.”

Andy pulled the cache topology. The corruption wasn’t random. It followed specific routes — /config, /session, /feature — across specific edge nodes.

All of them had been maintained by someone named Rin.

Andy searched for Rin in the company directory. No results.

evidence retained

The poisoned /session route did not originate from edge-01.
atlas was not affected through /feature.
A recovered fragment from Rin’s maintenance log: node=edge-03 route=/config — invalidated. Bob: “Funny how nobody can explain when exactly Rin left.”
edge-03 only served corrupted data to helios.
A partial audit fragment recovered from backup: tenant=orion route=/feature
A malformed telemetry line appears in the edge-03 log: cache.invalidate("truth"). Bob dismisses it as junk data. Andy does not.

messages bob

Bob 11:42

Don't use internal search for Rin. Half the records are gone.

Bob 11:44

Someone is cleaning logs faster than Security can archive them.

Bob 11:47

If this were an external breach, they'd already have announced it.

table cache topology

fault

Tenant data cross-contamination

type

Cache isolation failure

severity

SEV-1

	helios	orion	atlas	/config	/session	/feature
edge-01
edge-02
edge-03
/config
/session
/feature

answer input incident response

Question: Which tenant received poisoned data, through which edge node, on which route?

operator answer

04 / after the poison

Andy traced the corruption back through the cache invalidation logs.

Every affected route had the same pattern: a valid cache entry replaced by a stale replica. Not random. Surgical. Someone had rewritten the cache layer from the inside.

He searched for the original maintainer records. Every reference to Rin had been replaced:

[REDACTED BY POLICY]

But the cache diagnostic snapshots used a different index. Andy pulled the archived version.

cache-diag — edge-03 (archived)

  CACHE DIAGNOSTIC SNAPSHOT — edge-03
  ────────────────────────────────────
  route        maintainer       status
  ────────────────────────────────────
  /config      [REDACTED]       invalidated
  /session     [REDACTED]       invalidated
  /feature     [REDACTED]       invalidated
  ────────────────────────────────────
  last_deploy  [REDACTED]
  maintainer_key=r.sato
  ────────────────────────────────────
  ► all maintainer fields scrubbed
  ► key survived in legacy index

04 / loose thread

Rin had root-level infrastructure access. And someone was actively scrubbing every trace of that access from the system.

Andy closed the diagnostic window. A new telemetry fragment appeared in his terminal before he could log out:

you are being monitored too
— nix

He stared at it for a long time.

Then he closed his laptop.

NEXT INCIDENT →