Incident ADV-3 · Oct 15, 2026

`@acme-sim/drill-logger` 2.7.0

DrillClosed by a person · 12:04 UTC

At 11:14 a build installed this hijacked version of a logging package with install scripts on, so its hidden script could run. 12 minutes later the warning came in. By 12:04 Aftermath had found the 2 builds that installed it with scripts on, a person had approved the fixes, the exposed key was replaced and proven dead, the tainted image was deleted, and a person closed the case.

This is the run in the 3-minute demo video ↗.

Overview

2 of 7builds installed it with scripts on; the other 5 had scripts switched off

1 keyreplaced; the old one is now refused (HTTP 401)

1 imagedeleted: what the exposed build made. The live one was left alone.

4 min 24 sfor the whole clean-up, once a person said go

Three checkpoints: Aftermath stopped and waited for a person

Checkpoint 1What to look forasked 11:39:05 · approved 11:40:091 min 04 s
Checkpoint 2Approve the fixesasked 11:41:43 · approved 11:50:118 min 28 s
Checkpoint 3The receiptsasked 11:50:18 · approved 12:04:0113 min 43 sincludes the merges and the clean-up run

Where code couldn't see

An AI agent decided acme-web · build-image

Plain code can't see inside a Docker build, so it left this build undecided. An AI agent read that day's Dockerfile, and could only raise the verdict by quoting the line it relied on:

Dockerfile@3b94760c:4: RUN npm ci --ignore-scripts=false --omit=dev --no-audit --no-fund

Verdict: Likely ran · its decision, word for word ↗

Builds checked

4 pipelines ran in the window (Oct 15, 2026, 11:14–11:26 UTC). Every build that installed the bad version: its verdict, the rule and evidence lines it rests on, and the keys it could reach.

7 builds shown

Builds that installed @acme-sim/drill-logger 2.7.0
Build	Verdict and rule	Install and evidence	Keys in reach
acme-ui · test `feat/native-thumbnails`	Likely ran R1 Bad version locked, npm/yarn scripts on, install ran	`npm ci --no-audit --no-fund` package-lock.json@f4ee32e5:15: @acme-sim/drill-logger@2.7.0 job command: npm ci --no-audit --no-fund (npm 11) job log line 52: $ npm ci --no-audit --no-fund	CI_JOB_TOKEN
acme-web · build-image `main`	Likely ran R7 Install inside a Docker build or wrapper script: needs judgement decided by an AI agent	`docker build --platform linux/amd64 -t "$AR_REPO/acme-web:$CI_COMMIT_SHORT_SHA" .` Dockerfile@3b94760c:4: RUN npm ci --ignore-scripts=false --omit=dev --no-audit --no-fund package-lock.json@3b94760c:16: @acme-sim/drill-logger@2.7.0 job command: docker build --platform linux/amd64 -t "$AR_REPO/acme-web:$CI_COMMIT_SHORT_SHA" .	CI_JOB_TOKEN, GCP_ID_TOKEN
acme-api · test `main`	Safe: scripts were off R2 Install scripts disabled (--ignore-scripts / .npmrc / ignoreScripts)	`pnpm install --ignore-scripts --frozen-lockfile` pnpm-lock.yaml@fb972839:0: @acme-sim/drill-logger@2.7.0 job command: pnpm install --ignore-scripts --frozen-lockfile	CI_JOB_TOKEN
acme-ui · test `main`	Safe: scripts were off R2 Install scripts disabled (--ignore-scripts / .npmrc / ignoreScripts)	`npm ci --ignore-scripts --no-audit --no-fund` package-lock.json@d88531a7:15: @acme-sim/drill-logger@2.7.0 job command: npm ci --ignore-scripts --no-audit --no-fund	CI_JOB_TOKEN
acme-ui · lint `main`	Safe: scripts were off R2 Install scripts disabled (--ignore-scripts / .npmrc / ignoreScripts)	`npm ci --ignore-scripts --no-audit --no-fund` package-lock.json@d88531a7:15: @acme-sim/drill-logger@2.7.0 job command: npm ci --ignore-scripts --no-audit --no-fund	CI_JOB_TOKEN
acme-ui · lint `feat/native-thumbnails`	Safe: scripts were off R2 Install scripts disabled (--ignore-scripts / .npmrc / ignoreScripts)	`npm ci --ignore-scripts --no-audit --no-fund` package-lock.json@f4ee32e5:15: @acme-sim/drill-logger@2.7.0 job command: npm ci --ignore-scripts --no-audit --no-fund	CI_JOB_TOKEN
acme-web · install-test `main`	Safe: scripts were off R2 Install scripts disabled (--ignore-scripts / .npmrc / ignoreScripts)	`npm ci --ignore-scripts --no-audit --no-fund` package-lock.json@3b94760c:16: @acme-sim/drill-logger@2.7.0 job command: npm ci --ignore-scripts --no-audit --no-fund	CI_JOB_TOKEN

Clean-up

The protected respond pipeline, started when a person merged Aftermath's plan. Each line links the job that did it.

passed: Plan written by Aftermath, merged by a personnever a bot or the flow itselfsod-check ↗
passed: Emergency stop not pulledstop file absent, respond flag onkill switch ↗
passed: Rebuilt from the fixed codef73f1fb5rebuild ↗
passed: Deployed by digestsha256:927f9275…976ca6badeploy ↗
passed: Key replaced, old one refusedacme-web-api-key v3 → v4 · new key HTTP 200 · old key HTTP 401 · v3 disabledrotate ↗
passed: Tainted image deletedsha256:8ae6b10b…2f944e57, built by the exposed jobyank ↗
passed: Live image left alonesha256:927f9275…976ca6bayank ↗
passed: Receipts greenthe whole clean-up took 4 min 24 sreceipts ↗

Merge requests Aftermath wrote

acme-web!3 Pin @acme-sim/drill-logger to 2.7.1 and turn install scripts off (ADV-3)
acme-ui!4 Pin @acme-sim/drill-logger to 2.7.1 and turn install scripts off (ADV-3)
acme-api!3 Pin @acme-sim/drill-logger to 2.7.1 and turn install scripts off (ADV-3)
aftermath!6 Respond to ADV-3: rebuild, rotate and yank exactly what was exposed

Timeline

Oct 15, 2026, times in UTC. Blue steps are checkpoints, where Aftermath stopped and waited for a person.

a personAftermathan AI agenta CI pipeline

Something goes wrong

11:14a CI pipeline#
A build installs the bad version
acme-web's image build installs drill-logger 2.7.0 with install scripts allowed, so the package's own script could run inside the build, right next to the key that deploys the app. A test branch in acme-ui does the same. the build →

Finding out

11:26a person#
The warning arrives
Someone pastes the drill advisory ("malicious install script in @acme-sim/drill-logger 2.7.0") into a private GitLab issue. the issue →
11:37a person#
Aftermath is asked to look
A person mentions Aftermath on the issue, and it starts working. the comment →
11:39 → approved 11:40Aftermath#
Checkpoint 1: what to look for
Aftermath proposes the package, the bad version, the projects and the time window to search. A person checks it and says yes. the proposal →
11:40an AI agent#
An AI agent looks inside the Docker build
Plain code can't see inside a Docker build, so it marks it "can't tell". An agent opens that day's Dockerfile and decides the script likely ran. It is only allowed to say so by quoting the line (line 4). its words, verbatim →
11:41an AI agent#
A second agent double-checks
A verifier re-reads the raw evidence. It may only make a verdict more cautious, never call a build safe. It agrees. verbatim →
11:41Aftermath#
The list of exposed builds
Every build in the window, whether the bad version was installed with scripts on, and which keys it could reach. 2 of 7 were. see the list →

Fixing it

11:41 → approved 11:50Aftermath#
Checkpoint 2: approve the fixes
A person approves. Ordinary, tested code (not the AI) writes the changes. the plan →
11:50Aftermath#
Fixes written, not applied
One merge request per project (the safe version, scripts off, a guard against the bad version) and a clean-up plan: rebuild acme-web, replace one key, delete one image. the plan →
11:56a person#
A person merges the fixes
Every project moves to 2.7.1, and acme-web's build turns install scripts back off. acme-web's fix →
11:57a person#
A person gives the go-ahead
Aftermath can't merge its own plan. A person merging it is the approval for the clean-up. the merge →
11:57a CI pipeline#
The clean-up runs
It checks who wrote and who approved the plan and that nobody has pulled the emergency stop, rebuilds the app from clean code, replaces the key and tests that the old one is now refused, then deletes the tainted image. Done in 4 min 24 s. the pipeline →

Proving it

11:50 → approved 12:04Aftermath#
Checkpoint 3: the receipts
Aftermath checks the receipts from that exact clean-up run and posts the evidence. A person approves. the evidence →
12:04a person#
A person closes the case
Aftermath isn't allowed to close incidents. A person does, with the evidence in front of them. the record →

The run before this one went wrong

The first time this incident ran, Aftermath misread one setting: npm ci --ignore-scripts=false means scripts ON, and it read it as OFF. So it concluded nothing had run, and planned no clean-up at all. A person saw the empty plan at checkpoint 3, rejected it, and explained why on the issue. The fix, with tests, landed the same hour, and the run above is the re-run. The fix

Raw data

Download this incident (JSON)

The data this page is built from, with one field (the merging person's username) replaced by “a person”. Built 2026-10-17 07:48:42 UTC from GitLab records by tools/build_incident.py.

Check it yourself, no sign-in

Watch the old key get refused: the clean-up job's own log (new key 200, old key 401)Read the AI agent's decision, word for word from its session The full incident record, including the run that went wrong

Overview

Three checkpoints: Aftermath stopped and waited for a person

An AI agent decided acme-web · build-image

Builds checked

Clean-up

Merge requests Aftermath wrote

Timeline

Something goes wrong

A build installs the bad version

Finding out

The warning arrives

Aftermath is asked to look

Checkpoint 1: what to look for

An AI agent looks inside the Docker build

A second agent double-checks

The list of exposed builds

Fixing it

Checkpoint 2: approve the fixes

Fixes written, not applied

A person merges the fixes

A person gives the go-ahead

The clean-up runs

Proving it

Checkpoint 3: the receipts

A person closes the case

The run before this one went wrong

Raw data

Check it yourself, no sign-in