docs: workload registry plan; remove tests from both plans

Both plans now verify by build, vet and manual checks written into the
tasks. Spec verification sections updated to match so they no longer
describe tests that will not be written.
This commit is contained in:
2026-08-06 11:27:37 +01:00
parent d1ac3e98ce
commit 3afc4ab012
4 changed files with 1916 additions and 952 deletions
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -395,8 +395,9 @@ This adds one `notify` payload type and a `vuln_digest.html.tmpl` /
`vuln_digest.txt.tmpl` pair in `shared/mail`. Note that `shared/mail` templates
are parsed in `init()`, so a mistyped field is a boot-time panic — CLAUDE.md
describes a `render_test.go` guarding against exactly this, but **that file does
not exist**; the repository has no Go tests at all today. The implementation
plan adds it alongside this feature's own tests.
not exist**; the repository has no Go tests at all, and by instruction this
feature adds none. The template pair must therefore be verified by starting the
binary and sending one digest through a real channel.
---
@@ -470,30 +471,33 @@ quietly report all-clear.
---
## Testing
## Verification
**Version comparison, table-driven, per family.** The highest-value test in the
feature, because its failure mode is silent. The table must include the
backport case explicitly — installed `1:3.0.2-0ubuntu1.15` against advisory
fixed-in `1:3.0.2-0ubuntu1.15` resolving to *not vulnerable* — alongside tilde
ordering (`1.0~rc1` < `1.0`), epoch dominance (`1:1.0` > `2.0`) and
`1.9` < `1.10`.
**No automated tests.** The repository has none today, and by explicit
instruction this feature adds none — no `*_test.go`, no frontend test files.
That is a deliberate decision by the repository owner, recorded here so the
absence reads as a choice rather than an omission.
**Matcher golden tests.** A fixture package list and a small hand-built BoltDB
fixture asserting an exact finding set. No network: a test that pulls the real
database fails on a bad day and changes its expectations every six hours.
It does change the risk profile, and the places it changes it are worth naming,
because each fails by producing a **wrong answer rather than a crash**:
**State transitions.** `open → fixed → reopened`, acceptance expiry, and
`first_seen` surviving a rescan. The last is easy to break with an upsert that
overwrites, and nothing notices until a report claims everything was discovered
yesterday.
- **Version comparison.** The backport case — installed `1:3.0.2-0ubuntu1.15`
against advisory fixed-in `1:3.0.2-0ubuntu1.15` resolving to *not
vulnerable* — plus tilde ordering (`1.0~rc1` < `1.0`), epoch dominance
(`1:1.0` > `2.0`) and `1.9` < `1.10`. Wrong here means a vulnerable fleet
reported clean.
- **Source-package fan-out.** One advisory against `openssl` must flag
`libssl3`, `openssl` and `libssl-dev`. Matching on binary name alone silently
finds one of three.
- **`first_seen` preservation.** An upsert that overwrites it makes every
finding look discovered today, and nothing surfaces that until someone reads
a report.
- **Fixed-before-reopen ordering.** A finding both absent from a scan and past
its acceptance expiry must settle `fixed`, not reopen.
**Collector parsers**, one per package manager, from captured command output.
`updates.go` currently has no tests and its parsers are the same shape, so
these fixtures should cover both.
**Scoping and gating.** Findings queries scoped by `instance_id`, and the
entitlement gate proven to write nothing rather than merely hide.
The implementation plan carries a manual verification table for each, to be
walked before the relevant task is committed. They are the substitute for the
tests, not a formality.
---
@@ -377,22 +377,29 @@ Three rules that follow directly from the model:
---
## Testing
## Verification
Every test is pure, driven by captured fixtures, with no daemon, no database
and no network — the repository has no Go tests today and these must run under
plain `go test ./...`.
**No automated tests.** The repository has none today, and by explicit
instruction this feature adds none — no `*_test.go`, no frontend test files.
A deliberate decision by the repository owner, recorded so the absence reads as
a choice rather than an omission.
- `docker inspect` JSON fixture → `[]Workload`, asserting `RestartCount`,
health, and that the compose label becomes `Stack`.
- `systemctl` column fixtures → `[]Workload`, including the exclusion filter
dropping `systemd-*` and `user@*` while keeping `nginx.service`.
- Protected-set computation: `vantage-agent.service` marked, `nginx.service`
not.
- Hash order-independence, matching the package report's test.
- **Log capping in both directions**: 600 lines in → 500 out with `truncated`
set; a 300KB blob of fewer than 500 lines → capped, `truncated` set. The
second is the case a line-count-only implementation silently fails.
The behaviours that would otherwise have been tested are the ones that fail
quietly, and the implementation plan carries a manual check for each:
- **Parser output against real command output.** `docker inspect` must yield
`RestartCount`, health and the compose label as `Stack`; the `systemctl`
exclusion filter must drop `systemd-*` and `user@*` while keeping
`nginx.service`. Both are verified against a live host rather than a fixture.
- **Protected-set computation.** `vantage-agent.service` marked,
`nginx.service` not. Getting this wrong in the permissive direction lets a
server stop its own agent, which is unrecoverable from the UI.
- **Hash order-independence.** An ordering-sensitive hash resends the full list
every 60 seconds, which is invisible except as traffic.
- **Log capping in both directions.** 600 lines in → 500 out with `truncated`;
a 300KB blob of fewer than 500 lines → capped, `truncated`. The second is the
case a line-count-only implementation silently fails, and it fails by sending
megabytes rather than by erroring.
---