docs: workload registry plan; remove tests from both plans
Both plans now verify by build, vet and manual checks written into the tasks. Spec verification sections updated to match so they no longer describe tests that will not be written.
This commit is contained in:
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -395,8 +395,9 @@ This adds one `notify` payload type and a `vuln_digest.html.tmpl` /
|
||||
`vuln_digest.txt.tmpl` pair in `shared/mail`. Note that `shared/mail` templates
|
||||
are parsed in `init()`, so a mistyped field is a boot-time panic — CLAUDE.md
|
||||
describes a `render_test.go` guarding against exactly this, but **that file does
|
||||
not exist**; the repository has no Go tests at all today. The implementation
|
||||
plan adds it alongside this feature's own tests.
|
||||
not exist**; the repository has no Go tests at all, and by instruction this
|
||||
feature adds none. The template pair must therefore be verified by starting the
|
||||
binary and sending one digest through a real channel.
|
||||
|
||||
---
|
||||
|
||||
@@ -470,30 +471,33 @@ quietly report all-clear.
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
## Verification
|
||||
|
||||
**Version comparison, table-driven, per family.** The highest-value test in the
|
||||
feature, because its failure mode is silent. The table must include the
|
||||
backport case explicitly — installed `1:3.0.2-0ubuntu1.15` against advisory
|
||||
fixed-in `1:3.0.2-0ubuntu1.15` resolving to *not vulnerable* — alongside tilde
|
||||
ordering (`1.0~rc1` < `1.0`), epoch dominance (`1:1.0` > `2.0`) and
|
||||
`1.9` < `1.10`.
|
||||
**No automated tests.** The repository has none today, and by explicit
|
||||
instruction this feature adds none — no `*_test.go`, no frontend test files.
|
||||
That is a deliberate decision by the repository owner, recorded here so the
|
||||
absence reads as a choice rather than an omission.
|
||||
|
||||
**Matcher golden tests.** A fixture package list and a small hand-built BoltDB
|
||||
fixture asserting an exact finding set. No network: a test that pulls the real
|
||||
database fails on a bad day and changes its expectations every six hours.
|
||||
It does change the risk profile, and the places it changes it are worth naming,
|
||||
because each fails by producing a **wrong answer rather than a crash**:
|
||||
|
||||
**State transitions.** `open → fixed → reopened`, acceptance expiry, and
|
||||
`first_seen` surviving a rescan. The last is easy to break with an upsert that
|
||||
overwrites, and nothing notices until a report claims everything was discovered
|
||||
yesterday.
|
||||
- **Version comparison.** The backport case — installed `1:3.0.2-0ubuntu1.15`
|
||||
against advisory fixed-in `1:3.0.2-0ubuntu1.15` resolving to *not
|
||||
vulnerable* — plus tilde ordering (`1.0~rc1` < `1.0`), epoch dominance
|
||||
(`1:1.0` > `2.0`) and `1.9` < `1.10`. Wrong here means a vulnerable fleet
|
||||
reported clean.
|
||||
- **Source-package fan-out.** One advisory against `openssl` must flag
|
||||
`libssl3`, `openssl` and `libssl-dev`. Matching on binary name alone silently
|
||||
finds one of three.
|
||||
- **`first_seen` preservation.** An upsert that overwrites it makes every
|
||||
finding look discovered today, and nothing surfaces that until someone reads
|
||||
a report.
|
||||
- **Fixed-before-reopen ordering.** A finding both absent from a scan and past
|
||||
its acceptance expiry must settle `fixed`, not reopen.
|
||||
|
||||
**Collector parsers**, one per package manager, from captured command output.
|
||||
`updates.go` currently has no tests and its parsers are the same shape, so
|
||||
these fixtures should cover both.
|
||||
|
||||
**Scoping and gating.** Findings queries scoped by `instance_id`, and the
|
||||
entitlement gate proven to write nothing rather than merely hide.
|
||||
The implementation plan carries a manual verification table for each, to be
|
||||
walked before the relevant task is committed. They are the substitute for the
|
||||
tests, not a formality.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -377,22 +377,29 @@ Three rules that follow directly from the model:
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
## Verification
|
||||
|
||||
Every test is pure, driven by captured fixtures, with no daemon, no database
|
||||
and no network — the repository has no Go tests today and these must run under
|
||||
plain `go test ./...`.
|
||||
**No automated tests.** The repository has none today, and by explicit
|
||||
instruction this feature adds none — no `*_test.go`, no frontend test files.
|
||||
A deliberate decision by the repository owner, recorded so the absence reads as
|
||||
a choice rather than an omission.
|
||||
|
||||
- `docker inspect` JSON fixture → `[]Workload`, asserting `RestartCount`,
|
||||
health, and that the compose label becomes `Stack`.
|
||||
- `systemctl` column fixtures → `[]Workload`, including the exclusion filter
|
||||
dropping `systemd-*` and `user@*` while keeping `nginx.service`.
|
||||
- Protected-set computation: `vantage-agent.service` marked, `nginx.service`
|
||||
not.
|
||||
- Hash order-independence, matching the package report's test.
|
||||
- **Log capping in both directions**: 600 lines in → 500 out with `truncated`
|
||||
set; a 300KB blob of fewer than 500 lines → capped, `truncated` set. The
|
||||
second is the case a line-count-only implementation silently fails.
|
||||
The behaviours that would otherwise have been tested are the ones that fail
|
||||
quietly, and the implementation plan carries a manual check for each:
|
||||
|
||||
- **Parser output against real command output.** `docker inspect` must yield
|
||||
`RestartCount`, health and the compose label as `Stack`; the `systemctl`
|
||||
exclusion filter must drop `systemd-*` and `user@*` while keeping
|
||||
`nginx.service`. Both are verified against a live host rather than a fixture.
|
||||
- **Protected-set computation.** `vantage-agent.service` marked,
|
||||
`nginx.service` not. Getting this wrong in the permissive direction lets a
|
||||
server stop its own agent, which is unrecoverable from the UI.
|
||||
- **Hash order-independence.** An ordering-sensitive hash resends the full list
|
||||
every 60 seconds, which is invisible except as traffic.
|
||||
- **Log capping in both directions.** 600 lines in → 500 out with `truncated`;
|
||||
a 300KB blob of fewer than 500 lines → capped, `truncated`. The second is the
|
||||
case a line-count-only implementation silently fails, and it fails by sending
|
||||
megabytes rather than by erroring.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user