fix(patching): final review fixes
Chart Release / chart (push) Successful in 19s
Server Deploy / deploy (push) Successful in 6m12s

- no dispatch in the last 15 minutes of a window; no-result timeout from dispatch time
- per-server output moves to patch_run_outputs (16MB document limit)
- reboot proven by a changed boot time; RebootTimeout 45m, ResultGrace 20m
- window update and delete are server-scoped against the policies using them
- scheduler puts the claim back on an error after it, so the next tick retries
- cancelled runs with failures alert; MCP apply_updates audits per server
- apply-updates 503 body documented; openapi regenerated
- web: cleared numeric fields no longer save as 0; Run now asks for confirmation
This commit is contained in:
2026-09-15 13:49:28 +00:00
parent 3ecea7c39f
commit 3f2d20868e
22 changed files with 436 additions and 71 deletions
+21 -7
View File
@@ -175,11 +175,15 @@ parses is disabled rather than left spinning the loop every 30 seconds forever.
### Scheduled patching
Three collections: `maintenance_windows` (cron start, IANA zone, duration),
Four collections: `maintenance_windows` (cron start, IANA zone, duration),
`patch_policies` (selector, window, `all|security`, `never|if_required`,
concurrency cap, channels) and `patch_runs` (one per firing or manual Apply
updates, one `servers[]` entry per target). All three are in
`ScopedCollections`.
concurrency cap, channels), `patch_runs` (one per firing or manual Apply
updates, one `servers[]` entry per target) and `patch_run_outputs` (one per
run and server, holding the package manager's output tail). The output lives
apart from the run because a large run with up to 64KB per server would pass
MongoDB's 16MB document limit; `GetPatchRun` fills `servers[].output` back in
memory so the API shape is unchanged, and the tick paths never read it. All
four are in `ScopedCollections`.
**Runs are driven by database state, not goroutines.** A run can last hours; a
goroutine-driven run is stranded at `running` when its pod dies. `patchsched`
@@ -189,11 +193,21 @@ decision is a pure function in `internal/patchrun` (`Advance`, `ApplyResult`,
`VerifyReboot`, `Finalize`) and every write is guarded by the server run's
current status, so a result landing mid-tick is never overwritten.
**The window end never kills a package manager.** No server is dispatched in
the last `patchrun.LatestStartBeforeEnd` (15 minutes) of a window; queued and
waiting servers close at the window end as before. A server already patching
may finish past the end: its no-result timeout is its own dispatch time plus
`ManualTimeout` (2h, the agent's backstop) plus `ResultGrace` (20 minutes), for
windowed and manual runs alike. `RebootTimeout` is 45 minutes.
**Results do not cross the bus.** The pod holding the agent's stream writes
`PatchResult` straight into the run, found by `servers.command_id` and the
agent's own server ID. A reboot is settled by the first static inventory report
whose `boot_time_unix` is later than `rebooted_at`; a report during the
one-minute grace does not count.
whose boot time differs from (is later than) `boot_time_before`, the
`inventory.boot_time` recorded when the server moved to rebooting, so host and
server clock skew does not matter. Without `boot_time_before` the report's boot
time must be later than `rebooted_at`. A report during the one-minute grace
does not count either way.
**Old agents must never receive a scope.** An agent before
`patchrun.MinAgentVersion` ignores `scope` and installs everything, so policy
@@ -1053,7 +1067,7 @@ plane, each of which this codebase enforces:
## MongoDB Collections
`servers` · `keys` · `assignments` · `orgs` · `users` · `auth_providers` · `settings` · `secrets` · `workflows` · `workflow_steps` · `workflow_runs` · `workflow_log_lines` · `workflow_log_seq` · `monitors` · `incidents` · `monitor_rollups` · `notification_channels` · `console_sessions` · `audit_logs` · `server_packages` · `vuln_findings` · `vuln_alert_rules` · `vulndb_meta` · `server_workloads` · `api_tokens` · `status_pages` · `status_incidents` · `maintenance_windows` · `patch_policies` · `patch_runs` · `migrations`
`servers` · `keys` · `assignments` · `orgs` · `users` · `auth_providers` · `settings` · `secrets` · `workflows` · `workflow_steps` · `workflow_runs` · `workflow_log_lines` · `workflow_log_seq` · `monitors` · `incidents` · `monitor_rollups` · `notification_channels` · `console_sessions` · `audit_logs` · `server_packages` · `vuln_findings` · `vuln_alert_rules` · `vulndb_meta` · `server_workloads` · `api_tokens` · `status_pages` · `status_incidents` · `maintenance_windows` · `patch_policies` · `patch_runs` · `patch_run_outputs` · `migrations`
Every document except `migrations` carries `org_id`. Struct definitions are the source of truth - see `server/internal/models/`.