docs: design for maintenance windows and scheduled patching
This commit is contained in:
@@ -0,0 +1,533 @@
|
||||
# Maintenance windows and scheduled patching
|
||||
|
||||
Date: 2026-09-14
|
||||
|
||||
## Goal
|
||||
|
||||
Let an operator say "install security updates on every `env:prod` server,
|
||||
Sundays 02:00 to 04:00 Europe/London, and reboot them if the OS says a reboot
|
||||
is owed", and have Vantage do it, report exactly what happened per server, and
|
||||
say so loudly when it did not.
|
||||
|
||||
Today the only patching path is **Apply updates**, which runs immediately on one
|
||||
server, installs everything, never reboots, and reports nothing back: the agent
|
||||
logs a failure locally and the control plane never hears of it.
|
||||
|
||||
Out of scope, deliberately:
|
||||
|
||||
- **Update rings / staged rollout.** A policy runs all its targets within one
|
||||
window, bounded only by `max_concurrent`. Rings are their own feature and
|
||||
build on this one.
|
||||
- **Package holds and per-package allow-lists.** Scope is `all` or `security`.
|
||||
- **Live-streamed patch output.** A capped output tail is stored per server;
|
||||
`workflow_log_lines` is not reused.
|
||||
- **Alert muting during windows.** The window is a standalone object so this can
|
||||
reference it later, but v1 does not mute anything.
|
||||
- **Licence gating.** Patching is free on every tier, consistent with
|
||||
`applyUpdates` already being exempt from the licence gate.
|
||||
|
||||
## Current state
|
||||
|
||||
- `POST /api/servers/:id/apply-updates` calls `services.DispatchApplyUpdates`,
|
||||
which sends an empty `ApplyUpdatesCmd` and returns. No result is awaited or
|
||||
recorded.
|
||||
- Agent `handleApplyUpdates` runs `updates.ApplyAll()` and, on success only,
|
||||
sends an empty `ReportUpdates`. Output is discarded (`exec.Cmd.Run()`).
|
||||
- The apt path wraps `apt-get update` **and** `apt-get upgrade` in one 5-minute
|
||||
context, so a large upgrade can be killed partway. dnf, yum, zypper, pacman
|
||||
and apk run with no timeout at all.
|
||||
- The agent never reboots. `inventory.reboot_required` is set on the 15-minute
|
||||
static snapshot, and at agent start.
|
||||
- `workflowsched` already solves scheduling: 5-field cron, IANA timezone,
|
||||
`next_run_at` persisted and claimed atomically, recorded skips, run inside
|
||||
`bus.RunAsLeader("housekeeping", ...)`.
|
||||
- `default_steps/` ships `apply_package_updates` and `reboot_server` bash steps.
|
||||
They stay as they are; this feature does not use them.
|
||||
|
||||
## Approach
|
||||
|
||||
A native patch path, not a workflow convention. The agent gains a result-bearing
|
||||
update command with a scope and an opt-in reboot. Three new collections hold
|
||||
windows, policies and runs. A new `patchsched` loop fires policies and advances
|
||||
runs from state held in MongoDB, so a leader handover mid-window loses nothing.
|
||||
The manual button and the vulnerability page move onto the same run model, so
|
||||
every patch Vantage performs has a record.
|
||||
|
||||
Rejected alternatives:
|
||||
|
||||
- **Build on the workflow engine** (script steps per package manager). The patch
|
||||
logic would exist twice, as scripts and as the agent's `updates` package; the
|
||||
engine sends one step list to every target, so a mixed Linux/Windows policy
|
||||
needs per-OS branching it does not have; and the manual button would stay
|
||||
fire-and-forget.
|
||||
- **Native command, recorded as a `WorkflowRun`.** Saves one page, but binds
|
||||
patching to `steps_snapshot` and log sequencing that do not describe it, and
|
||||
breaks the rule that a run always shows the script that ran.
|
||||
|
||||
## Data model
|
||||
|
||||
All three collections carry `instance_id` and are added to
|
||||
`services.ScopedCollections`, so instance purge covers them.
|
||||
|
||||
### `maintenance_windows`
|
||||
|
||||
```go
|
||||
type MaintenanceWindow struct {
|
||||
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
|
||||
InstanceID string `bson:"instance_id" json:"instance_id"`
|
||||
WindowID string `bson:"window_id" json:"window_id"`
|
||||
Name string `bson:"name" json:"name"`
|
||||
Cron string `bson:"cron" json:"cron"` // 5-field, window start
|
||||
TZ string `bson:"tz" json:"tz"` // IANA name
|
||||
DurationMinutes int `bson:"duration_minutes" json:"duration_minutes"` // 15..720
|
||||
CreatedAt time.Time `bson:"created_at" json:"created_at"`
|
||||
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
|
||||
}
|
||||
```
|
||||
|
||||
A window answers "when" and nothing else: no targets, no behaviour. Cron and TZ
|
||||
are validated through `workflowsched.NextOccurrence`, exactly as workflow
|
||||
schedules are. Deleting a window referenced by any policy is refused with 409
|
||||
`window_in_use`. Editing a window recomputes `next_run_at` on every policy that
|
||||
references it, in the same service call.
|
||||
|
||||
### `patch_policies`
|
||||
|
||||
```go
|
||||
type PatchPolicy struct {
|
||||
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
|
||||
InstanceID string `bson:"instance_id" json:"instance_id"`
|
||||
PolicyID string `bson:"policy_id" json:"policy_id"`
|
||||
Name string `bson:"name" json:"name"`
|
||||
Enabled bool `bson:"enabled" json:"enabled"`
|
||||
WindowID string `bson:"window_id" json:"window_id"`
|
||||
TargetServerIDs []string `bson:"target_server_ids" json:"target_server_ids"`
|
||||
TargetTags map[string]string `bson:"target_tags,omitempty" json:"target_tags,omitempty"`
|
||||
Scope string `bson:"scope" json:"scope"` // "all" | "security"
|
||||
Reboot string `bson:"reboot" json:"reboot"` // "never" | "if_required"
|
||||
MaxConcurrent int `bson:"max_concurrent" json:"max_concurrent"` // 0 = no cap
|
||||
NotifyChannelIDs []string `bson:"notify_channel_ids,omitempty" json:"notify_channel_ids,omitempty"`
|
||||
NextRunAt *time.Time `bson:"next_run_at,omitempty" json:"next_run_at,omitempty"`
|
||||
LastRunAt *time.Time `bson:"last_run_at,omitempty" json:"last_run_at,omitempty"`
|
||||
LastSkipped *Skip `bson:"last_skipped,omitempty" json:"last_skipped,omitempty"`
|
||||
DisabledReason string `bson:"disabled_reason,omitempty" json:"disabled_reason,omitempty"`
|
||||
CreatedAt time.Time `bson:"created_at" json:"created_at"`
|
||||
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
|
||||
}
|
||||
```
|
||||
|
||||
Targets use the workflow selector semantics unchanged: the distinct union of
|
||||
named servers and tag matches, resolved through `services.ResolveTargets` at
|
||||
fire time, and an empty selector matches nothing (`ErrNoTargets` on save).
|
||||
`Skip` is the existing `models.Skip`.
|
||||
|
||||
Index: `{instance_id: 1, enabled: 1, next_run_at: 1}`.
|
||||
|
||||
### `patch_runs`
|
||||
|
||||
```go
|
||||
type PatchRun struct {
|
||||
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
|
||||
InstanceID string `bson:"instance_id" json:"instance_id"`
|
||||
RunID string `bson:"run_id" json:"run_id"`
|
||||
PolicyID string `bson:"policy_id,omitempty" json:"policy_id,omitempty"` // empty for manual
|
||||
PolicyName string `bson:"policy_name,omitempty" json:"policy_name,omitempty"`
|
||||
TriggeredBy string `bson:"triggered_by" json:"triggered_by"` // "schedule" | actor | "vulnerability:<cve>"
|
||||
Scope string `bson:"scope" json:"scope"` // snapshot
|
||||
Reboot string `bson:"reboot" json:"reboot"` // snapshot
|
||||
MaxConcurrent int `bson:"max_concurrent" json:"max_concurrent"` // snapshot
|
||||
WindowEnd *time.Time `bson:"window_end,omitempty" json:"window_end,omitempty"` // nil for manual
|
||||
Status string `bson:"status" json:"status"` // running | succeeded | partial | failed | cancelled
|
||||
StartedAt time.Time `bson:"started_at" json:"started_at"`
|
||||
FinishedAt *time.Time `bson:"finished_at,omitempty" json:"finished_at,omitempty"`
|
||||
Servers []PatchServerRun `bson:"servers" json:"servers"`
|
||||
}
|
||||
|
||||
type PatchServerRun struct {
|
||||
ServerID string `bson:"server_id" json:"server_id"`
|
||||
Hostname string `bson:"hostname" json:"hostname"`
|
||||
Status string `bson:"status" json:"status"`
|
||||
CommandID string `bson:"command_id,omitempty" json:"-"`
|
||||
PendingBefore int `bson:"pending_before" json:"pending_before"`
|
||||
PendingAfter *int `bson:"pending_after,omitempty" json:"pending_after,omitempty"`
|
||||
RebootedAt *time.Time `bson:"rebooted_at,omitempty" json:"rebooted_at,omitempty"`
|
||||
VerifiedAt *time.Time `bson:"verified_at,omitempty" json:"verified_at,omitempty"`
|
||||
Output string `bson:"output,omitempty" json:"output,omitempty"` // tail, max 64KB
|
||||
Error string `bson:"error,omitempty" json:"error,omitempty"`
|
||||
StartedAt *time.Time `bson:"started_at,omitempty" json:"started_at,omitempty"`
|
||||
FinishedAt *time.Time `bson:"finished_at,omitempty" json:"finished_at,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
Server run statuses:
|
||||
|
||||
| Status | Terminal | Meaning |
|
||||
| ----------------- | -------- | -------------------------------------------------------------------- |
|
||||
| `queued` | no | Waiting for a concurrency slot |
|
||||
| `waiting_offline` | no | Agent not connected; retried each tick while the window is open |
|
||||
| `patching` | no | Command dispatched, no result yet |
|
||||
| `rebooting` | no | Agent announced a reboot; awaiting a post-boot inventory report |
|
||||
| `succeeded` | yes | Patched, and rebooted and verified if a reboot was owed and allowed |
|
||||
| `failed` | yes | Package manager failed, no result, or reboot not verified |
|
||||
| `unsupported` | yes | Security-only requested on a host with no security metadata |
|
||||
| `agent_too_old` | yes | Agent predates patch results; not dispatched |
|
||||
| `missed_offline` | yes | Offline for the whole window |
|
||||
| `window_closed` | yes | Still queued when the window ended |
|
||||
| `cancelled` | yes | Run cancelled before this server was dispatched |
|
||||
|
||||
Scope, reboot and concurrency are snapshotted onto the run so editing a policy
|
||||
never changes what a past run shows. `PendingBefore` is the server's
|
||||
`available_updates` count at dispatch; `PendingAfter` comes from the result.
|
||||
|
||||
Indexes: `{instance_id: 1, policy_id: 1, started_at: -1}`,
|
||||
`{instance_id: 1, "servers.server_id": 1, started_at: -1}`,
|
||||
`{status: 1}` (the tick's scan), and `{"servers.command_id": 1}` (result lookup).
|
||||
Runs are swept under `workflow_log_retention_days` by the existing log sweeper.
|
||||
|
||||
## Wire contract
|
||||
|
||||
Changed in `vantage-shared` (`grpc/pb` and `proto/vantage/v1/vantage.proto` in
|
||||
the same commit):
|
||||
|
||||
```proto
|
||||
message ApplyUpdatesCmd {
|
||||
string scope = 1; // "" or "all" | "security"
|
||||
bool reboot_if_required = 2;
|
||||
int64 deadline_unix = 3; // 0 = none; the agent caps at 2h
|
||||
}
|
||||
|
||||
message PatchResult { // new AgentMessage oneof variant: PatchResult patch_result = 8;
|
||||
string command_id = 1;
|
||||
string status = 2; // ok | failed | unsupported | busy
|
||||
string message = 3;
|
||||
string output_tail = 4; // at most 64KB, newest bytes kept
|
||||
int32 pending_after = 5;
|
||||
bool reboot_required = 6;
|
||||
bool rebooting = 7;
|
||||
}
|
||||
|
||||
message InventoryReport {
|
||||
// existing fields ...
|
||||
int64 boot_time_unix = 11; // every report
|
||||
}
|
||||
```
|
||||
|
||||
The existing comment on `InventoryReport.reboot_required` ("The agent never
|
||||
reboots") is reworded in the same commit to match the new reboot rule.
|
||||
|
||||
An empty `ApplyUpdatesCmd` means `scope: all`, no reboot, no deadline: exactly
|
||||
today's behaviour, so a new agent under an old server is unaffected.
|
||||
|
||||
### Old agents
|
||||
|
||||
An old agent ignores the new fields. Under `scope: security` it would install
|
||||
**everything**, and it sends no `PatchResult`. The server therefore gates on
|
||||
`servers.agent_version`:
|
||||
|
||||
- `agentSupportsPatchResults(version)` is a semver comparison against the first
|
||||
agent release carrying this feature. Empty, unparseable and dev versions are
|
||||
treated as too old.
|
||||
- A policy run marks an older server `agent_too_old` without dispatching.
|
||||
- A manual Apply updates on an older server still dispatches the empty command,
|
||||
records the server run as `succeeded` immediately with
|
||||
`error: "no result reported: agent predates patch results"`, and the UI shows
|
||||
it as unverified. The button keeps working through the transition.
|
||||
|
||||
## Agent
|
||||
|
||||
`internal/updates`:
|
||||
|
||||
- `Apply(opts ApplyOptions) (Result, error)` replaces `ApplyAll()`.
|
||||
`ApplyOptions{Scope string; Deadline time.Time}`. `Result{Output []byte;
|
||||
Unsupported bool}`.
|
||||
- A package-level mutex: a second `Apply` while one runs returns `ErrBusy`.
|
||||
- Index refresh (`apt-get update`, and nothing else) keeps its own 5-minute
|
||||
timeout. The upgrade runs under a context ending at `Deadline`, or 2h from
|
||||
start when no deadline is given. The existing single 5-minute context is
|
||||
removed.
|
||||
- Combined stdout and stderr go to a 64KB ring buffer that keeps the newest
|
||||
bytes.
|
||||
- Non-interactive everywhere: `DEBIAN_FRONTEND=noninteractive` and
|
||||
`-o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold` for
|
||||
apt, `--non-interactive` for zypper, `--noconfirm` stays for pacman.
|
||||
|
||||
Security-only, per package manager:
|
||||
|
||||
| Manager | Command | No security metadata |
|
||||
| --------- | ---------------------------------------------------------------------------------------------------- | -------------------- |
|
||||
| apt | temp sources list of entries whose suite ends in `-security`, from `/etc/apt/sources.list`, `*.list` and deb822 `*.sources`; `apt-get update` and `apt-get upgrade -y` with `-o Dir::Etc::SourceList=<tmp> -o Dir::Etc::SourceParts=-` | no such entries: `unsupported` |
|
||||
| dnf / yum | `<pm> upgrade --security -y` | n/a |
|
||||
| zypper | `zypper --non-interactive patch --category security` | n/a |
|
||||
| pacman | none | always `unsupported` |
|
||||
| apk | none | always `unsupported` |
|
||||
| Windows | existing COM search, keeping updates whose `Categories` include Security Updates (`0FA1201D-4330-4FA8-8AE9-B877473B6441`) or Critical Updates (`E6CF1350-C01B-414D-A61F-263D14D133B4`) | n/a |
|
||||
|
||||
`unsupported` installs nothing. It never falls back to `all`.
|
||||
|
||||
apt security source filtering is a pure function over file contents
|
||||
(`securitySources(files map[string]string) (list string, ok bool)`) so it is
|
||||
testable on any platform.
|
||||
|
||||
`internal/sync` `handleApplyUpdates`:
|
||||
|
||||
1. `updates.Apply(...)` with the command's scope and deadline.
|
||||
2. Re-run `CheckAvailable`, `ReportUpdates` the result (on success **and**
|
||||
failure, so counts are fresh either way), read `RebootRequired()`.
|
||||
3. If `reboot_if_required`, a reboot is owed, and at least 5 minutes remain
|
||||
before the deadline: send `PatchResult{status: ok, rebooting: true, ...}` on
|
||||
the command stream, then run `shutdown -r +1` (Linux) or
|
||||
`shutdown /r /t 60 /c "Vantage patch policy"` (Windows). The one-minute
|
||||
grace lets the result leave before the host goes down.
|
||||
4. Otherwise send `PatchResult` with the outcome.
|
||||
|
||||
`internal/inventory` reports `boot_time_unix` on every report (Linux
|
||||
`/proc/stat` `btime`, Windows via the existing collection).
|
||||
|
||||
The agent's documented promise changes from "never reboots a host" to "never
|
||||
reboots a host unless the command explicitly asks and the OS reports a reboot
|
||||
is owed". `vantage-agent/CLAUDE.md` and this repository's CLAUDE.md are updated
|
||||
in the same change.
|
||||
|
||||
## Scheduler and run lifecycle
|
||||
|
||||
### `patchsched`
|
||||
|
||||
Runs inside the existing `RunAsLeader("housekeeping", ...)`, ticking every 30s.
|
||||
It must not import `services` (the same cycle `workflowsched` avoids); its
|
||||
dependencies are injected from `main.go` as `patchsched.Deps`.
|
||||
|
||||
Each tick does two things.
|
||||
|
||||
**Fire due policies.** For each enabled policy with `next_run_at <= now`, claim
|
||||
it with the `workflowsched` pattern: `UpdateOne` matching `policy_id` **and**
|
||||
the current `next_run_at`, setting the next occurrence. A zero match means
|
||||
another process claimed it. Then `Decide(due, windowEnd, now, running, nTargets)`:
|
||||
|
||||
| Condition | Result |
|
||||
| ----------------------------------------------------- | -------------------------------------------------- |
|
||||
| window missing or no longer parses | disable policy, set `disabled_reason`, audit |
|
||||
| `now >= windowEnd` or `now > due + 1h` | skip `missed` |
|
||||
| a run of this policy has `status: running` | skip `already_running` |
|
||||
| targets resolve to zero servers | skip `no_targets` |
|
||||
| otherwise | fire |
|
||||
|
||||
Skips set `last_skipped` and write `patch.skipped`, as workflow skips do. Firing
|
||||
creates a `patch_run` with every target `queued` (or `agent_too_old`),
|
||||
`WindowEnd = due + duration`, sets `last_run_at`, and writes `patch.run_started`
|
||||
with actor `schedule`.
|
||||
|
||||
**Advance running runs.** For each `patch_run` with `status: running`, load
|
||||
connection state for its servers and apply `Advance(run, now, connected)`, a
|
||||
pure function returning the transitions:
|
||||
|
||||
- While `now < WindowEnd` (or always, for a manual run): move `queued` and
|
||||
`waiting_offline` servers to `patching` up to `MaxConcurrent` in flight
|
||||
(`patching` plus `rebooting`), dispatching `ApplyUpdatesCmd` with
|
||||
`deadline_unix = WindowEnd`. A dispatch that fails (503, agent offline) moves
|
||||
the server to `waiting_offline`.
|
||||
- When `now >= WindowEnd`: `queued` to `window_closed`, `waiting_offline` to
|
||||
`missed_offline`.
|
||||
- `patching` with no result past `deadline + 10m` (manual: start + 2h + 10m):
|
||||
`failed`, `"no result from agent"`.
|
||||
- `rebooting` with no verifying report past `RebootedAt + 20m`: `failed`,
|
||||
`"did not come back within 20 minutes"`.
|
||||
- When every server is terminal, `Finalize` sets the run status: `succeeded` if
|
||||
all servers succeeded, `failed` if none did, otherwise `partial`.
|
||||
`unsupported`, `agent_too_old`, `missed_offline` and `window_closed` are not
|
||||
successes. Then notify (below) and write `patch.run_finished`.
|
||||
|
||||
Every transition is written with a filter on the server's current status, so a
|
||||
result arriving concurrently is never overwritten by a stale tick.
|
||||
|
||||
### Results
|
||||
|
||||
`CommandStream`, on whichever pod holds the agent's stream, handles
|
||||
`PatchResult` by updating the server run found by `servers.command_id`, via
|
||||
`ApplyResult(serverRun, result, now)`:
|
||||
|
||||
- `ok` without `rebooting`: `succeeded`, `PendingAfter`, output.
|
||||
- `ok` with `rebooting`: `rebooting`, `RebootedAt = now`, write `patch.reboot`.
|
||||
- `failed` or `busy`: `failed` with the message.
|
||||
- `unsupported`: `unsupported`.
|
||||
|
||||
Nothing awaits the result on the bus. The run document is the only state, so a
|
||||
pod or leader change mid-window loses nothing. A result for an unknown command
|
||||
ID is dropped and logged.
|
||||
|
||||
### Reboot verification
|
||||
|
||||
`ReportInventory` checks, for this server, any server run in `rebooting`, and
|
||||
only on reports with `include_static` set: `reboot_required` is only computed
|
||||
on static snapshots, so a metrics-only report would read as "no reboot owed".
|
||||
The agent sends a static snapshot at start, so the first report after a reboot
|
||||
qualifies.
|
||||
`VerifyReboot(serverRun, bootTime, rebootRequired, now)`:
|
||||
|
||||
- `bootTime <= RebootedAt`: not yet rebooted, no change. This is why boot time
|
||||
is used rather than "a report arrived": a static snapshot sent during the
|
||||
one-minute grace must not count.
|
||||
- `bootTime > RebootedAt` and `reboot_required` false: `succeeded`,
|
||||
`VerifiedAt = now`.
|
||||
- `bootTime > RebootedAt` and `reboot_required` still true: `failed`,
|
||||
`"still requires a reboot after restarting"`.
|
||||
|
||||
### Manual runs
|
||||
|
||||
`POST /servers/:id/apply-updates` and the vulnerability page's Apply updates
|
||||
create a one-server `patch_run` (`scope: all`, `reboot: never`, no window,
|
||||
`TriggeredBy` the actor or `vulnerability:<cve>`) and dispatch immediately in
|
||||
the handler rather than waiting for the next tick. A dispatch failure still
|
||||
answers 503, and the run is recorded as `failed` so the attempt is not lost.
|
||||
Timeouts and finalisation go through the same tick.
|
||||
|
||||
### Cancel
|
||||
|
||||
`POST /patch-runs/:runId/cancel` moves `queued` and `waiting_offline` servers to
|
||||
`cancelled`. Servers already `patching` or `rebooting` continue: interrupting a
|
||||
package manager mid-transaction is worse than letting it finish. The run
|
||||
finalises as `cancelled` once the in-flight servers settle, unless none were in
|
||||
flight, in which case immediately.
|
||||
|
||||
### Notifications
|
||||
|
||||
When a run finalises as `partial` or `failed`, one summary goes to each channel
|
||||
in `NotifyChannelIDs` through the existing `notify` dispatch: `[Vantage] Patch
|
||||
policy "Sunday prod" partial: 38 succeeded, 2 failed, 1 missed offline`. The
|
||||
webhook payload carries the counts as fields and the run ID. A clean run sends
|
||||
nothing.
|
||||
|
||||
## REST API
|
||||
|
||||
All under `/api`, each route registered in `routeScopes` (so
|
||||
`AssertScopeMapComplete` passes) and annotated for `swag`; `openapi.json` is
|
||||
regenerated and committed.
|
||||
|
||||
```
|
||||
maintenance-windows GET,POST /maintenance-windows · GET,PUT,DELETE /maintenance-windows/:id
|
||||
POST /maintenance-windows/preview {cron,tz,duration_minutes} -> next 3 {start,end}
|
||||
patch-policies GET,POST /patch-policies · GET,PUT,DELETE /patch-policies/:id
|
||||
POST /patch-policies/:id/run-now window = now .. now + window duration
|
||||
patch-runs GET /patch-runs?policy_id=&server_id=&limit= · GET /patch-runs/:runId
|
||||
POST /patch-runs/:runId/cancel
|
||||
servers POST /servers/:id/apply-updates 202 {run_id, message}
|
||||
```
|
||||
|
||||
- New scope resource **`patching`** (`:read`, `:write`). Windows, policies and
|
||||
runs use it. `apply-updates` stays on `servers:write`.
|
||||
- Creating, editing and deleting windows and policies, and run-now, are
|
||||
owner or admin (`RequireRole`). Viewing and cancelling runs are open to every
|
||||
role. Apply updates keeps its current access.
|
||||
- Saving a policy through a tag-restricted token is refused if its selector
|
||||
reaches outside the restriction, reusing `validateWorkflowTargetScope`
|
||||
unchanged. Scheduled firing passes a nil token scope, as workflow schedules do.
|
||||
- `apply-updates` keeps `message` in its response for existing scripts and adds
|
||||
`run_id`.
|
||||
- MCP `apply_updates` returns the `run_id`. No new MCP tools.
|
||||
|
||||
Errors: `400` validation (`invalid_cron`, `invalid_tz`, `invalid_duration`,
|
||||
`invalid_scope`, `invalid_reboot`, `no_targets`), `404` unknown window, policy
|
||||
or run, `409 window_in_use`, `503` agent offline on apply-updates.
|
||||
|
||||
## Audit
|
||||
|
||||
`maintenance_window.created|updated|deleted`, `patch_policy.created|updated|deleted|disabled`,
|
||||
`patch.run_started`, `patch.run_finished`, `patch.skipped`, `patch.cancelled`,
|
||||
`patch.reboot` (one per server Vantage reboots, naming the policy),
|
||||
and the existing `updates.applied` for manual runs, now carrying the run ID.
|
||||
|
||||
## Frontend
|
||||
|
||||
`web/`, dark tokens only, no hex, pills carry shape and label.
|
||||
|
||||
- Sidebar: **Patching** in the Fleet group.
|
||||
- `/patching`, three tabs:
|
||||
- **Policies**: name, next window in its own timezone, resolved target count
|
||||
via `web/lib/targets.ts`, scope and reboot chips, last-run pill, enabled
|
||||
toggle. A disabled policy shows its `disabled_reason`.
|
||||
- **Windows**: name, schedule in words, duration, policies using it.
|
||||
- **Runs**: newest first, filter by policy.
|
||||
- **Policy editor** (modal): name; window picker with inline "New window";
|
||||
targets with `DualListBox` plus tag rows; scope radio, noting apk and pacman
|
||||
hosts report unsupported for security-only; reboot radio, where "If required"
|
||||
shows "Up to N servers may reboot during this window"; max concurrent;
|
||||
notification channels. Shows "N of M targets need an agent update" linking to
|
||||
the servers when any target is too old.
|
||||
- **Window editor** (modal): name, presets writing cron (Nightly 02:00,
|
||||
Sunday 02:00, Saturday 22:00, Monthly 1st 02:00), cron field, timezone,
|
||||
duration; next 3 occurrences from `/maintenance-windows/preview`.
|
||||
- `/patching/runs/[runId]`: header with status, trigger, window end and counts
|
||||
per status; table of servers with status pill, updates installed
|
||||
(`PendingBefore - PendingAfter`), rebooted and verified times, error; each row
|
||||
expands to the output tail on the `--well` surface. Cancel while running. The
|
||||
page polls while the run is `running`.
|
||||
- **Server detail, OS updates panel**: "Covered by *Sunday prod*, next window
|
||||
Sun 21 Sep 02:00 BST" or "Not covered by any patch policy"; last patch run
|
||||
with link. Apply updates navigates to the new run.
|
||||
- **Vulnerabilities**: Apply updates navigates to the new run.
|
||||
|
||||
## Documentation
|
||||
|
||||
`vantage-docs`:
|
||||
|
||||
- New `docs/vantage/patching.md`: windows, policies, scope per package manager
|
||||
(including unsupported), reboot rule and verification, statuses table,
|
||||
offline and window-close behaviour, agent version requirement.
|
||||
- `vantage/servers.md`: remove the "Applying updates is not scheduled or staged"
|
||||
warning, describe the run record, update the reboot sentence.
|
||||
- `vantage/vulnerabilities.md`: Apply updates creates a run; link to patching.
|
||||
- `hq/licensing-and-entitlements.md`: patching is available on every tier.
|
||||
- `reference/api-tokens.md`: the `patching` scope.
|
||||
|
||||
`vantage-app/CLAUDE.md`: new "Scheduled patching" subsystem section, updated
|
||||
"Inventory and OS updates", collection list, REST list, `ServerCommand` and
|
||||
`AgentMessage` variants. `vantage-agent/CLAUDE.md`: the reboot promise.
|
||||
|
||||
## Testing
|
||||
|
||||
The repository tests pure functions without a database, so the logic is shaped
|
||||
for that and the Mongo layer is a thin shell.
|
||||
|
||||
Server:
|
||||
|
||||
- `patchsched.Decide`: on time, late within grace, past grace, past window
|
||||
end, already running, zero targets, and a DST case (`Europe/London`, last
|
||||
Sunday of October, 01:30 start occurring twice; `NextOccurrence` behaviour is
|
||||
asserted, not assumed).
|
||||
- `patchrun.Advance`: concurrency cap never exceeded counting `rebooting`;
|
||||
offline then online inside the window dispatches; offline to window end is
|
||||
`missed_offline`; queued at window end is `window_closed`; nothing dispatched
|
||||
after window end; no-result timeout; reboot timeout; cancelled runs dispatch
|
||||
nothing; manual runs ignore window rules.
|
||||
- `patchrun.ApplyResult` for each result status.
|
||||
- `patchrun.VerifyReboot`: boot time before, equal to and after `RebootedAt`,
|
||||
with reboot still owed and cleared.
|
||||
- `patchrun.Finalize`: every mix of terminal statuses.
|
||||
- `agentSupportsPatchResults`: older, equal, newer, pre-release, empty,
|
||||
unparseable.
|
||||
- Validation of windows and policies.
|
||||
- `scopes_test` covers the new routes; a token tag-restriction case in the
|
||||
style of `workflow_target_scope_test`.
|
||||
|
||||
Agent:
|
||||
|
||||
- `securitySources`: Debian and Ubuntu `.list`, Ubuntu 24.04 deb822 `.sources`,
|
||||
commented lines, a file set with no security suites (`ok == false`).
|
||||
- Ring buffer keeps the newest 64KB.
|
||||
- Windows security category filtering, as a parser test beside `winparse_test`.
|
||||
- Manual verification on Debian 12, Ubuntu 24.04, Rocky 9 and Windows Server
|
||||
2022: security-only and all, reboot and verification, window close
|
||||
mid-queue, agent busy.
|
||||
|
||||
## Rollout
|
||||
|
||||
1. `vantage-shared`: `ApplyUpdatesCmd` fields, `PatchResult`,
|
||||
`InventoryReport.boot_time_unix`, proto and `pb` together. Release a tag.
|
||||
2. `vantage-app`: bump the pin; ship collections, indexes, `patchsched`,
|
||||
results handling, API, UI, MCP change and docs. With no new agents yet,
|
||||
policies show every target as `agent_too_old` and the manual button works as
|
||||
before, now with a run record.
|
||||
3. `vantage-agent`: bump the pin, ship the agent changes, tag `agent/v*`. Set
|
||||
that version as the gate constant in the server in step 2 (the gate names a
|
||||
version that does not exist yet until step 3 ships, which is harmless: every
|
||||
agent reads as too old until it does).
|
||||
4. `vantage-docs`: publish the patching page.
|
||||
Reference in New Issue
Block a user