Compare commits
18
Commits
2b4b630ae0
...
3f2d20868e
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3f2d20868e | ||
|
|
3ecea7c39f | ||
|
|
68c613fd40 | ||
|
|
70c239021d | ||
|
|
3a7618f82f | ||
|
|
7809419202 | ||
|
|
59e7ef63fe | ||
|
|
b5bbf28c63 | ||
|
|
17c9f813fc | ||
|
|
139658864b | ||
|
|
1bb2ba7f2b | ||
|
|
fef886c93b | ||
|
|
3a1614066e | ||
|
|
c0e26d0493 | ||
|
|
b60daf0461 | ||
|
|
8f1ea6d5a0 | ||
|
|
e63e773cda | ||
|
|
48116bf737 |
@@ -173,6 +173,51 @@ Skips are recorded and surfaced, not just logged: past the 1h grace window is
|
||||
`missed`, an active run is `already_running`, and a schedule that no longer
|
||||
parses is disabled rather than left spinning the loop every 30 seconds forever.
|
||||
|
||||
### Scheduled patching
|
||||
|
||||
Four collections: `maintenance_windows` (cron start, IANA zone, duration),
|
||||
`patch_policies` (selector, window, `all|security`, `never|if_required`,
|
||||
concurrency cap, channels), `patch_runs` (one per firing or manual Apply
|
||||
updates, one `servers[]` entry per target) and `patch_run_outputs` (one per
|
||||
run and server, holding the package manager's output tail). The output lives
|
||||
apart from the run because a large run with up to 64KB per server would pass
|
||||
MongoDB's 16MB document limit; `GetPatchRun` fills `servers[].output` back in
|
||||
memory so the API shape is unchanged, and the tick paths never read it. All
|
||||
four are in `ScopedCollections`.
|
||||
|
||||
**Runs are driven by database state, not goroutines.** A run can last hours; a
|
||||
goroutine-driven run is stranded at `running` when its pod dies. `patchsched`
|
||||
ticks every 30s inside the housekeeping leader: it claims due policies with the
|
||||
workflowsched `next_run_at` pattern, then advances every running run. Every
|
||||
decision is a pure function in `internal/patchrun` (`Advance`, `ApplyResult`,
|
||||
`VerifyReboot`, `Finalize`) and every write is guarded by the server run's
|
||||
current status, so a result landing mid-tick is never overwritten.
|
||||
|
||||
**The window end never kills a package manager.** No server is dispatched in
|
||||
the last `patchrun.LatestStartBeforeEnd` (15 minutes) of a window; queued and
|
||||
waiting servers close at the window end as before. A server already patching
|
||||
may finish past the end: its no-result timeout is its own dispatch time plus
|
||||
`ManualTimeout` (2h, the agent's backstop) plus `ResultGrace` (20 minutes), for
|
||||
windowed and manual runs alike. `RebootTimeout` is 45 minutes.
|
||||
|
||||
**Results do not cross the bus.** The pod holding the agent's stream writes
|
||||
`PatchResult` straight into the run, found by `servers.command_id` and the
|
||||
agent's own server ID. A reboot is settled by the first static inventory report
|
||||
whose boot time differs from (is later than) `boot_time_before`, the
|
||||
`inventory.boot_time` recorded when the server moved to rebooting, so host and
|
||||
server clock skew does not matter. Without `boot_time_before` the report's boot
|
||||
time must be later than `rebooted_at`. A report during the one-minute grace
|
||||
does not count either way.
|
||||
|
||||
**Old agents must never receive a scope.** An agent before
|
||||
`patchrun.MinAgentVersion` ignores `scope` and installs everything, so policy
|
||||
runs mark it `agent_too_old` and do not dispatch. A manual Apply updates still
|
||||
sends such an agent the empty command and records "no result reported".
|
||||
|
||||
The next window starts after `max(now, windowEnd)`, so windows never overlap,
|
||||
including across a daylight-saving fall-back. `patchsched` must not import
|
||||
`services`; its dependencies are injected from `main.go`.
|
||||
|
||||
### Server tags and workflow targeting
|
||||
|
||||
A server carries `tags map[string]string` - lowercase `[a-z0-9_-]`, key ≤32,
|
||||
@@ -337,10 +382,12 @@ need a PowerShell Gallery install on every host and fails on an air-gapped
|
||||
fleet. `CurrentVersion` is empty on Windows and `NewVersion` carries the KB
|
||||
article ID: a Windows update is not a version bump of a named package.
|
||||
|
||||
**The agent never reboots a host.** `ApplyUpdatesCmd` installs and stops there;
|
||||
`inventory.reboot_required` reports that one is owed, set on the static snapshot
|
||||
every 15 minutes. Linux fills it too, from `/var/run/reboot-required` or
|
||||
`dnf needs-restarting -r`.
|
||||
**The agent reboots a host only when a patch command asks and a reboot is
|
||||
owed.** `ApplyUpdatesCmd` carries `scope`, `reboot_if_required` and
|
||||
`deadline_unix`; an empty command still means "everything, no reboot". The
|
||||
agent answers with `PatchResult` and, when rebooting, sends it first and then
|
||||
restarts after a one-minute grace. `inventory.reboot_required` is still set on
|
||||
the static snapshot every 15 minutes and at agent start.
|
||||
|
||||
### Package inventory and CVE findings
|
||||
|
||||
@@ -887,7 +934,7 @@ service Vantage {
|
||||
}
|
||||
```
|
||||
|
||||
`CommandStream` is the only streaming RPC: the agent authenticates once with `AgentReady`, then the server pushes `ServerCommand`s and the agent replies with `CommandResult`, `StepResult`, or `StepOutputChunk`.
|
||||
`CommandStream` is the only streaming RPC: the agent authenticates once with `AgentReady`, then the server pushes `ServerCommand`s and the agent replies with `CommandResult`, `StepResult`, `StepOutputChunk`, or `PatchResult`. `AgentMessage` now also carries `PatchResult`, the answer to `ApplyUpdatesCmd`.
|
||||
|
||||
`ServerCommand` variants: `GenerateKeyCmd`, `DeleteKeyCmd`, `UpdateAgentCmd`, `ApplyUpdatesCmd`, `RunStepCmd`, `CleanupWorkspaceCmd`, `OpenProxyCmd`, `PingCmd`, `RefreshWorkloadsCmd`, `ControlWorkloadCmd`,
|
||||
`WorkloadLogsCmd`.
|
||||
@@ -955,6 +1002,11 @@ status-pages GET,POST /status-pages · GET,PUT,DELETE /status-pages/:pageId (ow
|
||||
GET,POST /status-pages/:pageId/incidents
|
||||
PUT,DELETE /status-pages/:pageId/incidents/:incidentId
|
||||
POST /status-pages/:pageId/incidents/:incidentId/updates
|
||||
patching GET,POST /maintenance-windows · POST /maintenance-windows/preview
|
||||
GET,PUT,DELETE /maintenance-windows/:id (writes: owner|admin)
|
||||
GET,POST /patch-policies · GET,PUT,DELETE /patch-policies/:id
|
||||
POST /patch-policies/:id/run-now (writes: owner|admin)
|
||||
GET /patch-runs · GET /patch-runs/:runId · POST /patch-runs/:runId/cancel
|
||||
audit GET /audit
|
||||
agent GET /agent/latest-version
|
||||
settings GET,PUT /settings · POST /settings/secrets-token (owner|admin)
|
||||
@@ -1015,7 +1067,7 @@ plane, each of which this codebase enforces:
|
||||
|
||||
## MongoDB Collections
|
||||
|
||||
`servers` · `keys` · `assignments` · `orgs` · `users` · `auth_providers` · `settings` · `secrets` · `workflows` · `workflow_steps` · `workflow_runs` · `workflow_log_lines` · `workflow_log_seq` · `monitors` · `incidents` · `monitor_rollups` · `notification_channels` · `console_sessions` · `audit_logs` · `server_packages` · `vuln_findings` · `vuln_alert_rules` · `vulndb_meta` · `server_workloads` · `api_tokens` · `status_pages` · `status_incidents` · `migrations`
|
||||
`servers` · `keys` · `assignments` · `orgs` · `users` · `auth_providers` · `settings` · `secrets` · `workflows` · `workflow_steps` · `workflow_runs` · `workflow_log_lines` · `workflow_log_seq` · `monitors` · `incidents` · `monitor_rollups` · `notification_channels` · `console_sessions` · `audit_logs` · `server_packages` · `vuln_findings` · `vuln_alert_rules` · `vulndb_meta` · `server_workloads` · `api_tokens` · `status_pages` · `status_incidents` · `maintenance_windows` · `patch_policies` · `patch_runs` · `patch_run_outputs` · `migrations`
|
||||
|
||||
Every document except `migrations` carries `org_id`. Struct definitions are the source of truth - see `server/internal/models/`.
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,539 @@
|
||||
# Maintenance windows and scheduled patching
|
||||
|
||||
Date: 2026-09-14
|
||||
|
||||
## Goal
|
||||
|
||||
Let an operator say "install security updates on every `env:prod` server,
|
||||
Sundays 02:00 to 04:00 Europe/London, and reboot them if the OS says a reboot
|
||||
is owed", and have Vantage do it, report exactly what happened per server, and
|
||||
say so loudly when it did not.
|
||||
|
||||
Today the only patching path is **Apply updates**, which runs immediately on one
|
||||
server, installs everything, never reboots, and reports nothing back: the agent
|
||||
logs a failure locally and the control plane never hears of it.
|
||||
|
||||
Out of scope, deliberately:
|
||||
|
||||
- **Update rings / staged rollout.** A policy runs all its targets within one
|
||||
window, bounded only by `max_concurrent`. Rings are their own feature and
|
||||
build on this one.
|
||||
- **Package holds and per-package allow-lists.** Scope is `all` or `security`.
|
||||
- **Live-streamed patch output.** A capped output tail is stored per server;
|
||||
`workflow_log_lines` is not reused.
|
||||
- **Alert muting during windows.** The window is a standalone object so this can
|
||||
reference it later, but v1 does not mute anything.
|
||||
- **Licence gating.** Patching is free on every tier, consistent with
|
||||
`applyUpdates` already being exempt from the licence gate.
|
||||
|
||||
## Current state
|
||||
|
||||
- `POST /api/servers/:id/apply-updates` calls `services.DispatchApplyUpdates`,
|
||||
which sends an empty `ApplyUpdatesCmd` and returns. No result is awaited or
|
||||
recorded.
|
||||
- Agent `handleApplyUpdates` runs `updates.ApplyAll()` and, on success only,
|
||||
sends an empty `ReportUpdates`. Output is discarded (`exec.Cmd.Run()`).
|
||||
- The apt path wraps `apt-get update` **and** `apt-get upgrade` in one 5-minute
|
||||
context, so a large upgrade can be killed partway. dnf, yum, zypper, pacman
|
||||
and apk run with no timeout at all.
|
||||
- The agent never reboots. `inventory.reboot_required` is set on the 15-minute
|
||||
static snapshot, and at agent start.
|
||||
- `workflowsched` already solves scheduling: 5-field cron, IANA timezone,
|
||||
`next_run_at` persisted and claimed atomically, recorded skips, run inside
|
||||
`bus.RunAsLeader("housekeeping", ...)`.
|
||||
- `default_steps/` ships `apply_package_updates` and `reboot_server` bash steps.
|
||||
They stay as they are; this feature does not use them.
|
||||
|
||||
## Approach
|
||||
|
||||
A native patch path, not a workflow convention. The agent gains a result-bearing
|
||||
update command with a scope and an opt-in reboot. Three new collections hold
|
||||
windows, policies and runs. A new `patchsched` loop fires policies and advances
|
||||
runs from state held in MongoDB, so a leader handover mid-window loses nothing.
|
||||
The manual button and the vulnerability page move onto the same run model, so
|
||||
every patch Vantage performs has a record.
|
||||
|
||||
Rejected alternatives:
|
||||
|
||||
- **Build on the workflow engine** (script steps per package manager). The patch
|
||||
logic would exist twice, as scripts and as the agent's `updates` package; the
|
||||
engine sends one step list to every target, so a mixed Linux/Windows policy
|
||||
needs per-OS branching it does not have; and the manual button would stay
|
||||
fire-and-forget.
|
||||
- **Native command, recorded as a `WorkflowRun`.** Saves one page, but binds
|
||||
patching to `steps_snapshot` and log sequencing that do not describe it, and
|
||||
breaks the rule that a run always shows the script that ran.
|
||||
|
||||
## Data model
|
||||
|
||||
All three collections carry `instance_id` and are added to
|
||||
`services.ScopedCollections`, so instance purge covers them.
|
||||
|
||||
### `maintenance_windows`
|
||||
|
||||
```go
|
||||
type MaintenanceWindow struct {
|
||||
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
|
||||
InstanceID string `bson:"instance_id" json:"instance_id"`
|
||||
WindowID string `bson:"window_id" json:"window_id"`
|
||||
Name string `bson:"name" json:"name"`
|
||||
Cron string `bson:"cron" json:"cron"` // 5-field, window start
|
||||
TZ string `bson:"tz" json:"tz"` // IANA name
|
||||
DurationMinutes int `bson:"duration_minutes" json:"duration_minutes"` // 15..720
|
||||
CreatedAt time.Time `bson:"created_at" json:"created_at"`
|
||||
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
|
||||
}
|
||||
```
|
||||
|
||||
A window answers "when" and nothing else: no targets, no behaviour. Cron and TZ
|
||||
are validated through `workflowsched.NextOccurrence`, exactly as workflow
|
||||
schedules are. Deleting a window referenced by any policy is refused with 409
|
||||
`window_in_use`. Editing a window recomputes `next_run_at` on every policy that
|
||||
references it, in the same service call.
|
||||
|
||||
### `patch_policies`
|
||||
|
||||
```go
|
||||
type PatchPolicy struct {
|
||||
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
|
||||
InstanceID string `bson:"instance_id" json:"instance_id"`
|
||||
PolicyID string `bson:"policy_id" json:"policy_id"`
|
||||
Name string `bson:"name" json:"name"`
|
||||
Enabled bool `bson:"enabled" json:"enabled"`
|
||||
WindowID string `bson:"window_id" json:"window_id"`
|
||||
TargetServerIDs []string `bson:"target_server_ids" json:"target_server_ids"`
|
||||
TargetTags map[string]string `bson:"target_tags,omitempty" json:"target_tags,omitempty"`
|
||||
Scope string `bson:"scope" json:"scope"` // "all" | "security"
|
||||
Reboot string `bson:"reboot" json:"reboot"` // "never" | "if_required"
|
||||
MaxConcurrent int `bson:"max_concurrent" json:"max_concurrent"` // 0 = no cap
|
||||
NotifyChannelIDs []string `bson:"notify_channel_ids,omitempty" json:"notify_channel_ids,omitempty"`
|
||||
NextRunAt *time.Time `bson:"next_run_at,omitempty" json:"next_run_at,omitempty"`
|
||||
LastRunAt *time.Time `bson:"last_run_at,omitempty" json:"last_run_at,omitempty"`
|
||||
LastSkipped *Skip `bson:"last_skipped,omitempty" json:"last_skipped,omitempty"`
|
||||
DisabledReason string `bson:"disabled_reason,omitempty" json:"disabled_reason,omitempty"`
|
||||
CreatedAt time.Time `bson:"created_at" json:"created_at"`
|
||||
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
|
||||
}
|
||||
```
|
||||
|
||||
Targets use the workflow selector semantics unchanged: the distinct union of
|
||||
named servers and tag matches, resolved through `services.ResolveTargets` at
|
||||
fire time, and an empty selector matches nothing (`ErrNoTargets` on save).
|
||||
`Skip` is the existing `models.Skip`.
|
||||
|
||||
Index: `{instance_id: 1, enabled: 1, next_run_at: 1}`.
|
||||
|
||||
### `patch_runs`
|
||||
|
||||
```go
|
||||
type PatchRun struct {
|
||||
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
|
||||
InstanceID string `bson:"instance_id" json:"instance_id"`
|
||||
RunID string `bson:"run_id" json:"run_id"`
|
||||
PolicyID string `bson:"policy_id,omitempty" json:"policy_id,omitempty"` // empty for manual
|
||||
PolicyName string `bson:"policy_name,omitempty" json:"policy_name,omitempty"`
|
||||
TriggeredBy string `bson:"triggered_by" json:"triggered_by"` // "schedule" | actor
|
||||
Source string `bson:"source" json:"source"` // schedule | server | vulnerabilities | mcp | run_now
|
||||
CancelledAt *time.Time `bson:"cancelled_at,omitempty" json:"cancelled_at,omitempty"`
|
||||
Scope string `bson:"scope" json:"scope"` // snapshot
|
||||
Reboot string `bson:"reboot" json:"reboot"` // snapshot
|
||||
MaxConcurrent int `bson:"max_concurrent" json:"max_concurrent"` // snapshot
|
||||
WindowEnd *time.Time `bson:"window_end,omitempty" json:"window_end,omitempty"` // nil for manual
|
||||
Status string `bson:"status" json:"status"` // running | succeeded | partial | failed | cancelled
|
||||
StartedAt time.Time `bson:"started_at" json:"started_at"`
|
||||
FinishedAt *time.Time `bson:"finished_at,omitempty" json:"finished_at,omitempty"`
|
||||
Servers []PatchServerRun `bson:"servers" json:"servers"`
|
||||
}
|
||||
|
||||
type PatchServerRun struct {
|
||||
ServerID string `bson:"server_id" json:"server_id"`
|
||||
Hostname string `bson:"hostname" json:"hostname"`
|
||||
Status string `bson:"status" json:"status"`
|
||||
CommandID string `bson:"command_id,omitempty" json:"-"`
|
||||
PendingBefore int `bson:"pending_before" json:"pending_before"`
|
||||
PendingAfter *int `bson:"pending_after,omitempty" json:"pending_after,omitempty"`
|
||||
RebootedAt *time.Time `bson:"rebooted_at,omitempty" json:"rebooted_at,omitempty"`
|
||||
VerifiedAt *time.Time `bson:"verified_at,omitempty" json:"verified_at,omitempty"`
|
||||
Output string `bson:"output,omitempty" json:"output,omitempty"` // tail, max 64KB
|
||||
Error string `bson:"error,omitempty" json:"error,omitempty"`
|
||||
StartedAt *time.Time `bson:"started_at,omitempty" json:"started_at,omitempty"`
|
||||
FinishedAt *time.Time `bson:"finished_at,omitempty" json:"finished_at,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
Server run statuses:
|
||||
|
||||
| Status | Terminal | Meaning |
|
||||
| ----------------- | -------- | -------------------------------------------------------------------- |
|
||||
| `queued` | no | Waiting for a concurrency slot |
|
||||
| `waiting_offline` | no | Agent not connected; retried each tick while the window is open |
|
||||
| `patching` | no | Command dispatched, no result yet |
|
||||
| `rebooting` | no | Agent announced a reboot; awaiting a post-boot inventory report |
|
||||
| `succeeded` | yes | Patched, and rebooted and verified if a reboot was owed and allowed |
|
||||
| `failed` | yes | Package manager failed, no result, or reboot not verified |
|
||||
| `unsupported` | yes | Security-only requested on a host with no security metadata |
|
||||
| `agent_too_old` | yes | Agent predates patch results; not dispatched |
|
||||
| `missed_offline` | yes | Offline for the whole window |
|
||||
| `window_closed` | yes | Still queued when the window ended |
|
||||
| `cancelled` | yes | Run cancelled before this server was dispatched |
|
||||
|
||||
Scope, reboot and concurrency are snapshotted onto the run so editing a policy
|
||||
never changes what a past run shows. `PendingBefore` is the server's
|
||||
`available_updates` count at dispatch; `PendingAfter` comes from the result.
|
||||
|
||||
Indexes: `{instance_id: 1, policy_id: 1, started_at: -1}`,
|
||||
`{instance_id: 1, "servers.server_id": 1, started_at: -1}`,
|
||||
`{status: 1}` (the tick's scan), and `{"servers.command_id": 1}` (result lookup).
|
||||
Runs are swept under `workflow_log_retention_days` by the existing log sweeper.
|
||||
|
||||
## Wire contract
|
||||
|
||||
Changed in `vantage-shared` (`grpc/pb` and `proto/vantage/v1/vantage.proto` in
|
||||
the same commit):
|
||||
|
||||
```proto
|
||||
message ApplyUpdatesCmd {
|
||||
string scope = 1; // "" or "all" | "security"
|
||||
bool reboot_if_required = 2;
|
||||
int64 deadline_unix = 3; // 0 = none; the agent caps at 2h
|
||||
}
|
||||
|
||||
message PatchResult { // new AgentMessage oneof variant: PatchResult patch_result = 8;
|
||||
string command_id = 1;
|
||||
string status = 2; // ok | failed | unsupported | busy
|
||||
string message = 3;
|
||||
string output_tail = 4; // at most 64KB, newest bytes kept
|
||||
int32 pending_after = 5;
|
||||
bool reboot_required = 6;
|
||||
bool rebooting = 7;
|
||||
}
|
||||
|
||||
message InventoryReport {
|
||||
// existing fields ...
|
||||
int64 boot_time_unix = 11; // every report
|
||||
}
|
||||
```
|
||||
|
||||
The existing comment on `InventoryReport.reboot_required` ("The agent never
|
||||
reboots") is reworded in the same commit to match the new reboot rule.
|
||||
|
||||
An empty `ApplyUpdatesCmd` means `scope: all`, no reboot, no deadline: exactly
|
||||
today's behaviour, so a new agent under an old server is unaffected.
|
||||
|
||||
### Old agents
|
||||
|
||||
An old agent ignores the new fields. Under `scope: security` it would install
|
||||
**everything**, and it sends no `PatchResult`. The server therefore gates on
|
||||
`servers.agent_version`:
|
||||
|
||||
- `agentSupportsPatchResults(version)` is a semver comparison against the first
|
||||
agent release carrying this feature. Empty, unparseable and dev versions are
|
||||
treated as too old.
|
||||
- A policy run marks an older server `agent_too_old` without dispatching.
|
||||
- A manual Apply updates on an older server still dispatches the empty command,
|
||||
records the server run as `succeeded` immediately with
|
||||
`error: "no result reported: agent predates patch results"`, and the UI shows
|
||||
it as unverified. The button keeps working through the transition.
|
||||
|
||||
## Agent
|
||||
|
||||
`internal/updates`:
|
||||
|
||||
- `Apply(opts ApplyOptions) (Result, error)` replaces `ApplyAll()`.
|
||||
`ApplyOptions{Scope string; Deadline time.Time}`. `Result{Output []byte;
|
||||
Unsupported bool}`.
|
||||
- A package-level mutex: a second `Apply` while one runs returns `ErrBusy`.
|
||||
- Index refresh (`apt-get update`, and nothing else) keeps its own 5-minute
|
||||
timeout. The upgrade runs under a context ending at `Deadline`, or 2h from
|
||||
start when no deadline is given. The existing single 5-minute context is
|
||||
removed.
|
||||
- Combined stdout and stderr go to a 64KB ring buffer that keeps the newest
|
||||
bytes.
|
||||
- Non-interactive everywhere: `DEBIAN_FRONTEND=noninteractive` and
|
||||
`-o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold` for
|
||||
apt, `--non-interactive` for zypper, `--noconfirm` stays for pacman.
|
||||
|
||||
Security-only, per package manager:
|
||||
|
||||
| Manager | Command | No security metadata |
|
||||
| --------- | ---------------------------------------------------------------------------------------------------- | -------------------- |
|
||||
| apt | temp sources list of entries whose suite ends in `-security`, from `/etc/apt/sources.list`, `*.list` and deb822 `*.sources`; `apt-get update` and `apt-get upgrade -y` with `-o Dir::Etc::SourceList=<tmp> -o Dir::Etc::SourceParts=-` | no such entries: `unsupported` |
|
||||
| dnf / yum | `<pm> upgrade --security -y` | n/a |
|
||||
| zypper | `zypper --non-interactive patch --category security` | n/a |
|
||||
| pacman | none | always `unsupported` |
|
||||
| apk | none | always `unsupported` |
|
||||
| Windows | existing COM search, keeping updates whose `Categories` include Security Updates (`0FA1201D-4330-4FA8-8AE9-B877473B6441`) or Critical Updates (`E6CF1350-C01B-414D-A61F-263D14D133B4`) | n/a |
|
||||
|
||||
`unsupported` installs nothing. It never falls back to `all`.
|
||||
|
||||
apt security source filtering is a pure function over file contents
|
||||
(`securitySources(files map[string]string) (list string, ok bool)`) so it is
|
||||
testable on any platform.
|
||||
|
||||
`internal/sync` `handleApplyUpdates`:
|
||||
|
||||
1. `updates.Apply(...)` with the command's scope and deadline.
|
||||
2. Re-run `CheckAvailable`, `ReportUpdates` the result (on success **and**
|
||||
failure, so counts are fresh either way), read `RebootRequired()`.
|
||||
3. If `reboot_if_required`, a reboot is owed, and at least 5 minutes remain
|
||||
before the deadline: send `PatchResult{status: ok, rebooting: true, ...}` on
|
||||
the command stream, then run `shutdown -r +1` (Linux) or
|
||||
`shutdown /r /t 60 /c "Vantage patch policy"` (Windows). The one-minute
|
||||
grace lets the result leave before the host goes down.
|
||||
4. Otherwise send `PatchResult` with the outcome.
|
||||
|
||||
`internal/inventory` reports `boot_time_unix` on every report (Linux
|
||||
`/proc/stat` `btime`, Windows via the existing collection).
|
||||
|
||||
The agent's documented promise changes from "never reboots a host" to "never
|
||||
reboots a host unless the command explicitly asks and the OS reports a reboot
|
||||
is owed". `vantage-agent/CLAUDE.md` and this repository's CLAUDE.md are updated
|
||||
in the same change.
|
||||
|
||||
## Scheduler and run lifecycle
|
||||
|
||||
### `patchsched`
|
||||
|
||||
Runs inside the existing `RunAsLeader("housekeeping", ...)`, ticking every 30s.
|
||||
It must not import `services` (the same cycle `workflowsched` avoids); its
|
||||
dependencies are injected from `main.go` as `patchsched.Deps`.
|
||||
|
||||
Each tick does two things.
|
||||
|
||||
**Fire due policies.** For each enabled policy with `next_run_at <= now`, claim
|
||||
it with the `workflowsched` pattern: `UpdateOne` matching `policy_id` **and**
|
||||
the current `next_run_at`, setting the next occurrence. A zero match means
|
||||
another process claimed it. Then `Decide(due, windowEnd, now, running, nTargets)`:
|
||||
|
||||
| Condition | Result |
|
||||
| ----------------------------------------------------- | -------------------------------------------------- |
|
||||
| window missing or no longer parses | disable policy, set `disabled_reason`, audit |
|
||||
| `now >= windowEnd` or `now > due + 1h` | skip `missed` |
|
||||
| a run of this policy has `status: running` | skip `already_running` |
|
||||
| targets resolve to zero servers | skip `no_targets` |
|
||||
| otherwise | fire |
|
||||
|
||||
Skips set `last_skipped` and write `patch.skipped`, as workflow skips do. Firing
|
||||
creates a `patch_run` with every target `queued` (or `agent_too_old`),
|
||||
`WindowEnd = due + duration`, sets `last_run_at`, and writes `patch.run_started`
|
||||
with actor `schedule`.
|
||||
|
||||
**Advance running runs.** For each `patch_run` with `status: running`, load
|
||||
connection state for its servers and apply `Advance(run, now, connected)`, a
|
||||
pure function returning the transitions:
|
||||
|
||||
- While `now < WindowEnd` (or always, for a manual run): move `queued` and
|
||||
`waiting_offline` servers to `patching` up to `MaxConcurrent` in flight
|
||||
(`patching` plus `rebooting`), dispatching `ApplyUpdatesCmd` with
|
||||
`deadline_unix = WindowEnd`. A dispatch that fails (503, agent offline) moves
|
||||
the server to `waiting_offline`.
|
||||
- When `now >= WindowEnd`: `queued` to `window_closed`, `waiting_offline` to
|
||||
`missed_offline`.
|
||||
- `patching` with no result past `deadline + 10m` (manual: start + 2h + 10m):
|
||||
`failed`, `"no result from agent"`.
|
||||
- `rebooting` with no verifying report past `RebootedAt + 20m`: `failed`,
|
||||
`"did not come back within 20 minutes"`.
|
||||
- When every server is terminal, `Finalize` sets the run status: `succeeded` if
|
||||
all servers succeeded, `failed` if none did, otherwise `partial`.
|
||||
`unsupported`, `agent_too_old`, `missed_offline` and `window_closed` are not
|
||||
successes. Then notify (below) and write `patch.run_finished`.
|
||||
|
||||
Every transition is written with a filter on the server's current status, so a
|
||||
result arriving concurrently is never overwritten by a stale tick.
|
||||
|
||||
### Results
|
||||
|
||||
`CommandStream`, on whichever pod holds the agent's stream, handles
|
||||
`PatchResult` by updating the server run found by `servers.command_id`, via
|
||||
`ApplyResult(serverRun, result, now)`:
|
||||
|
||||
- `ok` without `rebooting`: `succeeded`, `PendingAfter`, output.
|
||||
- `ok` with `rebooting`: `rebooting`, `RebootedAt = now`, write `patch.reboot`.
|
||||
- `failed` or `busy`: `failed` with the message.
|
||||
- `unsupported`: `unsupported`.
|
||||
|
||||
Nothing awaits the result on the bus. The run document is the only state, so a
|
||||
pod or leader change mid-window loses nothing. A result for an unknown command
|
||||
ID is dropped and logged.
|
||||
|
||||
### Reboot verification
|
||||
|
||||
`ReportInventory` checks, for this server, any server run in `rebooting`, and
|
||||
only on reports with `include_static` set: `reboot_required` is only computed
|
||||
on static snapshots, so a metrics-only report would read as "no reboot owed".
|
||||
The agent sends a static snapshot at start, so the first report after a reboot
|
||||
qualifies.
|
||||
`VerifyReboot(serverRun, bootTime, rebootRequired, now)`:
|
||||
|
||||
- `bootTime <= RebootedAt`: not yet rebooted, no change. This is why boot time
|
||||
is used rather than "a report arrived": a static snapshot sent during the
|
||||
one-minute grace must not count.
|
||||
- `bootTime > RebootedAt` and `reboot_required` false: `succeeded`,
|
||||
`VerifiedAt = now`.
|
||||
- `bootTime > RebootedAt` and `reboot_required` still true: `failed`,
|
||||
`"still requires a reboot after restarting"`.
|
||||
|
||||
### Manual runs
|
||||
|
||||
`POST /servers/:id/apply-updates` and the vulnerability page's Apply updates
|
||||
create a one-server `patch_run` (`scope: all`, `reboot: never`, no window,
|
||||
`TriggeredBy` the actor, `Source` one of `server`, `vulnerabilities` or `mcp`)
|
||||
and dispatch immediately in
|
||||
the handler rather than waiting for the next tick. A dispatch failure still
|
||||
answers 503, and the run is recorded as `failed` so the attempt is not lost.
|
||||
Timeouts and finalisation go through the same tick.
|
||||
|
||||
### Cancel
|
||||
|
||||
`POST /patch-runs/:runId/cancel` moves `queued` and `waiting_offline` servers to
|
||||
`cancelled`. Servers already `patching` or `rebooting` continue: interrupting a
|
||||
package manager mid-transaction is worse than letting it finish. The run
|
||||
finalises as `cancelled` once the in-flight servers settle, unless none were in
|
||||
flight, in which case immediately.
|
||||
|
||||
### Notifications
|
||||
|
||||
When a run finalises as `partial` or `failed`, one summary goes to each channel
|
||||
in `NotifyChannelIDs` through the existing `notify` dispatch, as a new event
|
||||
type `patch`: `[Vantage] Patch policy "Sunday prod" partial: 38 succeeded,
|
||||
2 failed, 1 missed offline (run 3f2a...)`. The webhook payload is the existing
|
||||
event shape with that summary as its message; no new payload fields. A clean
|
||||
run sends nothing.
|
||||
|
||||
## REST API
|
||||
|
||||
All under `/api`, each route registered in `routeScopes` (so
|
||||
`AssertScopeMapComplete` passes) and annotated for `swag`; `openapi.json` is
|
||||
regenerated and committed.
|
||||
|
||||
```
|
||||
maintenance-windows GET,POST /maintenance-windows · GET,PUT,DELETE /maintenance-windows/:id
|
||||
POST /maintenance-windows/preview {cron,tz,duration_minutes} -> next 3 {start,end}
|
||||
patch-policies GET,POST /patch-policies · GET,PUT,DELETE /patch-policies/:id
|
||||
POST /patch-policies/:id/run-now window = now .. now + window duration
|
||||
patch-runs GET /patch-runs?policy_id=&server_id=&limit= · GET /patch-runs/:runId
|
||||
POST /patch-runs/:runId/cancel
|
||||
servers POST /servers/:id/apply-updates 202 {run_id, message}
|
||||
```
|
||||
|
||||
- New scope resource **`patching`** (`:read`, `:write`). Windows, policies and
|
||||
runs use it. `apply-updates` stays on `servers:write`.
|
||||
- Creating, editing and deleting windows and policies, and run-now, are
|
||||
owner or admin (`RequireRole`). Viewing and cancelling runs are open to every
|
||||
role. Apply updates keeps its current access.
|
||||
- Saving a policy through a tag-restricted token is refused if its selector
|
||||
reaches outside the restriction, reusing `validateWorkflowTargetScope`
|
||||
unchanged. Scheduled firing passes a nil token scope, as workflow schedules do.
|
||||
- `apply-updates` keeps `message` in its response for existing scripts and adds
|
||||
`run_id`.
|
||||
- MCP `apply_updates` returns the `run_id`. No new MCP tools.
|
||||
|
||||
Errors: `400` validation (`invalid_cron`, `invalid_tz`, `invalid_duration`,
|
||||
`invalid_scope`, `invalid_reboot`, `no_targets`), `404` unknown window, policy
|
||||
or run, `409 window_in_use`, `503` agent offline on apply-updates.
|
||||
|
||||
## Audit
|
||||
|
||||
Every event uses the `patch.` prefix so the audit page groups them under one
|
||||
category: `patch.window_created|window_updated|window_deleted`,
|
||||
`patch.policy_created|policy_updated|policy_deleted|policy_disabled`,
|
||||
`patch.run_started`, `patch.run_finished`, `patch.skipped`, `patch.cancelled`,
|
||||
`patch.reboot` (one per server Vantage reboots, naming the policy),
|
||||
and the existing `updates.applied` for manual runs, now carrying the run ID.
|
||||
|
||||
## Frontend
|
||||
|
||||
`web/`, dark tokens only, no hex, pills carry shape and label.
|
||||
|
||||
- Sidebar: **Patching** in the Fleet group.
|
||||
- `/patching`, three tabs:
|
||||
- **Policies**: name, next window in its own timezone, resolved target count
|
||||
via `web/lib/targets.ts`, scope and reboot chips, last-run pill, enabled
|
||||
toggle. A disabled policy shows its `disabled_reason`.
|
||||
- **Windows**: name, schedule in words, duration, policies using it.
|
||||
- **Runs**: newest first, filter by policy.
|
||||
- **Policy editor** (modal): name; window picker with inline "New window";
|
||||
targets with `DualListBox` plus tag rows; scope radio, noting apk and pacman
|
||||
hosts report unsupported for security-only; reboot radio, where "If required"
|
||||
shows "Up to N servers may reboot during this window"; max concurrent;
|
||||
notification channels. Shows "N of M targets need an agent update" linking to
|
||||
the servers when any target is too old.
|
||||
- **Window editor** (modal): name, presets writing cron (Nightly 02:00,
|
||||
Sunday 02:00, Saturday 22:00, Monthly 1st 02:00), cron field, timezone,
|
||||
duration; next 3 occurrences from `/maintenance-windows/preview`.
|
||||
- `/patching/runs/[runId]`: header with status, trigger, window end and counts
|
||||
per status; table of servers with status pill, updates installed
|
||||
(`PendingBefore - PendingAfter`), rebooted and verified times, error; each row
|
||||
expands to the output tail on the `--well` surface. Cancel while running. The
|
||||
page polls while the run is `running`.
|
||||
- **Server detail, OS updates panel**: "Covered by *Sunday prod*, next window
|
||||
Sun 21 Sep 02:00 BST" or "Not covered by any patch policy"; last patch run
|
||||
with link. Apply updates navigates to the new run.
|
||||
- **Vulnerabilities**: Apply updates navigates to the new run.
|
||||
|
||||
## Documentation
|
||||
|
||||
`vantage-docs`:
|
||||
|
||||
- New `docs/vantage/patching.md`: windows, policies, scope per package manager
|
||||
(including unsupported), reboot rule and verification, statuses table,
|
||||
offline and window-close behaviour, agent version requirement.
|
||||
- `vantage/servers.md`: remove the "Applying updates is not scheduled or staged"
|
||||
warning, describe the run record, update the reboot sentence.
|
||||
- `vantage/vulnerabilities.md`: Apply updates creates a run; link to patching.
|
||||
- `hq/licensing-and-entitlements.md`: patching is available on every tier.
|
||||
- `reference/api-tokens.md`: the `patching` scope.
|
||||
|
||||
`vantage-app/CLAUDE.md`: new "Scheduled patching" subsystem section, updated
|
||||
"Inventory and OS updates", collection list, REST list, `ServerCommand` and
|
||||
`AgentMessage` variants. `vantage-agent/CLAUDE.md`: the reboot promise.
|
||||
|
||||
## Testing
|
||||
|
||||
The repository tests pure functions without a database, so the logic is shaped
|
||||
for that and the Mongo layer is a thin shell.
|
||||
|
||||
Server:
|
||||
|
||||
- `patchsched.Decide`: on time, late within grace, past grace, past window
|
||||
end, already running, zero targets, and a DST case (`Europe/London`, last
|
||||
Sunday of October, 01:30 start occurring twice; `NextOccurrence` behaviour is
|
||||
asserted, not assumed).
|
||||
- `patchrun.Advance`: concurrency cap never exceeded counting `rebooting`;
|
||||
offline then online inside the window dispatches; offline to window end is
|
||||
`missed_offline`; queued at window end is `window_closed`; nothing dispatched
|
||||
after window end; no-result timeout; reboot timeout; cancelled runs dispatch
|
||||
nothing; manual runs ignore window rules.
|
||||
- `patchrun.ApplyResult` for each result status.
|
||||
- `patchrun.VerifyReboot`: boot time before, equal to and after `RebootedAt`,
|
||||
with reboot still owed and cleared.
|
||||
- `patchrun.Finalize`: every mix of terminal statuses.
|
||||
- `agentSupportsPatchResults`: older, equal, newer, pre-release, empty,
|
||||
unparseable.
|
||||
- Validation of windows and policies.
|
||||
- `scopes_test` covers the new routes; a token tag-restriction case in the
|
||||
style of `workflow_target_scope_test`.
|
||||
|
||||
Agent:
|
||||
|
||||
- `securitySources`: Debian and Ubuntu `.list`, Ubuntu 24.04 deb822 `.sources`,
|
||||
commented lines, a file set with no security suites (`ok == false`).
|
||||
- Ring buffer keeps the newest 64KB.
|
||||
- Windows security category filtering, as a parser test beside `winparse_test`.
|
||||
- Manual verification on Debian 12, Ubuntu 24.04, Rocky 9 and Windows Server
|
||||
2022: security-only and all, reboot and verification, window close
|
||||
mid-queue, agent busy.
|
||||
|
||||
## Rollout
|
||||
|
||||
1. `vantage-shared`: `ApplyUpdatesCmd` fields, `PatchResult`,
|
||||
`InventoryReport.boot_time_unix`, proto and `pb` together. Release a tag.
|
||||
2. `vantage-app`: bump the pin; ship collections, indexes, `patchsched`,
|
||||
results handling, API, UI, MCP change and docs. With no new agents yet,
|
||||
policies show every target as `agent_too_old` and the manual button works as
|
||||
before, now with a run record.
|
||||
3. `vantage-agent`: bump the pin, ship the agent changes, tag `agent/v*`. Set
|
||||
that version as the gate constant in the server in step 2 (the gate names a
|
||||
version that does not exist yet until step 3 ships, which is harmless: every
|
||||
agent reads as too old until it does).
|
||||
4. `vantage-docs`: publish the patching page.
|
||||
@@ -23,7 +23,9 @@ import (
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/db"
|
||||
grpcserver "gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/grpc"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/mcp"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/monitorsched"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/patchsched"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/services"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/vulnsched"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/workflowsched"
|
||||
@@ -179,6 +181,10 @@ func runSchemaSetup() {
|
||||
log.Printf("warning: failed to ensure status page indexes: %v", err)
|
||||
}
|
||||
|
||||
if err := services.EnsurePatchIndexes(); err != nil {
|
||||
log.Printf("warning: patch indexes: %v", err)
|
||||
}
|
||||
|
||||
if err := services.EnsureAuditIndexes(); err != nil {
|
||||
log.Printf("warning: failed to ensure audit indexes: %v", err)
|
||||
}
|
||||
@@ -253,6 +259,17 @@ func serve() {
|
||||
LogEvent: services.LogEvent,
|
||||
})
|
||||
|
||||
patchsched.Start(jobCtx, patchsched.Deps{
|
||||
LookupWindow: services.LookupWindow,
|
||||
CountTargets: services.CountPolicyTargets,
|
||||
StartPolicyRun: func(p models.PatchPolicy, windowEnd time.Time) error {
|
||||
_, err := services.StartPolicyRun(p, windowEnd, models.PatchSourceSchedule, "schedule")
|
||||
return err
|
||||
},
|
||||
AdvanceRuns: services.AdvancePatchRuns,
|
||||
LogEvent: services.LogEvent,
|
||||
})
|
||||
|
||||
vulnsched.Start(jobCtx, vulnsched.Deps{
|
||||
LogEvent: services.LogEvent,
|
||||
SendDigest: services.SendVulnDigest,
|
||||
|
||||
+2
-1
@@ -38,6 +38,7 @@ require (
|
||||
github.com/stretchr/objx v0.5.3 // indirect
|
||||
github.com/stretchr/testify v1.12.1 // indirect
|
||||
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
|
||||
github.com/yuin/goldmark v1.8.6 // indirect
|
||||
go.etcd.io/bbolt v1.5.0 // indirect
|
||||
go.opentelemetry.io/otel v1.46.0 // indirect
|
||||
go.opentelemetry.io/otel/trace v1.46.0 // indirect
|
||||
@@ -46,7 +47,7 @@ require (
|
||||
)
|
||||
|
||||
require (
|
||||
gitea.hostxtra.co.uk/vantage/vantage-shared v0.3.3
|
||||
gitea.hostxtra.co.uk/vantage/vantage-shared v0.5.0
|
||||
github.com/bytedance/sonic v1.15.3 // indirect
|
||||
github.com/bytedance/sonic/loader v0.5.2 // indirect
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
|
||||
+4
-2
@@ -1,5 +1,5 @@
|
||||
gitea.hostxtra.co.uk/vantage/vantage-shared v0.3.3 h1:+ak67Hj1C92hNfTE63nlF1c2wx4L8mML+1aI6VJnukc=
|
||||
gitea.hostxtra.co.uk/vantage/vantage-shared v0.3.3/go.mod h1:dWjeOFLltQ8sv9Pnn1xRxGfWGgqa2fkG0esuaJLoPXQ=
|
||||
gitea.hostxtra.co.uk/vantage/vantage-shared v0.5.0 h1:xwSIEkQKTd4Qk+BYHvoGN+h84Isr2h5qqnitUWF1m2w=
|
||||
gitea.hostxtra.co.uk/vantage/vantage-shared v0.5.0/go.mod h1:Zo66XhqF8No3dveIowLCepvMxVg8KnhsNMz0k0Xpuck=
|
||||
github.com/aquasecurity/bolt-fixtures v0.0.0-20200903104109-d34e7f983986 h1:2a30xLN2sUZcMXl50hg+PJCIDdJgIvIbVcKqLJ/ZrtM=
|
||||
github.com/aquasecurity/bolt-fixtures v0.0.0-20200903104109-d34e7f983986/go.mod h1:NT+jyeCzXk6vXR5MTkdn4z64TgGfE5HMLC8qfj5unl8=
|
||||
github.com/aquasecurity/trivy-db v0.0.0-20260813095258-0e0340a01b57 h1:A3Lz/9ip/qigafSxqBWcu7S8i+tJbQS7DB2V0XibOKs=
|
||||
@@ -152,6 +152,8 @@ github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT0
|
||||
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 h1:ilQV1hzziu+LLM3zUTJ0trRztfwgjqKnBWNtSRkbmwM=
|
||||
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78/go.mod h1:aL8wCCfTfSfmXjznFBSZNN13rSJjlIOI1fUNAtF7rmI=
|
||||
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
|
||||
github.com/yuin/goldmark v1.8.6 h1:d0VcaP1sx9GkFVkoW+KtggpGi2KZ965i14b0+bDQST4=
|
||||
github.com/yuin/goldmark v1.8.6/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg=
|
||||
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
|
||||
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
|
||||
go.etcd.io/bbolt v1.5.0 h1:S7GAl7Fxv12yohbwFfIbQCGDWbQbtDGPET4P/bD4lxU=
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -187,6 +187,7 @@ func RegisterRoutes(r *gin.Engine) {
|
||||
apiGroup.GET("/servers/:id/workloads/:wid/logs", auth.RequireRole("owner", "admin"), getWorkloadLogs)
|
||||
|
||||
registerStatusPageRoutes(apiGroup)
|
||||
registerPatchingRoutes(apiGroup)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -748,30 +749,41 @@ func updateAgent(c *gin.Context) {
|
||||
// applyUpdates godoc
|
||||
//
|
||||
// @Summary Apply pending OS updates on a server
|
||||
// @Description Dispatches ApplyUpdatesCmd. Exempt from the licence gate: security patching is never paywalled.
|
||||
// @Description Starts a manual patch run (all updates, no reboot) and returns its ID. Exempt from the licence gate: security patching is never paywalled.
|
||||
// @Tags servers
|
||||
// @Produce json
|
||||
// @Param id path string true "Server ID"
|
||||
// @Success 202 {object} MessageResponse
|
||||
// @Failure 404 {object} ErrorResponse
|
||||
// @Failure 503 {object} ErrorResponse
|
||||
// @Param id path string true "Server ID"
|
||||
// @Param source query string false "vulnerabilities when started from the vulnerabilities page"
|
||||
// @Success 202 {object} ApplyUpdatesResponse
|
||||
// @Failure 404 {object} ErrorResponse
|
||||
// @Failure 503 {object} ApplyUpdatesErrorResponse "agent offline; the attempt is recorded as run_id"
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /servers/{id}/apply-updates [post]
|
||||
func applyUpdates(c *gin.Context) {
|
||||
id := c.Param("id")
|
||||
s, err := services.GetServerScoped(auth.InstanceID(c), id, auth.ServerScope(c))
|
||||
instanceID := auth.InstanceID(c)
|
||||
s, err := services.GetServerScoped(instanceID, c.Param("id"), auth.ServerScope(c))
|
||||
if err != nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "server not found"})
|
||||
return
|
||||
}
|
||||
|
||||
if err := services.DispatchApplyUpdates(s.ServerID); err != nil {
|
||||
c.JSON(http.StatusServiceUnavailable, gin.H{"error": err.Error()})
|
||||
source := models.PatchSourceServer
|
||||
if c.Query("source") == models.PatchSourceVulnerabilities {
|
||||
source = models.PatchSourceVulnerabilities
|
||||
}
|
||||
run, err := services.StartManualRun(instanceID, s, actorFromCtx(c), source)
|
||||
if errors.Is(err, services.ErrAgentOffline) {
|
||||
// The attempt is still recorded, so it has a run ID to show.
|
||||
c.JSON(http.StatusServiceUnavailable, gin.H{"error": err.Error(), "run_id": run.RunID})
|
||||
return
|
||||
}
|
||||
services.LogEvent(auth.InstanceID(c), "updates.applied", actorFromCtx(c), s.ServerID, "", fmt.Sprintf("package update command dispatched to %s", s.Hostname))
|
||||
c.JSON(http.StatusAccepted, MessageResponse{Message: "apply updates command sent to agent"})
|
||||
if err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
services.LogEvent(instanceID, "updates.applied", actorFromCtx(c), s.ServerID, "",
|
||||
fmt.Sprintf("package update run %s started on %s", run.RunID, s.Hostname))
|
||||
c.JSON(http.StatusAccepted, ApplyUpdatesResponse{Message: "apply updates command sent to agent", RunID: run.RunID})
|
||||
}
|
||||
|
||||
// handleUpdateScript serves a dynamically generated shell script that
|
||||
|
||||
@@ -0,0 +1,455 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/auth"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/services"
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
// registerPatchingRoutes mounts maintenance windows, patch policies and patch
|
||||
// runs. Free on every tier: security patching is never paywalled, so no
|
||||
// RequireFeature here. Writes that decide what reboots and when are owner or
|
||||
// admin; watching and cancelling a run is open to every role.
|
||||
func registerPatchingRoutes(g *gin.RouterGroup) {
|
||||
admin := auth.RequireRole("owner", "admin")
|
||||
|
||||
g.GET("/maintenance-windows", listWindows)
|
||||
g.POST("/maintenance-windows", admin, createWindow)
|
||||
g.POST("/maintenance-windows/preview", previewWindow)
|
||||
g.GET("/maintenance-windows/:id", getWindow)
|
||||
g.PUT("/maintenance-windows/:id", admin, updateWindow)
|
||||
g.DELETE("/maintenance-windows/:id", admin, deleteWindow)
|
||||
|
||||
g.GET("/patch-policies", listPolicies)
|
||||
g.POST("/patch-policies", admin, createPolicy)
|
||||
g.GET("/patch-policies/:id", getPolicy)
|
||||
g.PUT("/patch-policies/:id", admin, updatePolicy)
|
||||
g.DELETE("/patch-policies/:id", admin, deletePolicy)
|
||||
g.POST("/patch-policies/:id/run-now", admin, runPolicyNow)
|
||||
|
||||
g.GET("/patch-runs", listPatchRuns)
|
||||
g.GET("/patch-runs/:runId", getPatchRun)
|
||||
g.POST("/patch-runs/:runId/cancel", cancelPatchRun)
|
||||
}
|
||||
|
||||
// patchError maps every patching service error once.
|
||||
func patchError(c *gin.Context, err error) {
|
||||
switch {
|
||||
case errors.Is(err, services.ErrWindowNotFound), errors.Is(err, services.ErrPolicyNotFound), errors.Is(err, services.ErrPatchRunNotFound):
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": err.Error()})
|
||||
case errors.Is(err, services.ErrWindowInUse), errors.Is(err, services.ErrPatchRunActive), errors.Is(err, services.ErrPatchRunFinished):
|
||||
c.JSON(http.StatusConflict, gin.H{"error": err.Error()})
|
||||
case errors.Is(err, services.ErrWindowInvalid), errors.Is(err, services.ErrPolicyInvalid),
|
||||
errors.Is(err, services.ErrNoTargets), errors.Is(err, services.ErrInvalidTag),
|
||||
errors.Is(err, services.ErrWorkflowTargetOutOfScope):
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
default:
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||
}
|
||||
}
|
||||
|
||||
// listWindows godoc
|
||||
//
|
||||
// @Summary List maintenance windows
|
||||
// @Tags patching
|
||||
// @Produce json
|
||||
// @Success 200 {array} models.MaintenanceWindow
|
||||
// @Failure 500 {object} ErrorResponse
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /maintenance-windows [get]
|
||||
func listWindows(c *gin.Context) {
|
||||
ws, err := services.ListWindows(auth.InstanceID(c))
|
||||
if err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, ws)
|
||||
}
|
||||
|
||||
// createWindow godoc
|
||||
//
|
||||
// @Summary Create a maintenance window
|
||||
// @Tags patching
|
||||
// @Accept json
|
||||
// @Produce json
|
||||
// @Param body body models.MaintenanceWindow true "Window"
|
||||
// @Success 201 {object} models.MaintenanceWindow
|
||||
// @Failure 400 {object} ErrorResponse
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /maintenance-windows [post]
|
||||
func createWindow(c *gin.Context) {
|
||||
var body models.MaintenanceWindow
|
||||
if err := c.ShouldBindJSON(&body); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
w, err := services.CreateWindow(auth.InstanceID(c), body)
|
||||
if err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
services.LogEvent(auth.InstanceID(c), "patch.window_created", actorFromCtx(c), "", "",
|
||||
fmt.Sprintf("maintenance window %s: %s %s for %d minutes", w.Name, w.Cron, w.TZ, w.DurationMinutes))
|
||||
c.JSON(http.StatusCreated, w)
|
||||
}
|
||||
|
||||
// previewWindow godoc
|
||||
//
|
||||
// @Summary Preview the next three maintenance windows
|
||||
// @Description Computed by the scheduler's own code, so the editor and the scheduler agree.
|
||||
// @Tags patching
|
||||
// @Accept json
|
||||
// @Produce json
|
||||
// @Param body body WindowPreviewRequest true "Schedule"
|
||||
// @Success 200 {array} services.WindowSpan
|
||||
// @Failure 400 {object} ErrorResponse
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /maintenance-windows/preview [post]
|
||||
func previewWindow(c *gin.Context) {
|
||||
var body WindowPreviewRequest
|
||||
if err := c.ShouldBindJSON(&body); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
spans, err := services.PreviewWindow(body.Cron, body.TZ, body.DurationMinutes, time.Now(), 3)
|
||||
if err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, spans)
|
||||
}
|
||||
|
||||
// getWindow godoc
|
||||
//
|
||||
// @Summary Get a maintenance window
|
||||
// @Tags patching
|
||||
// @Produce json
|
||||
// @Param id path string true "Window ID"
|
||||
// @Success 200 {object} models.MaintenanceWindow
|
||||
// @Failure 404 {object} ErrorResponse
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /maintenance-windows/{id} [get]
|
||||
func getWindow(c *gin.Context) {
|
||||
w, err := services.GetWindow(auth.InstanceID(c), c.Param("id"))
|
||||
if err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, w)
|
||||
}
|
||||
|
||||
// updateWindow godoc
|
||||
//
|
||||
// @Summary Update a maintenance window
|
||||
// @Description Moves the next run of every enabled policy using it.
|
||||
// @Tags patching
|
||||
// @Accept json
|
||||
// @Produce json
|
||||
// @Param id path string true "Window ID"
|
||||
// @Param body body models.MaintenanceWindow true "Window"
|
||||
// @Success 200 {object} models.MaintenanceWindow
|
||||
// @Failure 400 {object} ErrorResponse
|
||||
// @Failure 404 {object} ErrorResponse
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /maintenance-windows/{id} [put]
|
||||
func updateWindow(c *gin.Context) {
|
||||
var body models.MaintenanceWindow
|
||||
if err := c.ShouldBindJSON(&body); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
if err := services.CheckWindowScope(auth.InstanceID(c), c.Param("id"), auth.ServerScope(c)); err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
w, err := services.UpdateWindow(auth.InstanceID(c), c.Param("id"), body)
|
||||
if err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
services.LogEvent(auth.InstanceID(c), "patch.window_updated", actorFromCtx(c), "", "",
|
||||
fmt.Sprintf("maintenance window %s: %s %s for %d minutes", w.Name, w.Cron, w.TZ, w.DurationMinutes))
|
||||
c.JSON(http.StatusOK, w)
|
||||
}
|
||||
|
||||
// deleteWindow godoc
|
||||
//
|
||||
// @Summary Delete a maintenance window
|
||||
// @Tags patching
|
||||
// @Param id path string true "Window ID"
|
||||
// @Success 204
|
||||
// @Failure 404 {object} ErrorResponse
|
||||
// @Failure 409 {object} ErrorResponse "window_in_use"
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /maintenance-windows/{id} [delete]
|
||||
func deleteWindow(c *gin.Context) {
|
||||
if err := services.CheckWindowScope(auth.InstanceID(c), c.Param("id"), auth.ServerScope(c)); err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
if err := services.DeleteWindow(auth.InstanceID(c), c.Param("id")); err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
services.LogEvent(auth.InstanceID(c), "patch.window_deleted", actorFromCtx(c), "", "", "maintenance window "+c.Param("id")+" deleted")
|
||||
c.Status(http.StatusNoContent)
|
||||
}
|
||||
|
||||
// listPolicies godoc
|
||||
//
|
||||
// @Summary List patch policies
|
||||
// @Tags patching
|
||||
// @Produce json
|
||||
// @Success 200 {array} models.PatchPolicy
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /patch-policies [get]
|
||||
func listPolicies(c *gin.Context) {
|
||||
ps, err := services.ListPolicies(auth.InstanceID(c))
|
||||
if err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, ps)
|
||||
}
|
||||
|
||||
// createPolicy godoc
|
||||
//
|
||||
// @Summary Create a patch policy
|
||||
// @Tags patching
|
||||
// @Accept json
|
||||
// @Produce json
|
||||
// @Param body body models.PatchPolicy true "Policy"
|
||||
// @Success 201 {object} models.PatchPolicy
|
||||
// @Failure 400 {object} ErrorResponse
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /patch-policies [post]
|
||||
func createPolicy(c *gin.Context) {
|
||||
var body models.PatchPolicy
|
||||
if err := c.ShouldBindJSON(&body); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
p, err := services.CreatePolicy(auth.InstanceID(c), body, auth.ServerScope(c))
|
||||
if err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
services.LogEvent(auth.InstanceID(c), "patch.policy_created", actorFromCtx(c), "", "",
|
||||
fmt.Sprintf("patch policy %s: scope %s, reboot %s, enabled %v", p.Name, p.Scope, p.Reboot, p.Enabled))
|
||||
c.JSON(http.StatusCreated, p)
|
||||
}
|
||||
|
||||
// getPolicy godoc
|
||||
//
|
||||
// @Summary Get a patch policy
|
||||
// @Tags patching
|
||||
// @Produce json
|
||||
// @Param id path string true "Policy ID"
|
||||
// @Success 200 {object} models.PatchPolicy
|
||||
// @Failure 404 {object} ErrorResponse
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /patch-policies/{id} [get]
|
||||
func getPolicy(c *gin.Context) {
|
||||
p, err := services.GetPolicy(auth.InstanceID(c), c.Param("id"))
|
||||
if err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, p)
|
||||
}
|
||||
|
||||
// updatePolicy godoc
|
||||
//
|
||||
// @Summary Update a patch policy
|
||||
// @Tags patching
|
||||
// @Accept json
|
||||
// @Produce json
|
||||
// @Param id path string true "Policy ID"
|
||||
// @Param body body models.PatchPolicy true "Policy"
|
||||
// @Success 200 {object} models.PatchPolicy
|
||||
// @Failure 400 {object} ErrorResponse
|
||||
// @Failure 404 {object} ErrorResponse
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /patch-policies/{id} [put]
|
||||
func updatePolicy(c *gin.Context) {
|
||||
var body models.PatchPolicy
|
||||
if err := c.ShouldBindJSON(&body); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
p, err := services.UpdatePolicy(auth.InstanceID(c), c.Param("id"), body, auth.ServerScope(c))
|
||||
if err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
services.LogEvent(auth.InstanceID(c), "patch.policy_updated", actorFromCtx(c), "", "",
|
||||
fmt.Sprintf("patch policy %s: scope %s, reboot %s, enabled %v", p.Name, p.Scope, p.Reboot, p.Enabled))
|
||||
c.JSON(http.StatusOK, p)
|
||||
}
|
||||
|
||||
// deletePolicy godoc
|
||||
//
|
||||
// @Summary Delete a patch policy
|
||||
// @Tags patching
|
||||
// @Param id path string true "Policy ID"
|
||||
// @Success 204
|
||||
// @Failure 404 {object} ErrorResponse
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /patch-policies/{id} [delete]
|
||||
func deletePolicy(c *gin.Context) {
|
||||
instanceID := auth.InstanceID(c)
|
||||
p, err := services.GetPolicy(instanceID, c.Param("id"))
|
||||
if err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
if err := services.CheckPolicyScope(instanceID, *p, auth.ServerScope(c)); err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
if err := services.DeletePolicy(instanceID, p.PolicyID); err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
services.LogEvent(instanceID, "patch.policy_deleted", actorFromCtx(c), "", "", "patch policy "+p.Name+" deleted")
|
||||
c.Status(http.StatusNoContent)
|
||||
}
|
||||
|
||||
// runPolicyNow godoc
|
||||
//
|
||||
// @Summary Run a patch policy now
|
||||
// @Description Opens a window of the policy's usual length starting now.
|
||||
// @Tags patching
|
||||
// @Produce json
|
||||
// @Param id path string true "Policy ID"
|
||||
// @Success 202 {object} models.PatchRun
|
||||
// @Failure 404 {object} ErrorResponse
|
||||
// @Failure 409 {object} ErrorResponse
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /patch-policies/{id}/run-now [post]
|
||||
func runPolicyNow(c *gin.Context) {
|
||||
run, err := services.StartRunNow(auth.InstanceID(c), c.Param("id"), actorFromCtx(c), auth.ServerScope(c))
|
||||
if err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusAccepted, run)
|
||||
}
|
||||
|
||||
// listPatchRuns godoc
|
||||
//
|
||||
// @Summary List patch runs
|
||||
// @Tags patching
|
||||
// @Produce json
|
||||
// @Param policy_id query string false "Filter by policy"
|
||||
// @Param server_id query string false "Filter by server"
|
||||
// @Param limit query int false "At most 200, default 50"
|
||||
// @Success 200 {array} models.PatchRun
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /patch-runs [get]
|
||||
func listPatchRuns(c *gin.Context) {
|
||||
instanceID := auth.InstanceID(c)
|
||||
limit, _ := strconv.ParseInt(c.Query("limit"), 10, 64)
|
||||
runs, err := services.ListPatchRuns(instanceID, c.Query("policy_id"), c.Query("server_id"), limit)
|
||||
if err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
out := runs[:0]
|
||||
for i := range runs {
|
||||
before := len(runs[i].Servers)
|
||||
if err := services.ScopePatchRun(instanceID, &runs[i], auth.ServerScope(c)); err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
if before == 0 || len(runs[i].Servers) > 0 {
|
||||
out = append(out, runs[i])
|
||||
}
|
||||
}
|
||||
c.JSON(http.StatusOK, out)
|
||||
}
|
||||
|
||||
// getPatchRun godoc
|
||||
//
|
||||
// @Summary Get a patch run, with per-server output
|
||||
// @Tags patching
|
||||
// @Produce json
|
||||
// @Param runId path string true "Run ID"
|
||||
// @Success 200 {object} models.PatchRun
|
||||
// @Failure 404 {object} ErrorResponse
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /patch-runs/{runId} [get]
|
||||
func getPatchRun(c *gin.Context) {
|
||||
run, ok := scopedRun(c)
|
||||
if ok {
|
||||
c.JSON(http.StatusOK, run)
|
||||
}
|
||||
}
|
||||
|
||||
// cancelPatchRun godoc
|
||||
//
|
||||
// @Summary Cancel a patch run
|
||||
// @Description Stops further dispatch. Servers already patching finish.
|
||||
// @Tags patching
|
||||
// @Param runId path string true "Run ID"
|
||||
// @Success 204
|
||||
// @Failure 404 {object} ErrorResponse
|
||||
// @Failure 409 {object} ErrorResponse
|
||||
// @Security cookieAuth
|
||||
// @Security bearerAuth
|
||||
// @Router /patch-runs/{runId}/cancel [post]
|
||||
func cancelPatchRun(c *gin.Context) {
|
||||
run, ok := scopedRun(c)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if err := services.CancelPatchRun(auth.InstanceID(c), run.RunID); err != nil {
|
||||
patchError(c, err)
|
||||
return
|
||||
}
|
||||
services.LogEvent(auth.InstanceID(c), "patch.cancelled", actorFromCtx(c), "", "", "patch run "+run.RunID+" cancelled")
|
||||
c.Status(http.StatusNoContent)
|
||||
}
|
||||
|
||||
// scopedRun loads a run for a caller. A tag-restricted token that cannot see
|
||||
// every server in the run gets a 404, the same answer as a run that does not
|
||||
// exist, rather than a partial record it could act on.
|
||||
func scopedRun(c *gin.Context) (*models.PatchRun, bool) {
|
||||
instanceID := auth.InstanceID(c)
|
||||
run, err := services.GetPatchRun(instanceID, c.Param("runId"))
|
||||
if err != nil {
|
||||
patchError(c, err)
|
||||
return nil, false
|
||||
}
|
||||
before := len(run.Servers)
|
||||
if err := services.ScopePatchRun(instanceID, run, auth.ServerScope(c)); err != nil {
|
||||
patchError(c, err)
|
||||
return nil, false
|
||||
}
|
||||
if len(run.Servers) != before {
|
||||
patchError(c, services.ErrPatchRunNotFound)
|
||||
return nil, false
|
||||
}
|
||||
return run, true
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
package api
|
||||
|
||||
import "testing"
|
||||
|
||||
// Every patching route must carry a scope and a server-scope declaration, or
|
||||
// boot fails. Asserting the exact scope here keeps a copy-paste of
|
||||
// "patching:read" onto a write route from slipping through.
|
||||
func TestPatchingRouteScopes(t *testing.T) {
|
||||
want := map[string]string{
|
||||
"GET /api/maintenance-windows": "patching:read",
|
||||
"POST /api/maintenance-windows": "patching:write",
|
||||
"POST /api/maintenance-windows/preview": "patching:read",
|
||||
"GET /api/maintenance-windows/:id": "patching:read",
|
||||
"PUT /api/maintenance-windows/:id": "patching:write",
|
||||
"DELETE /api/maintenance-windows/:id": "patching:write",
|
||||
"GET /api/patch-policies": "patching:read",
|
||||
"POST /api/patch-policies": "patching:write",
|
||||
"GET /api/patch-policies/:id": "patching:read",
|
||||
"PUT /api/patch-policies/:id": "patching:write",
|
||||
"DELETE /api/patch-policies/:id": "patching:write",
|
||||
"POST /api/patch-policies/:id/run-now": "patching:write",
|
||||
"GET /api/patch-runs": "patching:read",
|
||||
"GET /api/patch-runs/:runId": "patching:read",
|
||||
"POST /api/patch-runs/:runId/cancel": "patching:write",
|
||||
}
|
||||
for route, scope := range want {
|
||||
if got := routeScopes[route]; got != scope {
|
||||
t.Errorf("%s: scope %q, want %q", route, got, scope)
|
||||
}
|
||||
if _, ok := serverScopedRoutes[route]; !ok {
|
||||
t.Errorf("%s: missing from serverScopedRoutes", route)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -175,6 +175,23 @@ var routeScopes = map[string]string{
|
||||
"PUT /api/status-pages/:pageId/incidents/:incidentId": "status:write",
|
||||
"DELETE /api/status-pages/:pageId/incidents/:incidentId": "status:write",
|
||||
"POST /api/status-pages/:pageId/incidents/:incidentId/updates": "status:write",
|
||||
|
||||
// Scheduled patching: maintenance windows, patch policies and patch runs.
|
||||
"GET /api/maintenance-windows": "patching:read",
|
||||
"POST /api/maintenance-windows": "patching:write",
|
||||
"POST /api/maintenance-windows/preview": "patching:read",
|
||||
"GET /api/maintenance-windows/:id": "patching:read",
|
||||
"PUT /api/maintenance-windows/:id": "patching:write",
|
||||
"DELETE /api/maintenance-windows/:id": "patching:write",
|
||||
"GET /api/patch-policies": "patching:read",
|
||||
"POST /api/patch-policies": "patching:write",
|
||||
"GET /api/patch-policies/:id": "patching:read",
|
||||
"PUT /api/patch-policies/:id": "patching:write",
|
||||
"DELETE /api/patch-policies/:id": "patching:write",
|
||||
"POST /api/patch-policies/:id/run-now": "patching:write",
|
||||
"GET /api/patch-runs": "patching:read",
|
||||
"GET /api/patch-runs/:runId": "patching:read",
|
||||
"POST /api/patch-runs/:runId/cancel": "patching:write",
|
||||
}
|
||||
|
||||
// RequireScopes enforces routeScopes for token-authenticated requests and does
|
||||
|
||||
@@ -398,6 +398,32 @@ var serverScopedRoutes = map[string]scopeDecl{
|
||||
// Each tool touching server data applies the caller's selector itself.
|
||||
"POST /api/mcp": exempt,
|
||||
"GET /api/mcp": exempt,
|
||||
|
||||
// Maintenance windows are a cron expression, a zone and a duration. They
|
||||
// name no server and return no server data, so reading and creating one
|
||||
// is exempt. Changing or deleting one moves or stops the patching of
|
||||
// every policy using it, so each is refused when any of those policies
|
||||
// targets servers outside the token's tag restriction.
|
||||
"GET /api/maintenance-windows": exempt,
|
||||
"POST /api/maintenance-windows": exempt,
|
||||
"POST /api/maintenance-windows/preview": exempt,
|
||||
"GET /api/maintenance-windows/:id": exempt,
|
||||
"PUT /api/maintenance-windows/:id": scoped,
|
||||
"DELETE /api/maintenance-windows/:id": scoped,
|
||||
// Reading a policy returns its selector (server IDs and tag pairs) and no
|
||||
// hostname, inventory or state, the same data a workflow's targets carry.
|
||||
"GET /api/patch-policies": exempt,
|
||||
"GET /api/patch-policies/:id": exempt,
|
||||
// Writes and run-now act on the policy's targets, so each is refused when
|
||||
// those targets reach outside the token's tag restriction.
|
||||
"POST /api/patch-policies": scoped,
|
||||
"PUT /api/patch-policies/:id": scoped,
|
||||
"DELETE /api/patch-policies/:id": scoped,
|
||||
"POST /api/patch-policies/:id/run-now": scoped,
|
||||
// Runs name hostnames; ScopePatchRun removes servers the token cannot see.
|
||||
"GET /api/patch-runs": scoped,
|
||||
"GET /api/patch-runs/:runId": scoped,
|
||||
"POST /api/patch-runs/:runId/cancel": scoped,
|
||||
}
|
||||
|
||||
// AssertServerScopeMapComplete refuses to boot when any registered /api route
|
||||
|
||||
@@ -297,3 +297,26 @@ type StatusIncidentUpdateRequest struct {
|
||||
Status string `json:"status" binding:"required"`
|
||||
Body string `json:"body" binding:"required"`
|
||||
}
|
||||
|
||||
// --- patching ---
|
||||
|
||||
// ApplyUpdatesResponse keeps the message existing scripts read and adds the
|
||||
// run that records what happened.
|
||||
type ApplyUpdatesResponse struct {
|
||||
Message string `json:"message"`
|
||||
RunID string `json:"run_id,omitempty"`
|
||||
}
|
||||
|
||||
// ApplyUpdatesErrorResponse is the 503 body of apply-updates: the agent is
|
||||
// offline, and the attempt is still recorded as a run.
|
||||
type ApplyUpdatesErrorResponse struct {
|
||||
Error string `json:"error"`
|
||||
RunID string `json:"run_id,omitempty"`
|
||||
}
|
||||
|
||||
// WindowPreviewRequest is the body of POST /maintenance-windows/preview.
|
||||
type WindowPreviewRequest struct {
|
||||
Cron string `json:"cron"`
|
||||
TZ string `json:"tz"`
|
||||
DurationMinutes int `json:"duration_minutes"`
|
||||
}
|
||||
|
||||
@@ -236,6 +236,12 @@ func (s *vantageServer) ReportInventory(ctx context.Context, req *pb.InventoryRe
|
||||
if err := services.StoreInventory(srv.ServerID, req); err != nil {
|
||||
log.Printf("store inventory for %s: %v", srv.ServerID, err)
|
||||
}
|
||||
// Only static snapshots compute reboot_required, so only they can settle a
|
||||
// reboot. The agent sends one at start, so the first report after a
|
||||
// reboot qualifies.
|
||||
if req.IncludeStatic && req.BootTimeUnix > 0 {
|
||||
services.VerifyPatchReboots(srv.InstanceID, srv.ServerID, time.Unix(req.BootTimeUnix, 0), req.RebootRequired)
|
||||
}
|
||||
return &pb.InventoryReportResponse{}, nil
|
||||
}
|
||||
|
||||
@@ -330,6 +336,9 @@ func (s *vantageServer) CommandStream(stream pb.Vantage_CommandStreamServer) err
|
||||
if m.WorkloadLogsResult != nil {
|
||||
services.WorkloadResults.Deliver(m.WorkloadLogsResult)
|
||||
}
|
||||
if m.PatchResult != nil {
|
||||
services.RecordPatchResult(srv.InstanceID, srv.ServerID, m.PatchResult)
|
||||
}
|
||||
if m.StepResult != nil {
|
||||
services.StepResults.Deliver(m.StepResult)
|
||||
}
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/services"
|
||||
)
|
||||
|
||||
@@ -156,16 +157,16 @@ type updateBatchResult struct {
|
||||
Servers int `json:"servers"`
|
||||
Succeeded []string `json:"succeeded"`
|
||||
Failed map[string]string `json:"failed,omitempty"`
|
||||
RunIDs map[string]string `json:"run_ids,omitempty"` // server ID -> patch run ID
|
||||
}
|
||||
|
||||
// apply_updates. The REST route (internal/api/handlers.go's applyUpdates) is
|
||||
// per-server: POST /servers/:id/apply-updates resolves one server with
|
||||
// services.GetServerScoped and calls services.DispatchApplyUpdates(serverID).
|
||||
// There is no fleet-wide variant of that service call to invoke once, so this
|
||||
// tool resolves the requested targets through ResolveTargetsScoped exactly as
|
||||
// the brief describes, then calls the same DispatchApplyUpdates the REST route
|
||||
// calls, once per resolved server - the identical dispatch, just looped
|
||||
// instead of hardcoded to one server_id from the URL.
|
||||
// services.GetServerScoped and calls services.StartManualRun. There is no
|
||||
// fleet-wide variant of that service call to invoke once, so this tool
|
||||
// resolves the requested targets through ResolveTargetsScoped, then calls the
|
||||
// same StartManualRun once per resolved server and writes the same
|
||||
// updates.applied audit event per server with its run ID.
|
||||
func init() {
|
||||
All().Register(Tool{
|
||||
Name: "apply_updates",
|
||||
@@ -178,9 +179,9 @@ func init() {
|
||||
Write: true,
|
||||
Scope: "servers:write",
|
||||
Description: "Apply pending OS package updates on real servers, selected by " +
|
||||
"server_ids and/or tags. This installs packages on real machines right now and " +
|
||||
"cannot be undone from here. A server may need a reboot afterward, which this " +
|
||||
"tool does not do.",
|
||||
"server_ids and/or tags. Starts one manual patch run per server. This installs " +
|
||||
"packages on real machines right now and cannot be undone from here. A server " +
|
||||
"may need a reboot afterward, which this tool does not do.",
|
||||
Handler: func(ctx context.Context, c Caller, args map[string]any) (any, error) {
|
||||
ids := stringSliceArg(args, "server_ids")
|
||||
targets, err := services.ResolveTargetsScoped(c.InstanceID, ids, tagArg(args), c.TokenScope)
|
||||
@@ -192,12 +193,20 @@ func init() {
|
||||
}
|
||||
|
||||
result := updateBatchResult{Servers: len(targets), Failed: map[string]string{}}
|
||||
for _, srv := range targets {
|
||||
if err := services.DispatchApplyUpdates(srv.ServerID); err != nil {
|
||||
for i := range targets {
|
||||
srv := targets[i]
|
||||
run, err := services.StartManualRun(c.InstanceID, &srv, "mcp:"+c.TokenName, models.PatchSourceMCP)
|
||||
if err != nil {
|
||||
result.Failed[srv.ServerID] = err.Error()
|
||||
continue
|
||||
}
|
||||
services.LogEvent(c.InstanceID, "updates.applied", "mcp:"+c.TokenName, srv.ServerID, "",
|
||||
fmt.Sprintf("package update run %s started on %s", run.RunID, srv.Hostname))
|
||||
result.Succeeded = append(result.Succeeded, srv.ServerID)
|
||||
if result.RunIDs == nil {
|
||||
result.RunIDs = map[string]string{}
|
||||
}
|
||||
result.RunIDs[srv.ServerID] = run.RunID
|
||||
}
|
||||
if len(result.Failed) == 0 {
|
||||
result.Failed = nil
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
package models
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
)
|
||||
|
||||
const (
|
||||
PatchScopeAll = "all"
|
||||
PatchScopeSecurity = "security"
|
||||
|
||||
PatchRebootNever = "never"
|
||||
PatchRebootIfRequired = "if_required"
|
||||
|
||||
PatchRunRunning = "running"
|
||||
PatchRunSucceeded = "succeeded"
|
||||
PatchRunPartial = "partial"
|
||||
PatchRunFailed = "failed"
|
||||
PatchRunCancelled = "cancelled"
|
||||
|
||||
PatchSrvQueued = "queued"
|
||||
PatchSrvWaitingOffline = "waiting_offline"
|
||||
PatchSrvPatching = "patching"
|
||||
PatchSrvRebooting = "rebooting"
|
||||
PatchSrvSucceeded = "succeeded"
|
||||
PatchSrvFailed = "failed"
|
||||
PatchSrvUnsupported = "unsupported"
|
||||
PatchSrvAgentTooOld = "agent_too_old"
|
||||
PatchSrvMissedOffline = "missed_offline"
|
||||
PatchSrvWindowClosed = "window_closed"
|
||||
PatchSrvCancelled = "cancelled"
|
||||
|
||||
PatchSourceSchedule = "schedule"
|
||||
PatchSourceRunNow = "run_now"
|
||||
PatchSourceServer = "server"
|
||||
PatchSourceVulnerabilities = "vulnerabilities"
|
||||
PatchSourceMCP = "mcp"
|
||||
)
|
||||
|
||||
// MaintenanceWindow answers "when" and nothing else. Policies reference it by
|
||||
// ID, so one window can later serve alert muting and status page maintenance
|
||||
// without a second definition of the same Sunday morning.
|
||||
type MaintenanceWindow struct {
|
||||
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
|
||||
InstanceID string `bson:"instance_id" json:"instance_id"`
|
||||
WindowID string `bson:"window_id" json:"window_id"`
|
||||
Name string `bson:"name" json:"name"`
|
||||
Cron string `bson:"cron" json:"cron"` // 5-field, window start
|
||||
TZ string `bson:"tz" json:"tz"` // IANA name
|
||||
DurationMinutes int `bson:"duration_minutes" json:"duration_minutes"`
|
||||
CreatedAt time.Time `bson:"created_at" json:"created_at"`
|
||||
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
|
||||
}
|
||||
|
||||
type PatchPolicy struct {
|
||||
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
|
||||
InstanceID string `bson:"instance_id" json:"instance_id"`
|
||||
PolicyID string `bson:"policy_id" json:"policy_id"`
|
||||
Name string `bson:"name" json:"name"`
|
||||
Enabled bool `bson:"enabled" json:"enabled"`
|
||||
WindowID string `bson:"window_id" json:"window_id"`
|
||||
TargetServerIDs []string `bson:"target_server_ids" json:"target_server_ids"`
|
||||
TargetTags map[string]string `bson:"target_tags,omitempty" json:"target_tags,omitempty"`
|
||||
Scope string `bson:"scope" json:"scope"`
|
||||
Reboot string `bson:"reboot" json:"reboot"`
|
||||
MaxConcurrent int `bson:"max_concurrent" json:"max_concurrent"` // 0 = no cap
|
||||
NotifyChannelIDs []string `bson:"notify_channel_ids,omitempty" json:"notify_channel_ids,omitempty"`
|
||||
NextRunAt *time.Time `bson:"next_run_at,omitempty" json:"next_run_at,omitempty"`
|
||||
LastRunAt *time.Time `bson:"last_run_at,omitempty" json:"last_run_at,omitempty"`
|
||||
LastSkipped *Skip `bson:"last_skipped,omitempty" json:"last_skipped,omitempty"`
|
||||
DisabledReason string `bson:"disabled_reason,omitempty" json:"disabled_reason,omitempty"`
|
||||
CreatedAt time.Time `bson:"created_at" json:"created_at"`
|
||||
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
|
||||
}
|
||||
|
||||
// PatchRun is one firing of a policy, or one manual Apply updates. Scope,
|
||||
// reboot and concurrency are copied from the policy at fire time so editing
|
||||
// the policy never rewrites what a past run shows.
|
||||
type PatchRun struct {
|
||||
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
|
||||
InstanceID string `bson:"instance_id" json:"instance_id"`
|
||||
RunID string `bson:"run_id" json:"run_id"`
|
||||
PolicyID string `bson:"policy_id,omitempty" json:"policy_id,omitempty"`
|
||||
PolicyName string `bson:"policy_name,omitempty" json:"policy_name,omitempty"`
|
||||
TriggeredBy string `bson:"triggered_by" json:"triggered_by"`
|
||||
Source string `bson:"source" json:"source"`
|
||||
Scope string `bson:"scope" json:"scope"`
|
||||
Reboot string `bson:"reboot" json:"reboot"`
|
||||
MaxConcurrent int `bson:"max_concurrent" json:"max_concurrent"`
|
||||
WindowEnd *time.Time `bson:"window_end,omitempty" json:"window_end,omitempty"`
|
||||
Status string `bson:"status" json:"status"`
|
||||
CancelledAt *time.Time `bson:"cancelled_at,omitempty" json:"cancelled_at,omitempty"`
|
||||
StartedAt time.Time `bson:"started_at" json:"started_at"`
|
||||
FinishedAt *time.Time `bson:"finished_at,omitempty" json:"finished_at,omitempty"`
|
||||
Servers []PatchServerRun `bson:"servers" json:"servers"`
|
||||
}
|
||||
|
||||
type PatchServerRun struct {
|
||||
ServerID string `bson:"server_id" json:"server_id"`
|
||||
Hostname string `bson:"hostname" json:"hostname"`
|
||||
Status string `bson:"status" json:"status"`
|
||||
CommandID string `bson:"command_id,omitempty" json:"-"`
|
||||
PendingBefore int `bson:"pending_before" json:"pending_before"`
|
||||
PendingAfter *int `bson:"pending_after,omitempty" json:"pending_after,omitempty"`
|
||||
RebootedAt *time.Time `bson:"rebooted_at,omitempty" json:"rebooted_at,omitempty"`
|
||||
VerifiedAt *time.Time `bson:"verified_at,omitempty" json:"verified_at,omitempty"`
|
||||
// BootTimeBefore is the host's reported boot time when the reboot was
|
||||
// announced. A later report with a different boot time proves the
|
||||
// restart without comparing the host clock to the server clock.
|
||||
BootTimeBefore *time.Time `bson:"boot_time_before,omitempty" json:"-"`
|
||||
Output string `bson:"output,omitempty" json:"output,omitempty"`
|
||||
Error string `bson:"error,omitempty" json:"error,omitempty"`
|
||||
StartedAt *time.Time `bson:"started_at,omitempty" json:"started_at,omitempty"`
|
||||
FinishedAt *time.Time `bson:"finished_at,omitempty" json:"finished_at,omitempty"`
|
||||
}
|
||||
@@ -42,6 +42,9 @@ type Inventory struct {
|
||||
RebootRequired bool `bson:"reboot_required,omitempty" json:"reboot_required,omitempty"`
|
||||
MetricsAt *time.Time `bson:"metrics_at,omitempty" json:"metrics_at,omitempty"`
|
||||
StaticAt *time.Time `bson:"static_at,omitempty" json:"static_at,omitempty"`
|
||||
// BootTime is the host's last reported boot time, stored on every report
|
||||
// that carries one so a patch reboot can be proven by a changed boot.
|
||||
BootTime *time.Time `bson:"boot_time,omitempty" json:"boot_time,omitempty"`
|
||||
}
|
||||
|
||||
type Server struct {
|
||||
|
||||
@@ -11,6 +11,10 @@ import (
|
||||
// monitor check. MonitorName carries the hostname in that case.
|
||||
const TypeServer = "server"
|
||||
|
||||
// TypePatch marks a patch run summary. Like a vulnerability digest it is a
|
||||
// headline, not a transition, so title() adds no verb.
|
||||
const TypePatch = "patch"
|
||||
|
||||
type Event struct {
|
||||
MonitorName string
|
||||
Type string
|
||||
@@ -26,7 +30,7 @@ func (e Event) title() string {
|
||||
verb = "is DOWN"
|
||||
}
|
||||
var s string
|
||||
if e.Type == TypeVuln {
|
||||
if e.Type == TypeVuln || e.Type == TypePatch {
|
||||
// A digest is not a transition. MonitorName already carries the whole
|
||||
// headline ("12 new critical across 4 servers"), so no verb applies.
|
||||
s = fmt.Sprintf("[Vantage] %s", e.MonitorName)
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
package notify
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
|
||||
)
|
||||
|
||||
// A patch summary is not a transition: it must not read "is DOWN" or
|
||||
// "recovered".
|
||||
func TestPatchEventTitle(t *testing.T) {
|
||||
ev := Event{MonitorName: `Patch policy "Sunday prod" partial`, Type: TypePatch, NewStatus: models.PatchRunPartial, Message: "38 succeeded, 2 failed"}
|
||||
want := `[Vantage] Patch policy "Sunday prod" partial: 38 succeeded, 2 failed`
|
||||
if got := ev.title(); got != want {
|
||||
t.Fatalf("got %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,270 @@
|
||||
// Package patchrun is the patch run state machine as pure functions. Nothing
|
||||
// here touches the database: the services layer loads a run, asks this
|
||||
// package what should change, and writes that change guarded by the status it
|
||||
// expected. That split is what makes the rules testable without MongoDB.
|
||||
package patchrun
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
|
||||
"gitea.hostxtra.co.uk/vantage/vantage-shared/grpc/pb"
|
||||
)
|
||||
|
||||
// MinAgentVersion is the first agent release that honours ApplyUpdatesCmd's
|
||||
// scope and answers with a PatchResult. An older agent ignores the scope and
|
||||
// installs everything, so a policy must never dispatch to one.
|
||||
const MinAgentVersion = "1.4.0"
|
||||
|
||||
const (
|
||||
// ResultGrace covers what the agent does after the upgrade and before it
|
||||
// answers: a pending-update re-check of up to 10 minutes (Windows Update
|
||||
// search) plus a 2 minute reboot check, with room to spare.
|
||||
ResultGrace = 20 * time.Minute
|
||||
// ManualTimeout is the agent's backstop for one started upgrade, counted
|
||||
// from its own start. The window end never stops a running upgrade.
|
||||
ManualTimeout = 2 * time.Hour
|
||||
// RebootTimeout is how long a rebooting server has to send a post-boot
|
||||
// inventory report. Windows cumulative updates routinely take over 20.
|
||||
RebootTimeout = 45 * time.Minute
|
||||
// LatestStartBeforeEnd is the tail of a window in which no server starts
|
||||
// patching: a late start would run long past the window end.
|
||||
LatestStartBeforeEnd = 15 * time.Minute
|
||||
)
|
||||
|
||||
// AgentSupportsPatchResults compares major.minor.patch. Empty, "dev" and
|
||||
// anything unparseable count as too old, and a pre-release of exactly the
|
||||
// minimum version counts as older than it, as semver orders them.
|
||||
func AgentSupportsPatchResults(version string) bool {
|
||||
have, pre, ok := parseVersion(version)
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
want, _, _ := parseVersion(MinAgentVersion)
|
||||
for i := 0; i < 3; i++ {
|
||||
if have[i] != want[i] {
|
||||
return have[i] > want[i]
|
||||
}
|
||||
}
|
||||
return !pre
|
||||
}
|
||||
|
||||
func parseVersion(v string) ([3]int, bool, bool) {
|
||||
var out [3]int
|
||||
v = strings.TrimPrefix(strings.TrimSpace(v), "v")
|
||||
v, _, _ = strings.Cut(v, "+")
|
||||
core, pre, hasPre := strings.Cut(v, "-")
|
||||
parts := strings.Split(core, ".")
|
||||
if len(parts) != 3 {
|
||||
return out, false, false
|
||||
}
|
||||
for i, p := range parts {
|
||||
n, err := strconv.Atoi(p)
|
||||
if err != nil || n < 0 {
|
||||
return out, false, false
|
||||
}
|
||||
out[i] = n
|
||||
}
|
||||
return out, hasPre && pre != "", true
|
||||
}
|
||||
|
||||
// IsTerminal reports whether a server run has finished.
|
||||
func IsTerminal(status string) bool {
|
||||
switch status {
|
||||
case models.PatchSrvQueued, models.PatchSrvWaitingOffline, models.PatchSrvPatching, models.PatchSrvRebooting:
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// Transition is one change Advance wants made. The caller writes it guarded
|
||||
// by From, so a result that arrived meanwhile is never overwritten. Dispatch
|
||||
// means: send ApplyUpdatesCmd, and set To only once the command is on its way.
|
||||
type Transition struct {
|
||||
ServerID string
|
||||
From string
|
||||
To string
|
||||
Error string
|
||||
Dispatch bool
|
||||
}
|
||||
|
||||
// Advance returns what should change on this tick. connected says which
|
||||
// agents hold a command stream right now.
|
||||
func Advance(run models.PatchRun, now time.Time, connected map[string]bool) []Transition {
|
||||
if run.Status != models.PatchRunRunning {
|
||||
return nil
|
||||
}
|
||||
windowOpen := run.WindowEnd == nil || now.Before(*run.WindowEnd)
|
||||
// In the last LatestStartBeforeEnd of a window nothing new starts: queued
|
||||
// and waiting servers simply wait, and close at WindowEnd as usual.
|
||||
mayStart := run.WindowEnd == nil || now.Before(run.WindowEnd.Add(-LatestStartBeforeEnd))
|
||||
|
||||
inFlight := 0
|
||||
for _, s := range run.Servers {
|
||||
if s.Status == models.PatchSrvPatching || s.Status == models.PatchSrvRebooting {
|
||||
inFlight++
|
||||
}
|
||||
}
|
||||
|
||||
var out []Transition
|
||||
for _, s := range run.Servers {
|
||||
switch s.Status {
|
||||
case models.PatchSrvQueued, models.PatchSrvWaitingOffline:
|
||||
switch {
|
||||
case run.CancelledAt != nil:
|
||||
out = append(out, Transition{ServerID: s.ServerID, From: s.Status, To: models.PatchSrvCancelled})
|
||||
case !windowOpen:
|
||||
to := models.PatchSrvWindowClosed
|
||||
if s.Status == models.PatchSrvWaitingOffline {
|
||||
to = models.PatchSrvMissedOffline
|
||||
}
|
||||
out = append(out, Transition{ServerID: s.ServerID, From: s.Status, To: to})
|
||||
case !mayStart:
|
||||
// The window tail: no dispatch, no transition.
|
||||
case run.MaxConcurrent > 0 && inFlight >= run.MaxConcurrent:
|
||||
// No slot this tick.
|
||||
case !connected[s.ServerID]:
|
||||
if s.Status == models.PatchSrvQueued {
|
||||
out = append(out, Transition{ServerID: s.ServerID, From: s.Status, To: models.PatchSrvWaitingOffline})
|
||||
}
|
||||
default:
|
||||
out = append(out, Transition{ServerID: s.ServerID, From: s.Status, To: models.PatchSrvPatching, Dispatch: true})
|
||||
inFlight++
|
||||
}
|
||||
case models.PatchSrvPatching:
|
||||
if now.After(resultDeadline(run, s)) {
|
||||
out = append(out, Transition{ServerID: s.ServerID, From: s.Status, To: models.PatchSrvFailed, Error: "no result from agent"})
|
||||
}
|
||||
case models.PatchSrvRebooting:
|
||||
if s.RebootedAt != nil && now.After(s.RebootedAt.Add(RebootTimeout)) {
|
||||
out = append(out, Transition{ServerID: s.ServerID, From: s.Status, To: models.PatchSrvFailed, Error: fmt.Sprintf("did not come back within %d minutes", int(RebootTimeout.Minutes()))})
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// resultDeadline is the same for windowed and manual runs: the agent lets a
|
||||
// started upgrade finish past the window end, so the window end says nothing
|
||||
// about when a result is due. The base is the server's dispatch time.
|
||||
func resultDeadline(run models.PatchRun, s models.PatchServerRun) time.Time {
|
||||
start := run.StartedAt
|
||||
if s.StartedAt != nil {
|
||||
start = *s.StartedAt
|
||||
}
|
||||
return start.Add(ManualTimeout + ResultGrace)
|
||||
}
|
||||
|
||||
// ApplyResult folds an agent's PatchResult into the server run. It only acts
|
||||
// on a server that is patching; anything else is a late or duplicate result.
|
||||
func ApplyResult(s models.PatchServerRun, r *pb.PatchResult, now time.Time) (models.PatchServerRun, bool) {
|
||||
if s.Status != models.PatchSrvPatching {
|
||||
return s, false
|
||||
}
|
||||
s.Output = r.OutputTail
|
||||
if r.PendingAfter >= 0 {
|
||||
v := int(r.PendingAfter)
|
||||
s.PendingAfter = &v
|
||||
}
|
||||
switch r.Status {
|
||||
case pb.PatchStatusOK:
|
||||
if r.Rebooting {
|
||||
s.Status = models.PatchSrvRebooting
|
||||
s.RebootedAt = &now
|
||||
return s, true
|
||||
}
|
||||
s.Status = models.PatchSrvSucceeded
|
||||
case pb.PatchStatusUnsupported:
|
||||
s.Status = models.PatchSrvUnsupported
|
||||
s.Error = r.Message
|
||||
default:
|
||||
s.Status = models.PatchSrvFailed
|
||||
s.Error = r.Message
|
||||
if s.Error == "" {
|
||||
s.Error = "agent reported a failure"
|
||||
}
|
||||
}
|
||||
s.FinishedAt = &now
|
||||
return s, true
|
||||
}
|
||||
|
||||
// VerifyReboot settles a rebooting server from a static inventory report.
|
||||
// When the boot time reported before the reboot is known, a later boot time
|
||||
// is the proof: both come from the host clock, so skew against the server
|
||||
// clock does not matter. Otherwise only a boot time later than the reboot
|
||||
// command counts. Either way a snapshot sent during the one-minute grace
|
||||
// period, before the host went down, does not.
|
||||
func VerifyReboot(s models.PatchServerRun, bootTime time.Time, rebootRequired bool, now time.Time) (models.PatchServerRun, bool) {
|
||||
if s.Status != models.PatchSrvRebooting || s.RebootedAt == nil {
|
||||
return s, false
|
||||
}
|
||||
proven := bootTime.After(*s.RebootedAt)
|
||||
if s.BootTimeBefore != nil {
|
||||
proven = bootTime.After(*s.BootTimeBefore)
|
||||
}
|
||||
if !proven {
|
||||
return s, false
|
||||
}
|
||||
if rebootRequired {
|
||||
s.Status = models.PatchSrvFailed
|
||||
s.Error = "still requires a reboot after restarting"
|
||||
} else {
|
||||
s.Status = models.PatchSrvSucceeded
|
||||
s.VerifiedAt = &now
|
||||
}
|
||||
s.FinishedAt = &now
|
||||
return s, true
|
||||
}
|
||||
|
||||
// Finalize says whether the run is over and how it ended. Only succeeded
|
||||
// counts as success: unsupported, agent_too_old and the window outcomes did
|
||||
// not patch anything.
|
||||
func Finalize(run models.PatchRun) (string, bool) {
|
||||
ok := 0
|
||||
for _, s := range run.Servers {
|
||||
if !IsTerminal(s.Status) {
|
||||
return "", false
|
||||
}
|
||||
if s.Status == models.PatchSrvSucceeded {
|
||||
ok++
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case run.CancelledAt != nil:
|
||||
return models.PatchRunCancelled, true
|
||||
case len(run.Servers) > 0 && ok == len(run.Servers):
|
||||
return models.PatchRunSucceeded, true
|
||||
case ok == 0:
|
||||
return models.PatchRunFailed, true
|
||||
default:
|
||||
return models.PatchRunPartial, true
|
||||
}
|
||||
}
|
||||
|
||||
var summaryOrder = []struct{ status, label string }{
|
||||
{models.PatchSrvSucceeded, "succeeded"},
|
||||
{models.PatchSrvFailed, "failed"},
|
||||
{models.PatchSrvUnsupported, "unsupported"},
|
||||
{models.PatchSrvAgentTooOld, "need an agent update"},
|
||||
{models.PatchSrvMissedOffline, "missed offline"},
|
||||
{models.PatchSrvWindowClosed, "window closed"},
|
||||
{models.PatchSrvCancelled, "cancelled"},
|
||||
}
|
||||
|
||||
// Summary is the one-line count used in alerts, e.g. "38 succeeded, 2 failed".
|
||||
func Summary(run models.PatchRun) string {
|
||||
counts := map[string]int{}
|
||||
for _, s := range run.Servers {
|
||||
counts[s.Status]++
|
||||
}
|
||||
var parts []string
|
||||
for _, o := range summaryOrder {
|
||||
if n := counts[o.status]; n > 0 {
|
||||
parts = append(parts, fmt.Sprintf("%d %s", n, o.label))
|
||||
}
|
||||
}
|
||||
return strings.Join(parts, ", ")
|
||||
}
|
||||
@@ -0,0 +1,322 @@
|
||||
package patchrun
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
|
||||
"gitea.hostxtra.co.uk/vantage/vantage-shared/grpc/pb"
|
||||
)
|
||||
|
||||
var t0 = time.Date(2026, 9, 20, 2, 0, 0, 0, time.UTC)
|
||||
|
||||
func tp(t time.Time) *time.Time { return &t }
|
||||
|
||||
func srv(id, status string) models.PatchServerRun {
|
||||
return models.PatchServerRun{ServerID: id, Hostname: id, Status: status}
|
||||
}
|
||||
|
||||
func windowRun(max int, servers ...models.PatchServerRun) models.PatchRun {
|
||||
return models.PatchRun{Status: models.PatchRunRunning, StartedAt: t0, WindowEnd: tp(t0.Add(2 * time.Hour)), MaxConcurrent: max, Servers: servers}
|
||||
}
|
||||
|
||||
func find(ts []Transition, id string) *Transition {
|
||||
for i := range ts {
|
||||
if ts[i].ServerID == id {
|
||||
return &ts[i]
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func TestAgentSupportsPatchResults(t *testing.T) {
|
||||
cases := map[string]bool{
|
||||
"1.4.0": true, "v1.4.0": true, "1.4.1": true, "1.10.0": true, "2.0.0": true,
|
||||
"1.3.5": false, "1.4.0-rc1": false, "1.5.0-rc1": true,
|
||||
"": false, "dev": false, "1.4": false, "x.y.z": false,
|
||||
}
|
||||
for v, want := range cases {
|
||||
if got := AgentSupportsPatchResults(v); got != want {
|
||||
t.Errorf("%q: got %v, want %v", v, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdvanceRespectsConcurrencyCountingRebooting(t *testing.T) {
|
||||
run := windowRun(2,
|
||||
srv("a", models.PatchSrvRebooting),
|
||||
srv("b", models.PatchSrvQueued),
|
||||
srv("c", models.PatchSrvQueued),
|
||||
)
|
||||
run.Servers[0].RebootedAt = tp(t0)
|
||||
ts := Advance(run, t0.Add(time.Minute), map[string]bool{"b": true, "c": true})
|
||||
if tr := find(ts, "b"); tr == nil || !tr.Dispatch || tr.To != models.PatchSrvPatching {
|
||||
t.Fatalf("b should dispatch, got %+v", tr)
|
||||
}
|
||||
if tr := find(ts, "c"); tr != nil {
|
||||
t.Fatalf("c must wait for a slot, got %+v", tr)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdvanceUnlimitedWhenZero(t *testing.T) {
|
||||
run := windowRun(0, srv("a", models.PatchSrvQueued), srv("b", models.PatchSrvQueued))
|
||||
ts := Advance(run, t0, map[string]bool{"a": true, "b": true})
|
||||
if len(ts) != 2 || !ts[0].Dispatch || !ts[1].Dispatch {
|
||||
t.Fatalf("both should dispatch: %+v", ts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdvanceOfflineWaitsThenDispatches(t *testing.T) {
|
||||
run := windowRun(0, srv("a", models.PatchSrvQueued))
|
||||
ts := Advance(run, t0, map[string]bool{})
|
||||
if len(ts) != 1 || ts[0].To != models.PatchSrvWaitingOffline || ts[0].Dispatch {
|
||||
t.Fatalf("offline should wait: %+v", ts)
|
||||
}
|
||||
run.Servers[0].Status = models.PatchSrvWaitingOffline
|
||||
if ts := Advance(run, t0.Add(time.Minute), map[string]bool{}); len(ts) != 0 {
|
||||
t.Fatalf("still offline must be a no-op, got %+v", ts)
|
||||
}
|
||||
ts = Advance(run, t0.Add(2*time.Minute), map[string]bool{"a": true})
|
||||
if len(ts) != 1 || !ts[0].Dispatch {
|
||||
t.Fatalf("back online should dispatch: %+v", ts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdvanceWindowCloses(t *testing.T) {
|
||||
run := windowRun(1, srv("q", models.PatchSrvQueued), srv("w", models.PatchSrvWaitingOffline))
|
||||
ts := Advance(run, t0.Add(2*time.Hour), map[string]bool{"q": true, "w": true})
|
||||
if tr := find(ts, "q"); tr == nil || tr.To != models.PatchSrvWindowClosed || tr.Dispatch {
|
||||
t.Fatalf("queued at window end: %+v", tr)
|
||||
}
|
||||
if tr := find(ts, "w"); tr == nil || tr.To != models.PatchSrvMissedOffline {
|
||||
t.Fatalf("waiting at window end: %+v", tr)
|
||||
}
|
||||
}
|
||||
|
||||
// A windowed run times out from the server's own dispatch time, not from the
|
||||
// window end: a server dispatched late in the window may finish past it.
|
||||
func TestAdvanceNoResultTimeoutFromDispatch(t *testing.T) {
|
||||
run := windowRun(0, srv("a", models.PatchSrvPatching))
|
||||
run.Servers[0].StartedAt = tp(t0.Add(90 * time.Minute))
|
||||
deadline := t0.Add(90*time.Minute + ManualTimeout + ResultGrace)
|
||||
if ts := Advance(run, run.WindowEnd.Add(ResultGrace+time.Minute), nil); len(ts) != 0 {
|
||||
t.Fatalf("past WindowEnd+grace but inside the dispatch timeout: %+v", ts)
|
||||
}
|
||||
if ts := Advance(run, deadline.Add(-time.Minute), nil); len(ts) != 0 {
|
||||
t.Fatalf("inside the dispatch timeout: %+v", ts)
|
||||
}
|
||||
ts := Advance(run, deadline.Add(time.Minute), nil)
|
||||
if len(ts) != 1 || ts[0].To != models.PatchSrvFailed || ts[0].Error == "" {
|
||||
t.Fatalf("past the dispatch timeout: %+v", ts)
|
||||
}
|
||||
}
|
||||
|
||||
// Without a server StartedAt the run's own start is the base.
|
||||
func TestAdvanceNoResultTimeoutFallsBackToRunStart(t *testing.T) {
|
||||
run := windowRun(0, srv("a", models.PatchSrvPatching))
|
||||
if ts := Advance(run, t0.Add(2*time.Hour+19*time.Minute), nil); len(ts) != 0 {
|
||||
t.Fatalf("inside timeout: %+v", ts)
|
||||
}
|
||||
if ts := Advance(run, t0.Add(2*time.Hour+21*time.Minute), nil); len(ts) != 1 || ts[0].To != models.PatchSrvFailed {
|
||||
t.Fatalf("past timeout: %+v", ts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdvanceManualRunTimeout(t *testing.T) {
|
||||
run := models.PatchRun{Status: models.PatchRunRunning, StartedAt: t0, Servers: []models.PatchServerRun{srv("a", models.PatchSrvPatching)}}
|
||||
run.Servers[0].StartedAt = tp(t0)
|
||||
if ts := Advance(run, t0.Add(2*time.Hour+19*time.Minute), nil); len(ts) != 0 {
|
||||
t.Fatalf("manual inside timeout: %+v", ts)
|
||||
}
|
||||
if ts := Advance(run, t0.Add(2*time.Hour+21*time.Minute), nil); len(ts) != 1 || ts[0].To != models.PatchSrvFailed {
|
||||
t.Fatalf("manual past timeout: %+v", ts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTimingConstants(t *testing.T) {
|
||||
if ResultGrace != 20*time.Minute || RebootTimeout != 45*time.Minute || LatestStartBeforeEnd != 15*time.Minute {
|
||||
t.Fatalf("ResultGrace=%v RebootTimeout=%v LatestStartBeforeEnd=%v", ResultGrace, RebootTimeout, LatestStartBeforeEnd)
|
||||
}
|
||||
}
|
||||
|
||||
// No server starts patching in the last 15 minutes of a window: it would
|
||||
// either be cut short or run long past the window end.
|
||||
func TestAdvanceNoDispatchInWindowTail(t *testing.T) {
|
||||
run := windowRun(0, srv("q", models.PatchSrvQueued), srv("w", models.PatchSrvWaitingOffline), srv("o", models.PatchSrvQueued))
|
||||
cutoff := run.WindowEnd.Add(-LatestStartBeforeEnd)
|
||||
online := map[string]bool{"q": true, "w": true}
|
||||
for _, at := range []time.Time{cutoff, cutoff.Add(time.Minute), run.WindowEnd.Add(-time.Second)} {
|
||||
if ts := Advance(run, at, online); len(ts) != 0 {
|
||||
t.Fatalf("at %v: nothing may change in the window tail, got %+v", at.Sub(t0), ts)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdvanceDispatchJustBeforeWindowTail(t *testing.T) {
|
||||
run := windowRun(0, srv("q", models.PatchSrvQueued))
|
||||
at := run.WindowEnd.Add(-LatestStartBeforeEnd - time.Second)
|
||||
ts := Advance(run, at, map[string]bool{"q": true})
|
||||
if len(ts) != 1 || !ts[0].Dispatch {
|
||||
t.Fatalf("dispatch must be allowed just before the tail: %+v", ts)
|
||||
}
|
||||
}
|
||||
|
||||
// A manual run has no window and no tail.
|
||||
func TestAdvanceManualRunHasNoTail(t *testing.T) {
|
||||
run := models.PatchRun{Status: models.PatchRunRunning, StartedAt: t0, Servers: []models.PatchServerRun{srv("a", models.PatchSrvQueued)}}
|
||||
if ts := Advance(run, t0.Add(10*time.Hour), map[string]bool{"a": true}); len(ts) != 1 || !ts[0].Dispatch {
|
||||
t.Fatalf("manual run must dispatch: %+v", ts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdvanceRebootTimeout(t *testing.T) {
|
||||
run := windowRun(0, srv("a", models.PatchSrvRebooting))
|
||||
run.Servers[0].RebootedAt = tp(t0)
|
||||
if ts := Advance(run, t0.Add(44*time.Minute), nil); len(ts) != 0 {
|
||||
t.Fatalf("inside reboot timeout: %+v", ts)
|
||||
}
|
||||
ts := Advance(run, t0.Add(46*time.Minute), nil)
|
||||
if len(ts) != 1 || ts[0].To != models.PatchSrvFailed || ts[0].Error != "did not come back within 45 minutes" {
|
||||
t.Fatalf("past reboot timeout: %+v", ts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdvanceCancelledDispatchesNothing(t *testing.T) {
|
||||
run := windowRun(0, srv("a", models.PatchSrvQueued), srv("b", models.PatchSrvWaitingOffline), srv("c", models.PatchSrvPatching))
|
||||
run.CancelledAt = tp(t0)
|
||||
ts := Advance(run, t0.Add(time.Minute), map[string]bool{"a": true, "b": true})
|
||||
for _, id := range []string{"a", "b"} {
|
||||
if tr := find(ts, id); tr == nil || tr.To != models.PatchSrvCancelled || tr.Dispatch {
|
||||
t.Errorf("%s: %+v", id, tr)
|
||||
}
|
||||
}
|
||||
if find(ts, "c") != nil {
|
||||
t.Error("an in-flight server must be left to finish")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdvanceIgnoresFinishedRun(t *testing.T) {
|
||||
run := windowRun(0, srv("a", models.PatchSrvQueued))
|
||||
run.Status = models.PatchRunSucceeded
|
||||
if ts := Advance(run, t0, map[string]bool{"a": true}); ts != nil {
|
||||
t.Fatalf("got %+v", ts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestApplyResult(t *testing.T) {
|
||||
now := t0.Add(10 * time.Minute)
|
||||
cases := []struct {
|
||||
r pb.PatchResult
|
||||
want string
|
||||
}{
|
||||
{pb.PatchResult{Status: pb.PatchStatusOK, PendingAfter: 0}, models.PatchSrvSucceeded},
|
||||
{pb.PatchResult{Status: pb.PatchStatusOK, Rebooting: true}, models.PatchSrvRebooting},
|
||||
{pb.PatchResult{Status: pb.PatchStatusFailed, Message: "apt broke"}, models.PatchSrvFailed},
|
||||
{pb.PatchResult{Status: pb.PatchStatusBusy, Message: "busy"}, models.PatchSrvFailed},
|
||||
{pb.PatchResult{Status: pb.PatchStatusUnsupported, Message: "no metadata"}, models.PatchSrvUnsupported},
|
||||
}
|
||||
for _, c := range cases {
|
||||
got, ok := ApplyResult(srv("a", models.PatchSrvPatching), &c.r, now)
|
||||
if !ok || got.Status != c.want {
|
||||
t.Errorf("%s: got %s ok=%v, want %s", c.r.Status, got.Status, ok, c.want)
|
||||
}
|
||||
if c.want == models.PatchSrvRebooting && (got.RebootedAt == nil || got.FinishedAt != nil) {
|
||||
t.Errorf("rebooting must set RebootedAt and leave FinishedAt nil: %+v", got)
|
||||
}
|
||||
}
|
||||
if _, ok := ApplyResult(srv("a", models.PatchSrvSucceeded), &pb.PatchResult{Status: pb.PatchStatusOK}, now); ok {
|
||||
t.Error("a result for a server not patching must be ignored")
|
||||
}
|
||||
got, _ := ApplyResult(srv("a", models.PatchSrvPatching), &pb.PatchResult{Status: pb.PatchStatusOK, PendingAfter: -1}, now)
|
||||
if got.PendingAfter != nil {
|
||||
t.Error("PendingAfter -1 means unknown and must stay nil")
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifyReboot(t *testing.T) {
|
||||
s := srv("a", models.PatchSrvRebooting)
|
||||
s.RebootedAt = tp(t0)
|
||||
now := t0.Add(5 * time.Minute)
|
||||
if _, ok := VerifyReboot(s, t0.Add(-time.Hour), false, now); ok {
|
||||
t.Error("boot before the reboot command is not proof")
|
||||
}
|
||||
if _, ok := VerifyReboot(s, t0, false, now); ok {
|
||||
t.Error("boot equal to the reboot command is not proof")
|
||||
}
|
||||
got, ok := VerifyReboot(s, t0.Add(2*time.Minute), false, now)
|
||||
if !ok || got.Status != models.PatchSrvSucceeded || got.VerifiedAt == nil {
|
||||
t.Errorf("clean reboot: %+v", got)
|
||||
}
|
||||
got, ok = VerifyReboot(s, t0.Add(2*time.Minute), true, now)
|
||||
if !ok || got.Status != models.PatchSrvFailed || got.Error == "" {
|
||||
t.Errorf("still owed: %+v", got)
|
||||
}
|
||||
if _, ok := VerifyReboot(srv("a", models.PatchSrvPatching), t0.Add(time.Hour), false, now); ok {
|
||||
t.Error("only rebooting servers verify")
|
||||
}
|
||||
}
|
||||
|
||||
// With the boot time recorded before the reboot, a changed boot time is the
|
||||
// proof, whatever the skew between the host clock and the server clock.
|
||||
func TestVerifyRebootChangedBoot(t *testing.T) {
|
||||
s := srv("a", models.PatchSrvRebooting)
|
||||
s.RebootedAt = tp(t0)
|
||||
s.BootTimeBefore = tp(t0.Add(-10 * 24 * time.Hour))
|
||||
now := t0.Add(5 * time.Minute)
|
||||
// The host clock runs 10 minutes slow: its new boot time reads earlier
|
||||
// than the server's RebootedAt, yet the boot did change.
|
||||
got, ok := VerifyReboot(s, t0.Add(-8*time.Minute), false, now)
|
||||
if !ok || got.Status != models.PatchSrvSucceeded {
|
||||
t.Fatalf("changed boot behind a slow clock must be proven: %+v ok=%v", got, ok)
|
||||
}
|
||||
// A fast host clock with an unchanged boot is not proof.
|
||||
if _, ok := VerifyReboot(s, *s.BootTimeBefore, false, now); ok {
|
||||
t.Error("an unchanged boot time is not proof")
|
||||
}
|
||||
if _, ok := VerifyReboot(s, s.BootTimeBefore.Add(-time.Minute), false, now); ok {
|
||||
t.Error("an earlier boot time is not proof")
|
||||
}
|
||||
}
|
||||
|
||||
func TestFinalize(t *testing.T) {
|
||||
mk := func(statuses ...string) models.PatchRun {
|
||||
r := windowRun(0)
|
||||
for i, s := range statuses {
|
||||
r.Servers = append(r.Servers, srv(string(rune('a'+i)), s))
|
||||
}
|
||||
return r
|
||||
}
|
||||
cases := []struct {
|
||||
run models.PatchRun
|
||||
want string
|
||||
done bool
|
||||
}{
|
||||
{mk(models.PatchSrvSucceeded, models.PatchSrvSucceeded), models.PatchRunSucceeded, true},
|
||||
{mk(models.PatchSrvSucceeded, models.PatchSrvFailed), models.PatchRunPartial, true},
|
||||
{mk(models.PatchSrvSucceeded, models.PatchSrvUnsupported), models.PatchRunPartial, true},
|
||||
{mk(models.PatchSrvSucceeded, models.PatchSrvAgentTooOld), models.PatchRunPartial, true},
|
||||
{mk(models.PatchSrvMissedOffline, models.PatchSrvWindowClosed), models.PatchRunFailed, true},
|
||||
{mk(models.PatchSrvSucceeded, models.PatchSrvPatching), "", false},
|
||||
{mk(), models.PatchRunFailed, true},
|
||||
}
|
||||
for i, c := range cases {
|
||||
got, done := Finalize(c.run)
|
||||
if got != c.want || done != c.done {
|
||||
t.Errorf("case %d: got %q/%v, want %q/%v", i, got, done, c.want, c.done)
|
||||
}
|
||||
}
|
||||
cancelled := mk(models.PatchSrvSucceeded, models.PatchSrvCancelled)
|
||||
cancelled.CancelledAt = tp(t0)
|
||||
if got, done := Finalize(cancelled); got != models.PatchRunCancelled || !done {
|
||||
t.Errorf("cancelled: %q/%v", got, done)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSummary(t *testing.T) {
|
||||
r := windowRun(0,
|
||||
srv("a", models.PatchSrvSucceeded), srv("b", models.PatchSrvSucceeded),
|
||||
srv("c", models.PatchSrvFailed), srv("d", models.PatchSrvMissedOffline))
|
||||
if got := Summary(r); got != "2 succeeded, 1 failed, 1 missed offline" {
|
||||
t.Fatalf("got %q", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
// Package patchsched fires patch policies at the start of their maintenance
|
||||
// window and advances running patch runs. Like workflowsched it runs under the
|
||||
// housekeeping leader lock and must not import services: services imports
|
||||
// this package for NextStart and WindowEnd.
|
||||
package patchsched
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/workflowsched"
|
||||
)
|
||||
|
||||
type Decision string
|
||||
|
||||
const (
|
||||
Fire Decision = "fire"
|
||||
SkipMissed Decision = "missed"
|
||||
SkipRunning Decision = "already_running"
|
||||
SkipNoTargets Decision = "no_targets"
|
||||
)
|
||||
|
||||
// Decide is the whole fire or skip rule for one due policy. Missed comes
|
||||
// first, as in workflowsched: a stale occurrence is recorded as missed
|
||||
// whatever else is true. A window that has already closed is missed even
|
||||
// inside the hour of grace, because there is no time left to patch in.
|
||||
func Decide(due, windowEnd, now time.Time, runActive bool, targets int) Decision {
|
||||
if !now.Before(windowEnd) || now.Sub(due) > workflowsched.GraceWindow {
|
||||
return SkipMissed
|
||||
}
|
||||
if runActive {
|
||||
return SkipRunning
|
||||
}
|
||||
if targets == 0 {
|
||||
return SkipNoTargets
|
||||
}
|
||||
return Fire
|
||||
}
|
||||
|
||||
func WindowEnd(start time.Time, durationMinutes int) time.Time {
|
||||
return start.Add(time.Duration(durationMinutes) * time.Minute)
|
||||
}
|
||||
|
||||
// NextStart is the first window start strictly after from. Callers pass
|
||||
// Later(now, currentWindowEnd) so windows never overlap, including across a
|
||||
// daylight-saving fall-back where the same wall-clock time occurs twice.
|
||||
func NextStart(cron, tz string, from time.Time) (time.Time, error) {
|
||||
return workflowsched.NextOccurrence(cron, tz, from)
|
||||
}
|
||||
|
||||
func Later(a, b time.Time) time.Time {
|
||||
if a.After(b) {
|
||||
return a
|
||||
}
|
||||
return b
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
package patchsched
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
var due = time.Date(2026, 9, 20, 2, 0, 0, 0, time.UTC)
|
||||
|
||||
func TestDecide(t *testing.T) {
|
||||
end := due.Add(2 * time.Hour)
|
||||
cases := []struct {
|
||||
name string
|
||||
now time.Time
|
||||
end time.Time
|
||||
running bool
|
||||
targets int
|
||||
want Decision
|
||||
}{
|
||||
{"on time", due, end, false, 3, Fire},
|
||||
{"late within grace", due.Add(59 * time.Minute), end, false, 3, Fire},
|
||||
{"past grace", due.Add(61 * time.Minute), end, false, 3, SkipMissed},
|
||||
{"window already over", due.Add(20 * time.Minute), due.Add(15 * time.Minute), false, 3, SkipMissed},
|
||||
{"missed wins over running", due.Add(2 * time.Hour), end, true, 3, SkipMissed},
|
||||
{"previous run active", due, end, true, 3, SkipRunning},
|
||||
{"no targets", due, end, false, 0, SkipNoTargets},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := Decide(due, c.end, c.now, c.running, c.targets); got != c.want {
|
||||
t.Errorf("%s: got %s, want %s", c.name, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Europe/London falls back on 25 October 2026, so 01:30 happens twice. The
|
||||
// next window is computed from the end of the current one, so a two-hour
|
||||
// window starting at the first 01:30 cannot fire again at the second.
|
||||
func TestNextStartAcrossFallBack(t *testing.T) {
|
||||
loc, _ := time.LoadLocation("Europe/London")
|
||||
from := time.Date(2026, 10, 24, 12, 0, 0, 0, time.UTC)
|
||||
first, err := NextStart("30 1 * * 0", "Europe/London", from)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if d := first.In(loc); d.Day() != 25 || d.Month() != time.October || d.Hour() != 1 || d.Minute() != 30 {
|
||||
t.Fatalf("first = %s", d)
|
||||
}
|
||||
end := WindowEnd(first, 120)
|
||||
next, err := NextStart("30 1 * * 0", "Europe/London", Later(first.Add(time.Minute), end))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if d := next.In(loc); d.Day() != 1 || d.Month() != time.November || d.Hour() != 1 || d.Minute() != 30 {
|
||||
t.Fatalf("next = %s, want 2026-11-01 01:30 London", d)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWindowEndAndLater(t *testing.T) {
|
||||
if got := WindowEnd(due, 90); !got.Equal(due.Add(90 * time.Minute)) {
|
||||
t.Fatalf("WindowEnd = %s", got)
|
||||
}
|
||||
a, b := due, due.Add(time.Second)
|
||||
if !Later(a, b).Equal(b) || !Later(b, a).Equal(b) {
|
||||
t.Fatal("Later must return the later time")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,159 @@
|
||||
package patchsched
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/db"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo/options"
|
||||
)
|
||||
|
||||
const tickInterval = 30 * time.Second
|
||||
|
||||
// Deps are injected from main.go: services imports this package, so this
|
||||
// package cannot import services.
|
||||
type Deps struct {
|
||||
// LookupWindow returns nil, nil when the window no longer exists.
|
||||
LookupWindow func(instanceID, windowID string) (*models.MaintenanceWindow, error)
|
||||
CountTargets func(p models.PatchPolicy) (int, error)
|
||||
StartPolicyRun func(p models.PatchPolicy, windowEnd time.Time) error
|
||||
AdvanceRuns func(ctx context.Context)
|
||||
LogEvent func(instanceID, eventType, actor, serverID, keyID, details string)
|
||||
}
|
||||
|
||||
// Start runs until ctx is cancelled, inside bus.RunAsLeader("housekeeping").
|
||||
// Each tick fires due policies, then advances every running run.
|
||||
func Start(ctx context.Context, deps Deps) {
|
||||
go func() {
|
||||
t := time.NewTicker(tickInterval)
|
||||
defer t.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-t.C:
|
||||
fireDue(ctx, deps, time.Now())
|
||||
deps.AdvanceRuns(ctx)
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
func fireDue(ctx context.Context, deps Deps, now time.Time) {
|
||||
cur, err := db.Col("patch_policies").Find(ctx, bson.M{"enabled": true, "next_run_at": bson.M{"$lte": now}})
|
||||
if err != nil {
|
||||
log.Printf("patchsched: find due: %v", err)
|
||||
return
|
||||
}
|
||||
var due []models.PatchPolicy
|
||||
if err := cur.All(ctx, &due); err != nil {
|
||||
log.Printf("patchsched: decode due: %v", err)
|
||||
return
|
||||
}
|
||||
for _, p := range due {
|
||||
if ctx.Err() != nil {
|
||||
return
|
||||
}
|
||||
process(ctx, deps, p, now)
|
||||
}
|
||||
}
|
||||
|
||||
func process(ctx context.Context, deps Deps, p models.PatchPolicy, now time.Time) {
|
||||
if p.NextRunAt == nil {
|
||||
return
|
||||
}
|
||||
due := *p.NextRunAt
|
||||
|
||||
w, err := deps.LookupWindow(p.InstanceID, p.WindowID)
|
||||
if err != nil {
|
||||
log.Printf("patchsched: policy %s: load window: %v", p.PolicyID, err)
|
||||
return // a database error is retried next tick, not treated as "gone"
|
||||
}
|
||||
if w == nil {
|
||||
disable(ctx, deps, p, "its maintenance window no longer exists")
|
||||
return
|
||||
}
|
||||
end := WindowEnd(due, w.DurationMinutes)
|
||||
next, err := NextStart(w.Cron, w.TZ, Later(now, end))
|
||||
if err != nil {
|
||||
disable(ctx, deps, p, "its maintenance window schedule is no longer valid: "+err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
// The claim, as in workflowsched: matching the current next_run_at means a
|
||||
// second process reaching this policy matches nothing.
|
||||
res, err := db.Col("patch_policies").UpdateOne(ctx,
|
||||
bson.M{"policy_id": p.PolicyID, "next_run_at": due},
|
||||
bson.M{"$set": bson.M{"next_run_at": next}})
|
||||
if err != nil || res.MatchedCount == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
n, err := deps.CountTargets(p)
|
||||
if err != nil {
|
||||
retryLater(ctx, deps, p, err, due, next, now)
|
||||
return
|
||||
}
|
||||
active, err := hasActiveRun(ctx, p)
|
||||
if err != nil {
|
||||
retryLater(ctx, deps, p, err, due, next, now)
|
||||
return
|
||||
}
|
||||
switch d := Decide(due, end, now, active, n); d {
|
||||
case Fire:
|
||||
if err := deps.StartPolicyRun(p, end); err != nil {
|
||||
retryLater(ctx, deps, p, err, due, next, now)
|
||||
return
|
||||
}
|
||||
_, _ = db.Col("patch_policies").UpdateOne(ctx, bson.M{"policy_id": p.PolicyID},
|
||||
bson.M{"$set": bson.M{"last_run_at": now}, "$unset": bson.M{"last_skipped": ""}})
|
||||
default:
|
||||
recordSkip(ctx, deps, p, string(d), due, now)
|
||||
}
|
||||
}
|
||||
|
||||
func hasActiveRun(ctx context.Context, p models.PatchPolicy) (bool, error) {
|
||||
err := db.Col("patch_runs").FindOne(ctx,
|
||||
bson.M{"instance_id": p.InstanceID, "policy_id": p.PolicyID, "status": models.PatchRunRunning},
|
||||
options.FindOne().SetProjection(bson.M{"_id": 1})).Err()
|
||||
if err == nil {
|
||||
return true, nil
|
||||
}
|
||||
if errors.Is(err, mongo.ErrNoDocuments) {
|
||||
return false, nil
|
||||
}
|
||||
return false, err
|
||||
}
|
||||
|
||||
// retryLater handles an error after the claim. The claim is put back, guarded
|
||||
// on the value just written so a concurrent edit to the policy is not undone,
|
||||
// and the next tick retries the same occurrence. Decide's missed rule bounds
|
||||
// the retries: once the occurrence is too late it is skipped as missed.
|
||||
func retryLater(ctx context.Context, deps Deps, p models.PatchPolicy, cause error, due, next, now time.Time) {
|
||||
if _, err := db.Col("patch_policies").UpdateOne(ctx,
|
||||
bson.M{"policy_id": p.PolicyID, "next_run_at": next},
|
||||
bson.M{"$set": bson.M{"next_run_at": due}}); err != nil {
|
||||
log.Printf("patchsched: policy %s: put back claim: %v", p.PolicyID, err)
|
||||
}
|
||||
recordSkip(ctx, deps, p, "error: "+cause.Error(), due, now)
|
||||
}
|
||||
|
||||
func recordSkip(ctx context.Context, deps Deps, p models.PatchPolicy, reason string, due, at time.Time) {
|
||||
_, _ = db.Col("patch_policies").UpdateOne(ctx, bson.M{"policy_id": p.PolicyID},
|
||||
bson.M{"$set": bson.M{"last_skipped": models.Skip{Reason: reason, Due: due, At: at}}})
|
||||
deps.LogEvent(p.InstanceID, "patch.skipped", "schedule", "", "",
|
||||
"patch policy "+p.Name+" skipped "+due.Format(time.RFC3339)+": "+reason)
|
||||
}
|
||||
|
||||
func disable(ctx context.Context, deps Deps, p models.PatchPolicy, reason string) {
|
||||
_, _ = db.Col("patch_policies").UpdateOne(ctx, bson.M{"policy_id": p.PolicyID}, bson.M{
|
||||
"$set": bson.M{"enabled": false, "disabled_reason": reason},
|
||||
"$unset": bson.M{"next_run_at": ""},
|
||||
})
|
||||
deps.LogEvent(p.InstanceID, "patch.policy_disabled", "schedule", "", "", "patch policy "+p.Name+" disabled: "+reason)
|
||||
}
|
||||
@@ -23,6 +23,11 @@ func StoreInventory(serverID string, r *pb.InventoryReport) error {
|
||||
set["inventory.memory.used_bytes"] = r.Memory.UsedBytes
|
||||
}
|
||||
set["inventory.swap_used_bytes"] = r.SwapUsed
|
||||
// Kept current so a patch reboot can be proven by a changed boot time,
|
||||
// independent of clock skew between the host and the control plane.
|
||||
if r.BootTimeUnix > 0 {
|
||||
set["inventory.boot_time"] = time.Unix(r.BootTimeUnix, 0).UTC()
|
||||
}
|
||||
|
||||
if r.IncludeStatic {
|
||||
set["inventory.static_at"] = now
|
||||
|
||||
@@ -48,6 +48,10 @@ var ScopedCollections = []string{
|
||||
"server_workloads",
|
||||
"status_pages",
|
||||
"status_incidents",
|
||||
"maintenance_windows",
|
||||
"patch_policies",
|
||||
"patch_runs",
|
||||
"patch_run_outputs",
|
||||
}
|
||||
|
||||
// collectionRenames maps the two collections whose names change. Ordered so the
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/db"
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo/options"
|
||||
)
|
||||
|
||||
// EnsurePatchIndexes builds the patching indexes. The command_id index is the
|
||||
// one that matters: every PatchResult is matched to its server run through it.
|
||||
func EnsurePatchIndexes() error {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
|
||||
if _, err := db.Col("maintenance_windows").Indexes().CreateOne(ctx, mongo.IndexModel{
|
||||
Keys: bson.D{{Key: "instance_id", Value: 1}, {Key: "window_id", Value: 1}},
|
||||
Options: options.Index().SetUnique(true),
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := db.Col("patch_policies").Indexes().CreateMany(ctx, []mongo.IndexModel{
|
||||
{Keys: bson.D{{Key: "instance_id", Value: 1}, {Key: "policy_id", Value: 1}}, Options: options.Index().SetUnique(true)},
|
||||
{Keys: bson.D{{Key: "enabled", Value: 1}, {Key: "next_run_at", Value: 1}}},
|
||||
{Keys: bson.D{{Key: "instance_id", Value: 1}, {Key: "window_id", Value: 1}}},
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := db.Col("patch_run_outputs").Indexes().CreateOne(ctx, mongo.IndexModel{
|
||||
Keys: bson.D{{Key: "instance_id", Value: 1}, {Key: "run_id", Value: 1}, {Key: "server_id", Value: 1}},
|
||||
Options: options.Index().SetUnique(true),
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err := db.Col("patch_runs").Indexes().CreateMany(ctx, []mongo.IndexModel{
|
||||
{Keys: bson.D{{Key: "instance_id", Value: 1}, {Key: "run_id", Value: 1}}, Options: options.Index().SetUnique(true)},
|
||||
{Keys: bson.D{{Key: "instance_id", Value: 1}, {Key: "policy_id", Value: 1}, {Key: "started_at", Value: -1}}},
|
||||
{Keys: bson.D{{Key: "instance_id", Value: 1}, {Key: "servers.server_id", Value: 1}, {Key: "started_at", Value: -1}}},
|
||||
{Keys: bson.D{{Key: "status", Value: 1}}},
|
||||
{Keys: bson.D{{Key: "servers.command_id", Value: 1}}},
|
||||
})
|
||||
return err
|
||||
}
|
||||
@@ -0,0 +1,254 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/db"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/patchsched"
|
||||
"github.com/google/uuid"
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo/options"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrPolicyInvalid = errors.New("invalid patch policy")
|
||||
ErrPolicyNotFound = errors.New("patch policy not found")
|
||||
)
|
||||
|
||||
// ValidatePolicy checks a policy without touching the database. The window's
|
||||
// existence and the token's tag scope are checked by Create and Update.
|
||||
func ValidatePolicy(p models.PatchPolicy) error {
|
||||
if n := strings.TrimSpace(p.Name); n == "" || len(n) > 100 {
|
||||
return fmt.Errorf("%w: name must be 1 to 100 characters", ErrPolicyInvalid)
|
||||
}
|
||||
if p.WindowID == "" {
|
||||
return fmt.Errorf("%w: a maintenance window is required", ErrPolicyInvalid)
|
||||
}
|
||||
if p.Scope != models.PatchScopeAll && p.Scope != models.PatchScopeSecurity {
|
||||
return fmt.Errorf("%w: scope must be %q or %q", ErrPolicyInvalid, models.PatchScopeAll, models.PatchScopeSecurity)
|
||||
}
|
||||
if p.Reboot != models.PatchRebootNever && p.Reboot != models.PatchRebootIfRequired {
|
||||
return fmt.Errorf("%w: reboot must be %q or %q", ErrPolicyInvalid, models.PatchRebootNever, models.PatchRebootIfRequired)
|
||||
}
|
||||
if p.MaxConcurrent < 0 || p.MaxConcurrent > 1000 {
|
||||
return fmt.Errorf("%w: max concurrent must be between 0 and 1000", ErrPolicyInvalid)
|
||||
}
|
||||
// Same rule as workflows: an empty selector matches nothing, and saying so
|
||||
// at save time beats a policy that silently patches nobody every Sunday.
|
||||
if len(p.TargetServerIDs) == 0 && len(p.TargetTags) == 0 {
|
||||
return ErrNoTargets
|
||||
}
|
||||
return ValidateTags(p.TargetTags)
|
||||
}
|
||||
|
||||
func nextRunFor(p models.PatchPolicy, w models.MaintenanceWindow, now time.Time) *time.Time {
|
||||
if !p.Enabled {
|
||||
return nil
|
||||
}
|
||||
start, err := patchsched.NextStart(w.Cron, w.TZ, now)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
return &start
|
||||
}
|
||||
|
||||
func ListPolicies(instanceID string) ([]models.PatchPolicy, error) {
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
cur, err := db.Col("patch_policies").Find(ctx, bson.M{"instance_id": instanceID}, options.Find().SetSort(bson.M{"name": 1}))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := []models.PatchPolicy{}
|
||||
return out, cur.All(ctx, &out)
|
||||
}
|
||||
|
||||
func GetPolicy(instanceID, policyID string) (*models.PatchPolicy, error) {
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
var p models.PatchPolicy
|
||||
err := db.Col("patch_policies").FindOne(ctx, bson.M{"instance_id": instanceID, "policy_id": policyID}).Decode(&p)
|
||||
if errors.Is(err, mongo.ErrNoDocuments) {
|
||||
return nil, ErrPolicyNotFound
|
||||
}
|
||||
return &p, err
|
||||
}
|
||||
|
||||
// preparePolicy runs every check a save needs and returns the policy's window.
|
||||
func preparePolicy(instanceID string, p *models.PatchPolicy, tokenScope map[string]string) (*models.MaintenanceWindow, error) {
|
||||
p.Name = strings.TrimSpace(p.Name)
|
||||
if p.TargetServerIDs == nil {
|
||||
p.TargetServerIDs = []string{}
|
||||
}
|
||||
if err := ValidatePolicy(*p); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
w, err := GetWindow(instanceID, p.WindowID)
|
||||
if errors.Is(err, ErrWindowNotFound) {
|
||||
return nil, fmt.Errorf("%w: that maintenance window does not exist", ErrPolicyInvalid)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// validateTargetServers returns a plain "target server X not found"; wrap
|
||||
// it so the handler can answer 400 without matching on text.
|
||||
if err := validateTargetServers(instanceID, p.TargetServerIDs, tokenScope); err != nil {
|
||||
return nil, fmt.Errorf("%w: %v", ErrPolicyInvalid, err)
|
||||
}
|
||||
if err := validateWorkflowTargetScope(instanceID, p.TargetServerIDs, p.TargetTags, tokenScope); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return w, nil
|
||||
}
|
||||
|
||||
func CreatePolicy(instanceID string, p models.PatchPolicy, tokenScope map[string]string) (*models.PatchPolicy, error) {
|
||||
w, err := preparePolicy(instanceID, &p, tokenScope)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
now := time.Now()
|
||||
p.InstanceID, p.PolicyID = instanceID, uuid.New().String()
|
||||
p.CreatedAt, p.UpdatedAt = now, now
|
||||
p.NextRunAt = nextRunFor(p, *w, now)
|
||||
p.LastRunAt, p.LastSkipped, p.DisabledReason = nil, nil, ""
|
||||
if _, err := db.Col("patch_policies").InsertOne(ctx, p); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &p, nil
|
||||
}
|
||||
|
||||
func UpdatePolicy(instanceID, policyID string, p models.PatchPolicy, tokenScope map[string]string) (*models.PatchPolicy, error) {
|
||||
// Missing policy answers 404 before any validation error.
|
||||
if _, err := GetPolicy(instanceID, policyID); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
w, err := preparePolicy(instanceID, &p, tokenScope)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
now := time.Now()
|
||||
set := bson.M{
|
||||
"name": p.Name, "enabled": p.Enabled, "window_id": p.WindowID,
|
||||
"target_server_ids": p.TargetServerIDs, "target_tags": p.TargetTags,
|
||||
"scope": p.Scope, "reboot": p.Reboot, "max_concurrent": p.MaxConcurrent,
|
||||
"notify_channel_ids": p.NotifyChannelIDs, "updated_at": now,
|
||||
}
|
||||
update := bson.M{"$set": set}
|
||||
if next := nextRunFor(p, *w, now); next != nil {
|
||||
set["next_run_at"] = *next
|
||||
// Re-enabling is the operator's answer to whatever disabled it.
|
||||
update["$unset"] = bson.M{"disabled_reason": ""}
|
||||
} else {
|
||||
update["$unset"] = bson.M{"next_run_at": ""}
|
||||
}
|
||||
if _, err := db.Col("patch_policies").UpdateOne(ctx, bson.M{"instance_id": instanceID, "policy_id": policyID}, update); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return GetPolicy(instanceID, policyID)
|
||||
}
|
||||
|
||||
func DeletePolicy(instanceID, policyID string) error {
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
res, err := db.Col("patch_policies").DeleteOne(ctx, bson.M{"instance_id": instanceID, "policy_id": policyID})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if res.DeletedCount == 0 {
|
||||
return ErrPolicyNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// recomputePolicySchedules moves every enabled policy on this window to the
|
||||
// window's next start.
|
||||
func recomputePolicySchedules(ctx context.Context, w models.MaintenanceWindow) error {
|
||||
start, err := patchsched.NextStart(w.Cron, w.TZ, time.Now())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = db.Col("patch_policies").UpdateMany(ctx,
|
||||
bson.M{"instance_id": w.InstanceID, "window_id": w.WindowID, "enabled": true},
|
||||
bson.M{"$set": bson.M{"next_run_at": start}})
|
||||
return err
|
||||
}
|
||||
|
||||
// CountPolicyTargets resolves the selector as a run would, at this moment.
|
||||
func CountPolicyTargets(p models.PatchPolicy) (int, error) {
|
||||
servers, err := ResolveTargets(p.InstanceID, p.TargetServerIDs, p.TargetTags)
|
||||
if errors.Is(err, ErrNoTargets) {
|
||||
return 0, nil
|
||||
}
|
||||
return len(servers), err
|
||||
}
|
||||
|
||||
var ErrPatchRunActive = errors.New("a run of this policy is already in progress")
|
||||
|
||||
// PoliciesForWindow lists every policy that uses a window, enabled or not.
|
||||
func PoliciesForWindow(instanceID, windowID string) ([]models.PatchPolicy, error) {
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
cur, err := db.Col("patch_policies").Find(ctx, bson.M{"instance_id": instanceID, "window_id": windowID})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := []models.PatchPolicy{}
|
||||
return out, cur.All(ctx, &out)
|
||||
}
|
||||
|
||||
// CheckWindowScope refuses a tag-restricted token changing a window when any
|
||||
// policy using it targets servers outside its restriction: moving or
|
||||
// deleting the window moves or stops those servers' patching.
|
||||
func CheckWindowScope(instanceID, windowID string, tokenScope map[string]string) error {
|
||||
if len(tokenScope) == 0 {
|
||||
return nil
|
||||
}
|
||||
ps, err := PoliciesForWindow(instanceID, windowID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, p := range ps {
|
||||
if err := CheckPolicyScope(instanceID, p, tokenScope); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// CheckPolicyScope refuses a tag-restricted token acting on a policy whose
|
||||
// targets reach outside its restriction, with the workflow rule unchanged.
|
||||
func CheckPolicyScope(instanceID string, p models.PatchPolicy, tokenScope map[string]string) error {
|
||||
return validateWorkflowTargetScope(instanceID, p.TargetServerIDs, p.TargetTags, tokenScope)
|
||||
}
|
||||
|
||||
// StartRunNow opens a window of the policy's usual length starting now. It is
|
||||
// how an operator tests a policy on a Tuesday afternoon.
|
||||
func StartRunNow(instanceID, policyID, actor string, tokenScope map[string]string) (*models.PatchRun, error) {
|
||||
p, err := GetPolicy(instanceID, policyID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := CheckPolicyScope(instanceID, *p, tokenScope); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
w, err := GetWindow(instanceID, p.WindowID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
if db.Col("patch_runs").FindOne(ctx, bson.M{"instance_id": instanceID, "policy_id": policyID, "status": models.PatchRunRunning}).Err() == nil {
|
||||
return nil, ErrPatchRunActive
|
||||
}
|
||||
return StartPolicyRun(*p, patchsched.WindowEnd(time.Now(), w.DurationMinutes), models.PatchSourceRunNow, actor)
|
||||
}
|
||||
@@ -0,0 +1,623 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/db"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/notify"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/patchrun"
|
||||
"gitea.hostxtra.co.uk/vantage/vantage-shared/grpc/pb"
|
||||
"github.com/google/uuid"
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo/options"
|
||||
)
|
||||
|
||||
// ErrAgentOffline is declared in consoleproxy.go and reused here: both mean
|
||||
// the same thing, the target's agent is not on the command stream.
|
||||
var (
|
||||
ErrPatchRunNotFound = errors.New("patch run not found")
|
||||
ErrPatchRunFinished = errors.New("patch run has already finished")
|
||||
)
|
||||
|
||||
const patchRunsCol = "patch_runs"
|
||||
|
||||
// patchRunOutputsCol holds each server's output tail, one document per
|
||||
// (run_id, server_id). Kept out of the run document because a large run with
|
||||
// up to 64KB of output per server would pass MongoDB's 16MB document limit.
|
||||
const patchRunOutputsCol = "patch_run_outputs"
|
||||
|
||||
// storePatchOutput upserts one server's output tail for a run.
|
||||
func storePatchOutput(ctx context.Context, instanceID, runID, serverID, output string) error {
|
||||
_, err := db.Col(patchRunOutputsCol).UpdateOne(ctx,
|
||||
bson.M{"instance_id": instanceID, "run_id": runID, "server_id": serverID},
|
||||
bson.M{"$set": bson.M{"output": output, "updated_at": time.Now()}},
|
||||
options.UpdateOne().SetUpsert(true))
|
||||
return err
|
||||
}
|
||||
|
||||
// fillPatchOutputs puts each server's stored output back on the run, so the
|
||||
// API response carries it as before. A run written before outputs moved out
|
||||
// keeps the output it has inline.
|
||||
func fillPatchOutputs(ctx context.Context, run *models.PatchRun) error {
|
||||
cur, err := db.Col(patchRunOutputsCol).Find(ctx, bson.M{"instance_id": run.InstanceID, "run_id": run.RunID})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var outs []struct {
|
||||
ServerID string `bson:"server_id"`
|
||||
Output string `bson:"output"`
|
||||
}
|
||||
if err := cur.All(ctx, &outs); err != nil {
|
||||
return err
|
||||
}
|
||||
byServer := make(map[string]string, len(outs))
|
||||
for _, o := range outs {
|
||||
byServer[o.ServerID] = o.Output
|
||||
}
|
||||
for i := range run.Servers {
|
||||
if out, ok := byServer[run.Servers[i].ServerID]; ok {
|
||||
run.Servers[i].Output = out
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// serverBootTime is the boot time the server last reported, or nil.
|
||||
func serverBootTime(ctx context.Context, instanceID, serverID string) *time.Time {
|
||||
var doc struct {
|
||||
Inventory struct {
|
||||
BootTime *time.Time `bson:"boot_time"`
|
||||
} `bson:"inventory"`
|
||||
}
|
||||
if err := db.Col("servers").FindOne(ctx, bson.M{"instance_id": instanceID, "server_id": serverID},
|
||||
options.FindOne().SetProjection(bson.M{"inventory.boot_time": 1})).Decode(&doc); err != nil {
|
||||
return nil
|
||||
}
|
||||
return doc.Inventory.BootTime
|
||||
}
|
||||
|
||||
func newServerRun(s models.Server, now time.Time) models.PatchServerRun {
|
||||
r := models.PatchServerRun{ServerID: s.ServerID, Hostname: s.Hostname, Status: models.PatchSrvQueued, PendingBefore: len(s.AvailableUpdates)}
|
||||
if !patchrun.AgentSupportsPatchResults(s.AgentVersion) {
|
||||
v := s.AgentVersion
|
||||
if v == "" {
|
||||
v = "unknown"
|
||||
}
|
||||
r.Status = models.PatchSrvAgentTooOld
|
||||
r.Error = fmt.Sprintf("agent %s predates patch results; update it to %s or later", v, patchrun.MinAgentVersion)
|
||||
r.FinishedAt = &now
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
// StartPolicyRun records a run for a policy and dispatches its first batch
|
||||
// immediately. The run document is written before any command leaves, so a
|
||||
// fast agent's result always finds it.
|
||||
func StartPolicyRun(p models.PatchPolicy, windowEnd time.Time, source, actor string) (*models.PatchRun, error) {
|
||||
targets, err := ResolveTargets(p.InstanceID, p.TargetServerIDs, p.TargetTags)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
now := time.Now()
|
||||
run := models.PatchRun{
|
||||
InstanceID: p.InstanceID, RunID: uuid.New().String(), PolicyID: p.PolicyID, PolicyName: p.Name,
|
||||
TriggeredBy: actor, Source: source, Scope: p.Scope, Reboot: p.Reboot, MaxConcurrent: p.MaxConcurrent,
|
||||
WindowEnd: &windowEnd, Status: models.PatchRunRunning, StartedAt: now,
|
||||
}
|
||||
for _, s := range targets {
|
||||
run.Servers = append(run.Servers, newServerRun(s, now))
|
||||
}
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
if _, err := db.Col(patchRunsCol).InsertOne(ctx, run); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
LogEvent(p.InstanceID, "patch.run_started", actor, "", "",
|
||||
fmt.Sprintf("patch policy %s started on %d servers (%s, reboot %s)", p.Name, len(run.Servers), p.Scope, p.Reboot))
|
||||
advanceRun(ctx, run.RunID)
|
||||
return &run, nil
|
||||
}
|
||||
|
||||
// StartManualRun is Apply updates on one server. An agent too old to answer
|
||||
// still gets the command it always got, and the run says it cannot know the
|
||||
// outcome rather than claiming one.
|
||||
func StartManualRun(instanceID string, srv *models.Server, actor, source string) (*models.PatchRun, error) {
|
||||
now := time.Now()
|
||||
sr := newServerRun(*srv, now)
|
||||
legacy := sr.Status == models.PatchSrvAgentTooOld
|
||||
sr.Status, sr.Error, sr.FinishedAt, sr.StartedAt = models.PatchSrvPatching, "", nil, &now
|
||||
run := models.PatchRun{
|
||||
InstanceID: instanceID, RunID: uuid.New().String(), TriggeredBy: actor, Source: source,
|
||||
Scope: models.PatchScopeAll, Reboot: models.PatchRebootNever, Status: models.PatchRunRunning, StartedAt: now,
|
||||
}
|
||||
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
|
||||
var dispatchErr error
|
||||
if !Dispatcher.IsConnected(srv.ServerID) {
|
||||
dispatchErr = ErrAgentOffline
|
||||
}
|
||||
if dispatchErr == nil && legacy {
|
||||
dispatchErr = DispatchApplyUpdates(srv.ServerID)
|
||||
if dispatchErr == nil {
|
||||
sr.Status = models.PatchSrvSucceeded
|
||||
sr.Error = "no result reported: agent predates patch results"
|
||||
sr.FinishedAt = &now
|
||||
}
|
||||
}
|
||||
if dispatchErr != nil {
|
||||
sr.Status, sr.Error, sr.FinishedAt = models.PatchSrvFailed, dispatchErr.Error(), &now
|
||||
} else if !legacy {
|
||||
sr.CommandID = uuid.New().String()
|
||||
}
|
||||
run.Servers = []models.PatchServerRun{sr}
|
||||
if _, err := db.Col(patchRunsCol).InsertOne(ctx, run); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if dispatchErr == nil && !legacy {
|
||||
if err := dispatchPatch(srv.ServerID, sr.CommandID, run); err != nil {
|
||||
_, _ = setServer(ctx, run.RunID, srv.ServerID, models.PatchSrvPatching,
|
||||
bson.M{"status": models.PatchSrvFailed, "error": err.Error(), "finished_at": time.Now()})
|
||||
dispatchErr = ErrAgentOffline
|
||||
}
|
||||
}
|
||||
finalizeRun(ctx, run.RunID)
|
||||
if dispatchErr != nil {
|
||||
return &run, ErrAgentOffline
|
||||
}
|
||||
return &run, nil
|
||||
}
|
||||
|
||||
func dispatchPatch(serverID, commandID string, run models.PatchRun) error {
|
||||
cmd := &pb.ApplyUpdatesCmd{Scope: run.Scope, RebootIfRequired: run.Reboot == models.PatchRebootIfRequired}
|
||||
if run.WindowEnd != nil {
|
||||
cmd.DeadlineUnix = run.WindowEnd.Unix()
|
||||
}
|
||||
return Dispatcher.dispatch(serverID, &pb.ServerCommand{CommandId: commandID, ApplyUpdates: cmd})
|
||||
}
|
||||
|
||||
// setServer updates one server run, but only if it is still in status from.
|
||||
// That guard is what stops a tick from overwriting a result that landed while
|
||||
// the tick was working.
|
||||
func setServer(ctx context.Context, runID, serverID, from string, set bson.M) (bool, error) {
|
||||
fields := bson.M{}
|
||||
for k, v := range set {
|
||||
fields["servers.$."+k] = v
|
||||
}
|
||||
res, err := db.Col(patchRunsCol).UpdateOne(ctx,
|
||||
bson.M{"run_id": runID, "servers": bson.M{"$elemMatch": bson.M{"server_id": serverID, "status": from}}},
|
||||
bson.M{"$set": fields})
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return res.MatchedCount > 0, nil
|
||||
}
|
||||
|
||||
// setServerForCommand is setServer scoped to one command: a late result from
|
||||
// an older, already-superseded command (say, a fresh dispatch to the same
|
||||
// server after a failed send) must not land on the new attempt.
|
||||
func setServerForCommand(ctx context.Context, runID, serverID, commandID, from string, set bson.M) (bool, error) {
|
||||
fields := bson.M{}
|
||||
for k, v := range set {
|
||||
fields["servers.$."+k] = v
|
||||
}
|
||||
res, err := db.Col(patchRunsCol).UpdateOne(ctx,
|
||||
bson.M{"run_id": runID, "servers": bson.M{"$elemMatch": bson.M{"server_id": serverID, "status": from, "command_id": commandID}}},
|
||||
bson.M{"$set": fields})
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return res.MatchedCount > 0, nil
|
||||
}
|
||||
|
||||
// claimServerForDispatch atomically moves one server from queued/waiting_offline
|
||||
// to patching, guarded on the run still being running and not cancelled: a
|
||||
// tick that loaded the run before CancelPatchRun wrote cancelled_at must not
|
||||
// dispatch to it.
|
||||
func claimServerForDispatch(ctx context.Context, runID, serverID, from, commandID string, now time.Time) (bool, error) {
|
||||
res, err := db.Col(patchRunsCol).UpdateOne(ctx,
|
||||
bson.M{
|
||||
"run_id": runID,
|
||||
"status": models.PatchRunRunning,
|
||||
"cancelled_at": bson.M{"$exists": false},
|
||||
"servers": bson.M{"$elemMatch": bson.M{"server_id": serverID, "status": from}},
|
||||
},
|
||||
bson.M{"$set": bson.M{
|
||||
"servers.$.status": models.PatchSrvPatching,
|
||||
"servers.$.command_id": commandID,
|
||||
"servers.$.started_at": now,
|
||||
}})
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return res.MatchedCount > 0, nil
|
||||
}
|
||||
|
||||
// loadRun reads a run for the tick and result paths, which never need output,
|
||||
// so any inline output left on an old document is not read either.
|
||||
func loadRun(ctx context.Context, filter bson.M) (*models.PatchRun, error) {
|
||||
var run models.PatchRun
|
||||
err := db.Col(patchRunsCol).FindOne(ctx, filter, options.FindOne().SetProjection(bson.M{"servers.output": 0})).Decode(&run)
|
||||
if errors.Is(err, mongo.ErrNoDocuments) {
|
||||
return nil, ErrPatchRunNotFound
|
||||
}
|
||||
return &run, err
|
||||
}
|
||||
|
||||
// AdvancePatchRuns is the scheduler's second job each tick.
|
||||
func AdvancePatchRuns(ctx context.Context) {
|
||||
cur, err := db.Col(patchRunsCol).Find(ctx, bson.M{"status": models.PatchRunRunning}, options.Find().SetProjection(bson.M{"run_id": 1}))
|
||||
if err != nil {
|
||||
log.Printf("patch runs: find running: %v", err)
|
||||
return
|
||||
}
|
||||
var ids []struct {
|
||||
RunID string `bson:"run_id"`
|
||||
}
|
||||
if err := cur.All(ctx, &ids); err != nil {
|
||||
log.Printf("patch runs: decode running: %v", err)
|
||||
return
|
||||
}
|
||||
for _, r := range ids {
|
||||
if ctx.Err() != nil {
|
||||
return
|
||||
}
|
||||
advanceRun(ctx, r.RunID)
|
||||
}
|
||||
}
|
||||
|
||||
func advanceRun(ctx context.Context, runID string) {
|
||||
run, err := loadRun(ctx, bson.M{"run_id": runID})
|
||||
if err != nil {
|
||||
log.Printf("patch run %s: load: %v", runID, err)
|
||||
return
|
||||
}
|
||||
now := time.Now()
|
||||
connected := map[string]bool{}
|
||||
for _, s := range run.Servers {
|
||||
if !patchrun.IsTerminal(s.Status) {
|
||||
connected[s.ServerID] = Dispatcher.IsConnected(s.ServerID)
|
||||
}
|
||||
}
|
||||
for _, tr := range patchrun.Advance(*run, now, connected) {
|
||||
if tr.Dispatch {
|
||||
// Each server's claim and any reset-after-failed-send get their
|
||||
// own fresh, short-lived context rather than sharing the caller's:
|
||||
// advanceRun can dispatch to many servers in one tick, each ack
|
||||
// waiting up to Dispatcher's own timeout, and a caller-supplied
|
||||
// context (StartPolicyRun's patchCtx, or a leader context that
|
||||
// AdvancePatchRuns loses partway through) would otherwise expire
|
||||
// mid-loop and silently drop a later server's writes.
|
||||
cmdID := uuid.New().String()
|
||||
claimCtx, claimCancel := patchCtx()
|
||||
ok, err := claimServerForDispatch(claimCtx, runID, tr.ServerID, tr.From, cmdID, now)
|
||||
claimCancel()
|
||||
if err != nil {
|
||||
log.Printf("patch run %s: server %s: claim: %v", runID, tr.ServerID, err)
|
||||
continue
|
||||
}
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if err := dispatchPatch(tr.ServerID, cmdID, *run); err != nil {
|
||||
// The agent dropped between the connection check and the send.
|
||||
// Back to waiting: the next tick tries again while the window
|
||||
// is open.
|
||||
resetCtx, resetCancel := patchCtx()
|
||||
if _, rerr := setServerForCommand(resetCtx, runID, tr.ServerID, cmdID, models.PatchSrvPatching,
|
||||
bson.M{"status": models.PatchSrvWaitingOffline, "command_id": ""}); rerr != nil {
|
||||
log.Printf("patch run %s: server %s: reset after failed dispatch: %v", runID, tr.ServerID, rerr)
|
||||
}
|
||||
resetCancel()
|
||||
}
|
||||
continue
|
||||
}
|
||||
set := bson.M{"status": tr.To}
|
||||
if tr.Error != "" {
|
||||
set["error"] = tr.Error
|
||||
}
|
||||
if patchrun.IsTerminal(tr.To) {
|
||||
set["finished_at"] = now
|
||||
}
|
||||
if _, err := setServer(ctx, runID, tr.ServerID, tr.From, set); err != nil {
|
||||
log.Printf("patch run %s: server %s: %v", runID, tr.ServerID, err)
|
||||
}
|
||||
}
|
||||
finalizeRun(ctx, runID)
|
||||
}
|
||||
|
||||
func finalizeRun(ctx context.Context, runID string) {
|
||||
run, err := loadRun(ctx, bson.M{"run_id": runID})
|
||||
if err != nil || run.Status != models.PatchRunRunning {
|
||||
return
|
||||
}
|
||||
status, done := patchrun.Finalize(*run)
|
||||
if !done {
|
||||
return
|
||||
}
|
||||
now := time.Now()
|
||||
res, err := db.Col(patchRunsCol).UpdateOne(ctx,
|
||||
bson.M{"run_id": runID, "status": models.PatchRunRunning},
|
||||
bson.M{"$set": bson.M{"status": status, "finished_at": now}})
|
||||
if err != nil || res.MatchedCount == 0 {
|
||||
return // finalised by someone else
|
||||
}
|
||||
run.Status = status
|
||||
name := run.PolicyName
|
||||
if name == "" {
|
||||
name = "manual update"
|
||||
}
|
||||
LogEvent(run.InstanceID, "patch.run_finished", run.TriggeredBy, "", "",
|
||||
fmt.Sprintf("patch run %s (%s) %s: %s", runID, name, status, patchrun.Summary(*run)))
|
||||
if status == models.PatchRunPartial || status == models.PatchRunFailed ||
|
||||
(status == models.PatchRunCancelled && hasBadOutcome(*run)) {
|
||||
notifyPatchRun(*run)
|
||||
}
|
||||
}
|
||||
|
||||
// hasBadOutcome reports whether any server failed or was not reached. A
|
||||
// cancelled run with such a result still alerts: cancelling does not make a
|
||||
// failure somebody else's problem.
|
||||
func hasBadOutcome(run models.PatchRun) bool {
|
||||
for _, s := range run.Servers {
|
||||
switch s.Status {
|
||||
case models.PatchSrvFailed, models.PatchSrvMissedOffline, models.PatchSrvWindowClosed:
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func notifyPatchRun(run models.PatchRun) {
|
||||
if run.PolicyID == "" {
|
||||
return // a manual run was watched by the person who clicked
|
||||
}
|
||||
p, err := GetPolicy(run.InstanceID, run.PolicyID)
|
||||
if err != nil || len(p.NotifyChannelIDs) == 0 {
|
||||
return
|
||||
}
|
||||
chs, err := GetChannels(run.InstanceID, p.NotifyChannelIDs)
|
||||
if err != nil {
|
||||
log.Printf("patch run %s: channels: %v", run.RunID, err)
|
||||
return
|
||||
}
|
||||
ev := notify.Event{
|
||||
MonitorName: fmt.Sprintf("Patch policy %q %s", run.PolicyName, run.Status),
|
||||
Type: notify.TypePatch,
|
||||
NewStatus: run.Status,
|
||||
Message: fmt.Sprintf("%s (run %s)", patchrun.Summary(run), run.RunID),
|
||||
Time: time.Now(),
|
||||
}
|
||||
for _, ch := range chs {
|
||||
if err := notify.Dispatch(ch, ev); err != nil {
|
||||
log.Printf("patch run %s: notify %s: %v", run.RunID, ch.Name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// RecordPatchResult is called by whichever pod holds the agent's stream. The
|
||||
// filter names this agent's own server, so one agent cannot answer for
|
||||
// another's command.
|
||||
func RecordPatchResult(instanceID, serverID string, r *pb.PatchResult) {
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
run, err := loadRun(ctx, bson.M{"instance_id": instanceID,
|
||||
"servers": bson.M{"$elemMatch": bson.M{"server_id": serverID, "command_id": r.CommandId}}})
|
||||
if err != nil {
|
||||
log.Printf("patch result %s from %s matches no run: %v", r.CommandId, serverID, err)
|
||||
return
|
||||
}
|
||||
for _, s := range run.Servers {
|
||||
if s.ServerID != serverID || s.CommandID != r.CommandId {
|
||||
continue
|
||||
}
|
||||
updated, ok := patchrun.ApplyResult(s, r, time.Now())
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
// The output goes to patch_run_outputs, not the run document.
|
||||
set := bson.M{"status": updated.Status, "error": updated.Error}
|
||||
if updated.PendingAfter != nil {
|
||||
set["pending_after"] = *updated.PendingAfter
|
||||
}
|
||||
if updated.RebootedAt != nil {
|
||||
set["rebooted_at"] = *updated.RebootedAt
|
||||
}
|
||||
if updated.Status == models.PatchSrvRebooting {
|
||||
// The boot time before the reboot: a later report with a
|
||||
// different one proves the restart.
|
||||
if bt := serverBootTime(ctx, instanceID, serverID); bt != nil {
|
||||
set["boot_time_before"] = *bt
|
||||
}
|
||||
}
|
||||
if updated.FinishedAt != nil {
|
||||
set["finished_at"] = *updated.FinishedAt
|
||||
}
|
||||
ok, err := setServerForCommand(ctx, run.RunID, serverID, r.CommandId, models.PatchSrvPatching, set)
|
||||
if err != nil {
|
||||
log.Printf("patch run %s: server %s: record result: %v", run.RunID, serverID, err)
|
||||
}
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if err := storePatchOutput(ctx, instanceID, run.RunID, serverID, updated.Output); err != nil {
|
||||
log.Printf("patch run %s: server %s: store output: %v", run.RunID, serverID, err)
|
||||
}
|
||||
if updated.Status == models.PatchSrvRebooting {
|
||||
LogEvent(instanceID, "patch.reboot", run.TriggeredBy, serverID, "",
|
||||
fmt.Sprintf("%s rebooting for patch policy %s", s.Hostname, run.PolicyName))
|
||||
}
|
||||
}
|
||||
finalizeRun(ctx, run.RunID)
|
||||
}
|
||||
|
||||
// VerifyPatchReboots settles any rebooting server run for this server from a
|
||||
// static inventory report.
|
||||
func VerifyPatchReboots(instanceID, serverID string, bootTime time.Time, rebootRequired bool) {
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
cur, err := db.Col(patchRunsCol).Find(ctx, bson.M{"instance_id": instanceID, "status": models.PatchRunRunning,
|
||||
"servers": bson.M{"$elemMatch": bson.M{"server_id": serverID, "status": models.PatchSrvRebooting}}})
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
var runs []models.PatchRun
|
||||
if err := cur.All(ctx, &runs); err != nil {
|
||||
return
|
||||
}
|
||||
now := time.Now()
|
||||
for _, run := range runs {
|
||||
for _, s := range run.Servers {
|
||||
if s.ServerID != serverID {
|
||||
continue
|
||||
}
|
||||
updated, ok := patchrun.VerifyReboot(s, bootTime, rebootRequired, now)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
set := bson.M{"status": updated.Status, "error": updated.Error, "finished_at": now}
|
||||
if updated.VerifiedAt != nil {
|
||||
set["verified_at"] = *updated.VerifiedAt
|
||||
}
|
||||
_, _ = setServer(ctx, run.RunID, serverID, models.PatchSrvRebooting, set)
|
||||
}
|
||||
finalizeRun(ctx, run.RunID)
|
||||
}
|
||||
}
|
||||
|
||||
// CancelPatchRun stops further dispatch. Servers already patching finish:
|
||||
// killing a package manager mid-transaction is worse than letting it end.
|
||||
func CancelPatchRun(instanceID, runID string) error {
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
res, err := db.Col(patchRunsCol).UpdateOne(ctx,
|
||||
bson.M{"instance_id": instanceID, "run_id": runID, "status": models.PatchRunRunning, "cancelled_at": bson.M{"$exists": false}},
|
||||
bson.M{"$set": bson.M{"cancelled_at": time.Now()}})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if res.MatchedCount == 0 {
|
||||
if _, err := GetPatchRun(instanceID, runID); err != nil {
|
||||
return err
|
||||
}
|
||||
return ErrPatchRunFinished
|
||||
}
|
||||
advanceRun(ctx, runID)
|
||||
return nil
|
||||
}
|
||||
|
||||
func GetPatchRun(instanceID, runID string) (*models.PatchRun, error) {
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
var run models.PatchRun
|
||||
err := db.Col(patchRunsCol).FindOne(ctx, bson.M{"instance_id": instanceID, "run_id": runID}).Decode(&run)
|
||||
if errors.Is(err, mongo.ErrNoDocuments) {
|
||||
return nil, ErrPatchRunNotFound
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := fillPatchOutputs(ctx, &run); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &run, nil
|
||||
}
|
||||
|
||||
// ListPatchRuns omits output: a list of fifty runs would otherwise carry up
|
||||
// to 64KB per server.
|
||||
func ListPatchRuns(instanceID, policyID, serverID string, limit int64) ([]models.PatchRun, error) {
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
filter := bson.M{"instance_id": instanceID}
|
||||
if policyID != "" {
|
||||
filter["policy_id"] = policyID
|
||||
}
|
||||
if serverID != "" {
|
||||
filter["servers.server_id"] = serverID
|
||||
}
|
||||
if limit <= 0 || limit > 200 {
|
||||
limit = 50
|
||||
}
|
||||
cur, err := db.Col(patchRunsCol).Find(ctx, filter, options.Find().
|
||||
SetSort(bson.M{"started_at": -1}).SetLimit(limit).SetProjection(bson.M{"servers.output": 0}))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := []models.PatchRun{}
|
||||
return out, cur.All(ctx, &out)
|
||||
}
|
||||
|
||||
// ScopePatchRun removes servers outside a tag-restricted token's reach, so a
|
||||
// run record cannot name a host the caller could not otherwise see.
|
||||
func ScopePatchRun(instanceID string, run *models.PatchRun, tokenScope map[string]string) error {
|
||||
if len(tokenScope) == 0 {
|
||||
return nil
|
||||
}
|
||||
ids := make([]string, 0, len(run.Servers))
|
||||
for _, s := range run.Servers {
|
||||
ids = append(ids, s.ServerID)
|
||||
}
|
||||
visible, err := ResolveTargetsScoped(instanceID, ids, nil, tokenScope)
|
||||
if err != nil && !errors.Is(err, ErrNoTargets) {
|
||||
return err
|
||||
}
|
||||
keep := map[string]bool{}
|
||||
for _, s := range visible {
|
||||
keep[s.ServerID] = true
|
||||
}
|
||||
kept := run.Servers[:0]
|
||||
for _, s := range run.Servers {
|
||||
if keep[s.ServerID] {
|
||||
kept = append(kept, s)
|
||||
}
|
||||
}
|
||||
run.Servers = kept
|
||||
return nil
|
||||
}
|
||||
|
||||
// sweepPatchRuns deletes finished runs past their instance's workflow log
|
||||
// retention: they are the same kind of record, and one setting governs both.
|
||||
func sweepPatchRuns() {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
|
||||
defer cancel()
|
||||
cur, err := db.Col(patchRunsCol).Find(ctx, bson.M{"finished_at": bson.M{"$ne": nil}},
|
||||
options.Find().SetProjection(bson.M{"run_id": 1, "instance_id": 1, "finished_at": 1}))
|
||||
if err != nil {
|
||||
log.Printf("patch run sweep: %v", err)
|
||||
return
|
||||
}
|
||||
defer cur.Close(ctx)
|
||||
cache := map[string]int{}
|
||||
now := time.Now()
|
||||
for cur.Next(ctx) {
|
||||
var r struct {
|
||||
RunID string `bson:"run_id"`
|
||||
InstanceID string `bson:"instance_id"`
|
||||
FinishedAt *time.Time `bson:"finished_at"`
|
||||
}
|
||||
if cur.Decode(&r) != nil || r.FinishedAt == nil {
|
||||
continue
|
||||
}
|
||||
days, ok := cache[r.InstanceID]
|
||||
if !ok {
|
||||
days = defaultRetentionDays
|
||||
if v, err := GetWorkflowLogRetentionDays(r.InstanceID); err == nil {
|
||||
days = v
|
||||
}
|
||||
cache[r.InstanceID] = days
|
||||
}
|
||||
if days <= 0 || !r.FinishedAt.Before(now.AddDate(0, 0, -days)) {
|
||||
continue
|
||||
}
|
||||
if _, err := db.Col(patchRunsCol).DeleteOne(ctx, bson.M{"run_id": r.RunID}); err == nil {
|
||||
_, _ = db.Col(patchRunOutputsCol).DeleteMany(ctx, bson.M{"instance_id": r.InstanceID, "run_id": r.RunID})
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
package services
|
||||
|
||||
import "testing"
|
||||
|
||||
// A tenant-scoped collection missing from ScopedCollections outlives its
|
||||
// instance when the instance is purged.
|
||||
func TestPatchCollectionsAreScoped(t *testing.T) {
|
||||
for _, name := range []string{"maintenance_windows", "patch_policies", "patch_runs", "patch_run_outputs"} {
|
||||
found := false
|
||||
for _, got := range ScopedCollections {
|
||||
if got == name {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Errorf("ScopedCollections is missing %q", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
|
||||
)
|
||||
|
||||
func goodWindow() models.MaintenanceWindow {
|
||||
return models.MaintenanceWindow{Name: "Sunday", Cron: "0 2 * * 0", TZ: "Europe/London", DurationMinutes: 120}
|
||||
}
|
||||
|
||||
func TestValidateWindow(t *testing.T) {
|
||||
if err := ValidateWindow(goodWindow()); err != nil {
|
||||
t.Fatalf("good window: %v", err)
|
||||
}
|
||||
bad := map[string]func(*models.MaintenanceWindow){
|
||||
"empty name": func(w *models.MaintenanceWindow) { w.Name = " " },
|
||||
"long name": func(w *models.MaintenanceWindow) { w.Name = strings.Repeat("a", 101) },
|
||||
"too short": func(w *models.MaintenanceWindow) { w.DurationMinutes = 14 },
|
||||
"too long": func(w *models.MaintenanceWindow) { w.DurationMinutes = 721 },
|
||||
"bad cron": func(w *models.MaintenanceWindow) { w.Cron = "every sunday" },
|
||||
"six-field cron": func(w *models.MaintenanceWindow) { w.Cron = "0 0 2 * * 0" },
|
||||
"bad tz": func(w *models.MaintenanceWindow) { w.TZ = "Mars/Olympus" },
|
||||
"no tz": func(w *models.MaintenanceWindow) { w.TZ = "" },
|
||||
}
|
||||
for name, mut := range bad {
|
||||
w := goodWindow()
|
||||
mut(&w)
|
||||
if err := ValidateWindow(w); !errors.Is(err, ErrWindowInvalid) {
|
||||
t.Errorf("%s: err = %v, want ErrWindowInvalid", name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPreviewWindowSpansDoNotOverlap(t *testing.T) {
|
||||
from := time.Date(2026, 9, 14, 12, 0, 0, 0, time.UTC)
|
||||
spans, err := PreviewWindow("0 * * * *", "UTC", 90, from, 3)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(spans) != 3 {
|
||||
t.Fatalf("got %d spans", len(spans))
|
||||
}
|
||||
for i := 1; i < len(spans); i++ {
|
||||
if spans[i].Start.Before(spans[i-1].End) {
|
||||
t.Fatalf("span %d starts %s before previous ends %s", i, spans[i].Start, spans[i-1].End)
|
||||
}
|
||||
}
|
||||
if !spans[0].End.Equal(spans[0].Start.Add(90 * time.Minute)) {
|
||||
t.Fatal("end must be start + duration")
|
||||
}
|
||||
}
|
||||
|
||||
func goodPolicy() models.PatchPolicy {
|
||||
return models.PatchPolicy{
|
||||
Name: "Sunday prod", WindowID: "w1", TargetTags: map[string]string{"env": "prod"},
|
||||
Scope: models.PatchScopeSecurity, Reboot: models.PatchRebootNever,
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidatePolicy(t *testing.T) {
|
||||
if err := ValidatePolicy(goodPolicy()); err != nil {
|
||||
t.Fatalf("good policy: %v", err)
|
||||
}
|
||||
bad := map[string]func(*models.PatchPolicy){
|
||||
"empty name": func(p *models.PatchPolicy) { p.Name = "" },
|
||||
"no window": func(p *models.PatchPolicy) { p.WindowID = "" },
|
||||
"bad scope": func(p *models.PatchPolicy) { p.Scope = "everything" },
|
||||
"bad reboot": func(p *models.PatchPolicy) { p.Reboot = "always" },
|
||||
"negative cap": func(p *models.PatchPolicy) { p.MaxConcurrent = -1 },
|
||||
"huge cap": func(p *models.PatchPolicy) { p.MaxConcurrent = 1001 },
|
||||
"uppercase tag": func(p *models.PatchPolicy) { p.TargetTags = map[string]string{"Env": "prod"} },
|
||||
}
|
||||
for name, mut := range bad {
|
||||
p := goodPolicy()
|
||||
mut(&p)
|
||||
if err := ValidatePolicy(p); err == nil {
|
||||
t.Errorf("%s: want an error", name)
|
||||
}
|
||||
}
|
||||
p := goodPolicy()
|
||||
p.TargetTags = nil
|
||||
if err := ValidatePolicy(p); !errors.Is(err, ErrNoTargets) {
|
||||
t.Errorf("empty selector: err = %v, want ErrNoTargets", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,166 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/db"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/patchsched"
|
||||
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/workflowsched"
|
||||
"github.com/google/uuid"
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo/options"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrWindowInvalid = errors.New("invalid maintenance window")
|
||||
ErrWindowNotFound = errors.New("maintenance window not found")
|
||||
ErrWindowInUse = errors.New("maintenance window is used by a patch policy")
|
||||
)
|
||||
|
||||
const (
|
||||
minWindowMinutes = 15
|
||||
maxWindowMinutes = 720
|
||||
)
|
||||
|
||||
func patchCtx() (context.Context, context.CancelFunc) {
|
||||
return context.WithTimeout(context.Background(), 10*time.Second)
|
||||
}
|
||||
|
||||
func ValidateWindow(w models.MaintenanceWindow) error {
|
||||
if n := strings.TrimSpace(w.Name); n == "" || len(n) > 100 {
|
||||
return fmt.Errorf("%w: name must be 1 to 100 characters", ErrWindowInvalid)
|
||||
}
|
||||
if w.DurationMinutes < minWindowMinutes || w.DurationMinutes > maxWindowMinutes {
|
||||
return fmt.Errorf("%w: duration must be between %d and %d minutes", ErrWindowInvalid, minWindowMinutes, maxWindowMinutes)
|
||||
}
|
||||
if _, err := workflowsched.ParseSchedule(w.Cron, w.TZ); err != nil {
|
||||
return fmt.Errorf("%w: %v", ErrWindowInvalid, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type WindowSpan struct {
|
||||
Start time.Time `json:"start"`
|
||||
End time.Time `json:"end"`
|
||||
}
|
||||
|
||||
// PreviewWindow returns the next n windows, computed exactly as the scheduler
|
||||
// computes them, so the editor cannot disagree with what will fire.
|
||||
func PreviewWindow(cron, tz string, durationMinutes int, from time.Time, n int) ([]WindowSpan, error) {
|
||||
if err := ValidateWindow(models.MaintenanceWindow{Name: "preview", Cron: cron, TZ: tz, DurationMinutes: durationMinutes}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := make([]WindowSpan, 0, n)
|
||||
for len(out) < n {
|
||||
start, err := patchsched.NextStart(cron, tz, from)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: %v", ErrWindowInvalid, err)
|
||||
}
|
||||
end := patchsched.WindowEnd(start, durationMinutes)
|
||||
out = append(out, WindowSpan{Start: start, End: end})
|
||||
from = patchsched.Later(start, end)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func ListWindows(instanceID string) ([]models.MaintenanceWindow, error) {
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
cur, err := db.Col("maintenance_windows").Find(ctx, bson.M{"instance_id": instanceID}, options.Find().SetSort(bson.M{"name": 1}))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := []models.MaintenanceWindow{}
|
||||
return out, cur.All(ctx, &out)
|
||||
}
|
||||
|
||||
func GetWindow(instanceID, windowID string) (*models.MaintenanceWindow, error) {
|
||||
w, err := LookupWindow(instanceID, windowID)
|
||||
if err == nil && w == nil {
|
||||
return nil, ErrWindowNotFound
|
||||
}
|
||||
return w, err
|
||||
}
|
||||
|
||||
// LookupWindow is GetWindow for the scheduler, which must tell "gone" (nil,
|
||||
// nil: disable the policy) from a database error (retry next tick).
|
||||
func LookupWindow(instanceID, windowID string) (*models.MaintenanceWindow, error) {
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
var w models.MaintenanceWindow
|
||||
err := db.Col("maintenance_windows").FindOne(ctx, bson.M{"instance_id": instanceID, "window_id": windowID}).Decode(&w)
|
||||
if errors.Is(err, mongo.ErrNoDocuments) {
|
||||
return nil, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &w, nil
|
||||
}
|
||||
|
||||
func CreateWindow(instanceID string, w models.MaintenanceWindow) (*models.MaintenanceWindow, error) {
|
||||
w.Name = strings.TrimSpace(w.Name)
|
||||
if err := ValidateWindow(w); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
w.InstanceID, w.WindowID = instanceID, uuid.New().String()
|
||||
w.CreatedAt = time.Now()
|
||||
w.UpdatedAt = w.CreatedAt
|
||||
if _, err := db.Col("maintenance_windows").InsertOne(ctx, w); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &w, nil
|
||||
}
|
||||
|
||||
// UpdateWindow saves the window and moves next_run_at on every enabled policy
|
||||
// that uses it, so an edited Sunday becomes the next Sunday everywhere at once.
|
||||
func UpdateWindow(instanceID, windowID string, w models.MaintenanceWindow) (*models.MaintenanceWindow, error) {
|
||||
w.Name = strings.TrimSpace(w.Name)
|
||||
if err := ValidateWindow(w); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
res, err := db.Col("maintenance_windows").UpdateOne(ctx,
|
||||
bson.M{"instance_id": instanceID, "window_id": windowID},
|
||||
bson.M{"$set": bson.M{"name": w.Name, "cron": w.Cron, "tz": w.TZ, "duration_minutes": w.DurationMinutes, "updated_at": time.Now()}})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if res.MatchedCount == 0 {
|
||||
return nil, ErrWindowNotFound
|
||||
}
|
||||
saved, err := GetWindow(instanceID, windowID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return saved, recomputePolicySchedules(ctx, *saved)
|
||||
}
|
||||
|
||||
func DeleteWindow(instanceID, windowID string) error {
|
||||
ctx, cancel := patchCtx()
|
||||
defer cancel()
|
||||
n, err := db.Col("patch_policies").CountDocuments(ctx, bson.M{"instance_id": instanceID, "window_id": windowID})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n > 0 {
|
||||
return ErrWindowInUse
|
||||
}
|
||||
res, err := db.Col("maintenance_windows").DeleteOne(ctx, bson.M{"instance_id": instanceID, "window_id": windowID})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if res.DeletedCount == 0 {
|
||||
return ErrWindowNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -11,8 +11,8 @@ import (
|
||||
// the vocabulary below.
|
||||
var ErrInvalidScope = errors.New("invalid scope")
|
||||
|
||||
// ScopeResources is the whole vocabulary. Ten resources, each with :read and
|
||||
// :write, and write implies read on the same resource.
|
||||
// ScopeResources is the whole vocabulary. Eleven resources, each with :read
|
||||
// and :write, and write implies read on the same resource.
|
||||
//
|
||||
// It is deliberately coarse. A scope per endpoint is a table nobody maintains,
|
||||
// and a route added without an entry either fails closed and breaks, or
|
||||
@@ -27,6 +27,7 @@ var ScopeResources = []string{
|
||||
"workloads",
|
||||
"settings",
|
||||
"status",
|
||||
"patching",
|
||||
// mcp:read is permission to reach the MCP endpoint at all; mcp:write is
|
||||
// permission for its write tools, which are not merely refused without it
|
||||
// but omitted from tools/list entirely.
|
||||
|
||||
@@ -332,6 +332,7 @@ func HasServerRunLog(runID, serverID string) bool {
|
||||
func StartLogSweeper(ctx context.Context) {
|
||||
go func() {
|
||||
sweepLogs()
|
||||
sweepPatchRuns()
|
||||
t := time.NewTicker(time.Hour)
|
||||
defer t.Stop()
|
||||
for {
|
||||
@@ -340,6 +341,7 @@ func StartLogSweeper(ctx context.Context) {
|
||||
return
|
||||
case <-t.C:
|
||||
sweepLogs()
|
||||
sweepPatchRuns()
|
||||
}
|
||||
}
|
||||
}()
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
"use client";
|
||||
|
||||
import { Suspense } from "react";
|
||||
import { useRouter, useSearchParams } from "next/navigation";
|
||||
import { useAuth } from "@/components/AuthProvider";
|
||||
import { Card, CenteredSpinner } from "@/components/ui";
|
||||
import { PolicyList } from "@/components/patching/PolicyList";
|
||||
import { WindowList } from "@/components/patching/WindowList";
|
||||
import { RunList } from "@/components/patching/RunList";
|
||||
|
||||
const TABS = [
|
||||
{ id: "policies", label: "Policies" },
|
||||
{ id: "windows", label: "Windows" },
|
||||
{ id: "runs", label: "Runs" },
|
||||
] as const;
|
||||
|
||||
type TabId = (typeof TABS)[number]["id"];
|
||||
|
||||
// useSearchParams is only allowed inside a Suspense boundary, as on the
|
||||
// servers page.
|
||||
export default function PatchingPage() {
|
||||
return (
|
||||
<Suspense fallback={<CenteredSpinner label="Loading patching" />}>
|
||||
<PatchingInner />
|
||||
</Suspense>
|
||||
);
|
||||
}
|
||||
|
||||
function PatchingInner() {
|
||||
const router = useRouter();
|
||||
const params = useSearchParams();
|
||||
const { isAdmin } = useAuth();
|
||||
const tab: TabId = (TABS.find((t) => t.id === params.get("tab"))?.id ?? "policies") as TabId;
|
||||
|
||||
return (
|
||||
<div className="p-4 sm:p-6 lg:p-8">
|
||||
<div className="mb-6">
|
||||
<h1 className="text-2xl font-semibold text-text-primary">Patching</h1>
|
||||
<p className="mt-1 max-w-2xl text-sm text-text-secondary">
|
||||
Patch servers inside maintenance windows. Every patch Vantage performs, scheduled or clicked, is recorded as a run with a result for each server.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div role="tablist" aria-label="Patching" className="mb-4 flex gap-1 border-b border-border">
|
||||
{TABS.map((t) => (
|
||||
<button
|
||||
key={t.id}
|
||||
role="tab"
|
||||
id={`patching-tab-${t.id}`}
|
||||
aria-selected={tab === t.id}
|
||||
onClick={() => router.replace(`/patching?tab=${t.id}`)}
|
||||
className={`-mb-px border-b-2 px-4 py-2 text-sm transition-colors focus:outline-none focus-visible:ring-2 focus-visible:ring-accent ${
|
||||
tab === t.id ? "border-accent text-text-primary" : "border-transparent text-text-secondary hover:text-text-primary"
|
||||
}`}
|
||||
>
|
||||
{t.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<Card padding={false} role="tabpanel" aria-labelledby={`patching-tab-${tab}`}>
|
||||
{tab === "policies" && <PolicyList canEdit={isAdmin} />}
|
||||
{tab === "windows" && <WindowList canEdit={isAdmin} />}
|
||||
{tab === "runs" && <RunList />}
|
||||
</Card>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,142 @@
|
||||
"use client";
|
||||
|
||||
import Link from "next/link";
|
||||
import { useState } from "react";
|
||||
import { useParams } from "next/navigation";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { api, PatchServerRun } from "@/lib/api";
|
||||
import { AsyncBoundary, Badge, Button, Card, CenteredSpinner, Table, Tbody, Td, Th, Thead, Tr, friendlyMessage, useToast } from "@/components/ui";
|
||||
import { RUN_STATUS, SERVER_STATUS } from "@/components/patching/status";
|
||||
|
||||
function installed(s: PatchServerRun): string {
|
||||
if (s.pending_after === undefined || s.pending_after === null) return "n/a";
|
||||
return String(Math.max(0, s.pending_before - s.pending_after));
|
||||
}
|
||||
|
||||
function ServerRow({ s }: { s: PatchServerRun }) {
|
||||
const [open, setOpen] = useState(false);
|
||||
const meta = SERVER_STATUS[s.status];
|
||||
return (
|
||||
<>
|
||||
<Tr>
|
||||
<Td label="Server">
|
||||
<Link href={`/servers/${s.server_id}`} className="font-mono text-sm text-accent hover:underline">
|
||||
{s.hostname}
|
||||
</Link>
|
||||
</Td>
|
||||
<Td label="Status">
|
||||
<Badge variant={meta.variant}>{meta.label}</Badge>
|
||||
</Td>
|
||||
<Td label="Installed">
|
||||
<span className="font-mono text-sm tabular-nums">{installed(s)}</span>
|
||||
</Td>
|
||||
<Td label="Reboot">
|
||||
<span className="text-xs text-text-secondary">
|
||||
{s.rebooted_at ? `rebooted ${new Date(s.rebooted_at).toLocaleTimeString()}` : "none"}
|
||||
{s.verified_at && `, back ${new Date(s.verified_at).toLocaleTimeString()}`}
|
||||
</span>
|
||||
</Td>
|
||||
<Td label="Detail">
|
||||
<div className="flex items-center justify-end gap-3">
|
||||
{s.error && <span className="max-w-xs truncate text-xs text-danger" title={s.error}>{s.error}</span>}
|
||||
{s.output && (
|
||||
<Button variant="ghost" size="sm" onClick={() => setOpen(!open)} aria-expanded={open}>
|
||||
{open ? "Hide output" : "Output"}
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
</Td>
|
||||
</Tr>
|
||||
{open && s.output && (
|
||||
<Tr>
|
||||
<Td colSpan={5}>
|
||||
<pre className="max-h-96 overflow-auto rounded bg-well px-4 py-3 font-mono text-[11.5px] leading-relaxed text-text-secondary">{s.output}</pre>
|
||||
</Td>
|
||||
</Tr>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
export default function PatchRunPage() {
|
||||
const { runId } = useParams<{ runId: string }>();
|
||||
const queryClient = useQueryClient();
|
||||
const toast = useToast();
|
||||
const { data: run, isLoading, error, refetch } = useQuery({
|
||||
queryKey: ["patch-run", runId],
|
||||
queryFn: () => api.getPatchRun(runId),
|
||||
refetchInterval: (q) => (q.state.data?.status === "running" ? 5_000 : false),
|
||||
});
|
||||
const cancel = useMutation({
|
||||
mutationFn: () => api.cancelPatchRun(runId),
|
||||
onSuccess: () => {
|
||||
toast.success("Cancelled. Servers already patching will finish; nothing further starts.");
|
||||
queryClient.invalidateQueries({ queryKey: ["patch-run", runId] });
|
||||
},
|
||||
onError: (e) => toast.error(friendlyMessage(e)),
|
||||
});
|
||||
|
||||
if (isLoading) return <CenteredSpinner />;
|
||||
|
||||
const counts = new Map<string, number>();
|
||||
run?.servers.forEach((s) => counts.set(s.status, (counts.get(s.status) ?? 0) + 1));
|
||||
|
||||
return (
|
||||
<div className="p-4 sm:p-6 lg:p-8">
|
||||
<AsyncBoundary isLoading={false} error={error} onRetry={refetch}>
|
||||
{run && (
|
||||
<>
|
||||
<Link href="/patching?tab=runs" className="text-sm text-text-secondary hover:text-accent">
|
||||
Back to patch runs
|
||||
</Link>
|
||||
<div className="mt-3 flex flex-wrap items-start justify-between gap-4">
|
||||
<div>
|
||||
<h1 className="text-2xl font-semibold text-text-primary">{run.policy_name ?? "Manual update"}</h1>
|
||||
<p className="mt-1 text-sm text-text-secondary">
|
||||
Started {new Date(run.started_at).toLocaleString()} by {run.triggered_by}.{" "}
|
||||
{run.scope === "security" ? "Security updates only" : "All pending updates"}, {run.reboot === "if_required" ? "reboot if required" : "no reboot"}.
|
||||
{run.window_end && ` Window ends ${new Date(run.window_end).toLocaleString()}.`}
|
||||
</p>
|
||||
</div>
|
||||
<div className="flex items-center gap-3">
|
||||
<Badge variant={RUN_STATUS[run.status].variant}>{RUN_STATUS[run.status].label}</Badge>
|
||||
{run.status === "running" && !run.cancelled_at && (
|
||||
<Button variant="secondary" size="sm" loading={cancel.isPending} onClick={() => cancel.mutate()}>
|
||||
Cancel run
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="mt-4 flex flex-wrap gap-x-6 gap-y-2">
|
||||
{[...counts.entries()].map(([status, n]) => (
|
||||
<span key={status} className="text-sm text-text-secondary">
|
||||
<span className="font-mono tabular-nums text-text-primary">{n}</span> {SERVER_STATUS[status as PatchServerRun["status"]].label}
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<Card padding={false} className="mt-6">
|
||||
<Table>
|
||||
<Thead>
|
||||
<Tr>
|
||||
<Th>Server</Th>
|
||||
<Th>Status</Th>
|
||||
<Th>Installed</Th>
|
||||
<Th>Reboot</Th>
|
||||
<Th />
|
||||
</Tr>
|
||||
</Thead>
|
||||
<Tbody>
|
||||
{run.servers.map((s) => (
|
||||
<ServerRow key={s.server_id} s={s} />
|
||||
))}
|
||||
</Tbody>
|
||||
</Table>
|
||||
</Card>
|
||||
</>
|
||||
)}
|
||||
</AsyncBoundary>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -130,7 +130,13 @@ export default function ServerDetailPage() {
|
||||
|
||||
const { mutate: applyUpdates, isPending: isApplying } = useMutation({
|
||||
mutationFn: () => api.applyUpdates(serverId),
|
||||
onSuccess: () => toast.success("Update command sent. Patching runs in the background and may take several minutes."),
|
||||
onSuccess: (res) => {
|
||||
if (res.run_id) {
|
||||
router.push(`/patching/runs/${res.run_id}`);
|
||||
} else {
|
||||
toast.success("Update command sent.");
|
||||
}
|
||||
},
|
||||
onError: toast.error,
|
||||
});
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
import { useMemo, useState } from "react";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { useRouter } from "next/navigation";
|
||||
import { api, vulnerabilities, type FindingState, type Severity, type VulnFinding } from "@/lib/api";
|
||||
import { useAuth } from "@/components/AuthProvider";
|
||||
import { Button, Card, Pagination, usePagination, useToast } from "@/components/ui";
|
||||
@@ -42,6 +43,7 @@ const FIX_FILTERS: { key: string; label: string; hasFix: boolean | undefined }[]
|
||||
export default function VulnerabilitiesPage() {
|
||||
const { isAdmin } = useAuth();
|
||||
const qc = useQueryClient();
|
||||
const router = useRouter();
|
||||
|
||||
const [state, setState] = useState<FindingState>("open");
|
||||
const [severity, setSeverity] = useState<Severity | "">("");
|
||||
@@ -112,10 +114,10 @@ export default function VulnerabilitiesPage() {
|
||||
});
|
||||
|
||||
const applyUpdates = useMutation({
|
||||
mutationFn: (serverId: string) => api.applyUpdates(serverId),
|
||||
// This one had no feedback of any kind: the button dispatched a patch
|
||||
// run to a whole server and the page did not change in any way.
|
||||
onSuccess: (_data, serverId) => toast.success(`Update command sent to ${serverName(serverId)}.`),
|
||||
mutationFn: (serverId: string) => api.applyUpdates(serverId, "vulnerabilities"),
|
||||
onSuccess: (res) => {
|
||||
if (res.run_id) router.push(`/patching/runs/${res.run_id}`);
|
||||
},
|
||||
onError: toast.error,
|
||||
});
|
||||
|
||||
|
||||
@@ -176,6 +176,14 @@ function TokenIcon() {
|
||||
);
|
||||
}
|
||||
|
||||
function PatchIcon() {
|
||||
return (
|
||||
<svg className="h-5 w-5" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}>
|
||||
<path strokeLinecap="round" strokeLinejoin="round" d="M6.75 3v2.25M17.25 3v2.25M3 18.75V7.5a2.25 2.25 0 012.25-2.25h13.5A2.25 2.25 0 0121 7.5v11.25m-18 0A2.25 2.25 0 005.25 21h13.5A2.25 2.25 0 0021 18.75m-18 0v-7.5A2.25 2.25 0 015.25 9h13.5A2.25 2.25 0 0121 11.25v7.5m-9-6l-1.5 3h3l-1.5 3" />
|
||||
</svg>
|
||||
);
|
||||
}
|
||||
|
||||
const navGroups: NavGroup[] = [
|
||||
{
|
||||
label: "Fleet",
|
||||
@@ -184,6 +192,7 @@ const navGroups: NavGroup[] = [
|
||||
{ href: "/workloads", label: "Workloads", icon: <WorkloadIcon /> },
|
||||
{ href: "/monitors", label: "Monitors", icon: <MonitorIcon /> },
|
||||
{ href: "/vulnerabilities", label: "Vulnerabilities", icon: <ShieldIcon /> },
|
||||
{ href: "/patching", label: "Patching", icon: <PatchIcon /> },
|
||||
],
|
||||
},
|
||||
{
|
||||
|
||||
@@ -0,0 +1,188 @@
|
||||
"use client";
|
||||
|
||||
import Link from "next/link";
|
||||
import { useState } from "react";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { useRouter } from "next/navigation";
|
||||
import { api, PatchPolicy } from "@/lib/api";
|
||||
import { resolveTargets } from "@/lib/targets";
|
||||
import { AsyncBoundary, Badge, Button, ConfirmDialog, EmptyState, Table, TableSkeleton, Tbody, Td, Th, Thead, Tr, friendlyMessage, useToast } from "@/components/ui";
|
||||
import { PolicyModal } from "./PolicyModal";
|
||||
import { RUN_STATUS } from "./status";
|
||||
|
||||
const SKIP_REASON: Record<string, string> = {
|
||||
missed: "the control plane was not running when the window opened",
|
||||
already_running: "the previous run was still going",
|
||||
no_targets: "no servers matched",
|
||||
};
|
||||
|
||||
// runNowBody says what clicking Run now does to real machines, before it does.
|
||||
function runNowBody(p: PatchPolicy, count: number): string {
|
||||
const servers = `${count} server${count === 1 ? "" : "s"}`;
|
||||
const installs = p.scope === "security" ? "security updates" : "all pending updates";
|
||||
const reboot = p.reboot === "if_required" ? "Servers that need a reboot will restart." : "No server will be rebooted.";
|
||||
return `This starts a window of the usual length now and installs ${installs} on ${servers}. ${reboot}`;
|
||||
}
|
||||
|
||||
export function PolicyList({ canEdit }: { canEdit: boolean }) {
|
||||
const router = useRouter();
|
||||
const queryClient = useQueryClient();
|
||||
const toast = useToast();
|
||||
const [editing, setEditing] = useState<PatchPolicy | "new" | null>(null);
|
||||
const [deleting, setDeleting] = useState<PatchPolicy | null>(null);
|
||||
const [running, setRunning] = useState<PatchPolicy | null>(null);
|
||||
const { data: policies, isLoading, error, refetch } = useQuery({ queryKey: ["patch-policies"], queryFn: () => api.listPatchPolicies() });
|
||||
const { data: windows } = useQuery({ queryKey: ["maintenance-windows"], queryFn: () => api.listMaintenanceWindows() });
|
||||
const { data: servers } = useQuery({ queryKey: ["servers"], queryFn: () => api.listServers() });
|
||||
const { data: runs } = useQuery({ queryKey: ["patch-runs", "all"], queryFn: () => api.listPatchRuns({ limit: 50 }) });
|
||||
|
||||
const runNow = useMutation({
|
||||
mutationFn: (p: PatchPolicy) => api.runPatchPolicyNow(p.policy_id),
|
||||
onSuccess: (run) => {
|
||||
setRunning(null);
|
||||
router.push(`/patching/runs/${run.run_id}`);
|
||||
},
|
||||
onError: (e) => {
|
||||
setRunning(null);
|
||||
toast.error(friendlyMessage(e));
|
||||
},
|
||||
});
|
||||
const remove = useMutation({
|
||||
mutationFn: (p: PatchPolicy) => api.deletePatchPolicy(p.policy_id),
|
||||
onSuccess: () => {
|
||||
queryClient.invalidateQueries({ queryKey: ["patch-policies"] });
|
||||
toast.success("Policy deleted.");
|
||||
setDeleting(null);
|
||||
},
|
||||
onError: (e) => toast.error(friendlyMessage(e)),
|
||||
});
|
||||
|
||||
const windowName = (id: string) => windows?.find((w) => w.window_id === id);
|
||||
const lastRun = (id: string) => runs?.find((r) => r.policy_id === id);
|
||||
|
||||
return (
|
||||
<>
|
||||
{canEdit && (
|
||||
<div className="flex justify-end border-b border-border px-6 py-3">
|
||||
<Button variant="primary" size="sm" onClick={() => setEditing("new")} disabled={(windows ?? []).length === 0} title={(windows ?? []).length === 0 ? "Create a maintenance window first" : undefined}>
|
||||
New policy
|
||||
</Button>
|
||||
</div>
|
||||
)}
|
||||
<AsyncBoundary
|
||||
isLoading={isLoading}
|
||||
error={error}
|
||||
onRetry={refetch}
|
||||
skeleton={<TableSkeleton columns={5} />}
|
||||
isEmpty={!policies || policies.length === 0}
|
||||
empty={<EmptyState title="No patch policies yet." description="A policy says which servers to patch, what to install and whether to reboot, inside a maintenance window." />}
|
||||
>
|
||||
<Table>
|
||||
<Thead>
|
||||
<Tr>
|
||||
<Th>Policy</Th>
|
||||
<Th>Next window</Th>
|
||||
<Th>Targets</Th>
|
||||
<Th>Installs</Th>
|
||||
<Th>Last run</Th>
|
||||
{canEdit && <Th />}
|
||||
</Tr>
|
||||
</Thead>
|
||||
<Tbody>
|
||||
{(policies ?? []).map((p) => {
|
||||
const w = windowName(p.window_id);
|
||||
const count = resolveTargets(servers ?? [], p.target_server_ids, p.target_tags ?? {}).length;
|
||||
const last = lastRun(p.policy_id);
|
||||
return (
|
||||
<Tr key={p.policy_id}>
|
||||
<Td label="Policy">
|
||||
<div className="flex flex-col gap-1">
|
||||
<span className="font-medium text-text-primary">{p.name}</span>
|
||||
{!p.enabled && (
|
||||
<span className="text-xs text-warning">
|
||||
Disabled{p.disabled_reason ? `: ${p.disabled_reason}` : ""}
|
||||
</span>
|
||||
)}
|
||||
{p.last_skipped && (
|
||||
<span className="text-xs text-warning">
|
||||
Skipped {new Date(p.last_skipped.due).toLocaleString()}: {SKIP_REASON[p.last_skipped.reason] ?? p.last_skipped.reason}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
</Td>
|
||||
<Td label="Next window">
|
||||
{p.enabled && p.next_run_at ? (
|
||||
<span className="text-sm">
|
||||
{new Date(p.next_run_at).toLocaleString(undefined, { timeZone: w?.tz, dateStyle: "medium", timeStyle: "short" })}
|
||||
<span className="block text-xs text-text-tertiary">{w?.name}</span>
|
||||
</span>
|
||||
) : (
|
||||
<span className="text-text-tertiary">none</span>
|
||||
)}
|
||||
</Td>
|
||||
<Td label="Targets">
|
||||
<span className="tabular-nums">{count}</span>
|
||||
</Td>
|
||||
<Td label="Installs">
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
<Badge variant="neutral">{p.scope === "security" ? "security only" : "all updates"}</Badge>
|
||||
{p.reboot === "if_required" && <Badge variant="accent">reboots</Badge>}
|
||||
</div>
|
||||
</Td>
|
||||
<Td label="Last run">
|
||||
{last ? (
|
||||
<Link href={`/patching/runs/${last.run_id}`}>
|
||||
<Badge variant={RUN_STATUS[last.status].variant}>{RUN_STATUS[last.status].label}</Badge>
|
||||
</Link>
|
||||
) : (
|
||||
<span className="text-text-tertiary">never</span>
|
||||
)}
|
||||
</Td>
|
||||
{canEdit && (
|
||||
<Td>
|
||||
<div className="flex justify-end gap-2">
|
||||
<Button variant="ghost" size="sm" loading={runNow.isPending && runNow.variables?.policy_id === p.policy_id} onClick={() => setRunning(p)}>
|
||||
Run now
|
||||
</Button>
|
||||
<Button variant="ghost" size="sm" onClick={() => setEditing(p)}>
|
||||
Edit
|
||||
</Button>
|
||||
<Button variant="ghost" size="sm" onClick={() => setDeleting(p)}>
|
||||
Delete
|
||||
</Button>
|
||||
</div>
|
||||
</Td>
|
||||
)}
|
||||
</Tr>
|
||||
);
|
||||
})}
|
||||
</Tbody>
|
||||
</Table>
|
||||
</AsyncBoundary>
|
||||
{editing && <PolicyModal initial={editing === "new" ? undefined : editing} onClose={() => setEditing(null)} />}
|
||||
{running && (
|
||||
<ConfirmDialog
|
||||
open
|
||||
title={`Run ${running.name} now?`}
|
||||
body={runNowBody(running, resolveTargets(servers ?? [], running.target_server_ids, running.target_tags ?? {}).length)}
|
||||
confirmLabel="Run now"
|
||||
destructive={false}
|
||||
loading={runNow.isPending}
|
||||
onConfirm={() => runNow.mutate(running)}
|
||||
onClose={() => setRunning(null)}
|
||||
/>
|
||||
)}
|
||||
{deleting && (
|
||||
<ConfirmDialog
|
||||
open
|
||||
title={`Delete ${deleting.name}?`}
|
||||
body="The policy stops running. Its past runs stay on the Runs tab."
|
||||
confirmLabel="Delete policy"
|
||||
loading={remove.isPending}
|
||||
onConfirm={() => remove.mutate(deleting)}
|
||||
onClose={() => setDeleting(null)}
|
||||
/>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,211 @@
|
||||
"use client";
|
||||
|
||||
import { useMemo, useState } from "react";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { api, PatchPolicy, PatchReboot, PatchScope } from "@/lib/api";
|
||||
import { resolveTargets } from "@/lib/targets";
|
||||
import { Button, Modal, friendlyMessage, useToast } from "@/components/ui";
|
||||
import { DualListBox } from "@/components/workflows/DualListBox";
|
||||
import { WindowModal } from "./WindowModal";
|
||||
import { agentSupportsPatchResults, describeCron, formatDuration, MIN_AGENT_VERSION, parseIntInRange } from "./status";
|
||||
|
||||
const inputClass = "w-full rounded-lg border border-border bg-surface-2 px-3 py-2 text-sm text-text-primary focus:border-accent/50 focus:outline-none focus:ring-1 focus:ring-accent/30";
|
||||
|
||||
function Radio<T extends string>({ name, value, current, onChange, title, hint }: { name: string; value: T; current: T; onChange: (v: T) => void; title: string; hint: string }) {
|
||||
return (
|
||||
<label className="flex items-start gap-3">
|
||||
<input type="radio" id={`${name}-${value}`} name={name} checked={current === value} onChange={() => onChange(value)} className="mt-0.5 h-4 w-4 accent-accent" />
|
||||
<span>
|
||||
<span className="block text-sm text-text-primary">{title}</span>
|
||||
<span className="mt-0.5 block text-xs text-text-tertiary">{hint}</span>
|
||||
</span>
|
||||
</label>
|
||||
);
|
||||
}
|
||||
|
||||
export function PolicyModal({ initial, onClose }: { initial?: PatchPolicy; onClose: () => void }) {
|
||||
const queryClient = useQueryClient();
|
||||
const toast = useToast();
|
||||
const [name, setName] = useState(initial?.name ?? "");
|
||||
const [enabled, setEnabled] = useState(initial?.enabled ?? true);
|
||||
const [windowId, setWindowId] = useState(initial?.window_id ?? "");
|
||||
const [targets, setTargets] = useState<string[]>(initial?.target_server_ids ?? []);
|
||||
const [tagRows, setTagRows] = useState<[string, string][]>(Object.entries(initial?.target_tags ?? {}));
|
||||
const [scope, setScope] = useState<PatchScope>(initial?.scope ?? "security");
|
||||
const [reboot, setReboot] = useState<PatchReboot>(initial?.reboot ?? "never");
|
||||
// Kept as the raw input so a cleared field is not silently read as 0 (no cap).
|
||||
const [maxConcurrentRaw, setMaxConcurrentRaw] = useState(String(initial?.max_concurrent ?? 0));
|
||||
const maxConcurrent = parseIntInRange(maxConcurrentRaw, 0, 1000);
|
||||
const [channels, setChannels] = useState<string[]>(initial?.notify_channel_ids ?? []);
|
||||
const [newWindow, setNewWindow] = useState(false);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
|
||||
const { data: windows } = useQuery({ queryKey: ["maintenance-windows"], queryFn: () => api.listMaintenanceWindows() });
|
||||
const { data: servers } = useQuery({ queryKey: ["servers"], queryFn: () => api.listServers() });
|
||||
const { data: knownTags } = useQuery({ queryKey: ["server-tags"], queryFn: () => api.listKnownTags(), staleTime: 60_000 });
|
||||
const { data: allChannels } = useQuery({ queryKey: ["channels"], queryFn: () => api.listChannels() });
|
||||
|
||||
const tags = useMemo(() => Object.fromEntries(tagRows.filter(([k, v]) => k && v)), [tagRows]);
|
||||
const resolved = useMemo(() => resolveTargets(servers ?? [], targets, tags), [servers, targets, tags]);
|
||||
const tooOld = resolved.filter((s) => !agentSupportsPatchResults(s.agent_version));
|
||||
|
||||
const { mutate: save, isPending } = useMutation({
|
||||
mutationFn: () => {
|
||||
const input = { name, enabled, window_id: windowId, target_server_ids: targets, target_tags: tags, scope, reboot, max_concurrent: maxConcurrent ?? 0, notify_channel_ids: channels };
|
||||
return initial ? api.updatePatchPolicy(initial.policy_id, input) : api.createPatchPolicy(input);
|
||||
},
|
||||
onSuccess: (p) => {
|
||||
queryClient.invalidateQueries({ queryKey: ["patch-policies"] });
|
||||
toast.success(initial ? `Saved ${p.name}.` : `Created ${p.name}.`);
|
||||
onClose();
|
||||
},
|
||||
onError: (e) => setError(friendlyMessage(e)),
|
||||
});
|
||||
|
||||
return (
|
||||
<>
|
||||
<Modal open title={initial ? "Edit patch policy" : "New patch policy"} onClose={onClose} wide>
|
||||
<div className="space-y-5">
|
||||
{error && (
|
||||
<div className="rounded border border-danger/30 bg-danger/10 px-3 py-2 text-sm text-danger" role="alert">
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="grid grid-cols-1 gap-4 sm:grid-cols-2">
|
||||
<label className="block">
|
||||
<span className="mb-1.5 block text-sm font-medium text-text-secondary">Name</span>
|
||||
<input id="policy-name" className={inputClass} value={name} onChange={(e) => setName(e.target.value)} placeholder="Sunday prod security" />
|
||||
</label>
|
||||
<label className="block">
|
||||
<span className="mb-1.5 block text-sm font-medium text-text-secondary">Maintenance window</span>
|
||||
<div className="flex gap-2">
|
||||
<select id="policy-window" className={inputClass} value={windowId} onChange={(e) => setWindowId(e.target.value)}>
|
||||
<option value="">Choose a window</option>
|
||||
{(windows ?? []).map((w) => (
|
||||
<option key={w.window_id} value={w.window_id}>
|
||||
{w.name}: {describeCron(w.cron)}, {formatDuration(w.duration_minutes)} ({w.tz})
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
<Button variant="secondary" size="sm" onClick={() => setNewWindow(true)}>
|
||||
New
|
||||
</Button>
|
||||
</div>
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<span className="mb-1 block text-xs uppercase text-text-secondary">Target servers</span>
|
||||
<DualListBox
|
||||
items={(servers ?? []).map((s) => ({ id: s.server_id, label: s.hostname, hint: s.status === "active" ? undefined : s.status }))}
|
||||
selected={targets}
|
||||
onChange={setTargets}
|
||||
selectedLabel="Targets"
|
||||
emptyAvailable="Every server is a target."
|
||||
emptySelected="No servers named."
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<span className="mb-1 block text-xs uppercase text-text-secondary">Target tags</span>
|
||||
<p className="mb-2 text-[11px] text-text-tertiary">Servers carrying every tag below are patched too. Tags are read when the window opens, so a server tagged later is included.</p>
|
||||
<datalist id="policy-tag-keys">
|
||||
{Object.keys(knownTags ?? {}).map((k) => (
|
||||
<option key={k} value={k} />
|
||||
))}
|
||||
</datalist>
|
||||
<div className="flex flex-col gap-2">
|
||||
{tagRows.map(([k, v], i) => (
|
||||
<div key={i} className="flex gap-2">
|
||||
<input id={`policy-tag-key-${i}`} list="policy-tag-keys" className={inputClass} value={k} placeholder="key" onChange={(e) => setTagRows(tagRows.map((r, j): [string, string] => (j === i ? [e.target.value, r[1]] : r)))} />
|
||||
<input id={`policy-tag-value-${i}`} className={inputClass} value={v} placeholder="value" onChange={(e) => setTagRows(tagRows.map((r, j): [string, string] => (j === i ? [r[0], e.target.value] : r)))} />
|
||||
<Button variant="ghost" size="sm" onClick={() => setTagRows(tagRows.filter((_, j) => j !== i))} aria-label={`Remove ${k || "tag"}`}>
|
||||
Remove
|
||||
</Button>
|
||||
</div>
|
||||
))}
|
||||
<div>
|
||||
<Button variant="secondary" size="sm" onClick={() => setTagRows([...tagRows, ["", ""]])}>
|
||||
Add tag
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
<p className="mt-2 text-xs text-text-secondary">
|
||||
{resolved.length} server{resolved.length === 1 ? "" : "s"} targeted now.
|
||||
{tooOld.length > 0 && (
|
||||
<span className="text-warning">
|
||||
{" "}
|
||||
{tooOld.length} of {resolved.length} need an agent update to {MIN_AGENT_VERSION} or later and will be skipped until then.
|
||||
</span>
|
||||
)}
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-1 gap-5 sm:grid-cols-2">
|
||||
<fieldset className="space-y-3">
|
||||
<legend className="mb-2 text-xs uppercase text-text-secondary">What to install</legend>
|
||||
<Radio name="scope" value="security" current={scope} onChange={setScope} title="Security updates only" hint="Servers using apk or pacman have no security metadata and report unsupported." />
|
||||
<Radio name="scope" value="all" current={scope} onChange={setScope} title="All pending updates" hint="Everything the package manager would upgrade." />
|
||||
</fieldset>
|
||||
<fieldset className="space-y-3">
|
||||
<legend className="mb-2 text-xs uppercase text-text-secondary">Reboots</legend>
|
||||
<Radio name="reboot" value="never" current={reboot} onChange={setReboot} title="Never reboot" hint="Servers that need one show reboot required." />
|
||||
<Radio name="reboot" value="if_required" current={reboot} onChange={setReboot} title="Reboot if required" hint="Only when the OS says so, and only with 5 minutes or more left in the window." />
|
||||
{reboot === "if_required" && resolved.length > 0 && (
|
||||
<p className="rounded-lg border border-warning/30 bg-warning/10 px-3 py-2 text-xs text-warning">
|
||||
Up to {maxConcurrent ? Math.min(maxConcurrent, resolved.length) : resolved.length} server{resolved.length === 1 ? "" : "s"} may be rebooting at the same time during this window.
|
||||
</p>
|
||||
)}
|
||||
</fieldset>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-1 gap-4 sm:grid-cols-2">
|
||||
<label className="block">
|
||||
<span className="mb-1.5 block text-sm font-medium text-text-secondary">At most this many at once</span>
|
||||
<input id="policy-max-concurrent" type="number" min={0} max={1000} className={inputClass} value={maxConcurrentRaw} aria-invalid={maxConcurrent === null} onChange={(e) => setMaxConcurrentRaw(e.target.value)} />
|
||||
{maxConcurrent === null ? (
|
||||
<span className="mt-1.5 block text-[11px] text-danger">Enter a whole number from 0 to 1000.</span>
|
||||
) : (
|
||||
<span className="mt-1.5 block text-[11px] text-text-tertiary">0 means no limit</span>
|
||||
)}
|
||||
</label>
|
||||
<div>
|
||||
<span className="mb-1.5 block text-sm font-medium text-text-secondary">Alert when a run is not clean</span>
|
||||
<div className="flex flex-col gap-1.5">
|
||||
{(allChannels ?? []).length === 0 && <p className="text-xs text-text-tertiary">No notification channels yet.</p>}
|
||||
{(allChannels ?? []).map((ch) => (
|
||||
<label key={ch.channel_id} className="flex items-center gap-2 text-sm text-text-primary">
|
||||
<input
|
||||
type="checkbox"
|
||||
id={`policy-channel-${ch.channel_id}`}
|
||||
checked={channels.includes(ch.channel_id)}
|
||||
onChange={(e) => setChannels(e.target.checked ? [...channels, ch.channel_id] : channels.filter((c) => c !== ch.channel_id))}
|
||||
className="h-4 w-4 accent-accent"
|
||||
/>
|
||||
{ch.name}
|
||||
</label>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<label className="flex items-center gap-3">
|
||||
<input type="checkbox" id="policy-enabled" checked={enabled} onChange={(e) => setEnabled(e.target.checked)} className="h-4 w-4 accent-accent" />
|
||||
<span className="text-sm text-text-primary">Enabled</span>
|
||||
</label>
|
||||
|
||||
<div className="flex justify-end gap-2">
|
||||
<Button variant="secondary" onClick={onClose}>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button variant="primary" loading={isPending} disabled={!name.trim() || !windowId || maxConcurrent === null} onClick={() => save()}>
|
||||
{initial ? "Save policy" : "Create policy"}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</Modal>
|
||||
{newWindow && <WindowModal onClose={() => setNewWindow(false)} onSaved={(w) => setWindowId(w.window_id)} />}
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
"use client";
|
||||
|
||||
import Link from "next/link";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { api } from "@/lib/api";
|
||||
import { AsyncBoundary, Badge, EmptyState, Table, TableSkeleton, Tbody, Td, Th, Thead, Tr } from "@/components/ui";
|
||||
import { RUN_STATUS } from "./status";
|
||||
|
||||
const SOURCE_LABEL: Record<string, string> = {
|
||||
schedule: "window",
|
||||
run_now: "run now",
|
||||
server: "server page",
|
||||
vulnerabilities: "vulnerabilities",
|
||||
mcp: "agent (MCP)",
|
||||
};
|
||||
|
||||
export function RunList({ policyId }: { policyId?: string }) {
|
||||
const { data: runs, isLoading, error, refetch } = useQuery({
|
||||
queryKey: ["patch-runs", policyId ?? "all"],
|
||||
queryFn: () => api.listPatchRuns({ policy_id: policyId, limit: 50 }),
|
||||
// A run in progress changes on every scheduler tick.
|
||||
refetchInterval: (q) => (q.state.data?.some((r) => r.status === "running") ? 10_000 : false),
|
||||
});
|
||||
|
||||
return (
|
||||
<AsyncBoundary
|
||||
isLoading={isLoading}
|
||||
error={error}
|
||||
onRetry={refetch}
|
||||
skeleton={<TableSkeleton columns={5} />}
|
||||
isEmpty={!runs || runs.length === 0}
|
||||
empty={<EmptyState title="No patch runs yet." description="A run is recorded every time a policy's window opens and every time someone clicks Apply updates." />}
|
||||
>
|
||||
<Table>
|
||||
<Thead>
|
||||
<Tr>
|
||||
<Th>Started</Th>
|
||||
<Th>Policy</Th>
|
||||
<Th>Status</Th>
|
||||
<Th>Servers</Th>
|
||||
<Th>Started by</Th>
|
||||
</Tr>
|
||||
</Thead>
|
||||
<Tbody>
|
||||
{(runs ?? []).map((r) => {
|
||||
const ok = r.servers.filter((s) => s.status === "succeeded").length;
|
||||
return (
|
||||
<Tr key={r.run_id}>
|
||||
<Td label="Started">
|
||||
<Link href={`/patching/runs/${r.run_id}`} className="font-mono text-sm text-accent hover:underline">
|
||||
{new Date(r.started_at).toLocaleString()}
|
||||
</Link>
|
||||
</Td>
|
||||
<Td label="Policy">{r.policy_name ?? <span className="text-text-tertiary">manual</span>}</Td>
|
||||
<Td label="Status">
|
||||
<Badge variant={RUN_STATUS[r.status].variant}>{RUN_STATUS[r.status].label}</Badge>
|
||||
</Td>
|
||||
<Td label="Servers">
|
||||
<span className="font-mono text-sm tabular-nums">
|
||||
{ok}/{r.servers.length}
|
||||
</span>
|
||||
</Td>
|
||||
<Td label="Started by">
|
||||
<span className="text-sm text-text-secondary">
|
||||
{r.triggered_by} <span className="text-text-tertiary">via {SOURCE_LABEL[r.source] ?? r.source}</span>
|
||||
</span>
|
||||
</Td>
|
||||
</Tr>
|
||||
);
|
||||
})}
|
||||
</Tbody>
|
||||
</Table>
|
||||
</AsyncBoundary>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
"use client";
|
||||
|
||||
import { useState } from "react";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { api, MaintenanceWindow } from "@/lib/api";
|
||||
import { AsyncBoundary, Button, ConfirmDialog, EmptyState, Table, TableSkeleton, Tbody, Td, Th, Thead, Tr, friendlyMessage, useToast } from "@/components/ui";
|
||||
import { WindowModal } from "./WindowModal";
|
||||
import { describeCron, formatDuration } from "./status";
|
||||
|
||||
export function WindowList({ canEdit }: { canEdit: boolean }) {
|
||||
const queryClient = useQueryClient();
|
||||
const toast = useToast();
|
||||
const [editing, setEditing] = useState<MaintenanceWindow | "new" | null>(null);
|
||||
const [deleting, setDeleting] = useState<MaintenanceWindow | null>(null);
|
||||
const { data: windows, isLoading, error, refetch } = useQuery({ queryKey: ["maintenance-windows"], queryFn: () => api.listMaintenanceWindows() });
|
||||
const { data: policies } = useQuery({ queryKey: ["patch-policies"], queryFn: () => api.listPatchPolicies() });
|
||||
|
||||
const { mutate: remove, isPending } = useMutation({
|
||||
mutationFn: (w: MaintenanceWindow) => api.deleteMaintenanceWindow(w.window_id),
|
||||
onSuccess: () => {
|
||||
queryClient.invalidateQueries({ queryKey: ["maintenance-windows"] });
|
||||
toast.success("Window deleted.");
|
||||
setDeleting(null);
|
||||
},
|
||||
// 409 window_in_use arrives here with the server's own sentence.
|
||||
onError: (e) => {
|
||||
toast.error(friendlyMessage(e));
|
||||
setDeleting(null);
|
||||
},
|
||||
});
|
||||
|
||||
const usedBy = (id: string) => (policies ?? []).filter((p) => p.window_id === id).length;
|
||||
|
||||
return (
|
||||
<>
|
||||
{canEdit && (
|
||||
<div className="flex justify-end border-b border-border px-6 py-3">
|
||||
<Button variant="primary" size="sm" onClick={() => setEditing("new")}>
|
||||
New window
|
||||
</Button>
|
||||
</div>
|
||||
)}
|
||||
<AsyncBoundary
|
||||
isLoading={isLoading}
|
||||
error={error}
|
||||
onRetry={refetch}
|
||||
skeleton={<TableSkeleton columns={4} />}
|
||||
isEmpty={!windows || windows.length === 0}
|
||||
empty={<EmptyState title="No maintenance windows yet." description="A window is a recurring time slot, such as Sundays 02:00 to 04:00. Patch policies run inside one." />}
|
||||
>
|
||||
<Table>
|
||||
<Thead>
|
||||
<Tr>
|
||||
<Th>Name</Th>
|
||||
<Th>When</Th>
|
||||
<Th>Length</Th>
|
||||
<Th>Used by</Th>
|
||||
{canEdit && <Th />}
|
||||
</Tr>
|
||||
</Thead>
|
||||
<Tbody>
|
||||
{(windows ?? []).map((w) => (
|
||||
<Tr key={w.window_id}>
|
||||
<Td label="Name">{w.name}</Td>
|
||||
<Td label="When">
|
||||
{describeCron(w.cron)} <span className="text-text-tertiary">({w.tz})</span>
|
||||
</Td>
|
||||
<Td label="Length">{formatDuration(w.duration_minutes)}</Td>
|
||||
<Td label="Used by">
|
||||
<span className="tabular-nums">{usedBy(w.window_id)}</span> {usedBy(w.window_id) === 1 ? "policy" : "policies"}
|
||||
</Td>
|
||||
{canEdit && (
|
||||
<Td>
|
||||
<div className="flex justify-end gap-2">
|
||||
<Button variant="ghost" size="sm" onClick={() => setEditing(w)}>
|
||||
Edit
|
||||
</Button>
|
||||
<Button variant="ghost" size="sm" onClick={() => setDeleting(w)} disabled={usedBy(w.window_id) > 0} title={usedBy(w.window_id) > 0 ? "Move its policies to another window first" : undefined}>
|
||||
Delete
|
||||
</Button>
|
||||
</div>
|
||||
</Td>
|
||||
)}
|
||||
</Tr>
|
||||
))}
|
||||
</Tbody>
|
||||
</Table>
|
||||
</AsyncBoundary>
|
||||
{editing && <WindowModal initial={editing === "new" ? undefined : editing} onClose={() => setEditing(null)} />}
|
||||
{deleting && (
|
||||
<ConfirmDialog
|
||||
open
|
||||
title={`Delete ${deleting.name}?`}
|
||||
body="The window is removed. No policy uses it, so nothing else changes."
|
||||
confirmLabel="Delete window"
|
||||
loading={isPending}
|
||||
onConfirm={() => remove(deleting)}
|
||||
onClose={() => setDeleting(null)}
|
||||
/>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,141 @@
|
||||
"use client";
|
||||
|
||||
import { useState } from "react";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { api, MaintenanceWindow } from "@/lib/api";
|
||||
import { Button, Modal, friendlyMessage, useToast } from "@/components/ui";
|
||||
import { parseIntInRange } from "./status";
|
||||
|
||||
/*
|
||||
* Presets write cron underneath, as the workflow schedule card does, and the
|
||||
* next three windows come from the server, so the editor cannot disagree with
|
||||
* the scheduler about when a window opens.
|
||||
*/
|
||||
const PRESETS: { label: string; cron: string }[] = [
|
||||
{ label: "Nightly, 02:00", cron: "0 2 * * *" },
|
||||
{ label: "Sunday, 02:00", cron: "0 2 * * 0" },
|
||||
{ label: "Saturday, 22:00", cron: "0 22 * * 6" },
|
||||
{ label: "Monthly, 1st 02:00", cron: "0 2 1 * *" },
|
||||
];
|
||||
|
||||
const ZONES = ["UTC", "Europe/London", "Europe/Berlin", "America/New_York", "America/Los_Angeles", "Asia/Singapore", "Australia/Sydney"];
|
||||
|
||||
const inputClass = "w-full rounded-lg border border-border bg-surface-2 px-3 py-2 text-sm text-text-primary focus:border-accent/50 focus:outline-none focus:ring-1 focus:ring-accent/30";
|
||||
|
||||
export function WindowModal({ initial, onClose, onSaved }: { initial?: MaintenanceWindow; onClose: () => void; onSaved?: (w: MaintenanceWindow) => void }) {
|
||||
const queryClient = useQueryClient();
|
||||
const toast = useToast();
|
||||
const [name, setName] = useState(initial?.name ?? "");
|
||||
const [cron, setCron] = useState(initial?.cron ?? "0 2 * * 0");
|
||||
const [tz, setTz] = useState(initial?.tz ?? Intl.DateTimeFormat().resolvedOptions().timeZone ?? "UTC");
|
||||
// Kept as the raw input so a cleared field never reaches the API as NaN.
|
||||
const [durationRaw, setDurationRaw] = useState(String(initial?.duration_minutes ?? 120));
|
||||
const duration = parseIntInRange(durationRaw, 15, 720);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
|
||||
const { data: preview, isError: previewFailed, error: previewError } = useQuery({
|
||||
queryKey: ["window-preview", cron, tz, duration],
|
||||
queryFn: () => api.previewMaintenanceWindow({ cron, tz, duration_minutes: duration ?? 0 }),
|
||||
enabled: duration !== null,
|
||||
retry: false,
|
||||
});
|
||||
|
||||
const { mutate: save, isPending } = useMutation({
|
||||
mutationFn: () => {
|
||||
const input = { name, cron, tz, duration_minutes: duration ?? 0 };
|
||||
return initial ? api.updateMaintenanceWindow(initial.window_id, input) : api.createMaintenanceWindow(input);
|
||||
},
|
||||
onSuccess: (w) => {
|
||||
queryClient.invalidateQueries({ queryKey: ["maintenance-windows"] });
|
||||
queryClient.invalidateQueries({ queryKey: ["patch-policies"] });
|
||||
toast.success(initial ? `Saved ${w.name}. Policies using it now follow the new times.` : `Created ${w.name}.`);
|
||||
onSaved?.(w);
|
||||
onClose();
|
||||
},
|
||||
onError: (e) => setError(friendlyMessage(e)),
|
||||
});
|
||||
|
||||
return (
|
||||
<Modal open title={initial ? "Edit maintenance window" : "New maintenance window"} onClose={onClose}>
|
||||
<div className="space-y-4">
|
||||
{error && (
|
||||
<div className="rounded border border-danger/30 bg-danger/10 px-3 py-2 text-sm text-danger" role="alert">
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
<label className="block">
|
||||
<span className="mb-1.5 block text-sm font-medium text-text-secondary">Name</span>
|
||||
<input id="window-name" className={inputClass} value={name} onChange={(e) => setName(e.target.value)} placeholder="Sunday early morning" />
|
||||
</label>
|
||||
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{PRESETS.map((p) => (
|
||||
<button
|
||||
key={p.cron}
|
||||
type="button"
|
||||
onClick={() => setCron(p.cron)}
|
||||
className={`rounded-lg border px-3 py-1.5 text-xs transition-colors focus:outline-none focus-visible:ring-2 focus-visible:ring-accent ${
|
||||
cron === p.cron ? "border-accent bg-accent/10 text-accent" : "border-border bg-surface-2 text-text-secondary hover:border-accent/40"
|
||||
}`}
|
||||
>
|
||||
{p.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-1 gap-4 sm:grid-cols-3">
|
||||
<label className="block">
|
||||
<span className="mb-1.5 block text-sm font-medium text-text-secondary">Starts (cron)</span>
|
||||
<input id="window-cron" className={`${inputClass} font-mono`} value={cron} onChange={(e) => setCron(e.target.value)} />
|
||||
<span className="mt-1.5 block font-mono text-[11px] text-text-tertiary">minute hour day month weekday</span>
|
||||
</label>
|
||||
<label className="block">
|
||||
<span className="mb-1.5 block text-sm font-medium text-text-secondary">Timezone</span>
|
||||
<select id="window-tz" className={inputClass} value={tz} onChange={(e) => setTz(e.target.value)}>
|
||||
{[...new Set([tz, ...ZONES])].map((z) => (
|
||||
<option key={z} value={z}>
|
||||
{z}
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
</label>
|
||||
<label className="block">
|
||||
<span className="mb-1.5 block text-sm font-medium text-text-secondary">Length (minutes)</span>
|
||||
<input id="window-duration" type="number" min={15} max={720} step={15} className={inputClass} value={durationRaw} aria-invalid={duration === null} onChange={(e) => setDurationRaw(e.target.value)} />
|
||||
{duration === null ? (
|
||||
<span className="mt-1.5 block text-[11px] text-danger">Enter a whole number from 15 to 720.</span>
|
||||
) : (
|
||||
<span className="mt-1.5 block text-[11px] text-text-tertiary">15 to 720</span>
|
||||
)}
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<div className="rounded-lg bg-well px-4 py-3">
|
||||
<p className="font-mono text-[10px] uppercase tracking-[0.16em] text-text-tertiary">Next three windows</p>
|
||||
{previewFailed ? (
|
||||
<p className="mt-1.5 font-mono text-[11.5px] text-danger">{friendlyMessage(previewError)}</p>
|
||||
) : preview ? (
|
||||
<ul className="mt-1.5 flex flex-col gap-0.5 font-mono text-[11.5px] text-text-secondary">
|
||||
{preview.map((s) => (
|
||||
<li key={s.start}>
|
||||
{new Date(s.start).toLocaleString()} to {new Date(s.end).toLocaleTimeString()}
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
) : (
|
||||
<p className="mt-1.5 font-mono text-[11.5px] text-text-tertiary">Working it out...</p>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="flex justify-end gap-2">
|
||||
<Button variant="secondary" onClick={onClose}>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button variant="primary" loading={isPending} disabled={previewFailed || !name.trim() || duration === null} onClick={() => save()}>
|
||||
{initial ? "Save window" : "Create window"}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</Modal>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
import type { PatchRunStatus, PatchServerStatus } from "@/lib/api";
|
||||
|
||||
type Variant = "success" | "warning" | "danger" | "neutral" | "accent";
|
||||
|
||||
/*
|
||||
* One place that says how every patch state reads. Badge adds a dot to the
|
||||
* three state variants, so each label here is also readable without colour.
|
||||
*/
|
||||
export const RUN_STATUS: Record<PatchRunStatus, { label: string; variant: Variant }> = {
|
||||
running: { label: "running", variant: "accent" },
|
||||
succeeded: { label: "succeeded", variant: "success" },
|
||||
partial: { label: "partial", variant: "warning" },
|
||||
failed: { label: "failed", variant: "danger" },
|
||||
cancelled: { label: "cancelled", variant: "neutral" },
|
||||
};
|
||||
|
||||
export const SERVER_STATUS: Record<PatchServerStatus, { label: string; variant: Variant }> = {
|
||||
queued: { label: "queued", variant: "neutral" },
|
||||
waiting_offline: { label: "waiting for agent", variant: "warning" },
|
||||
patching: { label: "patching", variant: "accent" },
|
||||
rebooting: { label: "rebooting", variant: "accent" },
|
||||
succeeded: { label: "succeeded", variant: "success" },
|
||||
failed: { label: "failed", variant: "danger" },
|
||||
unsupported: { label: "unsupported", variant: "warning" },
|
||||
agent_too_old: { label: "agent too old", variant: "warning" },
|
||||
missed_offline: { label: "missed, offline", variant: "danger" },
|
||||
window_closed: { label: "window closed", variant: "danger" },
|
||||
cancelled: { label: "cancelled", variant: "neutral" },
|
||||
};
|
||||
|
||||
/*
|
||||
* Mirrors patchrun.MinAgentVersion and AgentSupportsPatchResults in the
|
||||
* server. The server is the authority; this only lets the policy editor warn
|
||||
* before saving. Change both together.
|
||||
*/
|
||||
export const MIN_AGENT_VERSION = "1.4.0";
|
||||
|
||||
/** Mirrors patchrun.AgentSupportsPatchResults (and its parseVersion) in the server. */
|
||||
export function agentSupportsPatchResults(version?: string): boolean {
|
||||
const m = /^v?(\d+)\.(\d+)\.(\d+)(?:-([^+]*))?(?:\+.*)?$/.exec((version ?? "").trim());
|
||||
if (!m) return false;
|
||||
const have = [Number(m[1]), Number(m[2]), Number(m[3])];
|
||||
const want = MIN_AGENT_VERSION.split(".").map(Number);
|
||||
for (let i = 0; i < 3; i++) {
|
||||
if (have[i] !== want[i]) return have[i] > want[i];
|
||||
}
|
||||
const hasPrerelease = !!m[4];
|
||||
return !hasPrerelease;
|
||||
}
|
||||
|
||||
const DAYS = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"];
|
||||
|
||||
/** Plain words for the common shapes; anything else shows the expression itself. */
|
||||
export function describeCron(cron: string): string {
|
||||
const f = cron.trim().split(/\s+/);
|
||||
if (f.length !== 5 || !/^\d+$/.test(f[0]) || !/^\d+$/.test(f[1])) return cron;
|
||||
const at = `${f[1].padStart(2, "0")}:${f[0].padStart(2, "0")}`;
|
||||
const [, , dom, mon, dow] = f;
|
||||
if (dom === "*" && mon === "*" && dow === "*") return `Daily at ${at}`;
|
||||
if (dom === "*" && mon === "*" && /^[0-6]$/.test(dow)) return `${DAYS[Number(dow)]}s at ${at}`;
|
||||
if (/^\d+$/.test(dom) && mon === "*" && dow === "*") return `Day ${dom} of each month at ${at}`;
|
||||
return cron;
|
||||
}
|
||||
|
||||
export function formatDuration(minutes: number): string {
|
||||
const h = Math.floor(minutes / 60);
|
||||
const m = minutes % 60;
|
||||
if (h === 0) return `${m} min`;
|
||||
return m === 0 ? `${h} h` : `${h} h ${m} min`;
|
||||
}
|
||||
|
||||
// parseIntInRange reads a form field kept as its raw string. It returns null
|
||||
// for an empty, fractional or out-of-range value, so a cleared field is never
|
||||
// sent as 0 or NaN.
|
||||
export function parseIntInRange(raw: string, min: number, max: number): number | null {
|
||||
const t = raw.trim();
|
||||
if (!/^\d+$/.test(t)) return null;
|
||||
const n = Number(t);
|
||||
return n >= min && n <= max ? n : null;
|
||||
}
|
||||
@@ -1,8 +1,12 @@
|
||||
"use client";
|
||||
|
||||
import { useState } from "react";
|
||||
import Link from "next/link";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { api, ServerWithKeys } from "@/lib/api";
|
||||
import { Badge, Button, Card, ConfirmDialog, Table, Tbody, Td, Th, Thead, Tr } from "@/components/ui";
|
||||
import { matchesTags } from "@/lib/targets";
|
||||
import { RUN_STATUS } from "@/components/patching/status";
|
||||
|
||||
/*
|
||||
* Everything that changes what is installed on the machine: its OS packages,
|
||||
@@ -42,6 +46,17 @@ export function MaintenanceTab({
|
||||
const isWindows = server.os_info?.toLowerCase().includes("windows");
|
||||
const agentCurrent = !!latestVersion && !!server.agent_version && server.agent_version === latestVersion;
|
||||
|
||||
const { data: policies } = useQuery({ queryKey: ["patch-policies"], queryFn: () => api.listPatchPolicies() });
|
||||
const { data: windows } = useQuery({ queryKey: ["maintenance-windows"], queryFn: () => api.listMaintenanceWindows() });
|
||||
const { data: lastRuns } = useQuery({ queryKey: ["patch-runs", "server", server.server_id], queryFn: () => api.listPatchRuns({ server_id: server.server_id, limit: 1 }) });
|
||||
// Same selector rule as the scheduler: named, or carrying every tag.
|
||||
const covering = (policies ?? []).filter((p) => p.enabled && (p.target_server_ids.includes(server.server_id) || matchesTags(server, p.target_tags ?? {})));
|
||||
const next = covering
|
||||
.filter((p) => p.next_run_at)
|
||||
.sort((a, b) => (a.next_run_at! < b.next_run_at! ? -1 : 1))[0];
|
||||
const nextWindow = next && windows?.find((w) => w.window_id === next.window_id);
|
||||
const lastRun = lastRuns?.[0];
|
||||
|
||||
return (
|
||||
<div className="grid grid-cols-1 gap-6 xl:grid-cols-2">
|
||||
<Card padding={false}>
|
||||
@@ -55,6 +70,24 @@ export function MaintenanceTab({
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex flex-wrap items-center justify-between gap-2 border-b border-border px-6 py-3 text-xs text-text-secondary">
|
||||
{next && nextWindow ? (
|
||||
<span>
|
||||
Covered by <span className="text-text-primary">{next.name}</span>, next window{" "}
|
||||
{new Date(next.next_run_at!).toLocaleString(undefined, { timeZone: nextWindow.tz, dateStyle: "medium", timeStyle: "short" })} ({nextWindow.tz})
|
||||
</span>
|
||||
) : (
|
||||
<span>
|
||||
Not covered by any patch policy. <Link href="/patching" className="text-accent hover:underline">Set one up</Link>
|
||||
</span>
|
||||
)}
|
||||
{lastRun && (
|
||||
<Link href={`/patching/runs/${lastRun.run_id}`} className="flex items-center gap-2 hover:text-accent">
|
||||
Last run <Badge variant={RUN_STATUS[lastRun.status].variant}>{RUN_STATUS[lastRun.status].label}</Badge>
|
||||
</Link>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{updates.length === 0 ? (
|
||||
<p className="px-6 py-10 text-center text-sm text-text-secondary">
|
||||
{isWindows ? "No pending Windows updates. The agent checks hourly." : "No pending package updates. The agent checks hourly."}
|
||||
@@ -94,7 +127,7 @@ export function MaintenanceTab({
|
||||
<Button variant="primary" loading={isApplying} onClick={onApplyUpdates} disabled={server.status !== "active"} title={server.status !== "active" ? "Agent must be online to apply updates" : undefined}>
|
||||
Apply updates
|
||||
</Button>
|
||||
<p className="text-xs text-text-tertiary">Upgrade runs in the background and may take several minutes.</p>
|
||||
<p className="text-xs text-text-tertiary">Installs all pending updates now, without rebooting, and records a run.</p>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
|
||||
+153
-2
@@ -417,6 +417,99 @@ export interface WorkflowRun {
|
||||
server_runs: ServerRun[];
|
||||
}
|
||||
|
||||
export type PatchScope = "all" | "security";
|
||||
export type PatchReboot = "never" | "if_required";
|
||||
export type PatchRunStatus = "running" | "succeeded" | "partial" | "failed" | "cancelled";
|
||||
export type PatchServerStatus =
|
||||
| "queued"
|
||||
| "waiting_offline"
|
||||
| "patching"
|
||||
| "rebooting"
|
||||
| "succeeded"
|
||||
| "failed"
|
||||
| "unsupported"
|
||||
| "agent_too_old"
|
||||
| "missed_offline"
|
||||
| "window_closed"
|
||||
| "cancelled";
|
||||
|
||||
export interface MaintenanceWindow {
|
||||
window_id: string;
|
||||
name: string;
|
||||
cron: string;
|
||||
tz: string;
|
||||
duration_minutes: number;
|
||||
created_at: string;
|
||||
updated_at: string;
|
||||
}
|
||||
|
||||
export interface MaintenanceWindowInput {
|
||||
name: string;
|
||||
cron: string;
|
||||
tz: string;
|
||||
duration_minutes: number;
|
||||
}
|
||||
|
||||
export interface WindowSpan {
|
||||
start: string;
|
||||
end: string;
|
||||
}
|
||||
|
||||
export interface PatchPolicy {
|
||||
policy_id: string;
|
||||
name: string;
|
||||
enabled: boolean;
|
||||
window_id: string;
|
||||
target_server_ids: string[];
|
||||
target_tags?: Record<string, string>;
|
||||
scope: PatchScope;
|
||||
reboot: PatchReboot;
|
||||
max_concurrent: number;
|
||||
notify_channel_ids?: string[];
|
||||
next_run_at?: string;
|
||||
last_run_at?: string;
|
||||
last_skipped?: Skip;
|
||||
disabled_reason?: string;
|
||||
created_at: string;
|
||||
updated_at: string;
|
||||
}
|
||||
|
||||
export type PatchPolicyInput = Pick<
|
||||
PatchPolicy,
|
||||
"name" | "enabled" | "window_id" | "target_server_ids" | "target_tags" | "scope" | "reboot" | "max_concurrent" | "notify_channel_ids"
|
||||
>;
|
||||
|
||||
export interface PatchServerRun {
|
||||
server_id: string;
|
||||
hostname: string;
|
||||
status: PatchServerStatus;
|
||||
pending_before: number;
|
||||
pending_after?: number;
|
||||
rebooted_at?: string;
|
||||
verified_at?: string;
|
||||
output?: string;
|
||||
error?: string;
|
||||
started_at?: string;
|
||||
finished_at?: string;
|
||||
}
|
||||
|
||||
export interface PatchRun {
|
||||
run_id: string;
|
||||
policy_id?: string;
|
||||
policy_name?: string;
|
||||
triggered_by: string;
|
||||
source: "schedule" | "run_now" | "server" | "vulnerabilities" | "mcp";
|
||||
scope: PatchScope;
|
||||
reboot: PatchReboot;
|
||||
max_concurrent: number;
|
||||
window_end?: string;
|
||||
status: PatchRunStatus;
|
||||
cancelled_at?: string;
|
||||
started_at: string;
|
||||
finished_at?: string;
|
||||
servers: PatchServerRun[];
|
||||
}
|
||||
|
||||
export type Role = "owner" | "admin" | "member";
|
||||
|
||||
/** The session as returned by GET /auth/me mirrors auth.Session on the server. */
|
||||
@@ -842,12 +935,70 @@ export const api = {
|
||||
});
|
||||
},
|
||||
|
||||
applyUpdates(serverId: string): Promise<{ message: string }> {
|
||||
return request<{ message: string }>(`/servers/${serverId}/apply-updates`, {
|
||||
applyUpdates(serverId: string, source?: "vulnerabilities"): Promise<{ message: string; run_id?: string }> {
|
||||
const qs = source ? `?source=${source}` : "";
|
||||
return request<{ message: string; run_id?: string }>(`/servers/${serverId}/apply-updates${qs}`, {
|
||||
method: "POST",
|
||||
});
|
||||
},
|
||||
|
||||
listMaintenanceWindows(): Promise<MaintenanceWindow[]> {
|
||||
return request<MaintenanceWindow[]>("/maintenance-windows");
|
||||
},
|
||||
|
||||
createMaintenanceWindow(input: MaintenanceWindowInput): Promise<MaintenanceWindow> {
|
||||
return request<MaintenanceWindow>("/maintenance-windows", { method: "POST", body: JSON.stringify(input) });
|
||||
},
|
||||
|
||||
updateMaintenanceWindow(windowId: string, input: MaintenanceWindowInput): Promise<MaintenanceWindow> {
|
||||
return request<MaintenanceWindow>(`/maintenance-windows/${windowId}`, { method: "PUT", body: JSON.stringify(input) });
|
||||
},
|
||||
|
||||
deleteMaintenanceWindow(windowId: string): Promise<void> {
|
||||
return request<void>(`/maintenance-windows/${windowId}`, { method: "DELETE" });
|
||||
},
|
||||
|
||||
previewMaintenanceWindow(input: Omit<MaintenanceWindowInput, "name">): Promise<WindowSpan[]> {
|
||||
return request<WindowSpan[]>("/maintenance-windows/preview", { method: "POST", body: JSON.stringify(input) });
|
||||
},
|
||||
|
||||
listPatchPolicies(): Promise<PatchPolicy[]> {
|
||||
return request<PatchPolicy[]>("/patch-policies");
|
||||
},
|
||||
|
||||
createPatchPolicy(input: PatchPolicyInput): Promise<PatchPolicy> {
|
||||
return request<PatchPolicy>("/patch-policies", { method: "POST", body: JSON.stringify(input) });
|
||||
},
|
||||
|
||||
updatePatchPolicy(policyId: string, input: PatchPolicyInput): Promise<PatchPolicy> {
|
||||
return request<PatchPolicy>(`/patch-policies/${policyId}`, { method: "PUT", body: JSON.stringify(input) });
|
||||
},
|
||||
|
||||
deletePatchPolicy(policyId: string): Promise<void> {
|
||||
return request<void>(`/patch-policies/${policyId}`, { method: "DELETE" });
|
||||
},
|
||||
|
||||
runPatchPolicyNow(policyId: string): Promise<PatchRun> {
|
||||
return request<PatchRun>(`/patch-policies/${policyId}/run-now`, { method: "POST" });
|
||||
},
|
||||
|
||||
listPatchRuns(params: { policy_id?: string; server_id?: string; limit?: number } = {}): Promise<PatchRun[]> {
|
||||
const qs = new URLSearchParams();
|
||||
if (params.policy_id) qs.set("policy_id", params.policy_id);
|
||||
if (params.server_id) qs.set("server_id", params.server_id);
|
||||
if (params.limit) qs.set("limit", String(params.limit));
|
||||
const suffix = qs.toString();
|
||||
return request<PatchRun[]>(`/patch-runs${suffix ? `?${suffix}` : ""}`);
|
||||
},
|
||||
|
||||
getPatchRun(runId: string): Promise<PatchRun> {
|
||||
return request<PatchRun>(`/patch-runs/${runId}`);
|
||||
},
|
||||
|
||||
cancelPatchRun(runId: string): Promise<void> {
|
||||
return request<void>(`/patch-runs/${runId}/cancel`, { method: "POST" });
|
||||
},
|
||||
|
||||
listAuditEvents(params: AuditQuery = {}): Promise<AuditPage> {
|
||||
const qs = new URLSearchParams();
|
||||
if (params.q) qs.set("q", params.q);
|
||||
|
||||
@@ -36,6 +36,7 @@ export const AUDIT_CATEGORIES: { value: string; label: string }[] = [
|
||||
{ value: "console", label: "Console" },
|
||||
{ value: "agent", label: "Agents" },
|
||||
{ value: "updates", label: "OS updates" },
|
||||
{ value: "patch", label: "Patching" },
|
||||
{ value: "auth_provider", label: "Single sign-on" },
|
||||
{ value: "settings", label: "Settings" },
|
||||
{ value: "token", label: "API tokens" },
|
||||
|
||||
Reference in New Issue
Block a user