diff --git a/CLAUDE.md b/CLAUDE.md index a799d5d..6924625 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -360,6 +360,76 @@ Two environment variables: `VANTAGE_TRIVY_DB_REF` mirrors the artifact for air-gapped installs, and `VANTAGE_VULNDB_DISABLED` switches the puller and scheduler off entirely. +### Workload registry + +A **workload** is one Docker container or one systemd unit — one word for the +page, the collection and the commands, rather than saying "container or +service" in every identifier. Linux only, and **not gated by licence**: this +reads as core fleet management, so v1 ships everywhere with no `HasFeature` +check. If that changes the check belongs at `ReportWorkloads`, gating collection +rather than display, exactly as sub-project A does. + +Agents collect on a 60-second ticker and report through `ReportWorkloads` with +the **offer-then-send** handshake the package report already uses. The offer is +identified by an explicit `full` flag, **not by an empty workloads list**: a +host genuinely running nothing sends an empty list as its full report, and +inferring the offer from emptiness leaves that host answering `need_full` every +60 seconds forever and never storing anything. + +**The on-demand refresh returns no data.** `RefreshWorkloadsCmd` carries nothing +back; it makes the agent report through the normal RPC and the UI refetches. A +refresh that returned workloads inline would be a second writer for +`server_workloads`, arriving by a different route with its own serialisation and +its own opportunity to disagree with the periodic one. One writer, one shape. +Opening the panel dispatches a refresh because the panel has a Restart button on +it, and a stale row is a wrong action aimed at a container that already died. + +Two operations do answer back, both over the bus, both with `Await` called +**before** dispatch: control actions reuse the existing `CommandResult`, and log +reads get `WorkloadLogsResult`. `CommandStream` republishes **every** +`CommandResult` onto `bus.ResultChannel` — publishing with no subscriber is a +no-op, so this costs nothing and avoids a second result path. + +**The protected set is computed agent-side and enforced agent-side.** +`vantage-agent.service`, plus the container ID read from `/proc/self/cgroup` +should the agent ever run in a container. As with the console relay hardcoding +`127.0.0.1`, the control plane may name a target but the agent decides what it +will do to itself; a server-side denylist alone would be bypassed by the next +dispatch path someone adds, and the failure is unrecoverable from the UI. The +reported `Protected` flag is the courtesy that greys the button; the agent's own +check is the boundary. The API answers **409** when it fires — nothing failed. + +Collection avoids parsing English: `docker ps -aq` then +`docker inspect --format '{{json .}}'`, because `docker ps` reports health and +uptime inside a human `Status` string that is localised and reworded between +releases. Compose stacks come from the `com.docker.compose.project` label, never +from YAML on disk — a compose file there may not be what is running. systemd +uses **column** output, not `--output=json`, which needs systemd 246+. + +`DockerOK`/`DockerError` are two fields because there are three states: not +installed (common on this fleet, and not a fault), installed but not responding, +and running nothing. The UI must render the first as "not in use here" rather +than an empty list. + +Logs are capped at **500 lines and 256KB, whichever binds first** — a line count +alone does not bound size, and 500 lines of 4KB JSON is 2MB across the bus. The +cap is mirrored in `services.MaxWorkloadLogLines` because `agent/` is a separate +module with an `internal/` tree and the constant cannot be shared; change one, +change the other. There is **no follow mode**: the browser console already gives +a real terminal where `docker logs -f` works properly. Log reads and control +actions are **owner|admin and audited**, unlike the read-only snapshot — a +container's stdout is arbitrary and cannot be masked the way a workflow's can. + +`server_workloads` is one document per server, mirroring `server_packages`, and +is in `ScopedCollections` (which `scopedCollectionsForPurge` derives from). There +is no history: a workload list is state, not a record. + +**`proto/vantage/v1/vantage.proto` is documentation, not a generator input.** +Both `pb` packages are hand-written JSON-tagged structs over a custom codec, and +there are two copies — `agent/internal/grpc/pb` and `server/internal/grpc/pb`. +A message added to one must be added to the other and to the `.proto`, in the +same commit. + ### Agent self-update `UpdateAgentCmd` carries a target version and Gitea base URL; the agent downloads and replaces itself. @@ -512,6 +582,7 @@ service Vantage { rpc SyncKeys(SyncRequest) returns (SyncResponse); rpc UploadGeneratedKey(UploadKeyRequest) returns (UploadKeyResponse); rpc ReportUpdates(ReportUpdatesRequest) returns (ReportUpdatesResponse); + rpc ReportWorkloads(ReportWorkloadsRequest) returns (ReportWorkloadsResponse); rpc ReportInventory(InventoryReport) returns (InventoryReportResponse); rpc SyncMonitors(SyncMonitorsRequest) returns (SyncMonitorsResponse); rpc ReportChecks(ReportChecksRequest) returns (ReportChecksResponse); @@ -521,7 +592,8 @@ service Vantage { `CommandStream` is the only streaming RPC: the agent authenticates once with `AgentReady`, then the server pushes `ServerCommand`s and the agent replies with `CommandResult`, `StepResult`, or `StepOutputChunk`. -`ServerCommand` variants: `GenerateKeyCmd`, `DeleteKeyCmd`, `UpdateAgentCmd`, `ApplyUpdatesCmd`, `RunStepCmd`, `CleanupWorkspaceCmd`, `OpenProxyCmd`, `PingCmd`. +`ServerCommand` variants: `GenerateKeyCmd`, `DeleteKeyCmd`, `UpdateAgentCmd`, `ApplyUpdatesCmd`, `RunStepCmd`, `CleanupWorkspaceCmd`, `OpenProxyCmd`, `PingCmd`, `RefreshWorkloadsCmd`, `ControlWorkloadCmd`, +`WorkloadLogsCmd`. **`PingCmd` is a liveness beat, and it is not redundant with gRPC keepalive.** The server sends one every 20s on an otherwise idle command stream; the agent @@ -578,6 +650,10 @@ vulns GET /vulnerabilities · GET /vulnerabilities/summary GET /servers/:id/vulnerabilities · GET /servers/:id/packages GET /packages/search?name= GET,POST /vuln-rules · PUT,DELETE /vuln-rules/:id (owner|admin) +workloads GET /workloads · GET /servers/:id/workloads + POST /servers/:id/workloads/refresh + POST /servers/:id/workloads/:wid/action (owner|admin) + GET /servers/:id/workloads/:wid/logs (owner|admin) audit GET /audit agent GET /agent/latest-version settings GET,PUT /settings · POST /settings/secrets-token (owner|admin) @@ -658,7 +734,7 @@ Paddle is merchant of record; `admin/internal/paddle` is a thin REST client (no ## MongoDB Collections -`servers` · `keys` · `assignments` · `orgs` · `users` · `auth_providers` · `settings` · `secrets` · `workflows` · `workflow_steps` · `workflow_runs` · `workflow_log_lines` · `workflow_log_seq` · `monitors` · `incidents` · `monitor_rollups` · `notification_channels` · `console_sessions` · `audit_logs` · `server_packages` · `vuln_findings` · `vuln_alert_rules` · `vulndb_meta` · `migrations` +`servers` · `keys` · `assignments` · `orgs` · `users` · `auth_providers` · `settings` · `secrets` · `workflows` · `workflow_steps` · `workflow_runs` · `workflow_log_lines` · `workflow_log_seq` · `monitors` · `incidents` · `monitor_rollups` · `notification_channels` · `console_sessions` · `audit_logs` · `server_packages` · `vuln_findings` · `vuln_alert_rules` · `vulndb_meta` · `server_workloads` · `migrations` Every document except `migrations` carries `org_id`. Struct definitions are the source of truth — see `server/internal/models/`. diff --git a/docsite/docs/vantage/workloads.md b/docsite/docs/vantage/workloads.md new file mode 100644 index 0000000..bef6975 --- /dev/null +++ b/docsite/docs/vantage/workloads.md @@ -0,0 +1,90 @@ +--- +id: workloads +title: Workloads +sidebar_label: Workloads +--- + +A **workload** is one Docker container or one systemd service. Each Linux +server reports what it runs, and you can start, stop and restart those +workloads — and read a snapshot of their logs — without opening a console. + +Available on every instance. No licence feature is required. + +## What gets reported + +Linux servers only. Agents report every 60 seconds, and an unchanged list costs +a single small message rather than the whole thing again. + +- **Containers** — every container, running or not, with its image, published + ports, health, restart count and the compose stack it belongs to. +- **Services** — systemd units that are running or failed, plus units that are + enabled but currently stopped. The platform's own units (`systemd-*`, + `user@*`, `session-*`) are filtered out; a typical host has 300 of them and + they bury the ten you care about. + +Windows servers report no workloads at all. + +## Docker not in use is not an error + +Three different things look identical if you are careless, and only one of them +is a problem: + +| What you see | What it means | +| ------------ | ------------- | +| "Docker is not in use on this server" | Docker is not installed. Normal, and not a fault | +| "Docker is installed but not responding" | The daemon is down or the socket is unreachable | +| An empty container list | Docker is running and there are no containers | + +## Stacks are grouped + +Compose stacks appear first, grouped under the stack name, then loose +containers, then services. A stack is one thing even when it is six containers, +and a flat list turns one decision into six rows. + +The stack name comes from Docker's own `com.docker.compose.project` label. No +compose file is read from disk — a file on disk may not be what is running. + +## Controlling a workload + +Start, stop and restart are **owner or admin only**, and every action is +written to the audit log naming you, the server and the target. + +The agent refuses to act on itself. `vantage-agent.service` is shown with its +buttons disabled: a server that stops its own agent goes offline, and the only +way back is SSH or physical access — which is exactly what this page exists to +avoid needing. + +A stop that never finishes is not reported as success. Both `docker stop` and +`systemctl stop` run under a 90-second limit, and a timeout comes back as a +real error. + +## Reading logs + +Logs are **owner or admin only** and every read is audited. Unlike workflow +logs, a container's output cannot be masked: it is arbitrary, and a startup +banner or a stack trace may contain credentials nobody declared. + +A log read returns a snapshot of at most **500 lines or 256KB**, whichever +limit is reached first, with the most recent output kept. When either limit +binds, the dialog says so — a truncated log must never be read as a complete +one. + +There is no live following. The [browser console](./browser-console.md) already +gives you a real terminal on the same server, where `docker logs -f` works +properly with its own scrollback. + +## Refreshing + +Opening a server's Workloads panel asks its agent to report immediately, so +what is on screen is current rather than up to a minute old. That matters +because the panel has a Restart button on it: a stale row is not just a wrong +impression, it is a wrong action aimed at something that already died. + +If the agent is offline the refresh fails visibly rather than queueing. A +command whose target cannot be reached must say so. + +## Fleet view + +**Workloads** in the sidebar searches the whole fleet by image, stack or state +— "which of these servers is still on the old image" — and links each result +back to its server. diff --git a/docsite/sidebars.ts b/docsite/sidebars.ts index 716c1a4..7d5756d 100644 --- a/docsite/sidebars.ts +++ b/docsite/sidebars.ts @@ -27,6 +27,7 @@ const sidebars: SidebarsConfig = { "vantage/workflows", "vantage/monitors", "vantage/vulnerabilities", + "vantage/workloads", "vantage/notification-channels", "vantage/secrets", "vantage/browser-console", diff --git a/server/internal/services/migrate_instance.go b/server/internal/services/migrate_instance.go index a6b7d5e..77143ae 100644 --- a/server/internal/services/migrate_instance.go +++ b/server/internal/services/migrate_instance.go @@ -43,6 +43,7 @@ var ScopedCollections = []string{ "server_packages", "vuln_findings", "vuln_alert_rules", + "server_workloads", } // collectionRenames maps the two collections whose names change. Ordered so the