Vantage/vantage-app
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: replace em dashes with hyphens, add no-em-dash rule to CLAUDE.md
Chart Release / chart (push) Successful in 20s
Details
Server Deploy / deploy (push) Failing after 1m52s
Details

Browse Source
This commit is contained in:
mrhid6
2026-09-10 09:18:55 +00:00
parent b36a696d0e
commit 6ee203f5e9
172 changed files with 860 additions and 856 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+179 -175
View File
@@ -22,7 +22,7 @@ A central server (Go + Next.js + MongoDB + Redis) drives a lightweight Go agent
│ monitor scheduler · workflow runner │
│ guacd tunnel proxy for browser console │
└───────────────┬──────────────────────────────┘
│ gRPC (TLS) — outbound from agent only
│ gRPC (TLS) - outbound from agent only
┌───────────────▼──────────────────────────────┐
│ Go Agent (per server, Linux + Windows) │
│ polls SyncKeys · CommandStream │
@@ -66,7 +66,7 @@ left here is the control plane and its UI, and nothing else.
| Repository | What it holds |
| ---------------- | ------------------------------------------------------------------------------------------------- |
| `vantage-shared` | the private Go module below — `mail`, `license`, `models`, `provision`, `backup`, `grpc/pb`, … |
| `vantage-shared` | the private Go module below - `mail`, `license`, `models`, `provision`, `backup`, `grpc/pb`, … |
| `vantage-admin` | Vantage HQ: the licensing authority (`server/`, was `admin/`) and its console (`web/`, was `adminsite/`) |
| `vantage-site` | the marketing site (`web/`, was `site/`) and its contact-form service (`server/`, was `sitesvc/`) |
| `vantage-docs` | the user documentation, at the repository root (was `docsite/`) |
@@ -75,13 +75,13 @@ left here is the control plane and its UI, and nothing else.
**None of the three is a build dependency of anything here**, and nothing here
is a dependency of them. `vantage-site` and `vantage-docs` are wholly
independent — the contact-form service stores nothing and reads no database, so
independent - the contact-form service stores nothing and reads no database, so
the split cost nothing. `vantage-admin` is the only one with a live coupling,
and there is still no import in either direction, deliberately (see "Grants project, they do not
federate"). It reaches this codebase two ways at runtime, both by writing
directly into the control plane's MongoDB: `inject` for three licence fields
and `cloudprov` for instances and their owners. The parts of that contract this
repository must honour are documented where they bite — `users.auth_source ==
repository must honour are documented where they bite - `users.auth_source ==
"hq"` and `services.ErrHQManaged`, `POST /license` answering 409
`cloud_managed`, and `FREE_INSTANCE_REAP_AFTER` needing to match admin's value.
The rest lives in that repository's own CLAUDE.md.
@@ -92,7 +92,7 @@ email system: transport plus templates), `license/` (payload, sign, verify,
trusted keys, plans), `models/` (Instance, User, Settings), `provision/`,
`backup/`, `cryptobox/`, `indexes/`, `grpc/pb` + `grpc/codec`, and
`cmd/lkctl/`, and `proto/vantage/v1/vantage.proto`, which documents `grpc/pb`
and moved there to sit beside it. **One** module here depends on it — `server` —
and moved there to sit beside it. **One** module here depends on it - `server` -
pinning a version in its own `go.mod`, as do `vantage-admin`, `vantage-site`,
`vantage-agent` and `vantage-ctl`. It was a
directory in this repository until it was extracted with its history; the
@@ -103,13 +103,13 @@ with it.
directory, every service in a given commit built against exactly one `shared/`,
and a change there rebuilt several images at once whether or not they were ready
for it. Now a service moves when somebody bumps its pin, which is a commit under
that service's own directory — so the existing per-directory rebuild triggers
that service's own directory - so the existing per-directory rebuild triggers
already cover it, and there is no longer any way to ship a service against a
`shared/` it was never built against. The cost is the obvious one: a fix in
`vantage-shared` is live nowhere until each consumer's pin is bumped, and
nothing in this repository will remind you.
Every Go build now needs a credential for it — `GOPRIVATE=gitea.hostxtra.co.uk/*`
Every Go build now needs a credential for it - `GOPRIVATE=gitea.hostxtra.co.uk/*`
plus a netrc. CI writes one per job from `REGISTRY_USER` + `RELEASE_TOKEN`
(**that token needs read access to the `vantage` org, not only `mrhid6`**), and
the four Go Dockerfiles take it as a **BuildKit secret** rather than a build
@@ -138,24 +138,24 @@ Upload a public key, assign it per server, revoke softly. The agent diffs desire
A library of reusable **steps** (bash or PowerShell scripts with declared inputs, outputs, and secret refs) composed into **workflows** targeting a set of servers. Running one snapshots the resolved steps into a `WorkflowRun`, then dispatches `RunStepCmd` over the agent command stream. Step stdout/stderr streams back as `StepOutputChunk` and is written to MongoDB (`workflow_log_lines`, one document per line); the UI streams it live. Steps support `on_failure: stop|continue|retry`, per-run env passed between steps via `output_env`, and a per-run workspace directory the agent cleans up at the end.
Default steps are seeded per org at boot (`SeedDefaultSteps`) from `VANTAGE_DEFAULT_STEPS_DIR`, which `server/Dockerfile` bakes to `/opt/default-steps` from the repo's `default_steps/`. Deliberately **not** under `/data` — that is a bind mount, so the library would be editable from the host. Adding a step there means committing a file and rebuilding, which is why `default_steps/` is in the `server` rebuild trigger. **Steps with `source: "default"` are read-only**: `UpdateStep`/`DeleteStep` refuse with `ErrDefaultStep` (409), because seeding rewrites them on every boot, so an edit would silently revert and a delete would come back. `web/` mirrors this — the step modal opens read-only, Delete is hidden, and the designer's per-step script override is `readOnly` for a default library step — but as elsewhere, the API is the boundary and the UI is the courtesy. Seeding writes straight to the collection rather than through `UpdateStep`, so the guard does not lock out the seeder. Logs are swept by retention (`workflow_log_retention_days`; nil = 30 days, 0 = forever).
Default steps are seeded per org at boot (`SeedDefaultSteps`) from `VANTAGE_DEFAULT_STEPS_DIR`, which `server/Dockerfile` bakes to `/opt/default-steps` from the repo's `default_steps/`. Deliberately **not** under `/data` - that is a bind mount, so the library would be editable from the host. Adding a step there means committing a file and rebuilding, which is why `default_steps/` is in the `server` rebuild trigger. **Steps with `source: "default"` are read-only**: `UpdateStep`/`DeleteStep` refuse with `ErrDefaultStep` (409), because seeding rewrites them on every boot, so an edit would silently revert and a delete would come back. `web/` mirrors this - the step modal opens read-only, Delete is hidden, and the designer's per-step script override is `readOnly` for a default library step - but as elsewhere, the API is the boundary and the UI is the courtesy. Seeding writes straight to the collection rather than through `UpdateStep`, so the guard does not lock out the seeder. Logs are swept by retention (`workflow_log_retention_days`; nil = 30 days, 0 = forever).
### Scheduled workflows
A workflow may carry `schedule{enabled, cron, tz}` — standard **5-field** cron
A workflow may carry `schedule{enabled, cron, tz}` - standard **5-field** cron
and an IANA zone name, both validated at save time. `next_run_at` is
**persisted on the document, not held in memory**: a leader handover between
computing an occurrence and firing it would otherwise lose it or fire it twice,
the same argument that put `workflow_log_seq` in MongoDB.
`server/internal/workflowsched` ticks every 30s inside the **existing**
`bus.RunAsLeader("housekeeping", …)` alongside `monitorsched` and the sweepers —
`bus.RunAsLeader("housekeeping", …)` alongside `monitorsched` and the sweepers -
one role, one lock. **The atomic claim, not the lock, is what prevents a double
fire**: the `UpdateOne` matches on the document *and* its current `next_run_at`
while setting the recomputed one, so a second process reaching the same workflow
matches nothing and does nothing. The lock only makes it cheap.
`workflowsched` **must not import `services`** — `services` already imports it
`workflowsched` **must not import `services`** - `services` already imports it
for `SetSchedule`'s call to `NextOccurrence`, and Go has no cycles.
`TriggerWorkflow` and `LogEvent` are therefore injected as `workflowsched.Deps`
from `main.go`. Firing goes through the same `TriggerWorkflow` a person uses,
@@ -165,7 +165,7 @@ detail page needed no changes.
`main.go` imports `_ "time/tzdata"`, and it is load-bearing: `server/Dockerfile`
runs on `scratch`, which ships no zone database, so without it
`time.LoadLocation("Europe/London")` fails and every schedule silently falls
back to UTC — an hour wrong for half the year, in the direction nobody notices
back to UTC - an hour wrong for half the year, in the direction nobody notices
until a maintenance window lands in business hours. It works on a developer
machine either way, which is exactly why it gets forgotten.
@@ -175,12 +175,12 @@ parses is disabled rather than left spinning the loop every 30 seconds forever.
### Server tags and workflow targeting
A server carries `tags map[string]string` — lowercase `[a-z0-9_-]`, key ≤32,
A server carries `tags map[string]string` - lowercase `[a-z0-9_-]`, key ≤32,
value ≤64, 20 per server, `sys:` reserved. **There is no `tags` collection**: a
tag is a property of a server, not an entity, so `KnownTags` aggregates over
`servers` rather than reading a registry that would need reference counting to
know when a tag stopped existing. `PUT /api/servers/:id/tags` replaces the whole
map — last-write-wins over a small map beats merge semantics between two people
map - last-write-wins over a small map beats merge semantics between two people
editing one server. The index is `{instance_id: 1, "tags.$**": 1}`, wildcard
because the queried key is chosen by the user at request time and cannot be named
in advance; `EnsureServerIndexes` warns rather than being fatal, since a missing
@@ -188,19 +188,19 @@ index degrades tag filtering to a scan of a small collection and is no reason to
refuse to serve the fleet list.
`services.ResolveTargets` is the **single** answer to which servers a workflow
touches — the run path and validation both go through it, so the readout and the
touches - the run path and validation both go through it, so the readout and the
dispatch cannot disagree. It is the distinct union of `target_server_ids` and
`target_tags` (AND across keys), ordered by the fleet rather than by the
arguments, so two runs naming the same servers differently are still comparable
line by line. **An empty selector matches nothing** on purpose: "matches
everything" turns a cleared field in the designer into a fleet-wide run. Both
empty is `ErrNoTargets` (400), not a success over zero servers. Offline servers
are **not** filtered out — the dispatcher already answers 503 per server, and a
are **not** filtered out - the dispatcher already answers 503 per server, and a
patch run that silently omits an unreachable machine is worse than one that
visibly fails on it.
**Both halves of the selector are edited in `EditWorkflowModal`** — the named
servers in a `DualListBox`, the tag rows directly beneath it — and saved
**Both halves of the selector are edited in `EditWorkflowModal`** - the named
servers in a `DualListBox`, the tag rows directly beneath it - and saved
together by one `updateWorkflow`. The designer's Targets panel is **read-only**:
it reports the count and the tags and links to Edit. Splitting the two halves
across two screens meant a workflow's reach was decided in two places with no
@@ -209,10 +209,10 @@ one view showing both.
`web/lib/targets.ts` **duplicates the match logic in TypeScript** to draw the
resolved count without a round trip, since the browser already holds the fleet.
It is a second implementation of `UnionTargets` / `MatchesTags` and must change
in the same commit as the Go one — the same shape of hazard as the mirrored
in the same commit as the Go one - the same shape of hazard as the mirrored
token blocks. It is a shared module rather than inline in a component because
the logic had already been written twice, and the second copy — the workflows
list — counted `target_server_ids` alone, so a **tag-only workflow reported zero
the logic had already been written twice, and the second copy - the workflows
list - counted `target_server_ids` alone, so a **tag-only workflow reported zero
targets** while running fine.
The server picker is a hand-built two-pane list, not `<select multiple>`: a
@@ -239,7 +239,7 @@ upgrades to a WebSocket and proxies to **guacd** using `github.com/wwt/guac`.
guacd never dials the managed server. The server binds a single-use ephemeral
listener, pushes `OpenProxyCmd` down the agent's command stream, and the agent
opens a `ProxyStream` and relays the connection from its own **`127.0.0.1`** —
opens a `ProxyStream` and relays the connection from its own **`127.0.0.1`** -
the host is hardcoded agent-side, so the control plane can name only a port.
This is what makes the console work on Vantage Cloud, where the customer's
server is behind NAT on a private address. It also means the console now
@@ -250,12 +250,12 @@ server is behind NAT on a private address. It also means the console now
build the allow-list of sources permitted to claim a relay listener; a ClusterIP
resolves to the Service's virtual address while guacd connects from its *pod*
IP, so every relay connection is rejected and every session dies with
`waiting for guacd: i/o timeout`. Compose is immune — there the name resolves to
`waiting for guacd: i/o timeout`. Compose is immune - there the name resolves to
the address that connects.
SSH connections authenticate with a stored private key; RDP/VNC credentials are
encrypted, single-use, and consumed when the tunnel opens. None of them reach
the agent — the session is negotiated end-to-end between guacd and the target
the agent - the session is negotiated end-to-end between guacd and the target
daemon, so the agent relays bytes it cannot read.
### Running more than one server replica
@@ -266,7 +266,7 @@ replicas, the process asked to do something to an agent is almost never the
process holding that agent's stream.
`server/internal/bus` is the Redis message bus that closes the gap. It adds no
infrastructure — Redis was already required for sessions — and it is **not
infrastructure - Redis was already required for sessions - and it is **not
optional on a single-replica deployment**: dispatch takes the bus path always,
so the code running in production is the code running everywhere, rather than a
rare cross-pod branch that only fails under load.
@@ -274,16 +274,16 @@ rare cross-pod branch that only fails under load.
| Concern | How it crosses replicas |
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Which pod owns an agent | `vantage:agent:<server_id>` holds the owner's node ID with a 30s TTL, renewed every 10s. `Dispatcher.IsConnected` is an `EXISTS` on it |
| Sending a command | published to `vantage:cmd:<server_id>`; the owner pod acks on `vantage:ack:<command_id>`. **Request/ack, not a queue** — a command whose owner died must fail loudly (503) rather than queue. The envelope carries `node`, the presence holder resolved at publish time, and a pod ignores envelopes addressed elsewhere: the channel is a fan-out, and during a reconnect a half-open stream's pod is still subscribed. Unaddressed, it could ack first and queue the command onto a dead stream — the operator told it worked, the agent never seeing it. Presence renewal is owner-only (`RenewPresence`) for the same reason: a blind `SET` let the stale pod steal the key back every 10s |
| Sending a command | published to `vantage:cmd:<server_id>`; the owner pod acks on `vantage:ack:<command_id>`. **Request/ack, not a queue** - a command whose owner died must fail loudly (503) rather than queue. The envelope carries `node`, the presence holder resolved at publish time, and a pod ignores envelopes addressed elsewhere: the channel is a fan-out, and during a reconnect a half-open stream's pod is still subscribed. Unaddressed, it could ack first and queue the command onto a dead stream - the operator told it worked, the agent never seeing it. Presence renewal is owner-only (`RenewPresence`) for the same reason: a blind `SET` let the stale pod steal the key back every 10s |
| Step results | the owner pod publishes to `vantage:res:<command_id>`; the pod driving the run subscribes **before** dispatching, or a fast agent answers into a channel nobody has joined |
| Step output | never crosses. The dispatch envelope carries the secret mask list, so the owner pod masks and writes lines itself — unmasked bytes stay off the bus |
| Console relay | **not routed to the owner pod at all.** A `ProxyStream` is its own HTTP/2 request and an L7 proxy balances requests, not connections, so it does not follow the command stream — the listener therefore cannot be bound in advance. Whichever pod receives the stream binds it and announces **its own** address on `vantage:proxyaddr:<proxy_id>`; `vantage:proxypending:<proxy_id>` (30s, consumed atomically) is what authorises the claim, and the failure reason comes back on `vantage:proxyend:<proxy_id>` |
| Background jobs | `bus.RunAsLeader` — one Redis lock named `housekeeping` |
| Step output | never crosses. The dispatch envelope carries the secret mask list, so the owner pod masks and writes lines itself - unmasked bytes stay off the bus |
| Console relay | **not routed to the owner pod at all.** A `ProxyStream` is its own HTTP/2 request and an L7 proxy balances requests, not connections, so it does not follow the command stream - the listener therefore cannot be bound in advance. Whichever pod receives the stream binds it and announces **its own** address on `vantage:proxyaddr:<proxy_id>`; `vantage:proxypending:<proxy_id>` (30s, consumed atomically) is what authorises the claim, and the failure reason comes back on `vantage:proxyend:<proxy_id>` |
| Background jobs | `bus.RunAsLeader` - one Redis lock named `housekeeping` |
**Workflow logs are in MongoDB** (`workflow_log_lines`, one document per line,
with a `workflow_log_seq` counter document per run/server). Two pods write the
same log concurrently — the run's pod emits markers, the agent's pod emits
output — so ordering only means anything if both draw sequence numbers from the
same log concurrently - the run's pod emits markers, the agent's pod emits
output - so ordering only means anything if both draw sequence numbers from the
same counter. `StepRun.log_offset` is that sequence number now, not a byte
offset. Writes are batched (128 lines or 250ms) and capped: 8 KB per line,
200k lines per server-run, after which one final `[vantage] log truncated`
@@ -294,7 +294,7 @@ is a database incident. **Nothing writes to `/data` any more**, which is why
**Shutdown order is load-bearing.** `main` traps SIGTERM, stops gRPC
(`GracefulStop`, 10s cap) and only then drains HTTP. Each `CommandStream`
handler releases its agent's presence claim on return, so a killed process
leaves `vantage:agent:<server_id>` behind for the rest of its 30s TTL — during
leaves `vantage:agent:<server_id>` behind for the rest of its 30s TTL - during
which other replicas dispatch to a pod that has exited and the caller sees
`agent offline` for a perfectly healthy agent. Draining HTTP first would hold
those claims for the length of the drain, which is why gRPC goes first. The
@@ -305,7 +305,7 @@ handling buys nothing.
The agent side of the same failure: `runCommandStream` resets its backoff only
after a stream that survived `streamHealthyAfter`. `connectAndHandleStream`
returns an error on *every* stream end, healthy ones included, so without that
reset the backoff only ever climbed — an agent pinned itself at the ceiling
reset the backoff only ever climbed - an agent pinned itself at the ceiling
after a handful of ordinary deploys and stayed there. The ceiling is 30s, not
minutes, because while the stream is down the agent still polls `SyncKeys` and
still reads as `active` in the fleet list while answering no commands at all.
@@ -315,7 +315,7 @@ scheduler means each check fires N times, each incident notification reaches the
customer N times, and each hourly rollup is written N times; N reapers race to
purge the same Free instance. `monitorsched`, `StartReaper`, `StartLogSweeper`,
`StartAuditSweeper` and the offline sweep therefore all run inside one
`RunAsLeader("housekeeping", …)` — one role, one lock. Each takes a context
`RunAsLeader("housekeeping", …)` - one role, one lock. Each takes a context
cancelled the instant leadership is lost, and must return when it is.
Redis rather than a Kubernetes `Lease` so Compose takes the identical path: one
@@ -323,13 +323,13 @@ implementation to reason about, not two.
Two deployment requirements come with `replicaCount > 1`: every replica must
share **one** Redis (a per-pod Redis partitions the bus and every agent looks
offline to two thirds of the fleet), and `POD_IP` must be set — the chart does
it from the downward API — because `PROXY_ADVERTISE_HOST` names the Service, and
offline to two thirds of the fleet), and `POD_IP` must be set - the chart does
it from the downward API - because `PROXY_ADVERTISE_HOST` names the Service, and
a Service cannot address the one pod holding a console listener.
### Inventory and OS updates
Agents report CPU/memory/swap/partitions/kernel — metrics every 30s, full static snapshot every 15 min. They also check for pending OS package updates hourly and can apply them on command (`ApplyUpdatesCmd`).
Agents report CPU/memory/swap/partitions/kernel - metrics every 30s, full static snapshot every 15 min. They also check for pending OS package updates hourly and can apply them on command (`ApplyUpdatesCmd`).
Windows update checking and applying go through the Windows Update COM API
(`Microsoft.Update.Session`) rather than the PSWindowsUpdate module, which would
@@ -347,7 +347,7 @@ every 15 minutes. Linux fills it too, from `/var/run/reboot-required` or
Agents report their installed packages hourly; the control plane matches them
against distribution security feeds and raises findings that link to the
existing `ApplyUpdatesCmd` patching path. Gated by the `vuln_scanning` licence
feature, **checked at collection rather than display** — an ungated instance
feature, **checked at collection rather than display** - an ungated instance
stores no inventory, and storage is the expensive half.
**Matching uses distribution feeds, never NVD version ranges.** Distributions
@@ -369,7 +369,7 @@ replicas rescanning the same fleet and sending N digests. The tick is also the
digest's batch boundary, which is what makes "one message, not five hundred"
structural rather than a debounce someone maintains.
Findings are **never deleted when a package is patched** — the state moves to
Findings are **never deleted when a package is patched** - the state moves to
`fixed`, so "what did we remediate last quarter" stays answerable. Acceptance
requires a reason and an expiry, and reopens automatically: permanent dismissal
is where risk goes to be forgotten. An unsupported distribution reports
@@ -379,7 +379,7 @@ unknown is the same lie as a silently stale database, which is why
**`server/Dockerfile`'s runtime stage is `scratch`, so it carries an explicitly
copied `/tmp`.** The scheduler unpacks the database to a temporary directory,
and a scratch image has none — the failure is `vulnsched: temp dir: stat /tmp:
and a scratch image has none - the failure is `vulnsched: temp dir: stat /tmp:
no such file or directory`, logged once at boot while every other subsystem
runs normally, so the only symptom is a fleet that never reports a finding.
@@ -389,19 +389,19 @@ scheduler off entirely.
### Workload registry
A **workload** is one Docker container or one systemd unit — one word for the
A **workload** is one Docker container or one systemd unit - one word for the
page, the collection and the commands, rather than saying "container or
service" in every identifier.
On Windows a workload is a Docker container or a Windows **service**, reported
under the same `unit` kind and the same `systemd_ok` / `systemd_error` fields —
under the same `unit` kind and the same `systemd_ok` / `systemd_error` fields -
one wire shape, worded per platform in the UI, which is the only layer that
knows the host's OS. The platform split lives entirely in the agent, as build
tags (`systemd_linux.go` / `services_windows.go` and the matching `control_`
and `logs_` pairs); the control plane is OS-blind and needed no changes.
Windows collection runs PowerShell through the agent's `internal/winexec`. Every
script that reports data emits JSON that a build-tag-free parser reads, so
those parsers are tested on Linux — the agent module has no Windows CI. The
those parsers are tested on Linux - the agent module has no Windows CI. The
control verbs and `serviceDisplayName` emit no JSON and have no parser; they
are exercised only by running the agent on Windows.
@@ -428,7 +428,7 @@ it, and a stale row is a wrong action aimed at a container that already died.
Two operations do answer back, both over the bus, both with `Await` called
**before** dispatch: control actions reuse the existing `CommandResult`, and log
reads get `WorkloadLogsResult`. `CommandStream` republishes **every**
`CommandResult` onto `bus.ResultChannel` — publishing with no subscriber is a
`CommandResult` onto `bus.ResultChannel` - publishing with no subscriber is a
no-op, so this costs nothing and avoids a second result path.
**The protected set is computed agent-side and enforced agent-side.**
@@ -439,13 +439,13 @@ with the console relay hardcoding
will do to itself; a server-side denylist alone would be bypassed by the next
dispatch path someone adds, and the failure is unrecoverable from the UI. The
reported `Protected` flag is the courtesy that greys the button; the agent's own
check is the boundary. The API answers **409** when it fires — nothing failed.
check is the boundary. The API answers **409** when it fires - nothing failed.
Collection avoids parsing English: `docker ps -aq` then
`docker inspect --format '{{json .}}'`, because `docker ps` reports health and
uptime inside a human `Status` string that is localised and reworded between
releases. Compose stacks come from the `com.docker.compose.project` label, never
from YAML on disk — a compose file there may not be what is running. systemd
from YAML on disk - a compose file there may not be what is running. systemd
uses **column** output, not `--output=json`, which needs systemd 246+.
`DockerOK`/`DockerError` are two fields because there are three states: not
@@ -453,13 +453,13 @@ installed (common on this fleet, and not a fault), installed but not responding,
and running nothing. The UI must render the first as "not in use here" rather
than an empty list.
Logs are capped at **500 lines and 256KB, whichever binds first** — a line count
Logs are capped at **500 lines and 256KB, whichever binds first** - a line count
alone does not bound size, and 500 lines of 4KB JSON is 2MB across the bus. The
cap is mirrored in `services.MaxWorkloadLogLines` because the agent is a
separate module — a separate repository now — with an `internal/` tree, and the
separate module - a separate repository now - with an `internal/` tree, and the
constant cannot be shared; change one, change the other. There is **no follow mode**: the browser console already gives
a real terminal where `docker logs -f` works properly. Log reads and control
actions are **owner|admin and audited**, unlike the read-only snapshot — a
actions are **owner|admin and audited**, unlike the read-only snapshot - a
container's stdout is arbitrary and cannot be masked the way a workflow's can.
`server_workloads` is one document per server, mirroring `server_packages`, and
@@ -469,7 +469,7 @@ is no history: a workload list is state, not a record.
**The wire contract is not in this repository at all.** `shared/grpc/pb` is
hand-written JSON-tagged structs over the custom codec in `shared/grpc/codec`,
and `proto/vantage/v1/vantage.proto` is documentation of them rather than a
generator input — nothing compiles it. Both live in `vantage-shared`, together,
generator input - nothing compiles it. Both live in `vantage-shared`, together,
because that co-location is the only thing making "add the message to both in
the same commit" possible.
@@ -482,7 +482,7 @@ code, which the linker drops.
A message added to `vantage-shared` is not a message either side has until its
pin is bumped. What that buys is a mismatch that is a compile error rather than
two copies that both compiled and disagreed on the wire. What it costs is
ordering — **a wire change is three steps**: release `vantage-shared`, bump the
ordering - **a wire change is three steps**: release `vantage-shared`, bump the
pin in `server/` (live at the next push to main), bump the pin in `agent/` (live
only at the next `agent/v*` tag). The control plane runs ahead of the fleet in
between, which was true before too; it is now explicit in two `go.mod` files
@@ -490,7 +490,7 @@ rather than implicit in a shared directory.
### Status pages
Two collections: `status_pages` is the page itself — title, banner, published
Two collections: `status_pages` is the page itself - title, banner, published
flag, and an ordered list of sections each holding entries that pair a
`monitor_id` with a per-page display name. `status_incidents` holds both
operator-authored incidents and maintenance windows, sharing one document
@@ -505,7 +505,7 @@ one.** It takes a `snapshotInput` built from already-fetched
`StatusSnapshot` built entirely from a parallel, deliberately smaller
vocabulary (`PublicComponent`, `PublicIncident`, …) that has no field for a
target URL, host, port, expected status, keyword, failure message,
certificate expiry, latency, runner or notification channel — `models.Monitor`
certificate expiry, latency, runner or notification channel - `models.Monitor`
itself never reaches an anonymous caller, only the handful of fields
`assembleSnapshot` chooses to copy out of it. Being a pure function of already-
fetched data (no DB calls inside it) is what makes the boundary testable
@@ -515,7 +515,7 @@ a field to `PublicComponent` and that field being a hostname.
**An incident may only name components the page already carries.**
`services.checkAffectedOnPages` refuses an `affected_monitors` entry that no
page in the incident's `page_ids` lists, and the editor offers only the saved
page's components — labelled by their per-page display name, since that is the
page's components - labelled by their per-page display name, since that is the
name the reader sees. Naming an arbitrary monitor would publish a machine the
page deliberately does not, which is the same leak `assembleSnapshot`'s
redaction boundary exists to prevent, reached from the authoring side instead
@@ -531,7 +531,7 @@ snapshot assembly reads recent `incidents` for the page's monitors and folds
them into the timeline alongside the authored ones. There is no second
incidents table for automatic ones and no reconciliation between two records
of the same outage. A maintenance window in progress **repaints how a day is
drawn, never the uptime number** — `buildDays` computes each day's up/down
drawn, never the uptime number** - `buildDays` computes each day's up/down
state and the 90-day percentage from rollups first, and
`applyMaintenanceRepaint` only overwrites today's display state afterward, so
a component that stayed up throughout a maintenance window still shows as up
@@ -540,12 +540,12 @@ in its history.
The public route, `GET /public/status/:pageId`, is mounted on the gin **root**,
outside `/api`, on purpose: `/api` carries `auth.Middleware`, `RequireScopes`,
`RateLimitTokens` and `RequireActiveLicense` by virtue of where it is mounted,
and a public route living there would need four exemptions — each one a hole a
and a public route living there would need four exemptions - each one a hole a
later change to any of those four could widen back open. A missing page, an
unpublished page, and a page on the wrong host all answer the same 404;
inventing a distinct code for "exists but unpublished" would itself leak that
the page exists. A lapsed licence or a tier lacking `status_pages` answers 200
with `available:false` and a `reason`, never a 403 or a blank page — the
with `available:false` and a `reason`, never a 403 or a blank page - the
reader is a member of the public who can do nothing about either condition and
deserves an explanation, not a browser error.
@@ -556,7 +556,7 @@ saw `server:8080` and every status page 404'd on every deployment. `web/`
forwards the visitor's host in `X-Forwarded-Host` (and their address in
`X-Forwarded-For`, or the whole deployment shares one rate-limit bucket), and
`publicStatusInstance` honours that header **only when `c.RemoteIP()` is in
`TRUSTED_PROXIES`** — it selects a tenant, so an untrusted peer must not be
`TRUSTED_PROXIES`** - it selects a tenant, so an untrusted peer must not be
able to name one. It uses `RemoteIP()` and not `ClientIP()` deliberately: the
latter is reconstructed from the very headers being judged.
@@ -565,13 +565,13 @@ deployment.** `hostSlug` requires `<slug>.vantage.<tld>`; a self-hosted install
at `vantage.acme.com` or an IP has no slug and would otherwise 404 forever. It
has exactly one instance, resolved with the same count-then-read bootstrap
uses, cached alongside the slug lookups. More than one instance is a 404, not a
guess. A host that *does* name a slug which does not exist stays a 404 —
guess. A host that *does* name a slug which does not exist stays a 404 -
falling back there would serve one tenant's page on another's address.
Assembled snapshots are cached in Redis for **30 seconds**, keyed per
instance and page, and every authoring write (`UpdateStatusPage`,
`DeleteStatusPage`, and every incident mutation) invalidates its page's entry
immediately rather than waiting out the TTL — an operator posting an update
immediately rather than waiting out the TTL - an operator posting an update
mid-incident should not wonder for half a minute whether it saved. A cache
miss, on Redis being down or on any read error, degrades to reassembly rather
than an error: the status page has to survive the outage it exists to report.
@@ -581,7 +581,7 @@ pattern as `RateLimitTokens`.
**`TRUSTED_PROXIES` is load-bearing for that limiter, not cosmetic.** `main.go`
always calls `gin.SetTrustedProxies` with it; left unset, gin trusts no proxy
and `c.ClientIP()` falls back to the direct peer address — which, sat behind a
and `c.ClientIP()` falls back to the direct peer address - which, sat behind a
real reverse proxy, is the proxy's own address for every visitor. The rate
limiter then keys on one address for the whole fleet of readers, and the first
burst of legitimate traffic during an incident is what trips it. Set it to the
@@ -608,14 +608,14 @@ build that knows the new address, after which self-update works again.
The ordering matters. This server must be deployed with the new paths *before*
the one-liner is any use, because it is this server that hands out the URL.
The six generators — two install scripts, two update scripts,
The six generators - two install scripts, two update scripts,
`GET /api/agent/latest-version` in `services/dispatch.go`, and the tag lookup
inside each — all name that repository. They must agree with wherever
inside each - all name that repository. They must agree with wherever
`agent-release.yml` actually publishes, and nothing checks that they do.
### Backup and restore
`vantagectl` is `vantage-ctl` now — its own repository, with the command still
`vantagectl` is `vantage-ctl` now - its own repository, with the command still
named `vantagectl`. It is not a subcommand of `server` and never was: `server`
imports the whole control-plane dependency graph, `spf13/cobra` has no business
in a process that also terminates gRPC streams, and above all **it has to run
@@ -626,15 +626,15 @@ binary whose failure is the reason you reached for it.
Almost none of its logic is in that repository either: dump, restore, verify,
manifest and fingerprint are `shared/backup` in `vantage-shared`, and
`internal/cmd` holds only argument parsing and operator-facing output. That
split is what would let `server` import `shared/backup` later — a scheduled
in-process backup, say — without a second implementation to keep in sync.
split is what would let `server` import `shared/backup` later - a scheduled
in-process backup, say - without a second implementation to keep in sync.
`shared/cryptobox` is the same move one layer down: it is the **single**
AES-256-GCM implementation, and `server/internal/services/crypto.go` delegates
to it rather than keeping its own copy that `shared/backup` would otherwise have
had to duplicate to decrypt a probe value during `verify`.
**The one thing this repository owes it is `backup.ciphertextFields`**, which
lives in `vantage-shared` and mirrors `server/internal/models` **by hand** —
lives in `vantage-shared` and mirrors `server/internal/models` **by hand** -
`shared/` is a separate module and cannot import it. The map naming each
collection's `*_enc` fields (`keys`, `secrets`, `auth_providers`,
`console_sessions`) must change in the same commit as any of those bson tags,
@@ -642,47 +642,47 @@ and that commit is now in a different repository from the tags it tracks. Wrong
field names are **silent**: `verify`'s live probe finds no ciphertext and
reports "this database stores no ciphertext yet", so the one gate that catches
what a key fingerprint cannot becomes a no-op. `settings` is deliberately in
neither that map nor `CiphertextCollections()` — its ESO read token is a
neither that map nor `CiphertextCollections()` - its ESO read token is a
SHA-256 hash, not ciphertext.
The chart's optional backup CronJob runs that image, and **`backup.image` has
no default** — the template `fail`s without one rather than guessing, so a
no default** - the template `fail`s without one rather than guessing, so a
cluster set up before the move keeps working until someone changes the value.
It is `gitea.hostxtra.co.uk/vantage/vantage-ctl:latest` now, was
`mrhid6/vantage/vantagectl:latest`; `chart-release.yml`'s render checks name the
new path.
The rest — the refusals around `--force` and `--confirm-db`, the key
The rest - the refusals around `--force` and `--confirm-db`, the key
fingerprint, live collection enumeration, verbatim index replay, the `.partial`
rename — is documented in `vantage-ctl`.
rename - is documented in `vantage-ctl`.
### API tokens and OpenAPI
A token is `vt_` plus 32 random bytes hex, shown once at creation and stored
only as sha256 — the same shape as `servers.agent_token_hash` and the ESO read
only as sha256 - the same shape as `servers.agent_token_hash` and the ESO read
token, and for the same reason: nothing downstream ever needs the plaintext
back. It belongs to the user who created it, and its role can never exceed
theirs; see the `api_tokens` note under MongoDB Collections for how that stays
true across a demotion rather than only at issuance. Scopes are nine
resources — `servers`, `keys`, `secrets`, `workflows`, `monitors`, `vulns`,
`workloads`, `status`, `settings` — each split into `:read` and `:write`, with `:write`
resources - `servers`, `keys`, `secrets`, `workflows`, `monitors`, `vulns`,
`workloads`, `status`, `settings` - each split into `:read` and `:write`, with `:write`
satisfying a `:read` requirement on the same resource so a caller does not have
to hold both. Any signed-in member may mint and revoke their **own** tokens —
there is no `RequireRole` on `POST /tokens` or `DELETE /tokens/:id` — because
to hold both. Any signed-in member may mint and revoke their **own** tokens -
there is no `RequireRole` on `POST /tokens` or `DELETE /tokens/:id` - because
`roleRank` already bounds what a token can do to no more than its creator's
own role, so a member cannot use a token to reach past themselves. Owner and
admin additionally see and revoke every token in the instance — `all=true` on
admin additionally see and revoke every token in the instance - `all=true` on
`GET /tokens` is gated by `elevated()` in `api/tokens.go`, and
`RevokeAPIToken` in `services/tokens.go` checks the same owner/admin condition
before letting a revoke target somebody else's token — neither is a
before letting a revoke target somebody else's token - neither is a
`RequireRole` middleware.
The `settings:read`/`settings:write` entries in `routeScopes` govern a
**token-authenticated** caller reaching the token endpoints — `RequireScopes`
no-ops entirely for a cookie session — so they say nothing about which human
**token-authenticated** caller reaching the token endpoints - `RequireScopes`
no-ops entirely for a cookie session - so they say nothing about which human
role may call these routes with a session; that is `roleRank` and `elevated()`,
not the scope map. Expiry is optional per token; `settings.api_token_max_days` caps how
far out a new one may be set, and when that cap is set a token requested with
no expiry is refused rather than silently capped — the policy governs
no expiry is refused rather than silently capped - the policy governs
issuance only and never reaches back to invalidate a token already issued.
`RateLimitTokens` holds every token to 600 requests/minute in a Redis fixed
window, answering 429 with `Retry-After`; cookie sessions are untouched; it
@@ -695,12 +695,12 @@ The page is reachable at **every** role, which is the whole reason it is a page:
member has. `settings.api_token_max_days` stays on `/settings` because it is
instance policy rather than one person's credentials, and that split is exactly
what lets the page be ungated. The label differs from the identifiers on
purpose — the collection is `api_tokens`, the prefix is `vt_`, the routes are
purpose - the collection is `api_tokens`, the prefix is `vt_`, the routes are
`/api/tokens`, and renaming a published endpoint to match a nav label would
break every script already written against it.
`server/internal/api/docs/openapi.json` is a **generated, committed** OpenAPI
3.1 document — `swag v2` reading `@…` annotations off the handlers — served at
3.1 document - `swag v2` reading `@…` annotations off the handlers - served at
`GET /api/openapi.json` and rendered as a reference page by a vendored Scalar
bundle at `GET /api/docs`. `server-deploy.yml` regenerates it on every server
build and runs `git diff --exit-code` against the committed copy: a handler
@@ -708,7 +708,7 @@ whose annotation drifted from its code fails CI rather than shipping a
reference that lies. Scalar is vendored (`scalar.standalone.js`, served from
`GET /api/docs/scalar.js`) rather than pulled from a CDN, because the
reference page has to work on an air-gapped install with no outbound access at
all — the same requirement licence verification already meets.
all - the same requirement licence verification already meets.
### The public host
@@ -719,10 +719,10 @@ and its contact-form service are `vantage-site`; the documentation at `/docs` is
them on top of this one:
```bash
# self-hosted install — the control plane and nothing else
# self-hosted install - the control plane and nothing else
docker compose -f deploy/docker/docker-compose.yml up -d
# vantage.hostxtra.co.uk — every repository's fragment together
# vantage.hostxtra.co.uk - every repository's fragment together
docker compose \
-f vantage/deploy/docker/docker-compose.yml \
-f vantage-site/deploy/docker-compose.yml \
@@ -737,8 +737,8 @@ be a rule about which file a service went in; it is the repository boundary now.
**The reverse proxy in front is shared and belongs to none of them.** On
vantage.hostxtra.co.uk that is an Nginx Proxy Manager, and its routing spans
repositories: `/docs` to `vantage-docs` — a location that must sort **above**
the catch-all — and everything else on that host to `vantage-site`. A
repositories: `/docs` to `vantage-docs` - a location that must sort **above**
the catch-all - and everything else on that host to `vantage-site`. A
self-hosted install needs its own; see the compose note below for what it must
route.
@@ -749,7 +749,7 @@ a cloud instance from the portal.
### Signup and verification
Signup is **account-first**: it creates an HQ account and an unverified `customer_user` in admin's own database, nothing in the control plane. Only after a customer later creates a cloud instance from the portal (`POST /api/instances`, see Admin REST API) does an org, or rather an `instance`, come to exist — provisioned by `cloudprov`, with the owner's password hash copied from the HQ user rather than shared. `site_pending_signups` is gone; the contact-form service has no signup flow at
Signup is **account-first**: it creates an HQ account and an unverified `customer_user` in admin's own database, nothing in the control plane. Only after a customer later creates a cloud instance from the portal (`POST /api/instances`, see Admin REST API) does an org, or rather an `instance`, come to exist - provisioned by `cloudprov`, with the owner's password hash copied from the HQ user rather than shared. `site_pending_signups` is gone; the contact-form service has no signup flow at
all, and lives in another repository besides.
- The token is 32 random bytes; only its **SHA-256 hash** is stored, so a leaked database yields no working links.
@@ -759,7 +759,7 @@ all, and lives in another repository besides.
- Rate limited per client IP, plus a honeypot field.
An account is a team, not a person. `customer_users.account_role` is `owner`,
`admin` or `member` — the same three words as the control plane's roles, on
`admin` or `member` - the same three words as the control plane's roles, on
purpose. Owners and admins invite people, create instances and grant instance
access; billing is owner-only.
@@ -777,13 +777,13 @@ left unspent.
and `vantage-site` each import it and none of them builds a subject line, a MIME
part or a colour.
Before this existed the transport was copied three times, and the copies had
already diverged once — the 465-implicit-TLS fix landed in one of them while
already diverged once - the 465-implicit-TLS fix landed in one of them while
the others silently delivered nothing.
`Sender` is a value, not a singleton: `server/internal/notify` builds one per
notification channel from the channel document in Mongo, while `vantage-site`'s
service builds one at boot and `vantage-admin` holds one in its own
`internal/mail.Default`, alongside its other boot-time singletons. Callers only ever see typed methods —
`internal/mail.Default`, alongside its other boot-time singletons. Callers only ever see typed methods -
`SendVerification`, `SendExpiring`, `SendMonitorAlert`, `SendEnquiry` and the
rest, grouped by owner into `account.go`, `licence.go`, `billing.go`,
`monitor.go` and `contact.go`.
@@ -794,13 +794,13 @@ define `subject`, `title`, `pill` and `body`; `layout.html.tmpl` and
`layout.txt.tmpl` provide the chrome and the helper templates (`p`, `lead`,
`button`, `well`, `note`, `rows`, `chip`) that the bodies compose. One template
set is parsed per message rather than one big set, because every message
defines those same four names. **`subject` is defined in the txt file only** —
defines those same four names. **`subject` is defined in the txt file only** -
`html/template` would escape an ampersand in an instance name and mail clients
show subjects verbatim.
`shared/mail/render_test.go` renders all of them and fails if a template exists
that no case covers, which is the only thing standing between a mistyped field
and a boot-time panic — the templates are parsed in `init()`.
and a boot-time panic - the templates are parsed in `init()`.
### Shared provisioning
@@ -811,7 +811,7 @@ and a boot-time panic — the templates are parsed in `init()`.
Granting someone access to a cloud instance writes a real control-plane `users`
row through `cloudprov`, with `auth_source: "hq"` and `hq_user_id` set. The
instance authenticates it exactly as it authenticates anyone else, with **no
runtime dependency on admin**. Revoking deletes that row — the control plane has
runtime dependency on admin**. Revoking deletes that row - the control plane has
no disabled state, and a row that exists is a row that can sign in.
`instance_members` in admin's database is only admin's _index_ of those
@@ -826,7 +826,7 @@ The HQ password is the single source of truth for every `hq`-sourced row.
`PUT /api/account/password` rehashes and has `cloudprov` copy the hash to every
projected row; propagation is best-effort, and `admin/internal/hqsync` compares
and repairs every 15 minutes. It is **its own package rather than a pass inside
`inject`** — `inject` writes three licence fields and nothing else, and that
`inject`** - `inject` writes three licence fields and nothing else, and that
narrowness is what makes admin's reach into the control plane reviewable.
The control plane refuses to change an `hq`-sourced user's role or delete it
@@ -836,7 +836,7 @@ password-change endpoint at all, so there is no competing writer for the hash.
**A rename moves the host, and the licence does not care.** `PUT
/api/instances/:id/name` re-derives the slug from the new name through
`provision.RenameSlug` — the same rules that named the instance at creation —
`provision.RenameSlug` - the same rules that named the instance at creation -
and writes the control plane first, because `instances.slug`'s unique index is
what settles a race between two accounts reaching for one name. A taken slug is
a refusal, not an `acme-2`: creation appends a counter because any free slug
@@ -844,7 +844,7 @@ will do, and a rename is a request for one specific host. A licence binds the
instance UUID, so nothing is reissued and Paddle is not called. The old host
keeps resolving for up to 60s (`instancehost.go`'s cache, which admin cannot
reach into), and `km_session` is host-only, so the customer signs in again on
the new address — the portal says so rather than redirecting them into a login
the new address - the portal says so rather than redirecting them into a login
screen with no explanation. The 24h cooldown lives on `admin_instances.renamed_at`
because it is admin's policy; staff bypass it and must not write the field.
@@ -852,16 +852,16 @@ because it is admin's policy; staff bypass it and must not write the field.
## Auth and Orgs
- **Bootstrap** — first run has no users. `GET /auth/bootstrap-status` drives `/setup`, `POST /auth/bootstrap` creates the first org plus its owner.
- **Local auth** — email + password (bcrypt), `POST /auth/login`.
- **Auth providers** — configured _per instance_ in `auth_providers`, any number of them, each named and independently enabled. Issuer, client ID and an encrypted client secret per provider. `/auth/oidc/:providerId/start` → `/auth/oidc/:providerId/callback`. Presets (Entra, Google, Okta, GitHub) are a Go table in `server/internal/auth/presets.go` and expand to a real issuer on save, so nothing downstream knows a preset existed. GitHub is OAuth2 rather than OIDC and takes its own branch, requiring an address that is both primary **and** verified — an unverified address is not proof of control.
- **Local login** — `settings.local_login_enabled`, a `*bool` because absent must mean enabled; a plain bool would disable password sign-in fleet-wide at upgrade. `services.CheckLockout` refuses any change leaving neither local login nor an enabled provider, and is enforced in the service layer so the settings path and the provider path cannot disagree.
- **Sessions** — opaque 32-byte hex ID in the `km_session` cookie, session body stored in Redis with a 24h TTL.
- **Roles** — `owner`, `admin`, `member`. `/api/settings` and `/api/org/*` require owner or admin.
- **Host/org guard** — `APP_ROOT_LABEL` (default `vantage`) defines the app root label. A request to `<slug>.vantage.<tld>` resolves that org from the slug and rejects sessions belonging to a different one. Org lookups are cached for 60s.
- **Bootstrap** - first run has no users. `GET /auth/bootstrap-status` drives `/setup`, `POST /auth/bootstrap` creates the first org plus its owner.
- **Local auth** - email + password (bcrypt), `POST /auth/login`.
- **Auth providers** - configured _per instance_ in `auth_providers`, any number of them, each named and independently enabled. Issuer, client ID and an encrypted client secret per provider. `/auth/oidc/:providerId/start` → `/auth/oidc/:providerId/callback`. Presets (Entra, Google, Okta, GitHub) are a Go table in `server/internal/auth/presets.go` and expand to a real issuer on save, so nothing downstream knows a preset existed. GitHub is OAuth2 rather than OIDC and takes its own branch, requiring an address that is both primary **and** verified - an unverified address is not proof of control.
- **Local login** - `settings.local_login_enabled`, a `*bool` because absent must mean enabled; a plain bool would disable password sign-in fleet-wide at upgrade. `services.CheckLockout` refuses any change leaving neither local login nor an enabled provider, and is enforced in the service layer so the settings path and the provider path cannot disagree.
- **Sessions** - opaque 32-byte hex ID in the `km_session` cookie, session body stored in Redis with a 24h TTL.
- **Roles** - `owner`, `admin`, `member`. `/api/settings` and `/api/org/*` require owner or admin.
- **Host/org guard** - `APP_ROOT_LABEL` (default `vantage`) defines the app root label. A request to `<slug>.vantage.<tld>` resolves that org from the slug and rejects sessions belonging to a different one. Org lookups are cached for 60s.
Unique indexes are a **security property**, not an optimisation. `users` is
unique on `(instance_id, email)` — one address is one user _within_ an instance,
unique on `(instance_id, email)` - one address is one user _within_ an instance,
and the same address may hold a user in several instances, because an account's
people are projected into each instance they are granted. This is sufficient only
because **every lookup by email is scoped by instance**; there is deliberately no
@@ -898,7 +898,7 @@ treats 70s of silence as a dead stream and reconnects. Keepalive cannot do this
job behind an L7 proxy: the agent's HTTP/2 connection terminates at the proxy,
which answers pings on its own behalf, so a control-plane pod that dies leaves
the agent blocked in `Recv` on a stream that never delivers another message and
never errors — commands dispatched into it are silently lost while `SyncKeys`
never errors - commands dispatched into it are silently lost while `SyncKeys`
keeps succeeding and the fleet list still shows the server `active`. The agent's
watchdog arms only **after** it has seen a first ping, so an older server that
sends none is treated as working rather than put into a reconnect loop.
@@ -918,7 +918,7 @@ GET /update /update.ps1
GET /auth/bootstrap-status
POST /auth/bootstrap /auth/login /auth/logout
GET /auth/me
GET /auth/providers # {local_enabled, providers:[{id,name,preset}]} — no issuer, client ID or secret
GET /auth/providers # {local_enabled, providers:[{id,name,preset}]} - no issuer, client ID or secret
GET /api/secrets/:group/values # bearer token (ESO)
```
@@ -966,11 +966,11 @@ tokens GET /tokens · GET /tokens/scopes · POST /tokens · DELETE /tokens
GET /openapi.json · GET /docs
```
`GET /license` reports `deployment`, and **`POST /license` answers 409 `cloud_managed` when it is `cloud`**. A cloud instance's licence is written by `admin/internal/inject` straight into the database and never through this endpoint, so the refusal cannot break injection — it only stops a customer pasting over a licence they do not own. `web/` hides the paste form and points at the HQ portal instead, but as with `hq`-managed users, the API is the boundary and the UI is the courtesy.
`GET /license` reports `deployment`, and **`POST /license` answers 409 `cloud_managed` when it is `cloud`**. A cloud instance's licence is written by `admin/internal/inject` straight into the database and never through this endpoint, so the refusal cannot break injection - it only stops a customer pasting over a licence they do not own. `web/` hides the paste form and points at the HQ portal instead, but as with `hq`-managed users, the API is the boundary and the UI is the courtesy.
`POST /license` is also in `licenceExemptPaths`: pasting a valid licence has to work while the current one is expired, because it is the way out of degraded mode.
Free exists in both deployments, so it is no longer cloud-only by construction. The one-Free-per-account rule is enforced per account **and deployment**, in `licensing.checkFreeLimit` and in `createInstance`'s friendly pre-check — the two must stay scoped identically, because a pre-check stricter than the issuer refuses what would have worked. A self-hosted Free licence is claimed with `POST /api/instances/:id/claim-free` after the install is linked; the metered server count and per-instance feature toggles come from the instance's `entitlement`, which a licence snapshots at issue time (see the `catalogue`/`entitlements` note under MongoDB Collections).
Free exists in both deployments, so it is no longer cloud-only by construction. The one-Free-per-account rule is enforced per account **and deployment**, in `licensing.checkFreeLimit` and in `createInstance`'s friendly pre-check - the two must stay scoped identically, because a pre-check stricter than the issuer refuses what would have worked. A self-hosted Free licence is claimed with `POST /api/instances/:id/claim-free` after the install is linked; the metered server count and per-instance feature toggles come from the instance's `entitlement`, which a licence snapshots at issue time (see the `catalogue`/`entitlements` note under MongoDB Collections).
---
@@ -986,13 +986,13 @@ plane, each of which this codebase enforces:
- **Licences are injected, not pasted.** `inject` writes three licence fields
straight into `instances`. `GET /license` reports `deployment`, and **`POST
/license` answers 409 `cloud_managed` when it is `cloud`** — the refusal
/license` answers 409 `cloud_managed` when it is `cloud`** - the refusal
cannot break injection, it only stops a customer pasting over a licence they
do not own.
- **Instances and owners are provisioned through `shared/provision`**, the same
code path bootstrap uses, so there is one implementation of the slug rules
and reserved names rather than two — see "Shared provisioning".
- **Members are projected, not federated** — `users.auth_source: "hq"` with
and reserved names rather than two - see "Shared provisioning".
- **Members are projected, not federated** - `users.auth_source: "hq"` with
`hq_user_id` set, refused for role changes and deletion by
`services.ErrHQManaged`. See "Grants project, they do not federate", which is
the contract in full.
@@ -1004,7 +1004,7 @@ plane, each of which this codebase enforces:
`servers` · `keys` · `assignments` · `orgs` · `users` · `auth_providers` · `settings` · `secrets` · `workflows` · `workflow_steps` · `workflow_runs` · `workflow_log_lines` · `workflow_log_seq` · `monitors` · `incidents` · `monitor_rollups` · `notification_channels` · `console_sessions` · `audit_logs` · `server_packages` · `vuln_findings` · `vuln_alert_rules` · `vulndb_meta` · `server_workloads` · `api_tokens` · `status_pages` · `status_incidents` · `migrations`
Every document except `migrations` carries `org_id`. Struct definitions are the source of truth — see `server/internal/models/`.
Every document except `migrations` carries `org_id`. Struct definitions are the source of truth - see `server/internal/models/`.
Notes that are not obvious from the structs:
@@ -1015,15 +1015,15 @@ Notes that are not obvious from the structs:
- `workflow_runs.steps_snapshot` freezes the resolved steps so editing the library never rewrites history.
- `console_sessions.token_consumed_at` is set atomically to enforce one-time use.
- `auth_providers.provider_id` is a short random identifier, not the Mongo `_id`: it appears in the callback URL a customer pastes into their IdP, and an `_id` there would publish a database key. `callback_notice` marks a provider migrated from the old single-provider shape, whose redirect URI therefore changed.
- `workflow_log_lines` is keyed `(run_id, server_id, seq)` — the index is not an optimisation, every read is a range scan over it. `workflow_log_seq` holds one counter document per `run_id/server_id`, which is what lets two pods interleave into one ordered log. Neither carries `instance_id`: they are reached only through a run, and a run is already scoped.
- `workflow_log_lines` is keyed `(run_id, server_id, seq)` - the index is not an optimisation, every read is a range scan over it. `workflow_log_seq` holds one counter document per `run_id/server_id`, which is what lets two pods interleave into one ordered log. Neither carries `instance_id`: they are reached only through a run, and a run is already scoped.
- `users.auth_source` is `local`, `oidc` or `hq`. An `hq` user was projected from a Vantage HQ account and carries `hq_user_id`; HQ owns its role, password and existence.
- `server_packages` holds a server's whole package set in **one** document, not one per package. The hash already established that something changed, so a report is a single atomic upsert with no delta logic to get wrong; ~2000 packages is ~150KB, well inside the 16MB limit. `scan_pending` lives on the document rather than in memory so a leader handover cannot lose it.
- `vuln_findings` is unique on `(instance_id, server_id, cve_id, package_name)`. That key is what makes a rescan an idempotent upsert rather than a duplicate factory, and what lets `first_seen` survive one. An empty `fixed_in` means no vendor fix exists — a real state, never "not vulnerable".
- `vuln_findings` is unique on `(instance_id, server_id, cve_id, package_name)`. That key is what makes a rescan an idempotent upsert rather than a duplicate factory, and what lets `first_seen` survive one. An empty `fixed_in` means no vendor fix exists - a real state, never "not vulnerable".
- `vulndb_meta` is a singleton and deliberately carries **no** `instance_id`: the vulnerability database is a property of the deployment, not a tenant. Same reasoning as `migrations`, and the reason it is absent from `services.ScopedCollections`.
- **`services.ScopedCollections` is the canonical registry of tenant-scoped collections**, and `scopedCollectionsForPurge` derives instance deletion from it rather than keeping a second list. A new collection carrying `instance_id` must be added there or its rows outlive the instance.
- `api_tokens` stores only `sha256` of the token, like `servers.agent_token_hash`. A token's effective role is `min(user.role, token.role)` **recomputed per request**, so demoting somebody demotes their tokens; deleting the user deletes them. Scopes are enforced from a map keyed on the registered gin route pattern, and `AssertScopeMapComplete` **fails boot** when an `/api` route is missing from it — a route added without an entry would otherwise be silently unreachable by every token.
- `api_tokens` stores only `sha256` of the token, like `servers.agent_token_hash`. A token's effective role is `min(user.role, token.role)` **recomputed per request**, so demoting somebody demotes their tokens; deleting the user deletes them. Scopes are enforced from a map keyed on the registered gin route pattern, and `AssertScopeMapComplete` **fails boot** when an `/api` route is missing from it - a route added without an entry would otherwise be silently unreachable by every token.
Admin's database is its own and lives with `vantage-admin` — `accounts`,
Admin's database is its own and lives with `vantage-admin` - `accounts`,
`admin_instances`, `licenses`, `subscriptions`, `plans`, `catalogue`,
`entitlements`, `paddle_events`, `staff_users`, `customer_users`,
`instance_members`, `admin_audit`. Nothing here reads or writes it. Note in
@@ -1036,9 +1036,9 @@ database is the access.
`services.RunMigrations()` runs at boot, recording markers in `migrations`:
- `0001_default_org_backfill`
- `0002_settings_org_backfill` (must run before 0003 — 0003 can create a `default` org, which pushes 0002 into its ambiguous multi-org branch)
- `0002_settings_org_backfill` (must run before 0003 - 0003 can create a `default` org, which pushes 0002 into its ambiguous multi-org branch)
- `0003_missed_org_scopes`
- `0005_auth_providers` — copies each `instance_oidc` document into `auth_providers`, ciphertext verbatim rather than decrypted and re-encrypted, so it does not need `KEY_ENCRYPTION_KEY` and cannot strand an instance's SSO configuration that has none set.
- `0005_auth_providers` - copies each `instance_oidc` document into `auth_providers`, ciphertext verbatim rather than decrypted and re-encrypted, so it does not need `KEY_ENCRYPTION_KEY` and cannot strand an instance's SSO configuration that has none set.
Index builders (`EnsureAuthIndexes`, `EnsureSettingsIndexes`) are fatal on failure; `EnsureSecretIndexes` and `EnsureWorkflowIndexes` only warn.
@@ -1076,7 +1076,7 @@ tls: true
```
1. SyncKeys(server_id, agent_token, agent_version)
2. Non-Linux hosts stop here — the key-management steps below are Linux-only; a Windows agent's other work (workflow steps, inventory, OS updates, workloads) runs from the goroutines started above, not from this loop
2. Non-Linux hosts stop here - the key-management steps below are Linux-only; a Windows agent's other work (workflow steps, inventory, OS updates, workloads) runs from the goroutines started above, not from this loop
3. Diff desired keys against /root/.ssh/authorized_keys; unchanged → no write
4. Changed → write .tmp, os.Rename() over the real file, chmod 0600
```
@@ -1084,7 +1084,7 @@ tls: true
### Install
Linux: systemd unit at `/etc/systemd/system/vantage-agent.service`,
`Restart=always`, runs as root — written by the install script this server
`Restart=always`, runs as root - written by the install script this server
generates, not shipped as a file. Windows: MSI built by `vantage-agent`'s CI
(WiX), or its `installer/setup.ps1` registering the agent as a service via NSSM.
@@ -1110,10 +1110,10 @@ generates, not shipped as a file. Windows: MSI built by `vantage-agent`'s CI
| Name | Required | Notes |
| -------------------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `GRPC_HOST` | **yes** | `host:port` agents dial. Boot fails without it — there is no safe default; falling back to the web host would hand agents a port that does not speak gRPC. |
| `GRPC_HOST` | **yes** | `host:port` agents dial. Boot fails without it - there is no safe default; falling back to the web host would hand agents a port that does not speak gRPC. |
| `MONGO_URI` | no | default `mongodb://localhost:27017` |
| `MONGO_DB` | no | default `vantage` |
| `REDIS_USERNAME` | no | Redis 6+ ACL user. Leave empty for a legacy `requirepass` instance — go-redis then sends AUTH with one argument instead of two |
| `REDIS_USERNAME` | no | Redis 6+ ACL user. Leave empty for a legacy `requirepass` instance - go-redis then sends AUTH with one argument instead of two |
| `REDIS_PASSWORD` | no | empty for an unauthenticated Redis |
| `REDIS_ADDR` | no | default `localhost:6379` |
| `KEY_ENCRYPTION_KEY` | yes in practice | 64-char hex (32 bytes) for AES-256-GCM. Required for private keys, secrets, OIDC secrets, RDP credentials. |
@@ -1121,13 +1121,13 @@ generates, not shipped as a file. Windows: MSI built by `vantage-agent`'s CI
| `PROXY_ADVERTISE_HOST` | no | default `server`; the hostname guacd resolves the control plane by, handed to guacd as the relay's address. Wrong here and every console session fails at connect |
| `PROXY_LISTEN_HOST` | no | default `0.0.0.0`; the interface the ephemeral relay listener binds |
| `APP_ROOT_LABEL` | no | default `vantage`; wrong value disables the host/session org guard |
| `TRUSTED_PROXIES` | no | comma-separated CIDRs or addresses gin trusts for `X-Forwarded-For`. Empty means trust none: `c.ClientIP()` falls back to the direct peer address, which behind a real reverse proxy is that proxy's own address for every visitor — the public status page's per-address rate limit then keys on one address for the whole fleet of readers. Also the address recorded in `audit_logs` and `console_sessions`. Compose and the Helm chart default it to the RFC1918 ranges, right for their own bundled proxy and wrong the moment another one is inserted in front |
| `POD_IP` | no | this pod's own address, set by the Helm chart from the downward API. **Takes precedence over `PROXY_ADVERTISE_HOST`** — a console relay listener belongs to one replica, and a Service address names all of them |
| `TRUSTED_PROXIES` | no | comma-separated CIDRs or addresses gin trusts for `X-Forwarded-For`. Empty means trust none: `c.ClientIP()` falls back to the direct peer address, which behind a real reverse proxy is that proxy's own address for every visitor - the public status page's per-address rate limit then keys on one address for the whole fleet of readers. Also the address recorded in `audit_logs` and `console_sessions`. Compose and the Helm chart default it to the RFC1918 ranges, right for their own bundled proxy and wrong the moment another one is inserted in front |
| `POD_IP` | no | this pod's own address, set by the Helm chart from the downward API. **Takes precedence over `PROXY_ADVERTISE_HOST`** - a console relay listener belongs to one replica, and a Service address names all of them |
| `VANTAGE_MIGRATE_ONLY` | no | run schema setup (migrations, index builders, default-step seeding) and exit without serving. `GRPC_HOST` is not required in this mode. Set by the Helm chart's pre-upgrade Job |
| `VANTAGE_SKIP_MIGRATIONS` | no | serve without running schema setup, on the assumption a Job already did. Set by the chart's Deployment whenever `server.migrationJob.enabled`. Unset under Compose, where one process still migrates and then serves |
| `VANTAGE_TRIVY_DB_REF` | no | default `ghcr.io/aquasecurity/trivy-db:2`. Point at a mirror for an air-gapped install, or to avoid the anonymous ghcr rate limit |
| `VANTAGE_VULNDB_DISABLED` | no | `true` disables the vulnerability database puller and scan loop entirely. Findings already written are still served, and still shown as stale |
| `FREE_INSTANCE_REAP_AFTER` | no | duration past a Free licence's expiry before the instance and all its data are deleted. **Empty disables the reaper, and empty is the default.** Set to `336h` on vantage.hostxtra.co.uk only — a self-hosted deployment must never reap. Must match admin's value, which only names the date in warning emails |
| `FREE_INSTANCE_REAP_AFTER` | no | duration past a Free licence's expiry before the instance and all its data are deleted. **Empty disables the reaper, and empty is the default.** Set to `336h` on vantage.hostxtra.co.uk only - a self-hosted deployment must never reap. Must match admin's value, which only names the date in warning emails |
### Ingress (Helm, Traefik)
@@ -1139,23 +1139,23 @@ generates, not shipped as a file. Windows: MSI built by `vantage-agent`'s CI
| `ingress.api.paths` (when `api.enabled`) | `/api`, `/auth` → `<release>-server:8080`, bypassing the Next proxy |
| `ingress.grpc.host` | agents → a dedicated `<release>-server-grpc` Service on 9090, annotated `serversscheme: h2c` |
**`ingress.web.host` is normally a wildcard.** `*.vantage.example.com` is the per-tenant instance namespace — `APP_ROOT_LABEL` resolves the instance from the label. A Kubernetes wildcard host matches **exactly one** label, so it does not match the apex, and here that is correct rather than a gap: `vantage.hostxtra.co.uk` is the marketing site, which lives in `vantage-site` and which this chart does not deploy. `extraHosts` is for a genuine second name; adding the apex to it would put the control plane on the marketing host. Every host in the list gets identical paths.
**`ingress.web.host` is normally a wildcard.** `*.vantage.example.com` is the per-tenant instance namespace - `APP_ROOT_LABEL` resolves the instance from the label. A Kubernetes wildcard host matches **exactly one** label, so it does not match the apex, and here that is correct rather than a gap: `vantage.hostxtra.co.uk` is the marketing site, which lives in `vantage-site` and which this chart does not deploy. `extraHosts` is for a genuine second name; adding the apex to it would put the control plane on the marketing host. Every host in the list gets identical paths.
**`ingress.api.enabled` routes `/api`, `/auth`, `/public`, `/install*` and `/update*` straight to the server, and it is not optional.** It defaults to **true** and the chart refuses to render with it off, because `web` proxies nothing: with those prefixes unrouted the UI loads and every request it makes 404s against Next. The value survives only for an installation whose own terminator sits in front of this ingress and routes them there instead. Traefik derives router priority from rule length, so `PathPrefix(/api)` outranks the catch-all `/` with no priority annotation needed.
**The gRPC route needs its own Service.** The server terminates no TLS; it speaks plain h2c and always has, with TLS terminated by whatever sits in front. Traefik will not use h2c to a backend unless the *Service* says so, and that annotation applies to every port on the Service — so annotating the shared two-port `<release>-server` would force h2c on its HTTP port too.
**The gRPC route needs its own Service.** The server terminates no TLS; it speaks plain h2c and always has, with TLS terminated by whatever sits in front. Traefik will not use h2c to a backend unless the *Service* says so, and that annotation applies to every port on the Service - so annotating the shared two-port `<release>-server` would force h2c on its HTTP port too.
**`server.env.grpcHost` is not derived from `ingress.grpc.host`, and the chart refuses to render if they disagree.** Agents dial whatever `grpcHost` says, and it is baked into every install one-liner; left pointing at the in-cluster Service while agents arrive through the ingress, every install succeeds and every agent then fails to connect, with nothing in the control plane explaining why. Guessing at the port (443? 9090?) would be worse than stopping.
TLS is `ingress.tls.secretName` / `grpcSecretName` (pre-existing certificates) **or** `certResolver` (Traefik ACME). Setting neither while `tls.enabled` produces a TLS router with no certificate, so Traefik serves its self-signed default — which looks valid and is trusted by nothing. NOTES.txt warns on install rather than the chart failing, since it is a real if unusual choice behind another terminator.
TLS is `ingress.tls.secretName` / `grpcSecretName` (pre-existing certificates) **or** `certResolver` (Traefik ACME). Setting neither while `tls.enabled` produces a TLS router with no certificate, so Traefik serves its self-signed default - which looks valid and is trusted by nothing. NOTES.txt warns on install rather than the chart failing, since it is a real if unusual choice behind another terminator.
---
**The compose file ships an `nginx` service, and the UI does not work without it or an equivalent.** `web:3000` serves the UI only; a request to `/api` there is a Next 404. `deploy/docker/nginx/vantage.conf` routes `/api`, `/auth`, `/public`, `/install`, `/install.ps1`, `/update`, `/update.ps1` to `server:8080` and everything else to `web:3000`, on plain HTTP at `${NGINX_HTTP_PORT:-80}`. The self-hosted install docs reproduce that file, so a routing change there must change in `vantage-docs` too. On vantage.hostxtra.co.uk the Nginx Proxy Manager already in front does the same routing, so the host port must not collide with it.
`deploy/docker/docker-compose.yml` runs four services: `redis`, `guacd`, `server` (8080 + 9090), `web` (3000). MongoDB is external. **That is the whole of a self-hosted install**, and it is now the only compose file here. vantage.hostxtra.co.uk adds three fragments from three other repositories — `vantage-site` (`site` 3003, `sitesvc` 8082), `vantage-docs` (`docsite` 3005) and `vantage-admin` (`admin` 8083, `adminsite` 3004) — composed together as shown under "The public host".
`deploy/docker/docker-compose.yml` runs four services: `redis`, `guacd`, `server` (8080 + 9090), `web` (3000). MongoDB is external. **That is the whole of a self-hosted install**, and it is now the only compose file here. vantage.hostxtra.co.uk adds three fragments from three other repositories - `vantage-site` (`site` 3003, `sitesvc` 8082), `vantage-docs` (`docsite` 3005) and `vantage-admin` (`admin` 8083, `adminsite` 3004) - composed together as shown under "The public host".
`LICENSE_SIGNING_KEY` appears in **no compose file in this repository**, and must never be added to one: admin is the only signer, and it now lives in `vantage-admin` along with its own compose fragment. `docker-compose.yml` should never mention `admin` or `adminsite` again — the separation used to be a rule someone had to remember, and is the repository boundary now. `server` reads `REDIS_ADDR`/`REDIS_USERNAME`/`REDIS_PASSWORD` so a Kubernetes install can point at a managed Redis; the base compose still hardcodes an unauthenticated `redis:6379` for it.
`LICENSE_SIGNING_KEY` appears in **no compose file in this repository**, and must never be added to one: admin is the only signer, and it now lives in `vantage-admin` along with its own compose fragment. `docker-compose.yml` should never mention `admin` or `adminsite` again - the separation used to be a rule someone had to remember, and is the repository boundary now. `server` reads `REDIS_ADDR`/`REDIS_USERNAME`/`REDIS_PASSWORD` so a Kubernetes install can point at a managed Redis; the base compose still hardcodes an unauthenticated `redis:6379` for it.
---
@@ -1192,7 +1192,7 @@ repository they sit in.
`vantage-site`'s `web/app/globals.css` is the **origin**: it is the only one
carrying both light and dark values in full, and the other three copy its token
blocks **verbatim** — same names, same values. `web/` here holds the same tokens
blocks **verbatim** - same names, same values. `web/` here holds the same tokens
but only the **dark** values, since it does not switch. **Nothing enforces the
match, and the four now sit in four repositories, so "change them in the same
commit" is not merely unenforced but impossible.** The drift window is however
@@ -1203,7 +1203,7 @@ rather than a refactor.
everything below its token block maps Docusaurus's `--ifm-*` variables onto
them. Docusaurus already stamps `data-theme` on `<html>`, which is the selector
the dark block keys on, so the built-in toggle needed no wiring. The rule holds
all the same — no rule in that file outside the token blocks carries a hex. Its
all the same - no rule in that file outside the token blocks carries a hex. Its
one concession is a favicon, which must, for the same reason the email layout
must: a browser tab cannot read a token.
@@ -1224,15 +1224,15 @@ colour alone in either: every pill carries a distinct shape and a text label.
There is a **fifth** copy, and it is the one people forget:
`shared/mail/templates/layout.html.tmpl` in `vantage-shared` carries `web/`'s
dark values as literal hex. Email clients support neither `var()` nor a reliable
`prefers-color-scheme`, so the token indirection is simply not available there —
`prefers-color-scheme`, so the token indirection is simply not available there -
an email is read before the recipient clicks through to the control plane, and
the two should not look like different products.
`web/` stores its tokens as **RGB channel triplets** with the hex in a trailing comment, and derives `--token: rgb(var(--token-rgb))` from them. That is not a style preference: the console leans on Tailwind's opacity modifiers (`bg-danger/10`, `border-accent/50`, `ring-accent/30`) in a way the other two do not, and `<alpha-value>` only compiles against channels. Keep the hex comments — they are what lets the four token blocks still be diffed by eye, which matters more now that they cannot be diffed by `git`. `web/` also adds three tokens the marketing site has no use for: `--accent-hover` and `--down-hover` (it brightens with a CSS `filter`, which a Tailwind colour token cannot do) and `--well`, the floor beneath the ground for install one-liners, key blobs and run logs — surfaces showing machine output rather than interface.
`web/` stores its tokens as **RGB channel triplets** with the hex in a trailing comment, and derives `--token: rgb(var(--token-rgb))` from them. That is not a style preference: the console leans on Tailwind's opacity modifiers (`bg-danger/10`, `border-accent/50`, `ring-accent/30`) in a way the other two do not, and `<alpha-value>` only compiles against channels. Keep the hex comments - they are what lets the four token blocks still be diffed by eye, which matters more now that they cannot be diffed by `git`. `web/` also adds three tokens the marketing site has no use for: `--accent-hover` and `--down-hover` (it brightens with a CSS `filter`, which a Tailwind colour token cannot do) and `--well`, the floor beneath the ground for install one-liners, key blobs and run logs - surfaces showing machine output rather than interface.
`web/` is locked to dark and the HQ console defaults to **light**, and that pairing is the point: an operator with both open should never mistake one for the other before clicking Reissue. Now that both are drawn from the same palette the distinction rests **entirely** on the ground, so do not make dark the HQ console's default and do not give web/ a light theme. State never reads by colour alone in either: every pill carries a distinct shape and a text label. The same argument applies one level in: the **staff** masthead sits on `--panel-2` with a `STAFF` chip, so staff and customer screens are not identical either.
`web/` collapses Tailwind's radius scale — `md`, `lg` and `xl` all resolve to the shared 4px — rather than rewriting the ~140 `rounded-lg` classes across its pages. Every one of them meant "a panel corner", and `tailwind.config.ts` is now where that decision lives. `rounded-full` is untouched: status dots and pills still need it.
`web/` collapses Tailwind's radius scale - `md`, `lg` and `xl` all resolve to the shared 4px - rather than rewriting the ~140 `rounded-lg` classes across its pages. Every one of them meant "a panel corner", and `tailwind.config.ts` is now where that decision lives. `rounded-full` is untouched: status dots and pills still need it.
The HQ console's own shell, its `/staff/pricing` page and the catalogue coverage
ledger are documented in `vantage-admin`. They are still built from these
@@ -1250,18 +1250,18 @@ tokens, which is the only reason they are mentioned here at all.
| `/steps` | Reusable step library |
| `/monitors`, `/monitors/new`, `/monitors/[id][/edit]` | Checks, uptime, incidents |
| `/secrets`, `/secrets/[group]` | Vault |
| `/tokens` | Personal API keys — reachable at **every** role, unlike `/settings` |
| `/tokens` | Personal API keys - reachable at **every** role, unlike `/settings` |
| `/audit` | Audit log |
| `/settings`, `/settings/notifications`, `/settings/license` | Members, OIDC, alerts, retention, ESO token · channels · licence |
**The sidebar is grouped, and the groups are the nav's structure rather than
decoration.** `web/components/Sidebar.tsx` holds `navGroups` — Fleet, Access,
Automation, Instance — each rendered with a mono small-caps heading and a
decoration.** `web/components/Sidebar.tsx` holds `navGroups` - Fleet, Access,
Automation, Instance - each rendered with a mono small-caps heading and a
hairline rule above it, the first group excepted. Grouping is by what the
operator is doing, not by which service answers: SSH keys, vault secrets and
API keys sit together under Access because all three are credentials. A group
whose every item is `adminOnly` disappears **whole**, heading and rule
included, for a member — a labelled section with nothing under it reads as
included, for a member - a labelled section with nothing under it reads as
something that failed to load rather than something withheld.
**`/settings` is one page, not a section.** Members and single sign-on used to
@@ -1272,28 +1272,28 @@ behaves" made two half-pages and a nav entry called Instance that no one could
distinguish from Settings. `next.config.ts` keeps a permanent redirect from the
old path. The cards live in `web/components/settings/` rather than in the page,
which is also where the `Field`/`inputClass` pair the three of them share now
lives — one copy instead of the three that existed while they were apart.
lives - one copy instead of the three that existed while they were apart.
---
## CI/CD — Gitea Actions
## CI/CD - Gitea Actions
### `server-deploy.yml` — triggered on every push to `main`
### `server-deploy.yml` - triggered on every push to `main`
Builds and pushes **two** images to the Gitea container registry: `server` and `web`. That is now the whole of this workflow. Everything else that was once built here belongs to the repository that owns it — `vantage-site`, `vantage-docs`, `vantage-admin` and `vantage-ctl` each publish their own, and `vantage-agent` cuts releases rather than images.
Builds and pushes **two** images to the Gitea container registry: `server` and `web`. That is now the whole of this workflow. Everything else that was once built here belongs to the repository that owns it - `vantage-site`, `vantage-docs`, `vantage-admin` and `vantage-ctl` each publish their own, and `vantage-agent` cuts releases rather than images.
Note that despite the name, **this workflow does not deploy** — it only builds and pushes. There is no SSH step. Rolling images out is a separate manual step on the host:
Note that despite the name, **this workflow does not deploy** - it only builds and pushes. There is no SSH step. Rolling images out is a separate manual step on the host:
```bash
# self-hosted
cd /opt/vantage && docker compose -f deploy/docker/docker-compose.yml pull && \
docker compose -f deploy/docker/docker-compose.yml up -d --remove-orphans
# vantage.hostxtra.co.uk — all four repositories' fragments, see "The public host"
# vantage.hostxtra.co.uk - all four repositories' fragments, see "The public host"
```
**Each image only rebuilds when its own inputs changed.** A `git diff` against `github.event.before` decides, which is why the checkout uses `fetch-depth: 0` — the default shallow clone has one commit and nothing to diff — and why `git` is installed in the `docker:dind` container. The mapping follows the build contexts exactly:
**Each image only rebuilds when its own inputs changed.** A `git diff` against `github.event.before` decides, which is why the checkout uses `fetch-depth: 0` - the default shallow clone has one commit and nothing to diff - and why `git` is installed in the `docker:dind` container. The mapping follows the build contexts exactly:
| Image | Rebuilds when |
| ---------------------------- | -------------------------------- |
@@ -1311,7 +1311,7 @@ Every Go build in these workflows writes a netrc from `REGISTRY_USER` +
`RELEASE_TOKEN` before it runs, and sets `GOPRIVATE=gitea.hostxtra.co.uk/*`.
There is exactly **one** such place left here: `server-deploy.yml`'s single job.
The other repositories each carry their own, one per job, because jobs do not
share a filesystem — `vantage-agent`'s `msi` job is the one to remember, because
share a filesystem - `vantage-agent`'s `msi` job is the one to remember, because
it is Windows, where Go reads `%USERPROFILE%\_netrc` and not `.netrc`. The docker builds pass
it on as `--secret id=netrc`, never a build arg. **`RELEASE_TOKEN` needs read
access to the `vantage` org** on top of its existing scopes; without it every Go
@@ -1321,15 +1321,15 @@ a build arg is baked into the image. So does anything that leaves no
trustworthy base commit: a manual `workflow_dispatch`, a new branch, or a
force-push whose old head is gone.
The gap this leaves: **changing a repo variable pushes no commit, so nothing rebuilds.** After editing `HQ_URL`, run the workflow manually — that is what `workflow_dispatch` is there for. Base images also stop being refreshed on a service nobody touches; a periodic manual run covers that.
The gap this leaves: **changing a repo variable pushes no commit, so nothing rebuilds.** After editing `HQ_URL`, run the workflow manually - that is what `workflow_dispatch` is there for. Base images also stop being refreshed on a service nobody touches; a periodic manual run covers that.
### `chart-release.yml` — validates on every chart change, publishes on `chart/v*` tags
### `chart-release.yml` - validates on every chart change, publishes on `chart/v*` tags
Two jobs' worth of work in one, split by trigger. Any push or PR touching `deploy/chart/` lints the chart and renders it four ways: defaults, a multi-replica install, external Redis and MongoDB, and a set of values that **must be refused**. That last one is the point — every safety rail in this chart is a template `fail`, and `helm lint` happily accepts a chart whose templates never execute, so only rendering proves they still fire.
Two jobs' worth of work in one, split by trigger. Any push or PR touching `deploy/chart/` lints the chart and renders it four ways: defaults, a multi-replica install, external Redis and MongoDB, and a set of values that **must be refused**. That last one is the point - every safety rail in this chart is a template `fail`, and `helm lint` happily accepts a chart whose templates never execute, so only rendering proves they still fire.
Publishing runs only on a `chart/v*` tag, to the Gitea Helm registry at `/api/packages/<owner>/helm/api/charts`. **`Chart.yaml` is the source of truth for the version**; the tag only selects which one to publish, and a tag that disagrees with `Chart.yaml` fails rather than stamping over it — the alternative leaves the repository disagreeing with what shipped. A version already in the registry is rejected by Gitea, which is intended: published chart versions are immutable.
Publishing runs only on a `chart/v*` tag, to the Gitea Helm registry at `/api/packages/<owner>/helm/api/charts`. **`Chart.yaml` is the source of truth for the version**; the tag only selects which one to publish, and a tag that disagrees with `Chart.yaml` fails rather than stamping over it - the alternative leaves the repository disagreeing with what shipped. A version already in the registry is rejected by Gitea, which is intended: published chart versions are immutable.
The registry host comes from `github.server_url`, so it cannot drift from the instance the workflow is running on. It authenticates with `REGISTRY_USER` + **`RELEASE_TOKEN`** — the pair `server-deploy.yml` actually uses for `docker login`. `REGISTRY_PASSWORD` is listed in the secrets table below but set by no workflow; passing an unset secret yields an empty password and Gitea answers `401 Failed to authenticate user`, which reads like a scope problem on a token that was never sent. The publish step therefore checks both are non-empty before it calls curl. `RELEASE_TOKEN` needs `write:package` in addition to `write:release`.
The registry host comes from `github.server_url`, so it cannot drift from the instance the workflow is running on. It authenticates with `REGISTRY_USER` + **`RELEASE_TOKEN`** - the pair `server-deploy.yml` actually uses for `docker login`. `REGISTRY_PASSWORD` is listed in the secrets table below but set by no workflow; passing an unset secret yields an empty password and Gitea answers `401 Failed to authenticate user`, which reads like a scope problem on a token that was never sent. The publish step therefore checks both are non-empty before it calls curl. `RELEASE_TOKEN` needs `write:package` in addition to `write:release`.
```bash
helm repo add vantage https://gitea.hostxtra.co.uk/api/packages/mrhid6/helm
@@ -1352,17 +1352,17 @@ reads it programmatically.
| Name | Type | Value |
| ----------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `RELEASE_TOKEN` | Secret | Gitea API token. Needs `write:release` (agent releases), `write:package` (container images and the Helm chart) and **read access to the `vantage` org**, which is where the private `vantage-shared` module lives — without that last one every Go build fails at `go mod download` with what looks like a missing tag. **This is the only token any workflow authenticates with** — `docker login`, the chart publish and the module netrc all pair it with `REGISTRY_USER` |
| `RELEASE_TOKEN` | Secret | Gitea API token. Needs `write:release` (agent releases), `write:package` (container images and the Helm chart) and **read access to the `vantage` org**, which is where the private `vantage-shared` module lives - without that last one every Go build fails at `go mod download` with what looks like a missing tag. **This is the only token any workflow authenticates with** - `docker login`, the chart publish and the module netrc all pair it with `REGISTRY_USER` |
| `REGISTRY_USER` | Secret | Gitea username. Must own `RELEASE_TOKEN`, or basic auth is rejected |
| ~~`REGISTRY_PASSWORD`~~ | — | **Not used.** Named here historically; no workflow reads it. Referencing an unset secret yields an empty password and a `401 Failed to authenticate user` that looks like a token scope problem. Use `RELEASE_TOKEN` |
| ~~`REGISTRY_PASSWORD`~~ | - | **Not used.** Named here historically; no workflow reads it. Referencing an unset secret yields an empty password and a `401 Failed to authenticate user` that looks like a token scope problem. Use `RELEASE_TOKEN` |
| `DOCKER_HOST` | Variable | registry host used for image tags |
| ~~`API_URL`~~ | — | **Gone.** `web` proxies nothing and holds no address for the control plane. `/api`, `/auth`, `/public`, `/install*` and `/update*` must be routed to `server:8080` by the reverse proxy in front of both; everything else goes to `web:3000`. One variable that could name the wrong host was one request path too many — pointed at the marketing site, `/public/status/…` answered a Next 404 indistinguishable from a status page that does not exist. |
| ~~`API_URL`~~ | - | **Gone.** `web` proxies nothing and holds no address for the control plane. `/api`, `/auth`, `/public`, `/install*` and `/update*` must be routed to `server:8080` by the reverse proxy in front of both; everything else goes to `web:3000`. One variable that could name the wrong host was one request path too many - pointed at the marketing site, `/public/status/…` answered a Next 404 indistinguishable from a status page that does not exist. |
| `HQ_URL` | Variable | optional; browser URL of the HQ portal, baked into `web` so an `hq`-sourced member links to where they are managed. Empty on self-hosted, which renders a plain label instead. |
`SITE_URL`, `SITE_API_URL`, `SITE_CONTACT_EMAIL`, `ADMIN_API_URL`, `ADMIN_ENV`,
`DOCS_URL`, `DOCS_BASE_URL`, `APP_URL` and every `PADDLE_*` name are set on the
repository that bakes them in — `vantage-site`, `vantage-docs` or
`vantage-admin` — and none of them is read by anything here. Two are set in
repository that bakes them in - `vantage-site`, `vantage-docs` or
`vantage-admin` - and none of them is read by anything here. Two are set in
**two** repositories and must agree: `ADMIN_API_URL` (`vantage-site` bakes it
into the marketing site's signup form, `vantage-admin` into its own console) and
`SITE_URL`.
@@ -1372,28 +1372,28 @@ into the marketing site's signup form, `vantage-admin` into its own console) and
## Design Decisions
- **gRPC for agent traffic** — strong typing and cheap versioning; polling for state, one bidirectional stream for commands.
- **Outbound-only agents** — no inbound ports on managed servers, works behind NAT.
- **Poll for keys, push for commands** — a 30s key poll is fine, but running a workflow step should not wait up to 30s.
- **Atomic `authorized_keys` rewrite** — temp file plus `os.Rename()`; a machine that dies mid-write keeps the old file.
- **Fingerprint diffing before write** — no disk churn on unchanged state.
- **Soft revocation** — `revoked_at` rather than deletes; preserves audit history.
- **Run snapshots** — workflow runs freeze their resolved steps so editing a step never rewrites past runs.
- **Monitors run in two places** — server-side for external endpoints, agent-side for anything only reachable from inside the target network.
- **Redis for sessions only** — all durable state stays in MongoDB; losing Redis logs everyone out and nothing else.
- **guacd for console** — protocol handling is Guacamole's problem, not ours; we proxy the WebSocket and manage credentials.
- **`org_id` on every document** — isolation enforced at the query layer, not by separate databases.
- **root only** — manages `/root/.ssh/authorized_keys`; no per-user key management.
- **Windows agents cover the fleet-management path** — register, heartbeat, run
- **gRPC for agent traffic** - strong typing and cheap versioning; polling for state, one bidirectional stream for commands.
- **Outbound-only agents** - no inbound ports on managed servers, works behind NAT.
- **Poll for keys, push for commands** - a 30s key poll is fine, but running a workflow step should not wait up to 30s.
- **Atomic `authorized_keys` rewrite** - temp file plus `os.Rename()`; a machine that dies mid-write keeps the old file.
- **Fingerprint diffing before write** - no disk churn on unchanged state.
- **Soft revocation** - `revoked_at` rather than deletes; preserves audit history.
- **Run snapshots** - workflow runs freeze their resolved steps so editing a step never rewrites past runs.
- **Monitors run in two places** - server-side for external endpoints, agent-side for anything only reachable from inside the target network.
- **Redis for sessions only** - all durable state stays in MongoDB; losing Redis logs everyone out and nothing else.
- **guacd for console** - protocol handling is Guacamole's problem, not ours; we proxy the WebSocket and manage credentials.
- **`org_id` on every document** - isolation enforced at the query layer, not by separate databases.
- **root only** - manages `/root/.ssh/authorized_keys`; no per-user key management.
- **Windows agents cover the fleet-management path** - register, heartbeat, run
steps, report inventory, OS updates through the Windows Update COM API, and
workloads (services plus containers, with control and logs). They still do no
`authorized_keys` management, and no package inventory or CVE matching: a
Windows agent never calls `ReportPackages`, so no `server_packages` document
exists for it and it reports no package inventory at all — a different,
exists for it and it reports no package inventory at all - a different,
earlier state than the `unsupported` a Linux distribution reaches when its
family has no security feed.
- **Both `server` and `web` scale horizontally** — see "Running more than one server replica" below. `web` holds nothing; `server` holds per-agent state that is routed between replicas over Redis rather than duplicated.
- **Deletion lives in the control plane** — admin sends the warnings because it knows the billing address; the control plane performs the delete because it is the only service that knows which collections carry `instance_id`. Mirroring that list into admin would drift, and a drift there deletes the wrong rows.
- **Both `server` and `web` scale horizontally** - see "Running more than one server replica" below. `web` holds nothing; `server` holds per-agent state that is routed between replicas over Redis rather than duplicated.
- **Deletion lives in the control plane** - admin sends the warnings because it knows the billing address; the control plane performs the delete because it is the only service that knows which collections carry `instance_id`. Mirroring that list into admin would drift, and a drift there deletes the wrong rows.
## graphify
@@ -1404,3 +1404,7 @@ Rules:
- If graphify-out/wiki/index.md exists, use it for broad navigation instead of raw source browsing.
- Read graphify-out/GRAPH_REPORT.md only for broad architecture review or when query/path/explain do not surface enough context.
- After modifying code, run `graphify update .` to keep the graph current (AST-only, no API cost).
## Writing style
Never use em dashes (the long dash character) anywhere: code, comments, UI copy, docs, commit messages. Use a plain hyphen ` - `, a comma, a colon, or split the sentence instead.