mrhid6
653bd5a755
Server Deploy / deploy (push) Successful in 1m12s
/settings/instance held two cards — Members and Single sign-on — behind a sidebar entry called "Instance", next to one called "Settings". Nothing in either name told you which held what, and the split left both pages thin. They are now the Access group at the top of /settings, above Monitoring and Integrations. Six cards on one page needs sorting into groups or it reads as a pile, so each group carries site/'s keyed-label eyebrow on a hairline. The cards move to web/components/settings/ rather than into the page, which would have made it ~600 lines. That is also where the Field and inputClass pair now lives: the two pages each had a byte-identical copy, and folding them together is exactly when three copies would have started to drift. Members and SSO adopt SectionCard, which the rest of the page already used. A card that kept its own header treatment would read as a different kind of thing rather than another setting, which is the problem being fixed. next.config.ts keeps a permanent redirect from the old path, so bookmarks and any support reply linking it still land somewhere useful.
622 lines
48 KiB
Markdown
622 lines
48 KiB
Markdown
# Vantage
|
|
|
|
A self-hosted, multi-tenant infrastructure control plane. It started as SSH key management and has grown into fleet management: SSH key assignment, workflow/script execution, service monitoring, a secrets vault, a browser console (SSH/RDP/VNC), and OS update management.
|
|
|
|
A central server (Go + Next.js + MongoDB + Redis) drives a lightweight Go agent installed on each managed server. Agents poll over gRPC and also hold a bidirectional command stream for push-style commands.
|
|
|
|
---
|
|
|
|
## Architecture Overview
|
|
|
|
```
|
|
┌──────────────────────────────────────────────┐
|
|
│ Next.js 16 Frontend (web, :3000) │
|
|
│ servers · keys · workflows · monitors │
|
|
│ secrets · audit · console · settings │
|
|
└───────────────┬──────────────────────────────┘
|
|
│ REST + cookie session
|
|
┌───────────────▼──────────────────────────────┐
|
|
│ Go Backend (server) │
|
|
│ :8080 REST (gin) :9090 gRPC (agents) │
|
|
│ MongoDB (state) · Redis (sessions) │
|
|
│ monitor scheduler · workflow runner │
|
|
│ guacd tunnel proxy for browser console │
|
|
└───────────────┬──────────────────────────────┘
|
|
│ gRPC (TLS) — outbound from agent only
|
|
┌───────────────▼──────────────────────────────┐
|
|
│ Go Agent (per server, Linux + Windows) │
|
|
│ polls SyncKeys · CommandStream │
|
|
│ rewrites authorized_keys (Linux only) │
|
|
│ runs workflow steps · monitors · inventory │
|
|
└──────────────────────────────────────────────┘
|
|
```
|
|
|
|
Multi-tenancy: every domain document carries `org_id`, and every service query is scoped by it. Org is resolved from the session, and optionally cross-checked against the request host (`<slug>.vantage.<tld>`).
|
|
|
|
---
|
|
|
|
## Repository Structure
|
|
|
|
```
|
|
vantage/
|
|
├── agent/
|
|
│ ├── cmd/main.go # flags: -generate-key
|
|
│ └── internal/
|
|
│ ├── checker/ # monitor check execution
|
|
│ ├── config/ # config.yaml load/save
|
|
│ ├── exec/ # workflow step execution
|
|
│ ├── grpc/ # client + generated pb
|
|
│ ├── inventory/ # CPU/mem/disk collection (linux/other)
|
|
│ ├── keys/ # authorized_keys read/diff/write
|
|
│ ├── monitors/ # agent-run monitor loop
|
|
│ ├── sync/ # poll loop + command stream
|
|
│ └── updates/ # OS package update check/apply
|
|
├── server/
|
|
│ ├── cmd/main.go
|
|
│ └── internal/
|
|
│ ├── api/ # REST handlers
|
|
│ ├── auth/ # local, OIDC, session, middleware, orghost
|
|
│ ├── checker/ # server-run monitor checks
|
|
│ ├── db/ # mongo connect + Col()
|
|
│ ├── grpc/ # gRPC server + generated pb
|
|
│ ├── models/ # MongoDB documents
|
|
│ ├── monitorsched/ # server-side monitor scheduler
|
|
│ ├── notify/ # smtp, http, templating, dispatch
|
|
│ └── services/ # business logic + migrations
|
|
├── web/ # the application UI (authenticated)
|
|
│ ├── app/(app)/ # authed routes
|
|
│ ├── app/login, app/setup # unauthed routes
|
|
│ ├── components/ # ui/, workflows/, monitors/, Sidebar
|
|
│ └── lib/ # api client, guac console, query client
|
|
├── site/ # public marketing site
|
|
│ ├── app/ # one directory per route
|
|
│ ├── components/ # Nav, Footer, Logo, InstrumentPanel, forms
|
|
│ ├── assets/ # image sources, not served
|
|
│ └── Dockerfile # same shape as web/: standalone, node, 3000
|
|
├── sitesvc/ # public form: contact mail only
|
|
│ ├── cmd/main.go
|
|
│ └── internal/
|
|
│ ├── api/ # contact
|
|
│ ├── mail/ # SMTP
|
|
│ └── store/ # Mongo connect helper
|
|
├── admin/ # licensing authority: the only signer
|
|
│ ├── cmd/main.go # boot: two Mongo connections, reconciler, HTTP
|
|
│ ├── cmd/adminctl/ # staff-add; deliberately has no HTTP surface
|
|
│ └── internal/
|
|
│ ├── api/ # customer + staff handlers, route table
|
|
│ ├── auth/ # staff, HQ customer and cloud-owner sessions
|
|
│ ├── inject/ # licence write path into the control plane
|
|
│ ├── cloudprov/ # instance write path: creates instances + owners
|
|
│ ├── licensing/ # Issue, LinkInstance, Relink
|
|
│ ├── mail/ # verification and licence delivery
|
|
│ └── models/ # accounts, instances, licences, plans
|
|
├── adminsite/ # staff + customer console (vantage-hq)
|
|
│ ├── app/(customer)/ # overview, instance, link, billing
|
|
│ ├── app/(staff)/staff/ # operations, accounts, licences, plans, audit
|
|
│ ├── components/ # AppBar, PageHeader, PageFrame, InstanceRecord
|
|
│ └── lib/ # api client, session guards, formatters
|
|
├── shared/ # imported by server, sitesvc and admin
|
|
│ ├── license/ # payload, sign, verify, trusted keys, plans
|
|
│ ├── models/ # Instance, User, Settings
|
|
│ └── cmd/lkctl/ # issue and inspect licences by hand
|
|
├── proto/vantage/v1/vantage.proto
|
|
├── installer/ # Windows: setup.ps1, nssm.exe, WiX .wxs
|
|
├── deploy/ # docker-compose.yml, agent.service
|
|
└── .gitea/workflows/ # agent-release.yml, server-deploy.yml
|
|
```
|
|
|
|
---
|
|
|
|
## Subsystems
|
|
|
|
### SSH keys
|
|
|
|
Upload a public key, assign it per server, revoke softly. The agent diffs desired vs on-disk state and rewrites `/root/.ssh/authorized_keys` atomically. Keys can also be generated _on_ a server by the agent; the private half can optionally be uploaded and is stored AES-256-GCM encrypted.
|
|
|
|
### Workflows
|
|
|
|
A library of reusable **steps** (bash or PowerShell scripts with declared inputs, outputs, and secret refs) composed into **workflows** targeting a set of servers. Running one snapshots the resolved steps into a `WorkflowRun`, then dispatches `RunStepCmd` over the agent command stream. Step stdout/stderr streams back as `StepOutputChunk` and is written to a log file on disk; the UI streams it live. Steps support `on_failure: stop|continue|retry`, per-run env passed between steps via `output_env`, and a per-run workspace directory the agent cleans up at the end.
|
|
|
|
Default steps are seeded per org at boot (`SeedDefaultSteps`). Logs are swept by retention (`workflow_log_retention_days`; nil = 30 days, 0 = forever).
|
|
|
|
### Monitors
|
|
|
|
HTTP, TCP, ICMP and TLS checks. Each monitor has a `runner`: `"server"` (executed by the server-side scheduler) or a `server_id` (pushed to that agent, which runs it locally and reports results). Consecutive failures beyond `retries` flip state to `down`, open an `Incident`, and notify. Hourly `Rollup` documents back the uptime graphs.
|
|
|
|
### Notification channels
|
|
|
|
Per-org outbound destinations: `webhook`, `smtp`, `discord`, `slack`, `telegram`. Monitors reference channels by ID. Channels are testable from the UI.
|
|
|
|
### Secrets vault
|
|
|
|
Key/value pairs grouped by name, encrypted at rest with AES-256-GCM. Consumed two ways: referenced by workflow steps via `secret_refs` (injected as env at execution), and read by Kubernetes External Secrets Operator via `GET /api/secrets/:group/values` using a bearer token whose SHA-256 hash is stored in settings.
|
|
|
|
### Browser console
|
|
|
|
`POST /api/console/connect` mints a one-time session token; `GET /api/console/tunnel` upgrades to a WebSocket and proxies to **guacd** (Apache Guacamole daemon) using `github.com/wwt/guac`. SSH connections authenticate with a stored private key; RDP/VNC credentials are encrypted, single-use, and consumed when the tunnel opens.
|
|
|
|
### Inventory and OS updates
|
|
|
|
Agents report CPU/memory/swap/partitions/kernel — metrics every 30s, full static snapshot every 15 min. They also check for pending OS package updates hourly and can apply them on command (`ApplyUpdatesCmd`).
|
|
|
|
### Agent self-update
|
|
|
|
`UpdateAgentCmd` carries a target version and Gitea base URL; the agent downloads and replaces itself.
|
|
|
|
### Marketing site and sitesvc
|
|
|
|
`site/` is a separate Next.js app built exactly like `web/` — `output: "standalone"`, run by Node in a `node:26-alpine` image, listening on `3000` and published as `3003`. The contact form posts to `sitesvc`; account signup posts to `admin` (`NEXT_PUBLIC_ADMIN_API_URL`), which creates an HQ account, not an org — the control plane is not touched until the customer later creates a cloud instance from the portal.
|
|
|
|
`adminsite/` is built the same way and published as `3004`, served at **`vantage-hq.hostxtra.co.uk`** — deliberately *outside* `*.vantage.hostxtra.co.uk`, because that namespace is per-tenant instance subdomains and `APP_ROOT_LABEL` resolves an org from the label before `vantage`. It shares `site/`'s design tokens verbatim (see Frontend below) and, unlike `web/`, does **not** proxy through a Next rewrite: the browser calls `admin` directly, so `ADMIN_API_URL` must be browser-reachable. Authenticated requests work cross-origin only because both hosts share the registrable domain `hostxtra.co.uk`, which keeps `admin_session`'s `SameSite=Lax` cookie in play.
|
|
|
|
**`ADMIN_ORIGIN` must list every browser origin that calls admin — currently two**: `https://vantage-hq.hostxtra.co.uk` for the console, and `https://vantage.hostxtra.co.uk` because the marketing site's `/start` form posts account signups to admin directly. It is comma-separated. A missing origin does not produce a 403: `cors()` simply omits the `Access-Control-Allow-Origin` header and still answers the preflight `204`, so the browser blocks the request and **admin logs nothing at all**. Symptom is a CORS preflight failure on an endpoint that works fine under curl.
|
|
|
|
`sitesvc/` (port `8082`) now owns only the contact flow:
|
|
|
|
| Form | Endpoint | Effect |
|
|
| ------- | --------------------- | ----------------------------------------------------------------------- |
|
|
| Contact | `POST /api/contact` | Emails `support@hostxtra.co.uk`, `Reply-To` the sender. Nothing stored. |
|
|
|
|
Account signup lives in `admin` instead (`POST /auth/signup`, `GET /auth/verify?token=…`) — see Signup and verification below.
|
|
|
|
`site`, `sitesvc` and `admin` are deliberately **excluded from the self-hosted deployment**: `deploy/docker-compose.yml` mentions none of them, and they live in `deploy/docker-compose.site.yml` instead.
|
|
|
|
```bash
|
|
# self-hosted install — no marketing site, no sitesvc
|
|
docker compose up -d
|
|
|
|
# vantage.hostxtra.co.uk — control plane plus the public site
|
|
docker compose -f docker-compose.yml -f docker-compose.site.yml up -d
|
|
```
|
|
|
|
### Signup and verification
|
|
|
|
Signup is **account-first**: it creates an HQ account and an unverified `customer_user` in admin's own database, nothing in the control plane. Only after a customer later creates a cloud instance from the portal (`POST /api/instances`, see Admin REST API) does an org, or rather an `instance`, come to exist — provisioned by `cloudprov`, with the owner's password hash copied from the HQ user rather than shared. `site_pending_signups` is gone; sitesvc no longer has a signup flow at all.
|
|
|
|
- The token is 32 random bytes; only its **SHA-256 hash** is stored, so a leaked database yields no working links.
|
|
- Links expire after 24 hours (`VerifyWindow`).
|
|
- An unverified sign-in gets a distinct "check your email" error rather than the generic auth failure, because the address is already known to be theirs.
|
|
- If sending the verification email fails, the freshly inserted `customer_user` (and account, on first signup) is rolled back rather than left stranded holding the unique index on email.
|
|
- Rate limited per client IP, plus a honeypot field.
|
|
|
|
An account is a team, not a person. `customer_users.account_role` is `owner`,
|
|
`admin` or `member` — the same three words as the control plane's roles, on
|
|
purpose. Owners and admins invite people, create instances and grant instance
|
|
access; billing is owner-only.
|
|
|
|
An invitation creates a `customer_user` with an **empty password hash**, which
|
|
cannot authenticate, and the invitee sets their own at `/accept-invite`. An
|
|
inviter-chosen password would be a shared credential to every instance that
|
|
person is later granted. `GET /auth/verify` therefore peeks before it consumes:
|
|
a token belonging to a passwordless row answers `{"needs_password":true}` and is
|
|
left unspent.
|
|
|
|
### Shared provisioning
|
|
|
|
`shared/provision` (`instance.go`, `slug.go`, `user.go`) holds the slug rules, reserved names and instance/user creation logic that both `server` and `admin/internal/cloudprov` need, so there is no longer a second copy to drift: `cloudprov.CreateInstance` calls straight into it to create a control-plane instance and its owner from a customer request.
|
|
|
|
### Grants project, they do not federate
|
|
|
|
Granting someone access to a cloud instance writes a real control-plane `users`
|
|
row through `cloudprov`, with `auth_source: "hq"` and `hq_user_id` set. The
|
|
instance authenticates it exactly as it authenticates anyone else, with **no
|
|
runtime dependency on admin**. Revoking deletes that row — the control plane has
|
|
no disabled state, and a row that exists is a row that can sign in.
|
|
|
|
`instance_members` in admin's database is only admin's *index* of those
|
|
projections; the control-plane row is the access. That is why a failed
|
|
`instance_members` insert unwinds the projection, and why the boot backfill can
|
|
rebuild the index from the control plane but never the other way round.
|
|
|
|
**Self-hosted instances are never projected into.** All three mutating member
|
|
endpoints refuse when `deployment != cloud`.
|
|
|
|
The HQ password is the single source of truth for every `hq`-sourced row.
|
|
`PUT /api/account/password` rehashes and has `cloudprov` copy the hash to every
|
|
projected row; propagation is best-effort, and `admin/internal/hqsync` compares
|
|
and repairs every 15 minutes. It is **its own package rather than a pass inside
|
|
`inject`** — `inject` writes three licence fields and nothing else, and that
|
|
narrowness is what makes admin's reach into the control plane reviewable.
|
|
|
|
The control plane refuses to change an `hq`-sourced user's role or delete it
|
|
(`services.ErrHQManaged`, 409). `web/` shows those rows read-only with a link to
|
|
the portal, but the API is the boundary; the UI is a courtesy. There is no local
|
|
password-change endpoint at all, so there is no competing writer for the hash.
|
|
|
|
---
|
|
|
|
## Auth and Orgs
|
|
|
|
- **Bootstrap** — first run has no users. `GET /auth/bootstrap-status` drives `/setup`, `POST /auth/bootstrap` creates the first org plus its owner.
|
|
- **Local auth** — email + password (bcrypt), `POST /auth/login`.
|
|
- **OIDC** — configured _per org_ (`org_oidc`), issuer + client ID + encrypted client secret. `/auth/oidc/start` → `/auth/oidc/callback`.
|
|
- **Sessions** — opaque 32-byte hex ID in the `km_session` cookie, session body stored in Redis with a 24h TTL.
|
|
- **Roles** — `owner`, `admin`, `member`. `/api/settings` and `/api/org/*` require owner or admin.
|
|
- **Host/org guard** — `APP_ROOT_LABEL` (default `vantage`) defines the app root label. A request to `<slug>.vantage.<tld>` resolves that org from the slug and rejects sessions belonging to a different one. Org lookups are cached for 60s.
|
|
|
|
Unique indexes are a **security property**, not an optimisation. `users` is
|
|
unique on `(instance_id, email)` — one address is one user *within* an instance,
|
|
and the same address may hold a user in several instances, because an account's
|
|
people are projected into each instance they are granted. This is sufficient only
|
|
because **every lookup by email is scoped by instance**; there is deliberately no
|
|
unscoped lookup anywhere, and adding one would let the login path return an
|
|
arbitrary one of several matching users. Instance slug, settings instance and ESO
|
|
token hash remain globally unique.
|
|
|
|
---
|
|
|
|
## gRPC API
|
|
|
|
```protobuf
|
|
service Vantage {
|
|
rpc Register(RegisterRequest) returns (RegisterResponse);
|
|
rpc SyncKeys(SyncRequest) returns (SyncResponse);
|
|
rpc UploadGeneratedKey(UploadKeyRequest) returns (UploadKeyResponse);
|
|
rpc ReportUpdates(ReportUpdatesRequest) returns (ReportUpdatesResponse);
|
|
rpc ReportInventory(InventoryReport) returns (InventoryReportResponse);
|
|
rpc SyncMonitors(SyncMonitorsRequest) returns (SyncMonitorsResponse);
|
|
rpc ReportChecks(ReportChecksRequest) returns (ReportChecksResponse);
|
|
rpc CommandStream(stream AgentMessage) returns (stream ServerCommand);
|
|
}
|
|
```
|
|
|
|
`CommandStream` is the only streaming RPC: the agent authenticates once with `AgentReady`, then the server pushes `ServerCommand`s and the agent replies with `CommandResult`, `StepResult`, or `StepOutputChunk`.
|
|
|
|
`ServerCommand` variants: `GenerateKeyCmd`, `DeleteKeyCmd`, `UpdateAgentCmd`, `ApplyUpdatesCmd`, `RunStepCmd`, `CleanupWorkspaceCmd`.
|
|
|
|
Key-state polling stays on the 30s `SyncKeys` interval. Full message definitions live in `proto/vantage/v1/vantage.proto`.
|
|
|
|
---
|
|
|
|
## REST API
|
|
|
|
Unauthenticated:
|
|
|
|
```
|
|
GET /install /install.ps1 # dynamic agent install scripts
|
|
GET /update /update.ps1
|
|
GET /auth/bootstrap-status
|
|
POST /auth/bootstrap /auth/login /auth/logout
|
|
GET /auth/me /auth/oidc/start /auth/oidc/callback
|
|
GET /api/secrets/:group/values # bearer token (ESO)
|
|
```
|
|
|
|
Session-authed under `/api`:
|
|
|
|
```
|
|
servers GET,POST /servers · GET,POST /servers/new · GET,DELETE /servers/:id
|
|
POST /servers/:id/{generate-key,update-agent,apply-updates}
|
|
keys GET,POST /keys · GET,DELETE /keys/:id · GET /keys/:id/private-key
|
|
POST /keys/:id/assign · DELETE /keys/:id/assign/:serverId
|
|
workflows GET,POST /steps · PUT,DELETE /steps/:id · GET /steps/:id/export
|
|
POST /steps/{import,seed-defaults,parse} · GET /steps/usage
|
|
GET,POST /workflows · GET,PUT,DELETE /workflows/:id
|
|
POST /workflows/:id/run · GET /workflows/:id/runs
|
|
GET /runs/:runId · POST /runs/:runId/cancel
|
|
GET /runs/:runId/servers/:serverId/logs[/stream]
|
|
monitors GET,POST /monitors · GET,PUT,DELETE /monitors/:id
|
|
GET /monitors/:id/{incidents,uptime}
|
|
channels GET,POST /channels · PUT,DELETE /channels/:id · POST /channels/:id/test
|
|
secrets GET,POST /secrets · GET,PUT,DELETE /secrets/:group
|
|
POST /secrets/:group/reveal · DELETE /secrets/:group/:key
|
|
console POST /console/connect · GET /console/tunnel (websocket)
|
|
audit GET /audit
|
|
agent GET /agent/latest-version
|
|
settings GET,PUT /settings · POST /settings/secrets-token (owner|admin)
|
|
org GET,POST /org/users · PUT /org/users/:id/role · DELETE /org/users/:id
|
|
GET,PUT /org/oidc (owner|admin)
|
|
```
|
|
|
|
---
|
|
|
|
## Admin REST API (`admin`, :8083)
|
|
|
|
A separate service with its own session cookie (`admin_session`) and its own database. Unauthenticated:
|
|
|
|
```
|
|
GET /healthz
|
|
GET /auth/me # who am I; 401 drives the UI's redirects
|
|
POST /auth/staff/login /auth/login /auth/logout
|
|
POST /auth/signup # self-hosted only; honeypot + rate limited
|
|
GET /auth/verify?token=…
|
|
POST /auth/accept-invite # an invitee sets their own password
|
|
```
|
|
|
|
Customer-session (`/api`), every instance resolved through `ownedInstance`:
|
|
|
|
```
|
|
GET /account # account, instances, max_relinks
|
|
POST /instances # create a cloud instance (Free tier, capped at one per account)
|
|
POST /instances/:id/renew # Free renewal; refuses outside the renewal window
|
|
POST /instances/link · /instances/:id/relink
|
|
GET /instances/:id/license · /instances/:id/license/download
|
|
GET /subscriptions
|
|
GET,POST /account/users · PUT /account/users/:id/role · DELETE /account/users/:id
|
|
PUT /account/password # propagates to every projected user
|
|
GET,POST /instances/:id/members # cloud only
|
|
PUT /instances/:id/members/:uid/role · DELETE /instances/:id/members/:uid
|
|
```
|
|
|
|
Reading is open to any signed-in member; every mutation above except
|
|
`/account/password` (which is your own) sits behind `RequireAccountRole(owner,
|
|
admin)`. `:uid` is the **`customer_users.user_id`**, not the projected
|
|
control-plane user_id — the portal never has to know that one.
|
|
|
|
Staff-session (`/api/staff`):
|
|
|
|
```
|
|
GET,POST /accounts · GET /accounts/:id # search by name, email, Paddle ID or instance UUID
|
|
GET,POST /instances · GET /instances/:id # instance + account + licence history + injection state
|
|
POST /instances/:id/issue · /instances/:id/relink
|
|
GET /licenses · /subscriptions · /audit · /plans · PUT /plans/:tier
|
|
GET /health/injection
|
|
```
|
|
|
|
**Customer endpoints answer 404, never 403, for another account's resource** — a 403 confirms the resource exists. Route-group guards in `adminsite/` mirror this, but the backend is the layer that matters.
|
|
|
|
## MongoDB Collections
|
|
|
|
`servers` · `keys` · `assignments` · `orgs` · `users` · `org_oidc` · `settings` · `secrets` · `workflows` · `workflow_steps` · `workflow_runs` · `monitors` · `incidents` · `monitor_rollups` · `notification_channels` · `console_sessions` · `audit_logs` · `migrations`
|
|
|
|
Every document except `migrations` carries `org_id`. Struct definitions are the source of truth — see `server/internal/models/`.
|
|
|
|
Notes that are not obvious from the structs:
|
|
|
|
- `servers.agent_token_hash` stores SHA-256 of the token, never plaintext. `pre_reg_token` is cleared after `Register()`. `status` is `pending` → `active` on register, `offline` when `last_seen` passes the threshold (swept every 2 min).
|
|
- `servers.inventory` holds the latest metrics snapshot with separate `metrics_at` / `static_at` timestamps.
|
|
- `keys.private_key_enc` and `passphrase_enc` are AES-256-GCM; the JSON form exposes only `has_private_key` / `has_passphrase`.
|
|
- `assignments.revoked_at: null` means active. Revocation is soft, preserving audit history.
|
|
- `workflow_runs.steps_snapshot` freezes the resolved steps so editing the library never rewrites history.
|
|
- `console_sessions.token_consumed_at` is set atomically to enforce one-time use.
|
|
- `users.auth_source` is `local`, `oidc` or `hq`. An `hq` user was projected from a Vantage HQ account and carries `hq_user_id`; HQ owns its role, password and existence.
|
|
|
|
Admin's own database is separate and holds `accounts` · `admin_instances` · `licenses` · `subscriptions` · `plans` · `staff_users` · `customer_users` · `instance_members` · `admin_audit`. `instance_members` is unique on `(instance_id, customer_user_id)` — one person holds at most one user in one instance, which makes a grant idempotent-by-refusal rather than silently doubling a projection. It is an *index* of the control-plane rows, not the authority (see "Grants project, they do not federate"). Admin has no migrations collection; `models.Backfill` runs on every boot and is idempotent by filtering on the absence of what it writes.
|
|
|
|
### Migrations
|
|
|
|
`services.RunMigrations()` runs at boot, recording markers in `migrations`:
|
|
|
|
- `0001_default_org_backfill`
|
|
- `0002_settings_org_backfill` (must run before 0003 — 0003 can create a `default` org, which pushes 0002 into its ambiguous multi-org branch)
|
|
- `0003_missed_org_scopes`
|
|
|
|
Index builders (`EnsureAuthIndexes`, `EnsureSettingsIndexes`) are fatal on failure; `EnsureSecretIndexes` and `EnsureWorkflowIndexes` only warn.
|
|
|
|
---
|
|
|
|
## Agent Lifecycle
|
|
|
|
### Config file
|
|
|
|
Linux `/etc/vantage/config.yaml`, Windows `%ProgramData%\vantage\config.yaml`. Directory `0700`, file `0600`.
|
|
|
|
```yaml
|
|
server_url: "vantage.yourdomain.com:9090"
|
|
server_id: "<uuid>"
|
|
pre_reg_token: "<token>" # removed after first successful Register()
|
|
agent_token: "" # written by agent after Register()
|
|
poll_interval: 30s
|
|
tls: true
|
|
```
|
|
|
|
### Startup
|
|
|
|
```
|
|
1. Load config
|
|
2. If pre_reg_token present → Register() → save agent_token, clear pre_reg_token, reconnect
|
|
3. Start goroutines: command stream · update check (hourly) · inventory · monitors
|
|
4. Enter SyncKeys poll loop (default 30s)
|
|
```
|
|
|
|
### Poll loop
|
|
|
|
```
|
|
1. SyncKeys(server_id, agent_token, agent_version)
|
|
2. Non-Linux hosts stop here — Windows agents register and heartbeat only
|
|
3. Diff desired keys against /root/.ssh/authorized_keys; unchanged → no write
|
|
4. Changed → write .tmp, os.Rename() over the real file, chmod 0600
|
|
```
|
|
|
|
### Install
|
|
|
|
Linux: systemd unit at `/etc/systemd/system/vantage-agent.service`, `Restart=always`, runs as root.
|
|
Windows: MSI built by CI (WiX), or `installer/setup.ps1` registering the agent as a service via NSSM.
|
|
|
|
---
|
|
|
|
## Server Registration Flow
|
|
|
|
1. **Add Server** in the UI calls `POST /api/servers/new`, which generates a `server_id` and a pre-registration token (TTL 1 hour, single-use).
|
|
2. The UI shows a one-liner:
|
|
```bash
|
|
curl -fsSL https://vantage.yourdomain.com/install | \
|
|
bash -s -- --server-id=<id> --token=<token>
|
|
```
|
|
Windows gets the `/install.ps1` equivalent.
|
|
3. The script detects arch, downloads the agent from the Gitea release, verifies the SHA-256 checksum, writes the config, installs and starts the service.
|
|
4. The server flips to `active` on first sync.
|
|
|
|
`/install` is served dynamically, injecting the latest agent version from the Gitea API.
|
|
|
|
---
|
|
|
|
## Environment Variables (server)
|
|
|
|
| Name | Required | Notes |
|
|
| -------------------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `GRPC_HOST` | **yes** | `host:port` agents dial. Boot fails without it — there is no safe default; falling back to the web host would hand agents a port that does not speak gRPC. |
|
|
| `MONGO_URI` | no | default `mongodb://localhost:27017` |
|
|
| `MONGO_DB` | no | default `vantage` |
|
|
| `REDIS_ADDR` | no | default `localhost:6379` |
|
|
| `KEY_ENCRYPTION_KEY` | yes in practice | 64-char hex (32 bytes) for AES-256-GCM. Required for private keys, secrets, OIDC secrets, RDP credentials. |
|
|
| `GITEA_HOST` | yes | used to build install scripts and agent download URLs |
|
|
| `GUACD_ADDR` | no | default `guacd:4822` |
|
|
| `APP_ROOT_LABEL` | no | default `vantage`; wrong value disables the host/session org guard |
|
|
| `VANTAGE_WORKFLOW_LOG_DIR` | no | where run logs are written |
|
|
| `FREE_INSTANCE_REAP_AFTER` | no | duration past a Free licence's expiry before the instance and all its data are deleted. **Empty disables the reaper, and empty is the default.** Set to `336h` in `docker-compose.site.yml` only — a self-hosted deployment must never reap. Must match admin's value, which only names the date in warning emails |
|
|
|
|
**sitesvc** (`deploy/docker-compose.site.yml` only):
|
|
|
|
| Name | Required | Notes |
|
|
| --------------------------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
| `MONGO_URI` | yes | **must point at the control plane's database.** sitesvc no longer provisions orgs itself, but it still refuses to start (`RequireMigratedDatabase`) against a database that has not run migration `0004` (the `orgs` → `instances` rename), and it (re)declares the shared `users.email` / `instances.slug` indexes at boot. The database name is read from the URI path; a URI without one is refused rather than defaulted. Note this differs from the server, which takes `MONGO_DB` separately. |
|
|
| `SMTP_HOST` / `SMTP_FROM` | yes | without them the contact form refuses (503) rather than silently dropping |
|
|
| `SMTP_TO` | no | default `support@hostxtra.co.uk`; contact enquiries only |
|
|
| `SMTP_PORT` | no | default `587`; `465` uses implicit TLS |
|
|
| `SMTP_USERNAME` / `SMTP_PASSWORD` | no | auth skipped when username is empty |
|
|
| `SITE_ORIGIN` | yes in practice | comma-separated allowed origins; unset refuses every cross-origin browser request |
|
|
| `TRUST_PROXY` | no | only `true` behind a proxy that overwrites `X-Forwarded-For`, or clients spoof past the rate limiter |
|
|
|
|
`deploy/docker-compose.yml` runs four services: `redis`, `guacd`, `server` (8080 + 9090), `web` (3000). MongoDB is external. `deploy/docker-compose.site.yml` adds four more — `site` (3003), `sitesvc` (8082), `admin` (8083) and `adminsite` (3004) — and is only used on vantage.hostxtra.co.uk.
|
|
|
|
`LICENSE_SIGNING_KEY` appears in **exactly one service in exactly one compose file**: `admin` in `docker-compose.site.yml`. It must never be added to `server`, and the self-hosted `docker-compose.yml` must never mention `admin` or `adminsite` at all. Admin uses an external Redis via `REDIS_ADDR`/`REDIS_USERNAME`/`REDIS_PASSWORD`; the base compose hardcodes `redis:6379` for `server`, so those variables reach admin only.
|
|
|
|
---
|
|
|
|
## Security
|
|
|
|
- gRPC over TLS; agents connect outbound only, no inbound firewall holes on managed servers.
|
|
- Per-server agent token stored as SHA-256 on the server, plaintext only in the agent's `0600` config.
|
|
- Pre-registration tokens are short-lived (1 hour) and single-use.
|
|
- AES-256-GCM at rest for private keys, key passphrases, vault secrets, OIDC client secrets, RDP/VNC credentials.
|
|
- Console session tokens are one-time; RDP credentials are consumed on tunnel open.
|
|
- ESO read token stored as a SHA-256 hash and rotatable.
|
|
- Unique indexes on `(instance_id, email)`, instance slug, settings instance and the ESO token hash are load-bearing for tenant isolation. So is the absence of any unscoped lookup by email.
|
|
- `authorized_keys` written `0600`, owned by root. The agent runs as root because it must.
|
|
- Every mutating API path writes an audit event.
|
|
|
|
---
|
|
|
|
## Frontend
|
|
|
|
Next.js 16 (App Router) + React 18, Tailwind 3, TanStack Query. Guacamole client bundled locally in `web/lib/guacamole-common.js`.
|
|
|
|
All three apps are **one visual system**, anchored on the logo navy. What differs between them is which end of it they stand on:
|
|
|
|
| App | Ground | Accent | Themes |
|
|
| --- | --- | --- | --- |
|
|
| `web/` | `--ground` dark, `#071628` | `#5b9be8` | dark only, locked |
|
|
| `site/` | token-based | `#0b2a58` light / `#5b9be8` dark | light + dark |
|
|
| `adminsite/` | token-based | `#0b2a58` light / `#5b9be8` dark | light + dark, light default |
|
|
|
|
`adminsite/app/globals.css` holds `site/app/globals.css`'s token blocks **copied verbatim** — same names, same values. `web/app/globals.css` holds the same tokens too, but only the **dark** values, since it does not switch. **Change a token in all three files in the same commit; nothing enforces the match automatically**, the same shape of hazard as sitesvc's mirrored slug rules.
|
|
|
|
Tailwind in all three maps `var(--…)` references only, so **no component in any of them may carry a hex value**. The names differ per app on purpose, because each app has its own subject: `site/` calls the semantic three `--up`/`--pend`/`--down` for monitor state, `adminsite/` aliases them to `valid`/`warn`/`expired` for licence state, and `web/` to `success`/`warning`/`danger`. Same colours, honest names on each side.
|
|
|
|
`web/` stores its tokens as **RGB channel triplets** with the hex in a trailing comment, and derives `--token: rgb(var(--token-rgb))` from them. That is not a style preference: the console leans on Tailwind's opacity modifiers (`bg-danger/10`, `border-accent/50`, `ring-accent/30`) in a way the other two do not, and `<alpha-value>` only compiles against channels. Keep the hex comments — they are what lets the three token blocks still be diffed by eye. `web/` also adds three tokens site/ has no use for: `--accent-hover` and `--down-hover` (site/ brightens with a CSS `filter`, which a Tailwind colour token cannot do) and `--well`, the floor beneath the ground for install one-liners, key blobs and run logs — surfaces showing machine output rather than interface.
|
|
|
|
`web/` is locked to dark and `adminsite/` defaults to **light**, and that pairing is the point: an operator with both open should never mistake one for the other before clicking Reissue. Now that both are drawn from the same palette the distinction rests **entirely** on the ground, so do not make dark the adminsite default and do not give web/ a light theme. State never reads by colour alone in either: every pill carries a distinct shape and a text label. The same argument applies one level in: the **staff** masthead sits on `--panel-2` with a `STAFF` chip, so staff and customer screens are not identical either.
|
|
|
|
`web/` collapses Tailwind's radius scale — `md`, `lg` and `xl` all resolve to site/'s 4px — rather than rewriting the ~140 `rounded-lg` classes across its pages. Every one of them meant "a panel corner", and `tailwind.config.ts` is now where that decision lives. `rounded-full` is untouched: status dots and pills still need it.
|
|
|
|
**The `adminsite/` shell.** `AppBar` is the single masthead — identity, nav, environment, account menu — and it belongs to the two authenticated layouts, never to `app/layout.tsx`, so `/login` and `/accept-invite` do not render navigation they cannot use. Nav active state is derived from `usePathname`; do not hardcode it. `PageHeader` gives every screen the same back link, title, actions and **record line** (the reference number in mono, click-to-copy) — the reference is what people paste into support tickets, so it has a fixed slot rather than a per-page treatment. `PageFrame` is the main-plus-320px-rail split; the rail carries only what is true account-wide, which is why there is no plan card in it — **tier, limits and expiry belong to a licence, and a licence belongs to one instance**, so an account holding a Free cloud instance and a Professional self-hosted one has no single plan.
|
|
|
|
Customer nav is three destinations — Overview, People, Billing. Settings is in the account menu because it is your password, not a place, and appearance lives there too: `AccountMenu` is the only thing that sets `data-theme`, which the token blocks have always supported in both directions.
|
|
|
|
`InstanceRecord` is one component open or closed, and it **replaced** `InstanceCard`. Closed it is a row; open it adds licence contents, members and actions. It defaults open when the instance is the only one or needs attention, and a manual toggle is remembered per instance in `localStorage`. Do not reintroduce a second summary component — the split is what left a one-instance account showing a third of a row and nothing else.
|
|
|
|
| Route | Purpose |
|
|
| --------------------------------------------------------------- | ----------------------------------------------------------------------- |
|
|
| `/setup` | First-run bootstrap: create the first org and owner |
|
|
| `/login` | Local or OIDC sign-in |
|
|
| `/` | Fleet dashboard |
|
|
| `/servers`, `/servers/new`, `/servers/[id]` | Fleet list, install one-liner, server detail (keys, inventory, updates) |
|
|
| `/servers/[id]/console` | Browser SSH/RDP/VNC session |
|
|
| `/keys`, `/keys/[id]` | Key library; assign and revoke per server |
|
|
| `/workflows`, `/workflows/[id]`, `/workflows/[id]/runs[/runId]` | Compose, run, and follow live logs |
|
|
| `/steps` | Reusable step library |
|
|
| `/monitors`, `/monitors/new`, `/monitors/[id][/edit]` | Checks, uptime, incidents |
|
|
| `/secrets`, `/secrets/[group]` | Vault |
|
|
| `/audit` | Audit log |
|
|
| `/settings`, `/settings/notifications`, `/settings/license` | Members, OIDC, alerts, retention, ESO token · channels · licence |
|
|
|
|
**`/settings` is one page, not a section.** Members and single sign-on used to
|
|
live at `/settings/instance` with their own sidebar entry; they are now the
|
|
**Access** group at the top of `/settings`, above **Monitoring** and
|
|
**Integrations**. Splitting "who can sign in" away from "how this instance
|
|
behaves" made two half-pages and a nav entry called Instance that no one could
|
|
distinguish from Settings. `next.config.ts` keeps a permanent redirect from the
|
|
old path. The cards live in `web/components/settings/` rather than in the page,
|
|
which is also where the `Field`/`inputClass` pair the three of them share now
|
|
lives — one copy instead of the three that existed while they were apart.
|
|
|
|
---
|
|
|
|
## CI/CD — Gitea Actions
|
|
|
|
### `agent-release.yml` — triggered by `agent/v*` tags
|
|
|
|
Builds `linux/amd64`, `linux/arm64`, `windows/amd64`, writes `checksums.txt`, creates a Gitea release. A second `msi` job on `windows-2022` packages the WiX installer.
|
|
|
|
```bash
|
|
GOOS=linux GOARCH=amd64 go build \
|
|
-ldflags="-s -w -X main.Version=${VERSION}" \
|
|
-o dist/vantage-agent-linux-amd64 ./cmd
|
|
```
|
|
|
|
### `server-deploy.yml` — triggered on every push to `main`
|
|
|
|
Builds and pushes six images to the Gitea container registry: `server`, `web`, `site`, `sitesvc`, `admin` and `adminsite`.
|
|
|
|
Note that despite the name, **this workflow does not deploy** — it only builds and pushes. There is no SSH step. Rolling images out is a separate manual step on the host:
|
|
|
|
```bash
|
|
cd /opt/vantage && docker compose -f docker-compose.yml -f docker-compose.site.yml pull && \
|
|
docker compose -f docker-compose.yml -f docker-compose.site.yml up -d --remove-orphans
|
|
```
|
|
|
|
**Each image only rebuilds when its own inputs changed.** A `git diff` against `github.event.before` decides, which is why the checkout uses `fetch-depth: 0` — the default shallow clone has one commit and nothing to diff — and why `git` is installed in the `docker:dind` container. The mapping follows the build contexts exactly:
|
|
|
|
| Image | Rebuilds when |
|
|
| --- | --- |
|
|
| `server` | `server/`, `shared/`, `proto/`, `go.work` |
|
|
| `admin` | `admin/`, `shared/`, `go.work` |
|
|
| `sitesvc` | `sitesvc/`, `shared/`, `go.work` |
|
|
| `web` · `site` · `adminsite` | their own directory only |
|
|
|
|
`shared/` fans out to all three Go images because each of their Dockerfiles copies `shared/` from a root context — **if a fourth service ever imports `shared/`, add it to that list or it will ship stale**. A change to the workflow file rebuilds everything, since a build arg is baked into the image. So does anything that leaves no trustworthy base commit: a manual `workflow_dispatch`, a new branch, or a force-push whose old head is gone.
|
|
|
|
The gap this leaves: **changing a repo variable pushes no commit, so nothing rebuilds.** After editing `API_URL`, `ADMIN_API_URL`, `HQ_URL` or `ADMIN_ENV`, run the workflow manually — that is what `workflow_dispatch` is there for. Base images also stop being refreshed on a service nobody touches; a periodic manual run covers that.
|
|
|
|
### Tagging
|
|
|
|
```bash
|
|
git tag agent/v1.0.0 && git push origin agent/v1.0.0 # agent release
|
|
git push origin main # server + web deploy
|
|
```
|
|
|
|
### Secrets / variables
|
|
|
|
| Name | Type | Value |
|
|
| -------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `RELEASE_TOKEN` | Secret | Gitea API token, `write:release` |
|
|
| `REGISTRY_USER` | Secret | Gitea username |
|
|
| `REGISTRY_PASSWORD` | Secret | Gitea token, `write:packages` |
|
|
| `GITEA_HOST` | Variable | `gitea.hostxtra.co.uk` |
|
|
| `DOCKER_HOST` | Variable | registry host used for image tags |
|
|
| `API_URL` | Variable | baked into the `web` image at build time |
|
|
| `SITE_API_URL` | Variable | **browser-reachable** sitesvc URL, baked into the `site` image. Required — if empty, both forms report "not connected" and submit nowhere. Must also be in sitesvc's `SITE_ORIGIN`. |
|
|
| `SITE_CONTACT_EMAIL` | Variable | optional; address shown when a form is misconfigured |
|
|
| `ADMIN_API_URL` | Variable | **browser-reachable** admin URL, baked into **both** the `adminsite` and `site` images — `site/start` posts account signups straight to admin. Same footgun as `SITE_API_URL`: wrong here and every request fails at runtime with the not-connected panel. |
|
|
| `ADMIN_ENV` | Variable | `production` or `sandbox`; drives the persistent environment badge. Anything but `sandbox` reads as production. |
|
|
| `HQ_URL` | Variable | optional; browser URL of the HQ portal, baked into `web` so an `hq`-sourced member links to where they are managed. Empty on self-hosted, which renders a plain label instead. |
|
|
|
|
---
|
|
|
|
## Design Decisions
|
|
|
|
- **gRPC for agent traffic** — strong typing and cheap versioning; polling for state, one bidirectional stream for commands.
|
|
- **Outbound-only agents** — no inbound ports on managed servers, works behind NAT.
|
|
- **Poll for keys, push for commands** — a 30s key poll is fine, but running a workflow step should not wait up to 30s.
|
|
- **Atomic `authorized_keys` rewrite** — temp file plus `os.Rename()`; a machine that dies mid-write keeps the old file.
|
|
- **Fingerprint diffing before write** — no disk churn on unchanged state.
|
|
- **Soft revocation** — `revoked_at` rather than deletes; preserves audit history.
|
|
- **Run snapshots** — workflow runs freeze their resolved steps so editing a step never rewrites past runs.
|
|
- **Monitors run in two places** — server-side for external endpoints, agent-side for anything only reachable from inside the target network.
|
|
- **Redis for sessions only** — all durable state stays in MongoDB; losing Redis logs everyone out and nothing else.
|
|
- **guacd for console** — protocol handling is Guacamole's problem, not ours; we proxy the WebSocket and manage credentials.
|
|
- **`org_id` on every document** — isolation enforced at the query layer, not by separate databases.
|
|
- **root only** — manages `/root/.ssh/authorized_keys`; no per-user key management.
|
|
- **Windows agents are second-class by design** — register, heartbeat, run steps, report inventory; no `authorized_keys` management.
|
|
- **Deletion lives in the control plane** — admin sends the warnings because it knows the billing address; the control plane performs the delete because it is the only service that knows which collections carry `instance_id`. Mirroring that list into admin would drift, and a drift there deletes the wrong rows.
|