Compare commits

...
138 Commits
Author SHA1 Message Date
mrhid6 0c21765da3 feat: Added pagination
Chart Release / chart (push) Successful in 13s
Server Deploy / deploy (push) Successful in 1m39s
2026-08-07 09:59:50 +01:00
mrhid6 4ff8fc8d51 docs: document the workload registry
Chart Release / chart (push) Successful in 11s
Server Deploy / deploy (push) Successful in 8m45s
Agent Release / build (push) Successful in 10m54s
Agent Release / msi (push) Successful in 2m31s
2026-08-07 09:09:18 +01:00
mrhid6 483053b9a2 feat: workload registry UI 2026-08-07 09:06:36 +01:00
mrhid6 fd4c51f3db feat: workload registry REST API 2026-08-07 09:01:46 +01:00
mrhid6 1b351cfca4 feat: agent reports workloads and handles workload commands 2026-08-07 08:58:46 +01:00
mrhid6 cf9d85b3cd feat: store workload reports and route log results 2026-08-07 08:56:26 +01:00
mrhid6 6a4ef5b6c6 feat: workload registry proto messages 2026-08-07 08:53:33 +01:00
mrhid6 501cf4e733 feat: agent reads bounded workload logs 2026-08-07 08:49:42 +01:00
mrhid6 89c21d752a feat: agent control actions with self-protection 2026-08-07 08:48:27 +01:00
mrhid6 0e38d9d500 feat: agent enumerates systemd services 2026-08-07 08:46:59 +01:00
mrhid6 3511c34daa feat: agent enumerates docker containers 2026-08-07 08:46:03 +01:00
mrhid6 0838d1d735 feat: models and indexes for the workload registry 2026-08-07 08:45:00 +01:00
mrhid6 d1769fc886 feat: Updated vuln style
Chart Release / chart (push) Successful in 12s
Server Deploy / deploy (push) Successful in 1m36s
2026-08-06 16:35:04 +01:00
mrhid6 6dced22499 fix: Fixed agent collect packages
Chart Release / chart (push) Successful in 11s
Server Deploy / deploy (push) Successful in 21s
Agent Release / build (push) Successful in 58s
Agent Release / msi (push) Successful in 1m34s
2026-08-06 16:20:34 +01:00
mrhid6 5cee53dc5f feat: Better button description
Chart Release / chart (push) Successful in 20s
Server Deploy / deploy (push) Successful in 2m23s
2026-08-06 15:51:18 +01:00
mrhid6 81248bb159 style: bring the vulnerabilities page onto the house page shape
Every other page under app/(app) opens with `p-4 sm:p-6 lg:p-8` and the
layout adds none of its own, so this page alone sat flush against the shell
edge. Its h1 was text-xl where every other page is text-2xl.

The findings list was a stack of separately bordered cards; it is now rows
inside one Card, separated by border-border-soft, matching the monitors and
workflows lists. Loading is the shared spinner rather than a line of text,
the error is the shared danger strip, and the empty state uses the same
proportions as the monitors one.
2026-08-06 15:49:39 +01:00
mrhid6 6354d54de8 feat: added addon price to pricing page 2026-08-06 15:48:20 +01:00
mrhid6 da6d64f95c fix: give the scratch server image a /tmp for the vulnerability database
The runtime stage is FROM scratch, which has no /tmp, so vulnsched died at
startup with "temp dir: stat /tmp: no such file or directory" and no scan
ever ran. Nothing in the server wrote to a temporary directory before the
trivy-db puller, which is why this only appeared now.

scratch cannot mkdir its own, so the directory is staged in the builder at
1777 and copied in. Also corrects CLAUDE.md, which described this image as
Alpine; the time/tzdata import it justifies is if anything more load-bearing
on scratch.
2026-08-06 15:46:30 +01:00
mrhid6 9ba3d4a61f feat: Vulnerability Scanning feature on license page
Chart Release / chart (push) Successful in 12s
Server Deploy / deploy (push) Successful in 1m20s
2026-08-06 15:40:13 +01:00
mrhid6 eee236a072 fix: tidy server go.mod for the vulnerability database dependencies
Chart Release / chart (push) Successful in 12s
Server Deploy / deploy (push) Canceled after 2m11s
trivy-db, oras-go, the three version comparators and robfig/cron were
recorded as indirect with an incomplete go.sum, which builds locally
against a warm module cache but fails in CI with "updates to go.mod
needed". trivy-db pulls testify into the build graph, and its hashes
were missing entirely.
2026-08-06 15:18:57 +01:00
mrhid6 9df89e2db4 fix: surface vuln_scanning across licence, staff and pricing UI
Chart Release / chart (push) Successful in 11s
Server Deploy / deploy (push) Canceled after 59s
The catalogue row alone was not enough; the feature was invisible in
three places and mislabelled in a fourth.

PlanConfigurator rendered every key that was not "console" as "Single
sign-on", so the staff checkbox granting vulnerability scanning was
labelled single sign-on. Feature wording was duplicated between the staff
configurator and the purchase form and the copies had drifted, so it now
lives in adminsite/lib/features.ts and both read from it.

The customer licence panel showed raw keys; it now labels them.

Pricing gains a comparison row. The add-on block with a monthly price is
deliberately NOT added: that is a pricing decision, and the Paddle price
IDs for the new catalogue rows have to be pasted in before it can be sold
anyway.
2026-08-06 15:10:26 +01:00
mrhid6 f60c509b47 feat: vuln_scanning entitlement and documentation
Chart Release / chart (push) Successful in 12s
Server Deploy / deploy (push) Failing after 1m11s
Agent Release / build (push) Successful in 1m0s
Agent Release / msi (push) Successful in 2m18s
Adds license.FeatureVulnScanning as the one name for the feature and a
catalogue row per deployment/tier, following console and oidc: features
are opt-in per customer, so no plan bundles it.

Documents the subsystem in CLAUDE.md, including that ScopedCollections is
the canonical registry instance deletion derives from — there is no
separate deletion list, which the plan had wrong.
2026-08-06 14:44:11 +01:00
mrhid6 84dfcfeac7 feat: vulnerability findings UI
Fleet board grouped by CVE, a per-server section on server detail, and
alert rules beside the channels they consume.

The server detail page has no tab pattern despite the plan saying to
follow one, so this adds a section in the existing vertical stack.

Three states are kept visually distinct because they are identical if
handled carelessly and only one is good news: never reported, no advisory
feed for the distribution, and scanned-and-clean. Database freshness sits
with the findings rather than in settings for the same reason.
2026-08-06 14:40:37 +01:00
mrhid6 5dda3b5c4a feat: vulnerability scanning pipeline, matcher, scheduler and API
Completes tasks 10-15 and fixes what was outstanding:

- vulndb.Pull implemented with oras-go, streaming the ~50MB layer and
  staging both files before replacing either, so a failed pull leaves the
  previous database intact rather than a half-written one.
- db.go: Vulnerability.Severity is a string, not trivy Severity, so the
  int conversion did not compile. Severity now resolves vendor (highest
  when vendors disagree) then NVD then unknown, and CVSS is read too.
- findings.go: added sweepFixedFindings plus the fleet query, severity
  counts, rescan flag and accept/unaccept the API needs.
- vulnrules.go: added rule CRUD and the digest builder. ResolveTargets
  returns []models.Server, not []string, so filterByServers was wrong.
- api/vulnerabilities.go was an empty file while handlers.go registered
  twelve routes against it; written, grouped by CVE.
- shared/mail: added the missing sender. The templates were orphaned and
  the HTML one was a copy of the text one, defining "subject" (which
  html/template would escape) and emitting no markup. render.go parses
  every template in init(), so a bad one panics server, admin and sitesvc
  at boot — go build never runs init(), which is why nothing complained.
- notify: digests dispatch through their own path so SMTP gets the digest
  template rather than arriving dressed as a monitor alert.
2026-08-06 14:33:46 +01:00
mrhid6 db64320bd8 feat: agent reports installed packages on the hourly loop
SyncKeys now returns the whole response so the poll can carry
CollectPackages; a separate RPC for one boolean would be a message every
30 seconds for a value that changes when a licence does.

The flag is an atomic: the 30s poll writes it, the hourly package loop
reads it, and they are different goroutines.
2026-08-06 13:21:13 +01:00
mrhid6 583f60771c feat: store agent package reports and serve the collect flag
VulnScanningEnabled reads GetLicenseState(...).Feature("vuln_scanning")
and requires an active licence, never switching on tier. ReportPackages
re-checks it server-side: the agent flag is the optimisation, this is
the boundary.
2026-08-06 13:19:39 +01:00
mrhid6 a92c3190c2 feat: ReportPackages wire types with hash short-circuit
The pb packages are hand-written, not protoc-generated, and the wire
codec is JSON (encoding.RegisterCodec(JSONCodec{})). Field numbers in
the .proto are documentation; JSON field names are the contract. Both pb
packages edited by hand to match.

SyncResponse.collect_packages is omitempty and absent decodes as false,
so an older server leaves agents collecting nothing rather than
collecting without a licence.
2026-08-06 13:17:44 +01:00
mrhid6 3a6d24fe0e feat: models and indexes for package inventory and CVE findings
Adds server_packages, vuln_findings and vuln_alert_rules to
ScopedCollections rather than to a separate deletion list. purgeInstance
derives its collection list from that registry, so instance deletion
follows automatically and there is no second copy to drift.
2026-08-06 11:59:00 +01:00
mrhid6 c277ecff44 feat: agent collects installed packages per package manager 2026-08-06 11:56:41 +01:00
mrhid6 bd690c94c3 feat: agent parses /etc/os-release for distro identification 2026-08-06 11:55:45 +01:00
mrhid6 a22fdf197e feat: map OS family and version to trivy-db advisory buckets 2026-08-06 11:55:01 +01:00
mrhid6 bd24b03cac feat: version comparators for distro package ordering 2026-08-06 11:54:13 +01:00
mrhid6 3afc4ab012 docs: workload registry plan; remove tests from both plans
Both plans now verify by build, vet and manual checks written into the
tasks. Spec verification sections updated to match so they no longer
describe tests that will not be written.
2026-08-06 11:27:37 +01:00
mrhid6 d1ac3e98ce docs: design for the workload registry
Agents enumerate Docker containers, compose stacks and systemd services;
start/stop/restart and bounded log snapshots from the UI.

Sub-project B, Linux only. Live log following stays in the console.
2026-08-06 11:11:26 +01:00
mrhid6 5bba54f3e5 fix: Fixed style layout on workflow run page 2026-08-06 10:50:31 +01:00
mrhid6 fe7bc300e2 docs: implementation plan for package inventory and CVE findings
17 tasks, TDD where the logic is pure. Corrects two spec claims:
the server reads features via License.HasFeature rather than admin's
entitlement directly, and shared/mail/render_test.go does not exist.
2026-08-06 10:49:10 +01:00
mrhid6 00c03c365d docs: design for package inventory and CVE findings
Agents report installed packages; the control plane matches them against
trivy-db and raises findings that link to the existing ApplyUpdatesCmd
patching path.

Scoped to sub-project A, Linux only. Container registry, image scanning
and compliance baselines are separate specs.
2026-08-06 10:33:54 +01:00
mrhid6 dc8dd3dd58 fix: Fixed step descriptions
Chart Release / chart (push) Successful in 11s
Server Deploy / deploy (push) Successful in 35s
2026-08-04 17:45:31 +01:00
mrhid6 85a8865892 feat: restyle the steps table and add 22 default steps
Chart Release / chart (push) Successful in 11s
Server Deploy / deploy (push) Successful in 2m46s
2026-08-04 17:34:18 +01:00
mrhid6 50a9ac5fdc fix: count tag-matched servers in the workflows list 2026-08-04 17:28:01 +01:00
mrhid6 3388d2f895 fix: Fixed padding on add step button
Chart Release / chart (push) Successful in 22s
Server Deploy / deploy (push) Successful in 41s
2026-08-04 17:24:40 +01:00
mrhid6 3a77fc2abd feat: edit target servers and tags together in the workflow modal 2026-08-04 17:21:12 +01:00
mrhid6 3d59836d0c feat: dual list box for workflow target servers
Chart Release / chart (push) Successful in 11s
Server Deploy / deploy (push) Successful in 5m19s
2026-08-04 17:13:50 +01:00
mrhid6 d9184312aa fix: schedule card placement, preview state, and scheduled-workflow docs 2026-08-04 17:08:11 +01:00
mrhid6 b9802e6b04 docs: Updated docs 2026-08-04 17:03:29 +01:00
mrhid6 c2635ed51a fix: Fixed schedule workflow col
Chart Release / chart (push) Successful in 20s
Server Deploy / deploy (push) Successful in 41s
2026-08-04 14:42:10 +01:00
mrhid6 b21ac05547 feat: show workflow schedules in the list
Chart Release / chart (push) Successful in 24s
Server Deploy / deploy (push) Successful in 2m34s
2026-08-04 14:16:13 +01:00
mrhid6 484b620867 feat: schedule editor on the workflow page 2026-08-04 14:13:31 +01:00
mrhid6 439bc2ed7d feat: schedule methods on the web api client 2026-08-04 14:10:04 +01:00
mrhid6 a1e6986a64 feat: fire scheduled workflow runs from the housekeeping leader 2026-08-04 13:53:42 +01:00
mrhid6 d0e1cc4ad6 feat: cron arithmetic and persisted workflow schedules 2026-08-04 13:51:10 +01:00
mrhid6 b877024365 docs: server tags and workflow tag targeting 2026-08-04 13:44:16 +01:00
mrhid6 2de7ac116b feat: filter the fleet by tag and target workflows by tag selector 2026-08-04 13:42:47 +01:00
mrhid6 fa1fd14ed1 feat: view and edit server tags 2026-08-04 13:38:42 +01:00
mrhid6 d1b3cd2f74 feat: target workflow runs by tag selector 2026-08-04 13:36:43 +01:00
mrhid6 e00a0da5d9 feat: tag endpoints for servers 2026-08-04 13:34:22 +01:00
mrhid6 fef0b7c7a1 feat: read and write server tags, resolve targets from the database 2026-08-04 13:33:00 +01:00
mrhid6 efd29dc259 feat: parse tag filters and resolve targets as ids union tag selector 2026-08-04 13:31:04 +01:00
mrhid6 13cd41d202 feat: validate server tags and add the model field 2026-08-04 13:30:06 +01:00
mrhid6 3530ce6cb7 docs: implementation plans for server tags and scheduled workflows 2026-08-04 13:26:04 +01:00
mrhid6 09522c2566 docs: design for server tags and scheduled workflows 2026-08-04 13:13:12 +01:00
mrhid6 80f0afb28b feat: Updated monitors pages
Chart Release / chart (push) Successful in 13s
Server Deploy / deploy (push) Successful in 1m25s
2026-08-04 12:19:15 +01:00
mrhid6 287bd9657b fix: Fixed paddle relink sub
Chart Release / chart (push) Successful in 27s
Server Deploy / deploy (push) Successful in 1m17s
2026-08-03 17:34:39 +01:00
mrhid6 b5f684c4fe fix: Fixed paddle subs
Chart Release / chart (push) Successful in 28s
Server Deploy / deploy (push) Successful in 1m17s
2026-08-03 15:32:01 +01:00
mrhid6 1f08e90009 feat: Removed email alert settings
Chart Release / chart (push) Successful in 10s
Server Deploy / deploy (push) Successful in 5m15s
2026-08-03 14:40:15 +01:00
mrhid6 6881d92d0a fix: local-login toggle no longer reverts unsaved settings edits
Chart Release / chart (push) Successful in 26s
Server Deploy / deploy (push) Successful in 4m8s
server/internal/services/settings.go SaveSettings takes alerts and
email as required (non-pointer) values and writes them unconditionally
- absent fields would blank stored settings, not just leave them
alone. onLocalLoginChange was building its payload from the stale
loaded settings object instead of the in-progress form state
(thresholdMinutes/logRetentionDays) that handleSubmit uses, so editing
the offline threshold and then flipping the toggle silently reverted
the edit. Both paths now submit the same in-progress values.
2026-08-03 14:13:05 +01:00
mrhid6 5e016c6584 fix: audit ack_notice and stop misreporting DB errors as lockouts
ackAuthProviderNotice mutated callback_notice with no audit event; it
now writes auth_provider.ack_notice like create/update/delete.

guardProviderChange's callers turned any error from
CountEnabledAuthProviders into a 409 last_provider, so a transient
Mongo error was reported to the operator as an unremovable lockout.
Only services.ErrLockout now produces the 409; anything else is a 500.
2026-08-03 14:12:21 +01:00
mrhid6 537b8758ff fix: purge auth_providers when reaping an instance
auth_providers was missing from ScopedCollections, so reap.go's
scopedCollectionsForPurge() (derived from that list) never deleted an
instance's providers, leaving orphaned rows holding encrypted client
secrets forever. Verified migration 0004's $rename over org_id->instance_id
is a no-op here since auth_providers never carried org_id.
2026-08-03 14:11:42 +01:00
mrhid6 c03360333b fix: single source of truth for local-login lockout rescue
HandleLocalLogin and HandleListPublicProviders each computed their own
answer to whether password sign-in must stay available, and they could
disagree: an instance with local login off and a licence that lapses
loses its only provider and its password form in the same moment, with
no endpoint left to recover. services.LocalLoginPermitted is now the
one predicate both call.
2026-08-03 14:11:22 +01:00
mrhid6 fa7c5d341d docs: fix stale auth-provider references in rest-api and licensing docs 2026-08-03 11:08:22 +01:00
mrhid6 b6fc8c3f77 docs: document multiple auth providers and the callback URL change 2026-08-03 11:05:40 +01:00
mrhid6 37f2c1457e feat: manage multiple sign-in providers from settings 2026-08-03 11:00:25 +01:00
mrhid6 3a626922a5 feat: render one login button per configured auth provider 2026-08-03 10:56:03 +01:00
mrhid6 dde47de145 feat: auth provider REST API and public provider discovery 2026-08-03 10:51:35 +01:00
mrhid6 f3b9f6f286 feat: add GitHub OAuth2 provider branch 2026-08-03 10:48:19 +01:00
mrhid6 f1c3f67864 feat: per-provider SSO start and callback routes 2026-08-03 10:45:23 +01:00
mrhid6 8f5873afca refactor: carry provider id in the OIDC state token 2026-08-03 10:41:30 +01:00
mrhid6 e22faebfcd feat: migrate instance_oidc into auth_providers (0005) 2026-08-03 10:38:52 +01:00
mrhid6 e2b01b62a5 feat: add local_login_enabled setting with absent-means-on default 2026-08-03 10:36:48 +01:00
mrhid6 0858693d57 feat: add auth provider service layer and lockout guard 2026-08-03 10:33:36 +01:00
mrhid6 45f7c0c393 feat: add AuthProvider model and identity provider presets 2026-08-03 10:30:21 +01:00
mrhid6 c56bfb7270 docs: implementation plan for multiple auth providers 2026-08-03 10:23:35 +01:00
mrhid6 eb45072031 feat: Removed unused test units 2026-08-03 10:18:01 +01:00
mrhid6 1e2132c1a1 docs: Cleanup old specs and plans 2026-08-03 10:15:54 +01:00
mrhid6 19ef773690 docs: drop legacy OIDC callback from multi-provider design 2026-08-03 10:13:26 +01:00
mrhid6 c5aae0614a docs: design for multiple auth providers 2026-08-03 10:09:13 +01:00
mrhid6 17d97aaf52 feat: More logging for command stream
Chart Release / chart (push) Successful in 12s
Server Deploy / deploy (push) Successful in 1m24s
Agent Release / build (push) Successful in 10m37s
Agent Release / msi (push) Successful in 36s
2026-07-31 17:20:35 +01:00
mrhid6 1fb9bd827f feat: Added ping command
Chart Release / chart (push) Successful in 18s
Agent Release / build (push) Successful in 39s
Server Deploy / deploy (push) Successful in 55s
Agent Release / msi (push) Successful in 40s
2026-07-31 17:10:59 +01:00
mrhid6 8699dc5b7e fix: Renew presence on sub/pub
Chart Release / chart (push) Successful in 18s
Server Deploy / deploy (push) Successful in 56s
2026-07-31 16:58:50 +01:00
mrhid6 71240f183c fix: Fixes to server shutdown stream
Chart Release / chart (push) Successful in 21s
Server Deploy / deploy (push) Successful in 1m2s
Agent Release / build (push) Successful in 43s
Agent Release / msi (push) Successful in 49s
2026-07-31 16:44:47 +01:00
mrhid6 01e8b0ba44 feat: Better debugging for console
Chart Release / chart (push) Successful in 11s
Server Deploy / deploy (push) Successful in 1m25s
2026-07-31 16:31:19 +01:00
mrhid6 2aa4784518 feat: Better debugging for console
Chart Release / chart (push) Successful in 18s
Server Deploy / deploy (push) Successful in 59s
2026-07-31 16:13:51 +01:00
mrhid6 f611cae438 feat: Better debugging for console
Chart Release / chart (push) Successful in 11s
Server Deploy / deploy (push) Successful in 1m22s
2026-07-31 16:00:58 +01:00
mrhid6 1eb98ef962 feat: Better debugging for console
Chart Release / chart (push) Successful in 12s
Server Deploy / deploy (push) Successful in 1m22s
2026-07-31 15:51:26 +01:00
mrhid6 6f86496f10 fix: Ffixes to console
Chart Release / chart (push) Successful in 18s
Server Deploy / deploy (push) Successful in 1m9s
2026-07-31 15:05:21 +01:00
mrhid6 57a9b18102 fix: Guacd connection ip
Server Deploy / deploy (push) Successful in 9s
Chart Release / chart (push) Successful in 11s
2026-07-31 14:52:20 +01:00
mrhid6 36995fa62b fix: Fixed install and update scripts
Chart Release / chart (push) Successful in 9s
Server Deploy / deploy (push) Successful in 1m20s
2026-07-31 12:10:32 +01:00
mrhid6 9121fc461f fix: Fixed chart api routes for update
Server Deploy / deploy (push) Successful in 15s
Chart Release / chart (push) Successful in 10s
2026-07-31 12:03:45 +01:00
mrhid6 fc56bae5f9 chore: Bump chart version
Chart Release / chart (push) Successful in 18s
Server Deploy / deploy (push) Successful in 8s
Agent Release / build (push) Successful in 38s
Agent Release / msi (push) Successful in 57s
2026-07-31 11:53:08 +01:00
mrhid6 ac75b3ef76 feat: chart deployment Type added
Chart Release / chart (push) Successful in 20s
Server Deploy / deploy (push) Successful in 35s
2026-07-31 11:52:31 +01:00
mrhid6 e6fe463216 feat: Updated for api ingress routes
Chart Release / chart (push) Successful in 25s
Server Deploy / deploy (push) Successful in 4m26s
2026-07-31 11:21:15 +01:00
mrhid6 8528f14ed7 feat: Added ingress to chart
Chart Release / chart (push) Successful in 10s
Server Deploy / deploy (push) Successful in 1m24s
2026-07-31 10:49:20 +01:00
mrhid6 df1d9658f5 fix: Chart build
Chart Release / chart (push) Successful in 10s
Server Deploy / deploy (push) Successful in 1m20s
2026-07-31 10:41:41 +01:00
mrhid6 9f9b384481 fix: Fixed chart version
Chart Release / chart (push) Failing after 13s
2026-07-31 10:37:04 +01:00
mrhid6 165114471f fix: Fixes to running on kubernetes
Chart Release / chart (push) Failing after 13s
Server Deploy / deploy (push) Successful in 6m35s
2026-07-31 10:34:10 +01:00
mrhid6 de78688093 feat: authenticate the server's Redis connection
InitRedis now takes a username and password, read from REDIS_USERNAME and
REDIS_PASSWORD, matching what admin has always done. Both empty keeps an
unauthenticated Redis working; a password with an empty username is what a
legacy requirepass instance needs, since go-redis then sends AUTH with one
argument instead of two.

This is what lets a Kubernetes install point at a managed Redis instead of
the bundled one.
2026-07-31 09:36:31 +01:00
mrhid6 bbf9f72fd3 feat: Docker and helm charts
Server Deploy / deploy (push) Successful in 5m26s
Agent Release / build (push) Successful in 10m45s
Agent Release / msi (push) Successful in 1m31s
2026-07-31 09:28:54 +01:00
mrhid6 978b665aa6 fix: stop local relay teardown from logging a spurious proxy_failed reason
Session.Close now closing its own accepted conn (from the prior fix wave)
made net.ErrClosed on the guacd-side reader indistinguishable from a real
remote failure, so a normal browser-tab close could race the handler's
defer and intermittently log console.proxy_failed on a healthy session.
Add a closing flag, set before Close's sync.Once body actually tears
anything down, that setReason respects -- a deliberate local teardown can
no longer produce or race in a failure reason, while Close's own explicit
reason argument still wins normally.
2026-07-31 09:25:27 +01:00
mrhid6 1fe608f531 fix: bound and complete console relay teardown, restore proxy_failed audit
- Arm the unclaimed-relay watchdog in NewSession rather than Serve, so an
  agent that never opens its ProxyStream is bounded to 10s and reports
  reason "agent_timeout", per the design spec's failure-mode table.
- Session.Close now also closes the accepted net.Conn (stored via setConn),
  so ConsoleProxy.Close() is an unconditional kill of the whole relay chain
  instead of only closing an already-idle listener.
- Emit console.proxy_failed and end the console session from a defer in
  consoleTunnel guarded on relay.Reason(), since guac's OnDisconnect never
  runs when the connect callback errors -- which is the path every relay
  failure this feature introduces takes. Update the two docsite
  troubleshooting rows to match what the audit event can now actually show.
2026-07-31 09:21:07 +01:00
mrhid6 1e1546cb60 docs: document the agent-relayed console proxy
Every console session now rides the agent's outbound gRPC connection
instead of a direct guacd-to-target dial, so it works for servers
behind NAT and now requires a live agent (409 agent_offline
otherwise). Documents PROXY_ADVERTISE_HOST / PROXY_LISTEN_HOST and
corrects reachability claims across the docsite and CLAUDE.md.
2026-07-31 09:10:05 +01:00
mrhid6 119d8694d1 feat: Reap admin free instance license 2026-07-30 14:42:06 +01:00
mrhid6 8d43c689f5 feat: Reap admin free instance 2026-07-30 14:31:43 +01:00
mrhid6 05f10ed3c9 feat: record relay proxy_id and port in console audit events 2026-07-29 13:10:58 +01:00
mrhid6 c0bec3737b feat: route every console session through the agent relay 2026-07-29 13:07:30 +01:00
mrhid6 59d147fe4d feat: handle OpenProxyCmd in the agent command stream 2026-07-29 13:03:02 +01:00
mrhid6 9e38a01e3d feat: add agent-side console relay 2026-07-29 12:59:24 +01:00
mrhid6 20a302f84a feat: add OpenConsoleProxy service facade 2026-07-29 12:55:36 +01:00
mrhid6 ba2e263d00 fix: collapse ProxyStream auth failures into one indistinguishable response 2026-07-29 12:52:53 +01:00
mrhid6 a000703199 feat: add ProxyStream handler with scoped single-use auth 2026-07-29 12:50:20 +01:00
mrhid6 8fcda63742 fix: avoid closing proxy relay listener before validating remote source 2026-07-29 12:47:35 +01:00
mrhid6 3363ac9dad feat: add console proxy session relay 2026-07-29 12:43:04 +01:00
mrhid6 a7e338b171 feat: add console proxy session registry 2026-07-29 12:40:22 +01:00
mrhid6 bc79daab48 feat: add ProxyStream wire types for agent-relayed console 2026-07-29 12:37:37 +01:00
mrhid6 d3d8dba3ff docs: Implementation plan for agent-relayed console proxy 2026-07-29 12:26:16 +01:00
mrhid6 6d047e25ab docs: Design for agent-relayed console proxy 2026-07-29 12:16:45 +01:00
mrhid6 ed4c39650c feat: Removed hq signup page
Server Deploy / deploy (push) Successful in 1m44s
2026-07-29 10:42:51 +01:00
mrhid6 7b8fa4a8a0 feat: Updated hq login page
Server Deploy / deploy (push) Successful in 40s
2026-07-29 10:35:07 +01:00
mrhid6 8a02c35ec9 docs: Updated docs
Server Deploy / deploy (push) Successful in 55s
2026-07-28 16:54:39 +01:00
mrhid6 487de34a50 docs: Updated docs
Server Deploy / deploy (push) Failing after 1m57s
2026-07-28 16:53:42 +01:00
mrhid6 0424547dd4 docs: Updated docs 2026-07-28 16:53:38 +01:00
mrhid6 5be9ddb2e5 fix: Fixed baked api url
Server Deploy / deploy (push) Successful in 7m8s
2026-07-28 16:18:50 +01:00
mrhid6 bc6c7cdb5a feat: Updated docker file
Server Deploy / deploy (push) Successful in 2m35s
2026-07-28 16:04:51 +01:00
mrhid6 f46fb7fc0e feat: documentation site
Docusaurus 3 docs-only site at docsite/, served statically by nginx under
/docs on the marketing host. Covers getting started (self-hosted install
through first server and first licence), the control plane, Vantage HQ,
a reference section and operations.

Wired into docker-compose.site.yml as docsite (3005:80) and into the
image build workflow, rebuilding on its own directory only. Never added
to the self-hosted compose file.
2026-07-28 15:46:33 +01:00
mrhid6 d9945882e5 docs: serve the documentation site under /docs on the marketing host 2026-07-28 15:22:20 +01:00
mrhid6 9db16522e3 docs: design for the documentation site 2026-07-28 15:20:07 +01:00
mrhid6 6070972f9a chore: updated deps
Server Deploy / deploy (push) Successful in 36s
2026-07-28 15:07:19 +01:00
mrhid6 31e0000306 feat: default steps are read only
Server Deploy / deploy (push) Successful in 1m46s
2026-07-28 13:34:08 +01:00
mrhid6 26567766a9 feat: Move default steps into container 2026-07-28 13:28:43 +01:00
289 changed files with 52203 additions and 35635 deletions
+24
View File
@@ -0,0 +1,24 @@
{
"hooks": {
"PreToolUse": [
{
"matcher": "Bash|Grep",
"hooks": [
{
"type": "command",
"command": "C:/Python314/Scripts/graphify.EXE hook-guard search"
}
]
},
{
"matcher": "Read|Glob",
"hooks": [
{
"type": "command",
"command": "C:/Python314/Scripts/graphify.EXE hook-guard read"
}
]
}
]
}
}
+1
View File
@@ -0,0 +1 @@
graphify-out/graph.json merge=graphify
+221
View File
@@ -0,0 +1,221 @@
name: Chart Release
on:
# Every push that touches the chart is validated. Publishing is separate and
# deliberate: a chart version is immutable in the registry once pushed, so
# it must come from a tag someone chose, not from whatever landed on main.
# No `paths` filter on push, deliberately. A paths filter applies to tag
# pushes too, so tagging a commit that happened not to touch the chart
# would skip the publish entirely — a release that silently does nothing.
# Validation is seconds of helm rendering; running it on every push to main
# is cheaper than that failure mode.
push:
branches:
- main
tags:
- "chart/v*"
pull_request:
paths:
- "deploy/chart/**"
workflow_dispatch:
env:
CHART_DIR: deploy/chart/vantage
HELM_VERSION: v3.16.3
jobs:
chart:
runs-on: ubuntu-docker
container: alpine:3.21
steps:
# git for actions/checkout, curl for both the Helm download and the
# registry upload, tar because the Helm tarball is not self-extracting.
- name: Setup
run: apk add --no-cache bash curl git tar nodejs npm
- name: Install Helm
run: |
set -eu
curl -fsSL "https://get.helm.sh/helm-${HELM_VERSION}-linux-amd64.tar.gz" \
| tar -xz -C /tmp linux-amd64/helm
mv /tmp/linux-amd64/helm /usr/local/bin/helm
helm version --short
- name: Checkout
uses: actions/checkout@v4
- name: Lint
run: helm lint "$CHART_DIR"
# Rendering is the real test. `helm lint` accepts a chart whose
# templates fail to execute, and every guard in this chart is a
# template `fail` that only fires during rendering.
- name: Render default values
run: helm template test "$CHART_DIR" > /dev/null
- name: Render a multi-replica install
run: |
helm template test "$CHART_DIR" \
--set server.replicaCount=3 \
--set web.replicaCount=3 > /dev/null
# The reaper deletes whole instances, so "does this env appear only
# in cloud mode" is worth asserting rather than eyeballing.
- name: Check the reaper is cloud-only
run: |
set -eu
if helm template test "$CHART_DIR" | grep -q FREE_INSTANCE_REAP_AFTER; then
echo "FREE_INSTANCE_REAP_AFTER is set on a self-hosted render"
exit 1
fi
if ! helm template test "$CHART_DIR" \
--set server.env.deploymentType=cloud \
| grep -q FREE_INSTANCE_REAP_AFTER; then
echo "FREE_INSTANCE_REAP_AFTER is missing from a cloud render"
exit 1
fi
echo "ok: reaper configured in cloud mode only"
- name: Render against external Redis and MongoDB
run: |
helm template test "$CHART_DIR" \
--set redis.enabled=false \
--set redis.addr=redis.example.com:6379 \
--set mongo.enabled=false \
--set server.env.mongoUri=mongodb://mongo.example.com:27017/vantage > /dev/null
- name: Render with the Traefik ingress
run: |
helm template test "$CHART_DIR" \
--set ingress.enabled=true \
--set ingress.web.host=vantage.example.com \
--set ingress.grpc.host=agents.example.com \
--set ingress.tls.certResolver=letsencrypt \
--set server.env.grpcHost=agents.example.com:443 > /dev/null
# The shape the cloud deployment actually uses: a wildcard tenant
# namespace, /api and /auth routed at the edge, and no apex — that
# belongs to the marketing site, which this chart does not deploy.
- name: Render a wildcard host with edge-routed API paths
run: |
helm template test "$CHART_DIR" \
--set ingress.enabled=true \
--set 'ingress.web.host=*.vantage.example.com' \
--set ingress.api.enabled=true \
--set ingress.grpc.host=agents.example.com \
--set server.env.grpcHost=agents.example.com:443 \
--set ingress.tls.secretName=vantage-tls \
--set ingress.tls.grpcSecretName=agents-tls > /dev/null
# The guards are load-bearing, so their absence is a regression the
# same way a broken render is. Each of these must fail.
- name: Check the guards still refuse bad values
run: |
set -eu
refuses() {
desc="$1"; shift
if helm template test "$CHART_DIR" "$@" > /dev/null 2>&1; then
echo "GUARD MISSING: $desc was accepted"
exit 1
fi
echo "ok: refused $desc"
}
refuses "mongo disabled with an in-chart URI" \
--set mongo.enabled=false
refuses "redis disabled with no external address" \
--set redis.enabled=false
refuses "multiple replicas on a ReadWriteOnce volume" \
--set server.replicaCount=2 --set server.persistence.enabled=true
refuses "ingress with no web host" \
--set ingress.enabled=true
refuses "edge-routed API with an empty path list" \
--set ingress.enabled=true \
--set ingress.web.host=vantage.example.com \
--set ingress.grpc.enabled=false \
--set ingress.api.enabled=true \
--set 'ingress.api.paths=null'
refuses "gRPC ingress with no host" \
--set ingress.enabled=true \
--set ingress.web.host=vantage.example.com \
--set server.env.grpcHost=agents.example.com:443
refuses "gRPC ingress while grpcHost is still in-cluster" \
--set ingress.enabled=true \
--set ingress.web.host=vantage.example.com \
--set ingress.grpc.host=agents.example.com
- name: Read the chart version
id: chart
run: |
set -eu
VERSION="$(grep '^version:' "$CHART_DIR/Chart.yaml" | awk '{print $2}')"
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
echo "chart version is $VERSION"
# Chart.yaml is the source of truth for the version; the tag only
# says "publish this one". A mismatch is a mistake worth stopping
# for — the alternative is stamping the tag over Chart.yaml, which
# leaves the repository disagreeing with what was published.
- name: Check the tag matches Chart.yaml
if: startsWith(github.ref, 'refs/tags/chart/v')
run: |
set -eu
TAG_VERSION="${GITHUB_REF_NAME#chart/v}"
CHART_VERSION="${{ steps.chart.outputs.version }}"
if [ "$TAG_VERSION" != "$CHART_VERSION" ]; then
echo "tag chart/v$TAG_VERSION does not match Chart.yaml version $CHART_VERSION"
echo "bump version: in $CHART_DIR/Chart.yaml, or retag."
exit 1
fi
- name: Package
run: |
set -eu
mkdir -p dist
helm package "$CHART_DIR" --destination dist
ls -l dist
- name: Publish to the Gitea chart registry
if: startsWith(github.ref, 'refs/tags/chart/v')
env:
# github.server_url is this Gitea instance, so the registry
# host needs no variable of its own and cannot drift from it.
REGISTRY: ${{ github.server_url }}/api/packages/${{ github.repository_owner }}/helm/api/charts
# The same pair server-deploy.yml uses for `docker login`.
# RELEASE_TOKEN, not REGISTRY_PASSWORD: the latter is named in
# the docs but set by no workflow, and an unset secret becomes
# an empty password, which Gitea reports as "Failed to
# authenticate user" rather than as a missing credential.
REGISTRY_USER: ${{ secrets.REGISTRY_USER }}
REGISTRY_TOKEN: ${{ secrets.RELEASE_TOKEN }}
CHART_VERSION: ${{ steps.chart.outputs.version }}
run: |
set -eu
PKG="dist/vantage-${CHART_VERSION}.tgz"
test -f "$PKG"
# Checked explicitly, because the failure it prevents is a
# 401 that looks like a permissions problem on the token that
# was never sent.
if [ -z "${REGISTRY_USER}" ] || [ -z "${REGISTRY_TOKEN}" ]; then
echo "REGISTRY_USER or RELEASE_TOKEN is not set on this repository."
echo "RELEASE_TOKEN needs the write:package scope to publish a chart."
exit 1
fi
echo "publishing to ${REGISTRY} as ${REGISTRY_USER}"
# --fail-with-body so an HTTP error is a failed step with the
# server's explanation, rather than a green run that published
# nothing. A repeated version is rejected by the registry;
# that is the intended behaviour, not something to retry past.
curl --fail-with-body -sS \
--user "${REGISTRY_USER}:${REGISTRY_TOKEN}" \
-X POST \
--upload-file "$PKG" \
"$REGISTRY"
echo "published vantage ${CHART_VERSION}"
echo " helm repo add vantage ${{ github.server_url }}/api/packages/${{ github.repository_owner }}/helm"
echo " helm install vantage vantage/vantage --version ${CHART_VERSION}"
+17 -4
View File
@@ -77,15 +77,17 @@ jobs:
# generated pb is committed under server/, but a proto change
# that someone regenerates in the same push should not depend
# on that ordering.
flag server '^(server/|shared/|proto/|go\.work)'
flag server '^(server/|shared/|proto/|default_steps/|go\.work)'
flag sitesvc '^(sitesvc/|shared/|go\.work)'
flag admin '^(admin/|shared/|go\.work)'
# The three Next images use their own directory as the build
# context, so nothing outside it can affect them.
# The three Next images and the docs site use their own
# directory as the build context, so nothing outside it can
# affect them.
flag web '^web/'
flag site '^site/'
flag adminsite '^adminsite/'
flag docsite '^docsite/'
- name: Log in to registry
run: |
@@ -106,7 +108,6 @@ jobs:
run: |
IMAGE="${{ vars.DOCKER_HOST }}/${{ github.repository_owner }}/vantage/web:latest"
docker build \
--build-arg NEXT_PUBLIC_API_URL="${{ vars.API_URL }}" \
--build-arg NEXT_PUBLIC_HQ_URL="${{ vars.HQ_URL }}" \
-t "$IMAGE" \
-f web/Dockerfile web/
@@ -149,6 +150,18 @@ jobs:
--build-arg NEXT_PUBLIC_ADMIN_ENV="${{ vars.ADMIN_ENV }}" \
--build-arg NEXT_PUBLIC_PADDLE_CLIENT_TOKEN="${{ vars.PADDLE_CLIENT_TOKEN }}" \
--build-arg NEXT_PUBLIC_PADDLE_ENV="${{ vars.PADDLE_ENV }}" \
--build-arg NEXT_PUBLIC_SITE_URL="${{ vars.SITE_URL }}" \
-t "$IMAGE" \
-f adminsite/Dockerfile adminsite/
docker push "$IMAGE"
- name: Build and push docsite image
if: steps.changed.outputs.docsite == 'true'
run: |
IMAGE="${{ vars.DOCKER_HOST }}/${{ github.repository_owner }}/vantage/docsite:latest"
# DOCS_BASE_URL must match the proxy location that routes to
# this container and the directory the image serves from.
docker build \
-t "$IMAGE" \
-f docsite/Dockerfile docsite/
docker push "$IMAGE"
+4 -1
View File
@@ -3,6 +3,7 @@ dist
build
.env
.env.bck
.env.live
docs/*
!docs/superpowers/
.superpowers
@@ -11,4 +12,6 @@ installer/*.msi
installer/nssm.zip
installer/checksums-msi.txt
.next
*.tsbuildinfo
*.tsbuildinfo
graphify-out
docker-compose.live.yml
+1076
View File
File diff suppressed because it is too large Load Diff
+15 -10
View File
@@ -11,6 +11,7 @@ import (
"gitea.hostxtra.co.uk/mrhid6/vantage/admin/internal/billing"
"gitea.hostxtra.co.uk/mrhid6/vantage/admin/internal/catalogue"
"gitea.hostxtra.co.uk/mrhid6/vantage/admin/internal/db"
"gitea.hostxtra.co.uk/mrhid6/vantage/admin/internal/licensing"
"gitea.hostxtra.co.uk/mrhid6/vantage/admin/internal/models"
"gitea.hostxtra.co.uk/mrhid6/vantage/admin/internal/paddle"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/license"
@@ -245,19 +246,23 @@ func claimPlaceholderLink(c *gin.Context) {
placeholderID := inst.InstanceID
if _, err := db.Admin("admin_instances").UpdateOne(ctx,
bson.M{"instance_id": placeholderID},
bson.M{"$set": bson.M{
"instance_id": body.InstanceID,
"status": models.StatusActive,
"placeholder": false,
}}); err != nil {
bson.M{
"$set": bson.M{
"instance_id": body.InstanceID,
"status": models.StatusActive,
"placeholder": false,
},
"$addToSet": bson.M{"previous_instance_ids": placeholderID},
}); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
// Re-point the subscription from the placeholder id to the real UUID so
// billing.IssueForInstance (and every later webhook) finds it.
if _, err := db.Admin("subscriptions").UpdateMany(ctx,
bson.M{"instance_id": placeholderID},
bson.M{"$set": bson.M{"instance_id": body.InstanceID}}); err != nil {
// Re-point the subscription rows from the placeholder id to the real UUID so
// billing.IssueForInstance finds it, and rewrite Paddle's own copy of
// custom_data — written at checkout, it still names the placeholder, and every
// later event on this subscription is decoded from it.
if err := licensing.RepointSubscriptions(ctx, placeholderID, body.InstanceID, inst.AccountID); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
+40 -11
View File
@@ -2,6 +2,7 @@ package billing
import (
"context"
"errors"
"fmt"
"time"
@@ -14,6 +15,7 @@ import (
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/license"
"github.com/google/uuid"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
@@ -67,9 +69,21 @@ func handleSubscription(ctx context.Context, ev Event) error {
return fmt.Errorf("resolve items for subscription %s: %w", d.ID, err)
}
// Resolve BEFORE recording. A self-hosted subscription's custom_data is
// written at checkout and names the placeholder; the claim rewrote the
// instance's identity to the install's real UUID and patched Paddle, but that
// patch is best-effort and any event already in flight still carries the old
// id. Writing it straight through would revert the linked subscription row and
// then fail to find the instance, wedging every renewal.
instanceID, inst, err := resolveInstance(ctx, d.CustomData.InstanceID)
if err != nil {
return fmt.Errorf("subscription %s names unknown instance %s: %w",
d.ID, d.CustomData.InstanceID, err)
}
sub := models.Subscription{
AccountID: d.CustomData.AccountID,
InstanceID: d.CustomData.InstanceID,
InstanceID: instanceID,
PaddleSubscriptionID: d.ID,
Tier: match.Tier,
Term: match.Term,
@@ -88,13 +102,6 @@ func handleSubscription(ctx context.Context, ev Event) error {
bson.M{"$set": bson.M{"paddle_customer_id": d.CustomerID}})
}
var inst models.Instance
if err := db.Admin("admin_instances").FindOne(ctx,
bson.M{"instance_id": d.CustomData.InstanceID}).Decode(&inst); err != nil {
return fmt.Errorf("subscription %s names unknown instance %s: %w",
d.ID, d.CustomData.InstanceID, err)
}
// Placeholders are the payment-first path: the instance does not exist until
// this confirmed-payment event. A cloud placeholder is provisioned here and
// then issued (first term). A self-hosted placeholder has no UUID to bind to
@@ -116,6 +123,27 @@ func handleSubscription(ctx context.Context, ev Event) error {
return promoteAndIssue(ctx, &inst, match, reason)
}
// resolveInstance finds the instance a webhook's custom_data names, following the
// identity trail when the id is one a placeholder claim or a relink has since
// replaced. It returns the instance's CURRENT id, which is the only id anything
// else should be written against.
func resolveInstance(ctx context.Context, customDataID string) (string, models.Instance, error) {
var inst models.Instance
err := db.Admin("admin_instances").FindOne(ctx,
bson.M{"instance_id": customDataID}).Decode(&inst)
if err == nil {
return inst.InstanceID, inst, nil
}
if !errors.Is(err, mongo.ErrNoDocuments) {
return "", inst, err
}
if err := db.Admin("admin_instances").FindOne(ctx,
bson.M{"previous_instance_ids": customDataID}).Decode(&inst); err != nil {
return "", inst, err
}
return inst.InstanceID, inst, nil
}
// promoteAndIssue promotes desired→granted from the resolved match, then signs a
// licence from granted. This is the only promotion path other than the staff
// grant, and it exists because a webhook is a confirmed payment.
@@ -280,9 +308,10 @@ func billingEmailFor(ctx context.Context, accountID string) string {
// instanceNameFor is a best-effort display name for an email subject.
func instanceNameFor(ctx context.Context, instanceID string) string {
var inst models.Instance
if err := db.Admin("admin_instances").FindOne(ctx,
bson.M{"instance_id": instanceID}).Decode(&inst); err != nil || inst.Name == "" {
// Alias-aware: a cancellation can name a placeholder id, and "your instance"
// in place of the name the customer chose reads like the wrong email.
_, inst, err := resolveInstance(ctx, instanceID)
if err != nil || inst.Name == "" {
return "your instance"
}
return inst.Name
+57 -1
View File
@@ -4,11 +4,13 @@ import (
"context"
"errors"
"fmt"
"log"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/admin/internal/audit"
"gitea.hostxtra.co.uk/mrhid6/vantage/admin/internal/db"
"gitea.hostxtra.co.uk/mrhid6/vantage/admin/internal/models"
"gitea.hostxtra.co.uk/mrhid6/vantage/admin/internal/paddle"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/license"
"github.com/google/uuid"
"go.mongodb.org/mongo-driver/v2/bson"
@@ -57,6 +59,48 @@ func LinkInstance(ctx context.Context, accountID, instanceID, name string) (*mod
return &inst, nil
}
// RepointSubscriptions follows an instance identity rewrite: it moves every
// subscription row from the old id to the new one, then rewrites Paddle's copy
// of custom_data so future webhooks decode to the new id.
//
// The local rewrite is returned as an error — issuance reads the subscription
// back, so a half-moved row is worth failing on. The Paddle patch only logs: the
// customer must not be blocked from linking or relinking by an outbound API
// failure, and the caller has already recorded the old id in
// previous_instance_ids, which is what makes the webhook path correct whether or
// not the patch lands.
func RepointSubscriptions(ctx context.Context, oldID, newID, accountID string) error {
if _, err := db.Admin("subscriptions").UpdateMany(ctx,
bson.M{"instance_id": oldID},
bson.M{"$set": bson.M{"instance_id": newID}}); err != nil {
return fmt.Errorf("repoint %s -> %s: %w", oldID, newID, err)
}
cur, err := db.Admin("subscriptions").Find(ctx, bson.M{"instance_id": newID})
if err != nil {
log.Printf("repoint %s -> %s: read subscriptions: %v", oldID, newID, err)
return nil
}
var subs []models.Subscription
if err := cur.All(ctx, &subs); err != nil {
log.Printf("repoint %s -> %s: decode subscriptions: %v", oldID, newID, err)
return nil
}
for _, s := range subs {
if s.PaddleSubscriptionID == "" {
continue
}
// Paddle replaces the whole custom_data object on a PATCH, so account_id
// is sent alongside rather than dropped.
if err := paddle.Get().UpdateSubscriptionCustomData(ctx, s.PaddleSubscriptionID,
map[string]string{"account_id": accountID, "instance_id": newID}); err != nil {
log.Printf("repoint %s -> %s: patch custom_data on %s: %v",
oldID, newID, s.PaddleSubscriptionID, err)
}
}
return nil
}
// Relink moves a licence to a rebuilt server's new UUID.
//
// The replacement covers the REMAINING term, not a fresh one — relinking is not
@@ -96,13 +140,25 @@ func Relink(ctx context.Context, accountID, oldID, newID string, staff bool) (*m
if _, err := db.Admin("admin_instances").UpdateOne(ctx,
bson.M{"instance_id": oldID},
bson.M{"$set": bson.M{"instance_id": newID}, "$inc": bson.M{"relink_count": 1}}); err != nil {
bson.M{
"$set": bson.M{"instance_id": newID},
"$inc": bson.M{"relink_count": 1},
"$addToSet": bson.M{"previous_instance_ids": oldID},
}); err != nil {
if mongo.IsDuplicateKeyError(err) {
return nil, ErrAlreadyLinked
}
return nil, fmt.Errorf("relink: %w", err)
}
// A relink rewrites the instance's identity exactly as a placeholder claim
// does, so the same two things have to follow it: the subscription rows that
// named the old id, and Paddle's own copy of custom_data. Without this a
// renewal after a relink cannot find its instance and the term never extends.
if err := RepointSubscriptions(ctx, oldID, newID, accountID); err != nil {
return nil, err
}
actor := accountID
if staff {
actor = "staff"
+10
View File
@@ -77,6 +77,16 @@ func Run(ctx context.Context) error {
}
}
if lic.ExpiresAt.Add(reapAfter).Before(now) {
if _, err := db.Admin("admin_instances").DeleteOne(ctx, bson.M{"instance_id": inst.InstanceID}); err != nil {
log.Printf("lifecycle: delete instance %s: %v", inst.InstanceID, err)
}
if _, err := db.Admin("licenses").DeleteMany(ctx, bson.M{"instance_id": inst.InstanceID}); err != nil {
log.Printf("lifecycle: delete licenses for instance %s: %v", inst.InstanceID, err)
}
}
due := dueNotice(now, lic.ExpiresAt, inst.NoticesSent)
if due == "" {
continue
+69
View File
@@ -4,6 +4,7 @@ import (
"context"
"errors"
"log"
"strings"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/admin/internal/db"
@@ -12,6 +13,7 @@ import (
"github.com/google/uuid"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
// MigrateLegacyPlans re-keys the pre-spec-7 plan rows and MUST run before
@@ -210,6 +212,73 @@ func Backfill(ctx context.Context) error {
if err := backfillEntitlements(ctx); err != nil {
return err
}
// Pass 6: instances whose identity was rewritten before previous_instance_ids
// existed carry no trail, and Paddle's custom_data still names the id they
// were rewritten FROM — so their next webhook resolves to nothing. Both
// rewrites wrote an audit entry naming the old id, which is the only surviving
// record of it, so reconstruct the trail from those.
if err := backfillInstanceIDHistory(ctx); err != nil {
return err
}
return nil
}
// backfillInstanceIDHistory rebuilds previous_instance_ids from the audit entries
// the two identity rewrites leave behind: a placeholder claim
// ("instance.placeholder_linked", detail "from placeholder <id>") and a relink
// ("instance.relinked", detail "was <id>").
//
// $addToSet is what makes it idempotent, and it also means a chain of relinks
// accumulates rather than the last one winning. Entries are walked NEWEST first,
// matching on the current id or an already-recovered one: an instance relinked
// A→B→C answers to neither A nor B by the time this runs, so the C entry has to
// record B before the B entry has anything to attach A to.
func backfillInstanceIDHistory(ctx context.Context) error {
prefixes := map[string]string{
"instance.placeholder_linked": "from placeholder ",
"instance.relinked": "was ",
}
actions := make(bson.A, 0, len(prefixes))
for action := range prefixes {
actions = append(actions, action)
}
cur, err := db.Admin("admin_audit").Find(ctx,
bson.M{"action": bson.M{"$in": actions}},
options.Find().SetSort(bson.D{{Key: "created_at", Value: -1}}))
if err != nil {
return err
}
var entries []AuditEntry
if err := cur.All(ctx, &entries); err != nil {
return err
}
recorded := 0
for _, e := range entries {
prefix := prefixes[e.Action]
if e.Target == "" || !strings.HasPrefix(e.Detail, prefix) {
continue
}
oldID := strings.TrimSpace(strings.TrimPrefix(e.Detail, prefix))
if oldID == "" || oldID == e.Target {
continue
}
res, err := db.Admin("admin_instances").UpdateOne(ctx,
bson.M{"$or": bson.A{
bson.M{"instance_id": e.Target},
bson.M{"previous_instance_ids": e.Target},
}},
bson.M{"$addToSet": bson.M{"previous_instance_ids": oldID}})
if err != nil {
return err
}
recorded += int(res.ModifiedCount)
}
if recorded > 0 {
log.Printf("backfill: recovered %d instance id rewrites from the audit log", recorded)
}
return nil
}
+3 -1
View File
@@ -67,7 +67,8 @@ func (r CatalogueRow) Priced(env string) bool {
return false
}
// SeedCatalogue inserts the sixteen rows the four PAID plans need.
// SeedCatalogue inserts the twenty rows the four PAID plans need: a base, a
// server limit, and one row per feature key.
//
// The two Free plans get no rows at all, and that absence is what keeps Free
// outside Paddle: with nothing to price, no checkout can be built for it. Do not
@@ -84,6 +85,7 @@ func SeedCatalogue(ctx context.Context) error {
{Kind: KindLimit, Deployment: deployment, Tier: tier, LimitKey: LimitKeyServers},
{Kind: KindFeature, Deployment: deployment, Tier: tier, FeatureKey: license.FeatureConsole},
{Kind: KindFeature, Deployment: deployment, Tier: tier, FeatureKey: license.FeatureOIDC},
{Kind: KindFeature, Deployment: deployment, Tier: tier, FeatureKey: license.FeatureVulnScanning},
}
for _, r := range rows {
filter := bson.M{
+8
View File
@@ -147,6 +147,14 @@ type Instance struct {
// checkout has something to attach custom_data to, before the customer has
// pasted their install's real UUID. Cleared when the instance is linked.
Placeholder bool `bson:"placeholder,omitempty" json:"placeholder,omitempty"`
// PreviousInstanceIDs is every id this row has carried before its current one.
// A self-hosted row's identity is rewritten twice over its life — once when a
// paid placeholder is claimed, and again on each relink to a rebuilt server —
// and Paddle keeps its own copy of custom_data written at checkout. That copy
// is patched on each rewrite, but the patch is best-effort and any event
// already in flight still names an old id, so this is what lets a webhook
// resolve to the right instance instead of erroring as unknown.
PreviousInstanceIDs []string `bson:"previous_instance_ids,omitempty" json:"-"`
// PendingOwnerUserID is the customer_user who bought a paid-cloud placeholder,
// remembered so the confirmed-payment webhook can provision the instance with
// them as owner. Cleared once provisioned. Only ever set on a cloud placeholder.
+4
View File
@@ -25,6 +25,10 @@ type Client interface {
// immediately by Paddle. This is the one outbound mutation, used when a
// customer changes their server count or features on an existing plan.
UpdateSubscriptionItems(ctx context.Context, paddleSubscriptionID string, items []LineItem) error
// UpdateSubscriptionCustomData replaces a subscription's custom_data. Used
// when a self-hosted placeholder is claimed: the checkout attached the
// placeholder id, and every later webhook must name the real install UUID.
UpdateSubscriptionCustomData(ctx context.Context, paddleSubscriptionID string, data map[string]string) error
// PortalSession returns a customer-portal URL for managing billing.
PortalSession(ctx context.Context, paddleCustomerID string) (string, error)
// Env is "sandbox" or "production", the same value catalogue price lookups
+11
View File
@@ -99,6 +99,17 @@ func (c *httpClient) UpdateSubscriptionItems(ctx context.Context, subID string,
}, nil)
}
// UpdateSubscriptionCustomData patches custom_data only. Paddle replaces the
// whole object, so callers pass every key they want to keep.
func (c *httpClient) UpdateSubscriptionCustomData(ctx context.Context, subID string, data map[string]string) error {
if subID == "" {
return fmt.Errorf("paddle: empty subscription id")
}
return c.do(ctx, http.MethodPatch, "/subscriptions/"+subID, struct {
CustomData map[string]string `json:"custom_data"`
}{CustomData: data}, nil)
}
func (c *httpClient) PortalSession(ctx context.Context, customerID string) (string, error) {
if customerID == "" {
return "", fmt.Errorf("paddle: empty customer id")
+5
View File
@@ -26,6 +26,11 @@ ENV NEXT_PUBLIC_PADDLE_CLIENT_TOKEN=$NEXT_PUBLIC_PADDLE_CLIENT_TOKEN
ARG NEXT_PUBLIC_PADDLE_ENV=sandbox
ENV NEXT_PUBLIC_PADDLE_ENV=$NEXT_PUBLIC_PADDLE_ENV
# Marketing site origin. Signup lives there (/start), not here; empty renders no
# link at all rather than one that 404s.
ARG NEXT_PUBLIC_SITE_URL=
ENV NEXT_PUBLIC_SITE_URL=$NEXT_PUBLIC_SITE_URL
RUN npm run build
FROM node:26-alpine AS runner
@@ -12,6 +12,7 @@ import { StatePill } from "@/components/StatePill";
import { PageFrame, RailCard, RailFacts } from "@/components/PageFrame";
import { PageHeader } from "@/components/PageHeader";
import { formatDate, licenceState, limitLabel } from "@/lib/format";
import { featureLabel } from "@/lib/features";
export default function InstancePage() {
const id = String(useParams().id);
@@ -104,7 +105,11 @@ export default function InstancePage() {
},
{
label: "Features",
value: lic.features.join(", ") || "none",
// Labelled, not raw keys: this is
// the customer's own licence, and
// "vuln_scanning" is not a name
// anyone bought.
value: lic.features.map(featureLabel).join(", ") || "none",
},
]}
/>
@@ -6,25 +6,16 @@ import Link from "next/link";
import { useMutation, useQuery } from "@tanstack/react-query";
import { ApiError, api, lineItemsFor, type CatalogueRow, type CheckoutOptions, type Deployment, type Plan, type Term, type Tier } from "@/lib/api";
import { initPaddle, previewPrices, type PricePreview } from "@/lib/paddle";
import { featureDesc, featureLabel } from "@/lib/features";
/* Tiers in the order a customer reads them, cheapest first. */
const TIER_ORDER: Tier[] = ["free", "professional", "enterprise"];
const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
/* Human labels for feature keys. The catalogue names them by key; this is the
* one place the customer-facing wording lives. */
const FEATURE_LABEL: Record<string, string> = {
console: "Browser console",
oidc: "Single sign-on",
};
const FEATURE_DESC: Record<string, string> = {
console: "In-browser SSH, RDP and VNC sessions",
oidc: "OIDC sign-in for your whole team",
};
function featureLabel(key: string) {
return FEATURE_LABEL[key] ?? key;
}
/* Feature wording lives in lib/features.ts, shared with the staff
* configurator. It was duplicated here and there, and the two copies had
* already drifted. */
interface Choice {
tier: Tier;
@@ -294,7 +285,7 @@ export function PurchaseForm() {
{featureKeys.map((key) => {
const st = featureStateFor(plan, rows, options.env, choice.term, key);
return (
<Row key={key} title={featureLabel(key)} desc={FEATURE_DESC[key] ?? ""} dim={st === "absent"}>
<Row key={key} title={featureLabel(key)} desc={featureDesc(key)} dim={st === "absent"}>
{st === "included" ? (
<span className="text-[0.72rem] font-semibold uppercase tracking-[0.06em] text-valid">Included</span>
) : st === "absent" ? (
+1 -1
View File
@@ -10,7 +10,7 @@ export const metadata: Metadata = {
/*
* The masthead deliberately does NOT live here. It belongs to the authenticated
* layouts, so /login, /signup, /verify and /accept-invite stop rendering a bar
* layouts, so /login, /verify and /accept-invite stop rendering a bar
* whose navigation and account menu they cannot use.
*/
export default function RootLayout({ children }: { children: React.ReactNode }) {
+65 -35
View File
@@ -2,12 +2,13 @@
import { useRouter } from "next/navigation";
import { useState } from "react";
import Link from "next/link";
import { API_BASE, ApiError, NotConnected, api } from "@/lib/api";
import { NotConnectedPanel } from "@/components/NotConnected";
import { Button } from "@/components/Button";
import { Field } from "@/components/Field";
const SITE_URL = (process.env.NEXT_PUBLIC_SITE_URL ?? "").replace(/\/$/, "");
export default function LoginPage() {
const router = useRouter();
const [email, setEmail] = useState("");
@@ -44,46 +45,75 @@ export default function LoginPage() {
return (
<Main>
<h1 className="text-3xl">Sign in</h1>
<form onSubmit={submit} className="mt-6 grid gap-4">
<Field
label="Email"
type="email"
autoComplete="username"
required
value={email}
onChange={(e) => setEmail(e.target.value)}
/>
<Field
label="Password"
type="password"
autoComplete="current-password"
required
value={password}
onChange={(e) => setPassword(e.target.value)}
error={error ?? undefined}
/>
<label className="flex items-center gap-2 text-[0.82rem] text-ink-2">
<input
type="checkbox"
checked={staff}
onChange={(e) => setStaff(e.target.checked)}
{/* The masthead's lockup, unlinked: there is nowhere to go yet. */}
<div className="mb-7 flex flex-col items-center gap-2 text-center">
<span className="flex items-baseline gap-2 text-[1.5rem] font-extrabold tracking-[-0.02em]">
Vantage
<span className="font-mono text-[0.78rem] font-normal uppercase tracking-[0.14em] text-ink-3">
HQ
</span>
</span>
<h1 className="text-[1.16rem]">Sign in</h1>
<p className="font-mono text-[0.72rem] uppercase tracking-[0.1em] text-ink-3">
Licences · instances · billing
</p>
</div>
<div className="rounded border border-rule bg-panel p-6 shadow-[var(--shadow)]">
<form onSubmit={submit} className="grid gap-4">
<Field
label="Email"
type="email"
autoComplete="username"
required
value={email}
onChange={(e) => setEmail(e.target.value)}
/>
I work at Vantage
</label>
<div className="flex flex-wrap items-center gap-3">
<Button type="submit" disabled={busy}>
<Field
label="Password"
type="password"
autoComplete="current-password"
required
value={password}
onChange={(e) => setPassword(e.target.value)}
error={error ?? undefined}
/>
<label className="flex items-center gap-2 text-[0.82rem] text-ink-2">
<input
type="checkbox"
checked={staff}
onChange={(e) => setStaff(e.target.checked)}
className="accent-[var(--accent)]"
/>
I work at Vantage
</label>
<Button type="submit" disabled={busy} className="w-full justify-center">
{busy ? "Signing in…" : "Sign in"}
</Button>
<Link href="/signup" className="text-[0.82rem] text-accent underline">
Create an account for a self-hosted licence
</Link>
</div>
</form>
</form>
{SITE_URL && (
<>
<div className="my-5 h-px bg-rule-soft" />
{/* Signup lives on the marketing site's /start, not here. */}
<p className="text-center text-[0.82rem] text-ink-3">
No account?{" "}
<a href={`${SITE_URL}/start`} className="text-accent underline">
Create one
</a>
</p>
</>
)}
</div>
</Main>
);
}
function Main({ children }: { children: React.ReactNode }) {
return <main className="mx-auto max-w-rail px-5 py-12">{children}</main>;
return (
<main className="mx-auto flex min-h-screen w-full max-w-[26rem] flex-col justify-center px-5 py-12">
{children}
</main>
);
}
-92
View File
@@ -1,92 +0,0 @@
"use client";
import { useState } from "react";
import { ApiError, NotConnected, api } from "@/lib/api";
import { Button } from "@/components/Button";
import { Field } from "@/components/Field";
export default function SignupPage() {
const [form, setForm] = useState({ name: "", email: "", password: "", website: "" });
const [state, setState] = useState<"idle" | "busy" | "sent">("idle");
const [error, setError] = useState<string | null>(null);
async function submit(e: React.FormEvent) {
e.preventDefault();
setState("busy");
setError(null);
try {
await api.signup(form);
setState("sent");
} catch (err) {
setState("idle");
setError(
err instanceof NotConnected
? "The licensing service is not reachable from this page."
: err instanceof ApiError
? err.message
: "Could not create the account. Try again.",
);
}
}
return (
<main className="mx-auto max-w-rail px-5 py-12">
{state === "sent" ? (
<div className="grid max-w-xl gap-3">
<h1 className="text-3xl">Check your email</h1>
<p className="text-ink-2">
We sent a link to {form.email}. Open it to finish setting up your account
it expires in 24 hours. Nothing is created until you do.
</p>
</div>
) : (
<>
<h1 className="text-3xl">Create an account</h1>
<p className="mt-2 max-w-xl text-ink-2">
For self-hosted licences. If you run on our cloud, sign in with the same
details you use for your Vantage instance.
</p>
<form onSubmit={submit} className="mt-6 grid gap-4">
<Field
label="Organisation"
required
value={form.name}
onChange={(e) => setForm({ ...form, name: e.target.value })}
/>
<Field
label="Email"
type="email"
required
value={form.email}
onChange={(e) => setForm({ ...form, email: e.target.value })}
/>
<Field
label="Password"
type="password"
required
minLength={12}
hint="At least 12 characters."
value={form.password}
onChange={(e) => setForm({ ...form, password: e.target.value })}
error={error ?? undefined}
/>
{/* Honeypot: off-screen, unlabelled for humans, irresistible to bots. */}
<input
type="text"
name="website"
tabIndex={-1}
autoComplete="off"
aria-hidden="true"
value={form.website}
onChange={(e) => setForm({ ...form, website: e.target.value })}
className="absolute left-[-9999px] h-0 w-0"
/>
<Button type="submit" disabled={state === "busy"}>
{state === "busy" ? "Creating…" : "Create account"}
</Button>
</form>
</>
)}
</main>
);
}
+2 -1
View File
@@ -2,6 +2,7 @@
import { useMemo } from "react";
import type { CatalogueRow, Deployment, Plan, Term, Tier } from "@/lib/api";
import { featureLabel } from "@/lib/features";
export interface PlanChoice {
tier: Tier;
@@ -159,7 +160,7 @@ export default function PlanConfigurator({
})
}
/>
<span>{key === "console" ? "Browser console" : "Single sign-on"}</span>
<span>{featureLabel(key)}</span>
<span className="text-[0.72rem] text-ink-3">
{priced ? "paid add-on" : "included"}
</span>
-2
View File
@@ -296,8 +296,6 @@ export const api = {
staffLogin: (email: string, password: string) =>
post<Session>("/auth/staff/login", { email, password }),
logout: () => post<{ ok: boolean }>("/auth/logout"),
signup: (payload: { name: string; email: string; password: string; website?: string }) =>
post<{ pending: boolean }>("/auth/signup", payload),
verify: (token: string) =>
req<{ verified: boolean; needs_password?: boolean }>(
`/auth/verify?token=${encodeURIComponent(token)}`,
+27
View File
@@ -0,0 +1,27 @@
/* Human wording for licence feature keys.
*
* One place, because there were two and they disagreed: the staff configurator
* rendered every key that was not "console" as "Single sign-on", so adding a
* third feature silently mislabelled the checkbox that grants it. A map with a
* fallback degrades to the raw key, which is ugly but never wrong.
*
* Keys must match shared/license/license.go. */
export const FEATURE_LABEL: Record<string, string> = {
console: "Browser console",
oidc: "Single sign-on",
vuln_scanning: "Vulnerability scanning",
};
export const FEATURE_DESC: Record<string, string> = {
console: "In-browser SSH, RDP and VNC sessions",
oidc: "OIDC sign-in for your whole team",
vuln_scanning: "Package inventory matched against distribution security advisories",
};
export function featureLabel(key: string): string {
return FEATURE_LABEL[key] ?? key;
}
export function featureDesc(key: string): string {
return FEATURE_DESC[key] ?? "";
}
-20
View File
@@ -1,20 +0,0 @@
package config
import (
"runtime"
"strings"
"testing"
)
func TestConfigDirByOS(t *testing.T) {
d := ConfigDir()
if runtime.GOOS == "windows" {
if !strings.Contains(strings.ToLower(d), "programdata") {
t.Fatalf("windows config dir = %q, want ProgramData path", d)
}
} else {
if d != "/etc/vantage" {
t.Fatalf("unix config dir = %q, want /etc/vantage", d)
}
}
}
+37 -2
View File
@@ -80,7 +80,11 @@ func (c *Client) Register(serverID, preRegToken, hostname, ipAddress, osInfo str
return resp.AgentToken, nil
}
func (c *Client) SyncKeys(serverID, agentToken, version string) ([]string, error) {
// SyncKeys returns the whole response rather than just the keys: the poll now
// also carries CollectPackages, and a second RPC purely to learn one boolean
// would be a message every 30 seconds for a value that changes at most when a
// licence does.
func (c *Client) SyncKeys(serverID, agentToken, version string) (*pb.SyncResponse, error) {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
@@ -92,7 +96,34 @@ func (c *Client) SyncKeys(serverID, agentToken, version string) ([]string, error
if err != nil {
return nil, err
}
return resp.PublicKeys, nil
return resp, nil
}
// ReportPackages sends a package report and returns whether the server wants
// the full list. Given a longer deadline than the other unary calls because the
// full body is ~150KB on a slow link.
func (c *Client) ReportPackages(req *pb.ReportPackagesRequest) (bool, error) {
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
resp, err := c.client.ReportPackages(ctx, req)
if err != nil {
return false, err
}
return resp.NeedFull, nil
}
// ReportWorkloads sends a workload report and returns whether the server wants
// the full list.
func (c *Client) ReportWorkloads(req *pb.ReportWorkloadsRequest) (bool, error) {
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
resp, err := c.client.ReportWorkloads(ctx, req)
if err != nil {
return false, err
}
return resp.NeedFull, nil
}
func (c *Client) UploadGeneratedKey(serverID, agentToken, publicKey, privateKey, label string) (string, error) {
@@ -151,3 +182,7 @@ func (c *Client) ReportChecks(serverID, agentToken string, results []pb.CheckRes
func (c *Client) CommandStream(ctx context.Context) (pb.Vantage_CommandStreamClient, error) {
return c.client.CommandStream(ctx)
}
func (c *Client) ProxyStream(ctx context.Context) (pb.Vantage_ProxyStreamClient, error) {
return c.client.ProxyStream(ctx)
}
+143
View File
@@ -30,6 +30,45 @@ type SyncRequest struct {
type SyncResponse struct {
PublicKeys []string `json:"public_keys"`
// CollectPackages tells the agent whether this instance's licence grants
// vulnerability scanning. Absent decodes as false, which is the safe
// direction: an older server leaves agents collecting nothing.
CollectPackages bool `json:"collect_packages,omitempty"`
}
type OSRelease struct {
Family string `json:"family"`
// VersionId is not optional: Ubuntu 22.04 and 24.04 publish different fixed
// versions for the same CVE, so a scan without it is guesswork.
VersionId string `json:"version_id"`
Arch string `json:"arch,omitempty"`
}
type InstalledPackage struct {
Name string `json:"name"`
Version string `json:"version"`
Epoch int32 `json:"epoch,omitempty"`
Arch string `json:"arch,omitempty"`
// SourceName is what the Debian and Ubuntu feeds are keyed on: one advisory
// against "openssl" covers libssl3, openssl and libssl-dev.
SourceName string `json:"source_name,omitempty"`
}
// ReportPackagesRequest carries a server's installed package set.
//
// The agent calls twice at most: first with Packages empty, offering only the
// hash. If the server already holds it, NeedFull is false and the ~150KB body
// is never sent.
type ReportPackagesRequest struct {
ServerId string `json:"server_id"`
AgentToken string `json:"agent_token"`
Hash string `json:"hash"`
Os OSRelease `json:"os"`
Packages []InstalledPackage `json:"packages,omitempty"`
}
type ReportPackagesResponse struct {
NeedFull bool `json:"need_full"`
}
type UploadKeyRequest struct {
@@ -131,6 +170,32 @@ type ReportChecksResponse struct{}
type ApplyUpdatesCmd struct{}
type OpenProxyCmd struct {
ProxyId string `json:"proxy_id"`
Port uint32 `json:"port"`
}
type ProxyOpen struct {
ServerId string `json:"server_id"`
AgentToken string `json:"agent_token"`
ProxyId string `json:"proxy_id"`
}
type ProxyClose struct {
Reason string `json:"reason,omitempty"`
}
type ProxyClientMsg struct {
Open *ProxyOpen `json:"open,omitempty"`
Data []byte `json:"data,omitempty"`
Close *ProxyClose `json:"close,omitempty"`
}
type ProxyServerMsg struct {
Data []byte `json:"data,omitempty"`
Close *ProxyClose `json:"close,omitempty"`
}
type ServerCommand struct {
CommandId string `json:"command_id"`
GenerateKey *GenerateKeyCmd `json:"generate_key,omitempty"`
@@ -139,8 +204,19 @@ type ServerCommand struct {
ApplyUpdates *ApplyUpdatesCmd `json:"apply_updates,omitempty"`
RunStep *RunStepCmd `json:"run_step,omitempty"`
CleanupWorkspace *CleanupWorkspaceCmd `json:"cleanup_workspace,omitempty"`
OpenProxy *OpenProxyCmd `json:"open_proxy,omitempty"`
Ping *PingCmd `json:"ping,omitempty"`
RefreshWorkloads *RefreshWorkloadsCmd `json:"refresh_workloads,omitempty"`
ControlWorkload *ControlWorkloadCmd `json:"control_workload,omitempty"`
WorkloadLogs *WorkloadLogsCmd `json:"workload_logs,omitempty"`
}
// PingCmd is a server-originated liveness beat. It carries nothing and expects
// no reply: its arrival is the entire message. See the .proto for why gRPC
// keepalive is not sufficient on its own.
type PingCmd struct{}
type CleanupWorkspaceCmd struct {
@@ -171,6 +247,8 @@ type AgentMessage struct {
Result *CommandResult `json:"result,omitempty"`
StepResult *StepResult `json:"step_result,omitempty"`
StepOutput *StepOutputChunk `json:"step_output,omitempty"`
WorkloadLogsResult *WorkloadLogsResult `json:"workload_logs_result,omitempty"`
}
type AgentReady struct{}
@@ -254,15 +332,63 @@ func (s *keyManagerCommandStreamServer) Recv() (*AgentMessage, error) {
return m, nil
}
type Vantage_ProxyStreamServer interface {
Send(*ProxyServerMsg) error
Recv() (*ProxyClientMsg, error)
grpc.ServerStream
}
type vantageProxyStreamServer struct {
grpc.ServerStream
}
func (s *vantageProxyStreamServer) Send(m *ProxyServerMsg) error {
return s.ServerStream.SendMsg(m)
}
func (s *vantageProxyStreamServer) Recv() (*ProxyClientMsg, error) {
m := new(ProxyClientMsg)
if err := s.ServerStream.RecvMsg(m); err != nil {
return nil, err
}
return m, nil
}
type Vantage_ProxyStreamClient interface {
Send(*ProxyClientMsg) error
Recv() (*ProxyServerMsg, error)
CloseSend() error
grpc.ClientStream
}
type vantageProxyStreamClient struct {
grpc.ClientStream
}
func (c *vantageProxyStreamClient) Send(m *ProxyClientMsg) error {
return c.ClientStream.SendMsg(m)
}
func (c *vantageProxyStreamClient) Recv() (*ProxyServerMsg, error) {
m := new(ProxyServerMsg)
if err := c.ClientStream.RecvMsg(m); err != nil {
return nil, err
}
return m, nil
}
type VantageClient interface {
Register(ctx context.Context, in *RegisterRequest, opts ...grpc.CallOption) (*RegisterResponse, error)
SyncKeys(ctx context.Context, in *SyncRequest, opts ...grpc.CallOption) (*SyncResponse, error)
UploadGeneratedKey(ctx context.Context, in *UploadKeyRequest, opts ...grpc.CallOption) (*UploadKeyResponse, error)
ReportUpdates(ctx context.Context, in *ReportUpdatesRequest, opts ...grpc.CallOption) (*ReportUpdatesResponse, error)
ReportPackages(ctx context.Context, in *ReportPackagesRequest, opts ...grpc.CallOption) (*ReportPackagesResponse, error)
ReportWorkloads(ctx context.Context, in *ReportWorkloadsRequest, opts ...grpc.CallOption) (*ReportWorkloadsResponse, error)
ReportInventory(ctx context.Context, in *InventoryReport, opts ...grpc.CallOption) (*InventoryReportResponse, error)
SyncMonitors(ctx context.Context, in *SyncMonitorsRequest, opts ...grpc.CallOption) (*SyncMonitorsResponse, error)
ReportChecks(ctx context.Context, in *ReportChecksRequest, opts ...grpc.CallOption) (*ReportChecksResponse, error)
CommandStream(ctx context.Context, opts ...grpc.CallOption) (Vantage_CommandStreamClient, error)
ProxyStream(ctx context.Context, opts ...grpc.CallOption) (Vantage_ProxyStreamClient, error)
}
type UnimplementedVantageServer struct{}
@@ -317,6 +443,14 @@ func (c *keyManagerClient) ReportUpdates(ctx context.Context, in *ReportUpdatesR
return out, nil
}
func (c *keyManagerClient) ReportPackages(ctx context.Context, in *ReportPackagesRequest, opts ...grpc.CallOption) (*ReportPackagesResponse, error) {
out := new(ReportPackagesResponse)
if err := c.cc.Invoke(ctx, "/vantage.v1.Vantage/ReportPackages", in, out, opts...); err != nil {
return nil, err
}
return out, nil
}
func (c *keyManagerClient) ReportInventory(ctx context.Context, in *InventoryReport, opts ...grpc.CallOption) (*InventoryReportResponse, error) {
out := new(InventoryReportResponse)
if err := c.cc.Invoke(ctx, "/vantage.v1.Vantage/ReportInventory", in, out, opts...); err != nil {
@@ -349,3 +483,12 @@ func (c *keyManagerClient) CommandStream(ctx context.Context, opts ...grpc.CallO
}
return &vantageCommandStreamClient{stream}, nil
}
func (c *keyManagerClient) ProxyStream(ctx context.Context, opts ...grpc.CallOption) (Vantage_ProxyStreamClient, error) {
desc := &grpc.StreamDesc{StreamName: "ProxyStream", ServerStreams: true, ClientStreams: true}
stream, err := c.cc.NewStream(ctx, desc, "/vantage.v1.Vantage/ProxyStream", opts...)
if err != nil {
return nil, err
}
return &vantageProxyStreamClient{stream}, nil
}
+80
View File
@@ -0,0 +1,80 @@
package pb
import (
"context"
"google.golang.org/grpc"
)
// Workload registry messages. Hand-written like the rest of this package: the
// .proto is the contract, this file is the Go side of it, and the two must be
// changed together.
// Workload is one container or one systemd unit.
type Workload struct {
Kind string `json:"kind"`
Id string `json:"id"`
Name string `json:"name"`
State string `json:"state"`
Health string `json:"health,omitempty"`
Image string `json:"image,omitempty"`
Stack string `json:"stack,omitempty"`
Ports []string `json:"ports,omitempty"`
Restarts int32 `json:"restarts,omitempty"`
StartedAt string `json:"started_at,omitempty"` // RFC3339, empty when not running
Protected bool `json:"protected,omitempty"`
}
// ReportWorkloadsRequest carries what a server is running.
//
// Offer-then-send, the same handshake as ReportPackages: the agent calls once
// with Workloads empty, and resends with the body only if NeedFull is set.
type ReportWorkloadsRequest struct {
ServerId string `json:"server_id"`
AgentToken string `json:"agent_token"`
Hash string `json:"hash"`
DockerOk bool `json:"docker_ok"`
DockerError string `json:"docker_error,omitempty"`
SystemdOk bool `json:"systemd_ok"`
SystemdError string `json:"systemd_error,omitempty"`
Workloads []Workload `json:"workloads,omitempty"` // empty on the offer call
// Full marks the second call. It is not inferred from an empty Workloads
// slice: a host running nothing sends an empty list as its full report.
Full bool `json:"full,omitempty"`
}
type ReportWorkloadsResponse struct {
NeedFull bool `json:"need_full"`
}
// RefreshWorkloadsCmd carries no payload back. It makes the agent report
// immediately through ReportWorkloads, so there is exactly one writer for the
// server_workloads collection rather than two arriving by different routes.
type RefreshWorkloadsCmd struct{}
type ControlWorkloadCmd struct {
Kind string `json:"kind"`
Id string `json:"id"`
Action string `json:"action"` // start | stop | restart
}
type WorkloadLogsCmd struct {
Kind string `json:"kind"`
Id string `json:"id"`
Tail int32 `json:"tail,omitempty"`
}
type WorkloadLogsResult struct {
CommandId string `json:"command_id"`
Text string `json:"text,omitempty"`
Truncated bool `json:"truncated,omitempty"`
Error string `json:"error,omitempty"`
}
func (c *keyManagerClient) ReportWorkloads(ctx context.Context, in *ReportWorkloadsRequest, opts ...grpc.CallOption) (*ReportWorkloadsResponse, error) {
out := new(ReportWorkloadsResponse)
if err := c.cc.Invoke(ctx, "/vantage.v1.Vantage/ReportWorkloads", in, out, opts...); err != nil {
return nil, err
}
return out, nil
}
+63
View File
@@ -0,0 +1,63 @@
package packages
import (
"bufio"
"errors"
"io"
"os"
"runtime"
"strings"
)
// OSRelease identifies the distribution well enough to select an advisory
// feed. VersionID is not optional: Ubuntu 22.04 and 24.04 publish different
// fixed versions for the same CVE.
type OSRelease struct {
Family string
VersionID string
Arch string
}
// ParseOSRelease reads the os-release format: KEY=value, one per line, with
// values optionally quoted, and # comments.
//
// The quote stripping handles both ID=ubuntu and ID="rocky", which real
// distributions both emit.
func ParseOSRelease(r io.Reader) (OSRelease, error) {
out := OSRelease{Arch: runtime.GOARCH}
sc := bufio.NewScanner(r)
for sc.Scan() {
line := strings.TrimSpace(sc.Text())
if line == "" || strings.HasPrefix(line, "#") {
continue
}
key, val, ok := strings.Cut(line, "=")
if !ok {
continue
}
val = strings.Trim(strings.TrimSpace(val), `"'`)
switch strings.TrimSpace(key) {
case "ID":
out.Family = strings.ToLower(val)
case "VERSION_ID":
out.VersionID = val
}
}
if err := sc.Err(); err != nil {
return OSRelease{}, err
}
if out.Family == "" {
return OSRelease{}, errors.New("os-release has no ID")
}
return out, nil
}
// DetectOS reads /etc/os-release.
func DetectOS() (OSRelease, error) {
f, err := os.Open("/etc/os-release")
if err != nil {
return OSRelease{}, err
}
defer f.Close()
return ParseOSRelease(f)
}
+73
View File
@@ -0,0 +1,73 @@
package packages
import (
"context"
"fmt"
"os/exec"
"runtime"
"time"
)
const collectTimeout = 2 * time.Minute
// Collect enumerates installed packages. Linux only: Windows agents are
// second-class by design, and vulnerability scanning there needs a different
// source, a different collector and a different matcher, all out of scope.
//
// The format strings below are raw string literals on purpose. The "\t" and
// "\n" reach dpkg-query and rpm as two characters each, and those tools do the
// interpreting themselves — Go must not consume the escapes first.
func Collect() (OSRelease, []Package, error) {
if runtime.GOOS != "linux" {
return OSRelease{}, nil, fmt.Errorf("package collection is linux-only, got %s", runtime.GOOS)
}
osrel, err := DetectOS()
if err != nil {
return OSRelease{}, nil, fmt.Errorf("detect os: %w", err)
}
ctx, cancel := context.WithTimeout(context.Background(), collectTimeout)
defer cancel()
switch {
case have("dpkg-query"):
out, err := run(ctx, "dpkg-query", "-W", "-f",
`${Package}\t${Version}\t${Architecture}\t${source:Package}\n`)
if err != nil {
return osrel, nil, err
}
return osrel, ParseDpkg(out), nil
case have("rpm"):
out, err := run(ctx, "rpm", "-qa", "--qf",
`%{NAME}\t%{EPOCH}\t%{VERSION}-%{RELEASE}\t%{ARCH}\t%{SOURCERPM}\n`)
if err != nil {
return osrel, nil, err
}
return osrel, ParseRPM(out), nil
case have("apk"):
out, err := run(ctx, "apk", "info", "-v")
if err != nil {
return osrel, nil, err
}
return osrel, ParseAPK(out), nil
default:
return osrel, nil, fmt.Errorf("no supported package manager found")
}
}
func have(bin string) bool {
_, err := exec.LookPath(bin)
return err == nil
}
func run(ctx context.Context, name string, args ...string) (string, error) {
out, err := exec.CommandContext(ctx, name, args...).Output()
if err != nil {
return "", fmt.Errorf("%s: %w", name, err)
}
return string(out), nil
}
+146
View File
@@ -0,0 +1,146 @@
package packages
import (
"crypto/sha256"
"encoding/hex"
"sort"
"strconv"
"strings"
)
// Package is one installed package as the distribution reports it. Version is
// the distribution's own version string, verbatim — never normalised, because
// the advisory feeds are keyed on exactly this form.
type Package struct {
Name string
Version string
Epoch int
Arch string
SourceName string
}
// ParseDpkg reads tab-separated output of
// dpkg-query -W -f '${Package}\t${Version}\t${Architecture}\t${source:Package}\n'
//
// SourceName is why the fourth column is requested at all: Debian and Ubuntu
// advisories are keyed on the SOURCE package, so one CVE against "openssl"
// covers the binaries libssl3, openssl and libssl-dev. Matching on binary name
// alone finds one of the three.
func ParseDpkg(out string) []Package {
var pkgs []Package
for _, line := range strings.Split(out, "\n") {
if strings.TrimSpace(line) == "" {
continue
}
f := strings.Split(line, "\t")
if len(f) < 3 {
continue
}
p := Package{Name: f[0], Version: f[1], Arch: f[2]}
if len(f) > 3 && f[3] != "" {
p.SourceName = f[3]
} else {
p.SourceName = p.Name
}
pkgs = append(pkgs, p)
}
return pkgs
}
// ParseRPM reads tab-separated output of
// rpm -qa --qf '%{NAME}\t%{EPOCH}\t%{VERSION}-%{RELEASE}\t%{ARCH}\t%{SOURCERPM}\n'
func ParseRPM(out string) []Package {
var pkgs []Package
for _, line := range strings.Split(out, "\n") {
if strings.TrimSpace(line) == "" {
continue
}
f := strings.Split(line, "\t")
if len(f) < 4 {
continue
}
epoch := 0
// rpm prints "(none)" rather than omitting the field when a package has
// no epoch. That must become 0, not fail the line.
if f[1] != "" && f[1] != "(none)" {
if n, err := strconv.Atoi(f[1]); err == nil {
epoch = n
}
}
p := Package{Name: f[0], Epoch: epoch, Version: f[2], Arch: f[3]}
if len(f) > 4 {
p.SourceName = srcRPMName(f[4])
}
if p.SourceName == "" {
p.SourceName = p.Name
}
pkgs = append(pkgs, p)
}
return pkgs
}
// srcRPMName reduces "openssl-3.0.7-24.el9.src.rpm" to "openssl" by dropping
// the trailing ".src.rpm" and then the version and release segments, which are
// the last two hyphen-separated fields.
func srcRPMName(s string) string {
s = strings.TrimSuffix(s, ".src.rpm")
parts := strings.Split(s, "-")
if len(parts) <= 2 {
return s
}
return strings.Join(parts[:len(parts)-2], "-")
}
// ParseAPK reads "apk info -v" output: one "name-version-rREV" per line.
// Alpine has no separate source package, so SourceName mirrors Name.
func ParseAPK(out string) []Package {
var pkgs []Package
for _, line := range strings.Split(out, "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
name, version := splitAPK(line)
if name == "" {
continue
}
pkgs = append(pkgs, Package{Name: name, Version: version, SourceName: name})
}
return pkgs
}
// splitAPK finds the version boundary from the RIGHT. The version is always the
// last two hyphen-separated fields ("<version>-r<rev>"), which is reliable
// where scanning from the left is not: package names legitimately contain
// digits and underscores, so "musl" in "musl-1.2.4_git20230717-r4" cannot be
// found by looking for the first digit.
func splitAPK(s string) (name, version string) {
last := strings.LastIndex(s, "-")
if last <= 0 {
return "", ""
}
prev := strings.LastIndex(s[:last], "-")
if prev <= 0 {
return "", ""
}
return s[:prev], s[prev+1:]
}
// Hash fingerprints a package set so an unchanged set never has to be sent.
//
// It sorts first: the ordering of dpkg or rpm output is not guaranteed stable,
// and an ordering-sensitive hash would resend the full ~150KB list every hour
// for no reason — a cost visible only as traffic.
func Hash(pkgs []Package) string {
lines := make([]string, 0, len(pkgs))
for _, p := range pkgs {
lines = append(lines, p.Name+"\x00"+strconv.Itoa(p.Epoch)+"\x00"+p.Version+"\x00"+p.Arch)
}
sort.Strings(lines)
h := sha256.New()
for _, l := range lines {
h.Write([]byte(l))
h.Write([]byte("\n"))
}
return hex.EncodeToString(h.Sum(nil))
}
+114
View File
@@ -0,0 +1,114 @@
// Package agentproxy relays a single TCP connection between a local service and
// the control plane, so a control plane that cannot route to this host's network
// can still open a console session.
//
// The dial host is hardcoded to loopback. The control plane supplies only a
// port, and nothing in this package can be made to dial anywhere else.
package agentproxy
import (
"errors"
"fmt"
"io"
"net"
"strconv"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/grpc/pb"
)
const (
loopbackHost = "127.0.0.1"
chunkSize = 32 * 1024
dialTimeout = 10 * time.Second
)
// Stream is the agent's half of a ProxyStream.
type Stream interface {
Send(*pb.ProxyClientMsg) error
Recv() (*pb.ProxyServerMsg, error)
CloseSend() error
}
// Open dials the local port, announces itself on the stream, and relays until
// either side ends. A refused dial is reported as an explicit close so the
// operator sees a reason rather than a hang.
func Open(stream Stream, serverID, agentToken, proxyID string, port uint32) error {
conn, dialErr := net.DialTimeout("tcp",
net.JoinHostPort(loopbackHost, strconv.Itoa(int(port))), dialTimeout)
if err := stream.Send(&pb.ProxyClientMsg{Open: &pb.ProxyOpen{
ServerId: serverID,
AgentToken: agentToken,
ProxyId: proxyID,
}}); err != nil {
if conn != nil {
_ = conn.Close()
}
return fmt.Errorf("send open: %w", err)
}
if dialErr != nil {
_ = stream.Send(&pb.ProxyClientMsg{Close: &pb.ProxyClose{
Reason: "dial_refused: " + dialErr.Error(),
}})
_ = stream.CloseSend()
return fmt.Errorf("dial 127.0.0.1:%d: %w", port, dialErr)
}
defer conn.Close()
return relay(conn, stream)
}
func relay(conn net.Conn, stream Stream) error {
errCh := make(chan error, 2)
// local service -> control plane
go func() {
buf := make([]byte, chunkSize)
for {
n, err := conn.Read(buf)
if n > 0 {
chunk := make([]byte, n)
copy(chunk, buf[:n])
if sendErr := stream.Send(&pb.ProxyClientMsg{Data: chunk}); sendErr != nil {
errCh <- sendErr
return
}
}
if err != nil {
errCh <- err
return
}
}
}()
// control plane -> local service
go func() {
for {
msg, err := stream.Recv()
if err != nil {
errCh <- err
return
}
if msg.Close != nil {
errCh <- fmt.Errorf("server closed relay: %s", msg.Close.Reason)
return
}
if len(msg.Data) > 0 {
if _, err := conn.Write(msg.Data); err != nil {
errCh <- err
return
}
}
}
}()
err := <-errCh
_ = conn.Close()
_ = stream.CloseSend()
if errors.Is(err, io.EOF) {
return nil
}
return err
}
+125
View File
@@ -0,0 +1,125 @@
package agentsync
import (
"context"
"log"
"runtime"
"sync"
"sync/atomic"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/config"
grpcclient "gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/grpc"
"gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/grpc/pb"
"gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/packages"
)
// collectPackagesFlag is written by the 30s key poll and read by the hourly
// package loop — two different goroutines, hence the atomic.
//
// It defaults to false, so an agent that has not yet completed a poll, or is
// talking to a server too old to send the field, collects nothing. Off is the
// safe default: collecting without a licence costs the customer storage they
// are not paying for.
var collectPackagesFlag atomic.Bool
// firstPoll closes once a SyncKeys response has set the flag above.
//
// Without it the boot-time package report loses a race it can only lose: the
// hourly loop starts before the first poll, reads a flag that is still false by
// construction, and skips — so a freshly installed agent reports no packages for
// an hour and the server shows nothing to scan.
// How long the boot package report waits for that first poll. Two poll
// intervals plus slack: long enough to cover one failed attempt, short enough
// that a dead control plane does not hold the OS-update report hostage.
const firstPollWait = 90 * time.Second
var (
firstPoll = make(chan struct{})
firstPollOnce sync.Once
)
func markFirstPoll() { firstPollOnce.Do(func() { close(firstPoll) }) }
// waitFirstPoll blocks until the flag is known, or gives up. The wait is
// bounded because this loop also reports OS updates, which do not depend on the
// flag at all — a control plane that cannot be polled must not silence those too.
func waitFirstPoll(ctx context.Context, limit time.Duration) {
t := time.NewTimer(limit)
defer t.Stop()
select {
case <-firstPoll:
case <-t.C:
log.Printf("package collection: no SyncKeys response within %s, collecting nothing this round", limit)
case <-ctx.Done():
}
}
func collectPackagesEnabled() bool { return collectPackagesFlag.Load() }
// reportPackages offers a hash of the installed package set and sends the full
// list only if the server does not already hold it.
//
// It runs on the same hourly cadence as the update check because a package set
// changes on roughly the same schedule, and reusing that loop means one timer
// rather than two.
func reportPackages(client *grpcclient.Client, cfg *config.Config) {
if runtime.GOOS != "linux" {
return
}
if !collectPackagesEnabled() {
return
}
osrel, pkgs, err := packages.Collect()
if err != nil {
log.Printf("package collection error: %v", err)
return
}
pbOS := pb.OSRelease{
Family: osrel.Family,
VersionId: osrel.VersionID,
Arch: osrel.Arch,
}
hash := packages.Hash(pkgs)
// The offer: hash only, no body. On an unchanged host this is the whole
// exchange, which is the point of the handshake.
needFull, err := client.ReportPackages(&pb.ReportPackagesRequest{
ServerId: cfg.ServerID,
AgentToken: cfg.AgentToken,
Hash: hash,
Os: pbOS,
})
if err != nil {
log.Printf("ReportPackages offer error: %v", err)
return
}
if !needFull {
return
}
pbPkgs := make([]pb.InstalledPackage, len(pkgs))
for i, p := range pkgs {
pbPkgs[i] = pb.InstalledPackage{
Name: p.Name,
Version: p.Version,
Epoch: int32(p.Epoch),
Arch: p.Arch,
SourceName: p.SourceName,
}
}
if _, err := client.ReportPackages(&pb.ReportPackagesRequest{
ServerId: cfg.ServerID,
AgentToken: cfg.AgentToken,
Hash: hash,
Os: pbOS,
Packages: pbPkgs,
}); err != nil {
log.Printf("ReportPackages full error: %v", err)
return
}
log.Printf("reported %d installed packages", len(pkgs))
}
+211 -18
View File
@@ -24,6 +24,7 @@ import (
"gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/inventory"
"gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/keys"
"gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/monitors"
agentproxy "gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/proxy"
"gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/updates"
)
@@ -69,6 +70,8 @@ func Run(ctx context.Context, cfg *config.Config, version string) error {
go runInventory(ctx, cfg)
go runWorkloads(ctx, cfg)
go monitors.Run(ctx, cfg)
ticker := time.NewTicker(cfg.PollInterval)
@@ -91,11 +94,19 @@ func Run(ctx context.Context, cfg *config.Config, version string) error {
}
func poll(client *grpcclient.Client, cfg *config.Config, version string) error {
desired, err := client.SyncKeys(cfg.ServerID, cfg.AgentToken, version)
resp, err := client.SyncKeys(cfg.ServerID, cfg.AgentToken, version)
if err != nil {
return fmt.Errorf("SyncKeys: %w", err)
}
// Stored atomically: the hourly package loop reads this from another
// goroutine. Absent on the wire decodes as false, so an older server leaves
// collection off rather than on.
collectPackagesFlag.Store(resp.CollectPackages)
markFirstPoll()
desired := resp.PublicKeys
if runtime.GOOS != "linux" {
return nil
}
@@ -117,9 +128,36 @@ func poll(client *grpcclient.Client, cfg *config.Config, version string) error {
return nil
}
// How long a command stream must survive before it counts as having worked.
// Past this, the next drop is treated as a fresh incident rather than as the
// continuation of a run of failures.
const streamHealthyAfter = time.Minute
// Stream staleness. The server beats every 20s, so 70s tolerates three missed
// beats before the stream is written off — high enough that a slow network or a
// briefly busy server does not cost a reconnect, low enough that an agent is
// not uncommandable for minutes after a control-plane restart.
const (
streamStaleAfter = 70 * time.Second
streamStaleCheck = 10 * time.Second
// How often a healthy stream reports itself. Also the interval at which an
// agent talking to a control plane too old to send heartbeats says so —
// that agent is running without a watchdog, and the journal should not be
// silent about it.
pingSummaryInterval = 5 * time.Minute
)
func runCommandStream(ctx context.Context, cfg *config.Config) {
backoff := time.Second
const maxBackoff = 2 * time.Minute
// Two minutes was the old ceiling, and it was reached far too easily. The
// command stream is what makes this agent controllable at all: while it is
// down, workflows and console sessions fail as "agent offline" even though
// SyncKeys keeps polling happily and the fleet list still shows the server
// active. A shorter ceiling costs a few reconnect attempts; the old one cost
// two minutes of an agent that looks fine and answers nothing.
const maxBackoff = 30 * time.Second
for {
select {
@@ -128,22 +166,45 @@ func runCommandStream(ctx context.Context, cfg *config.Config) {
default:
}
if err := connectAndHandleStream(ctx, cfg); err != nil {
if ctx.Err() != nil {
return
}
log.Printf("command stream error: %v, reconnecting in %s", err, backoff)
select {
case <-ctx.Done():
return
case <-time.After(backoff):
}
if backoff < maxBackoff {
backoff *= 2
}
} else {
started := time.Now()
err := connectAndHandleStream(ctx, cfg)
if ctx.Err() != nil {
return
}
// A stream that stayed up is evidence the control plane is reachable,
// whatever ended it. Without this the backoff only ever climbed:
// connectAndHandleStream returns an error on *every* stream end,
// including a healthy one dropped by a routine deploy, so an agent
// pinned itself at the ceiling after a handful of ordinary restarts and
// stayed there for the rest of its life.
if time.Since(started) >= streamHealthyAfter {
backoff = time.Second
}
// The uptime is in the line because it is what distinguishes a stream
// that never worked from one that ran for hours and was dropped by a
// deploy — and it is the same measure that decides whether the backoff
// resets, so a reader can see why the delay is what it is.
up := time.Since(started).Truncate(time.Second)
if err != nil {
log.Printf("command stream error after %s: %v, reconnecting in %s", up, err, backoff)
} else {
log.Printf("command stream closed after %s, reconnecting in %s", up, backoff)
}
select {
case <-ctx.Done():
return
case <-time.After(backoff):
}
if backoff < maxBackoff {
backoff *= 2
if backoff > maxBackoff {
backoff = maxBackoff
}
}
}
}
@@ -154,7 +215,13 @@ func connectAndHandleStream(ctx context.Context, cfg *config.Config) error {
}
defer client.Close()
stream, err := client.CommandStream(ctx)
// Cancelling this context is what unblocks Recv when the stream has gone
// quiet. Without it the watchdog below would have no way to interrupt a
// read that is never going to return.
streamCtx, abandon := context.WithCancel(ctx)
defer abandon()
stream, err := client.CommandStream(streamCtx)
if err != nil {
return fmt.Errorf("open stream: %w", err)
}
@@ -167,7 +234,7 @@ func connectAndHandleStream(ctx context.Context, cfg *config.Config) error {
return fmt.Errorf("send auth: %w", err)
}
log.Println("command stream connected")
log.Printf("command stream connected to %s", cfg.ServerURL)
var sendMu sync.Mutex
send := func(msg *pb.AgentMessage) error {
@@ -176,11 +243,93 @@ func connectAndHandleStream(ctx context.Context, cfg *config.Config) error {
return stream.Send(msg)
}
// Stream liveness, tracked here rather than left to gRPC keepalive.
//
// Keepalive operates on the transport, and behind an L7 proxy the transport
// ends at the proxy: it answers pings whether or not the server behind it
// is still running. A control-plane pod that dies therefore leaves this
// agent blocked in Recv on a stream that will never deliver another message
// and never error, while the control plane dispatches commands into it and
// the operator watches nothing happen.
//
// The watchdog only arms once a ping has actually been seen. A server too
// old to send them must not be treated as dead — that would put the agent
// in a reconnect loop against a control plane that is working perfectly.
var (
lastMu sync.Mutex
lastRecv = time.Now()
pinged bool
beats int
)
markRecv := func(isPing bool) {
lastMu.Lock()
lastRecv = time.Now()
if isPing {
beats++
// Logged once per stream, because it is the moment the agent starts
// holding the control plane to account: before this the watchdog is
// disarmed and a dead stream would go unnoticed indefinitely.
if !pinged {
pinged = true
log.Printf("command stream heartbeat detected, watchdog armed (%s threshold)", streamStaleAfter)
}
}
lastMu.Unlock()
}
go func() {
t := time.NewTicker(streamStaleCheck)
defer t.Stop()
// Reported periodically rather than per beat: at one every 20s the
// journal would be nothing else. The count is what makes a partial
// failure visible — beats arriving but fewer than expected is a
// different problem from beats stopping altogether.
summary := time.NewTicker(pingSummaryInterval)
defer summary.Stop()
for {
select {
case <-streamCtx.Done():
return
case <-summary.C:
lastMu.Lock()
n, armed := beats, pinged
beats = 0
lastMu.Unlock()
if armed {
log.Printf("command stream healthy, %d heartbeats in the last %s", n, pingSummaryInterval)
} else {
log.Printf("command stream up but sending no heartbeats; "+
"control plane predates them, watchdog stays disarmed (last message %s ago)",
time.Since(lastRecv).Truncate(time.Second))
}
case <-t.C:
lastMu.Lock()
idle, armed := time.Since(lastRecv), pinged
lastMu.Unlock()
if armed && idle > streamStaleAfter {
log.Printf("command stream silent for %s (threshold %s), assuming it is dead and reconnecting",
idle.Truncate(time.Second), streamStaleAfter)
abandon()
return
}
}
}
}()
for {
cmd, err := stream.Recv()
if err != nil {
return fmt.Errorf("recv: %w", err)
}
markRecv(cmd.Ping != nil)
// Pings carry nothing and are not acknowledged; being received is their
// whole purpose.
if cmd.Ping != nil {
continue
}
if cmd.GenerateKey != nil {
go handleGenerateKey(cfg, cmd)
@@ -197,6 +346,18 @@ func connectAndHandleStream(ctx context.Context, cfg *config.Config) error {
if cmd.CleanupWorkspace != nil {
go handleCleanupWorkspace(cmd)
}
if cmd.OpenProxy != nil {
go handleOpenProxy(ctx, cfg, cmd.OpenProxy)
}
if cmd.RefreshWorkloads != nil {
go handleRefreshWorkloads(cfg)
}
if cmd.ControlWorkload != nil {
go handleControlWorkload(send, cfg, cmd.CommandId, cmd.ControlWorkload)
}
if cmd.WorkloadLogs != nil {
go handleWorkloadLogs(send, cfg, cmd.CommandId, cmd.WorkloadLogs)
}
if cmd.RunStep != nil {
go func(rc *pb.RunStepCmd, cid string) {
emit := func(seq uint64, data []byte) {
@@ -253,8 +414,17 @@ func runUpdateCheck(ctx context.Context, cfg *config.Config) {
return
}
log.Printf("reported %d available OS updates", len(pkgs))
// Same hourly cadence, same connection. A package set changes on
// roughly the schedule available updates do, so this needs no timer of
// its own.
reportPackages(client, cfg)
}
// The boot round only: after this the flag has long been set, and every
// later tick is an hour past a poll that runs every 30s.
waitFirstPoll(ctx, firstPollWait)
doCheck()
ticker := time.NewTicker(interval)
defer ticker.Stop()
@@ -326,6 +496,29 @@ func handleCleanupWorkspace(cmd *pb.ServerCommand) {
log.Printf("removed run workspace %s (cmd=%s)", dir, cmd.CommandId)
}
// handleOpenProxy relays one console connection. It uses its own gRPC
// connection so console traffic never shares a stream with commands, key sync
// or workflow output.
func handleOpenProxy(ctx context.Context, cfg *config.Config, cmd *pb.OpenProxyCmd) {
client, err := grpcclient.New(cfg.ServerURL, cfg.TLS)
if err != nil {
log.Printf("proxy %s: dial control plane: %v", cmd.ProxyId, err)
return
}
defer client.Close()
stream, err := client.ProxyStream(ctx)
if err != nil {
log.Printf("proxy %s: open stream: %v", cmd.ProxyId, err)
return
}
log.Printf("proxy %s: relaying 127.0.0.1:%d", cmd.ProxyId, cmd.Port)
if err := agentproxy.Open(stream, cfg.ServerID, cfg.AgentToken, cmd.ProxyId, cmd.Port); err != nil {
log.Printf("proxy %s: %v", cmd.ProxyId, err)
}
}
func handleDeleteKey(cmd *pb.ServerCommand) {
label := cmd.DeleteKey.Label
keyPath := fmt.Sprintf("/root/.ssh/vantage_%s", strings.ReplaceAll(label, " ", "_"))
+161
View File
@@ -0,0 +1,161 @@
package agentsync
import (
"context"
"log"
"runtime"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/config"
grpcclient "gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/grpc"
"gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/grpc/pb"
"gitea.hostxtra.co.uk/mrhid6/vantage/agent/internal/workloads"
)
// workloadInterval is the report cadence. Sixty seconds is affordable because
// an unchanged list costs one small offer message, not the body.
const workloadInterval = 60 * time.Second
// runWorkloads reports what this host runs, on its own ticker.
func runWorkloads(ctx context.Context, cfg *config.Config) {
if runtime.GOOS != "linux" {
return
}
reportWorkloads(cfg)
ticker := time.NewTicker(workloadInterval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
reportWorkloads(cfg)
}
}
}
// reportWorkloads offers a hash of the current workload set and sends the full
// list only if the server does not already hold it.
//
// This is the ONLY writer of the server_workloads collection. RefreshWorkloadsCmd
// calls straight into here rather than answering with data of its own.
func reportWorkloads(cfg *config.Config) {
if runtime.GOOS != "linux" {
return
}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
res := workloads.Collect(ctx)
hash := workloads.Hash(res.Workloads)
client, err := grpcclient.New(cfg.ServerURL, cfg.TLS)
if err != nil {
log.Printf("workload report dial error: %v", err)
return
}
defer client.Close()
base := func() *pb.ReportWorkloadsRequest {
return &pb.ReportWorkloadsRequest{
ServerId: cfg.ServerID,
AgentToken: cfg.AgentToken,
Hash: hash,
DockerOk: res.DockerOK,
DockerError: res.DockerError,
SystemdOk: res.SystemdOK,
SystemdError: res.SystemdError,
}
}
// The offer: hash only, no body. On an unchanged host this is the whole
// exchange, which is the point of the handshake.
needFull, err := client.ReportWorkloads(base())
if err != nil {
log.Printf("ReportWorkloads offer error: %v", err)
return
}
if !needFull {
return
}
req := base()
req.Full = true
req.Workloads = make([]pb.Workload, len(res.Workloads))
for i, w := range res.Workloads {
req.Workloads[i] = pb.Workload{
Kind: w.Kind,
Id: w.ID,
Name: w.Name,
State: w.State,
Health: w.Health,
Image: w.Image,
Stack: w.Stack,
Ports: w.Ports,
Restarts: int32(w.Restarts),
Protected: w.Protected,
}
if !w.StartedAt.IsZero() {
req.Workloads[i].StartedAt = w.StartedAt.Format(time.RFC3339)
}
}
if _, err := client.ReportWorkloads(req); err != nil {
log.Printf("ReportWorkloads error: %v", err)
return
}
log.Printf("reported %d workload(s)", len(res.Workloads))
}
// handleRefreshWorkloads makes the agent report immediately. It sends nothing
// back beyond the stream ack: the refresh is a nudge, not a channel, so there
// is one writer for the collection rather than two.
func handleRefreshWorkloads(cfg *config.Config) {
reportWorkloads(cfg)
}
// handleControlWorkload starts, stops or restarts a workload and answers with
// the ordinary CommandResult.
//
// The agent's own protected check inside workloads.Control is the boundary; the
// Protected flag it reports is only there so the UI can grey the button.
func handleControlWorkload(send func(*pb.AgentMessage) error, cfg *config.Config, commandID string, cmd *pb.ControlWorkloadCmd) {
err := workloads.Control(context.Background(), cmd.Kind, cmd.Id, cmd.Action)
res := &pb.CommandResult{CommandId: commandID, Success: err == nil}
if err != nil {
res.Message = err.Error()
log.Printf("workload %s %s failed (cmd=%s): %v", cmd.Action, cmd.Id, commandID, err)
} else {
res.Message = cmd.Action + " " + cmd.Id + " ok"
}
_ = send(&pb.AgentMessage{
ServerId: cfg.ServerID,
AgentToken: cfg.AgentToken,
Result: res,
})
// Report straight away on success so the UI's refetch shows the new state
// rather than the old one.
if err == nil {
reportWorkloads(cfg)
}
}
func handleWorkloadLogs(send func(*pb.AgentMessage) error, cfg *config.Config, commandID string, cmd *pb.WorkloadLogsCmd) {
text, truncated, err := workloads.Logs(context.Background(), cmd.Kind, cmd.Id, int(cmd.Tail))
res := &pb.WorkloadLogsResult{CommandId: commandID, Text: text, Truncated: truncated}
if err != nil {
res.Error = err.Error()
}
_ = send(&pb.AgentMessage{
ServerId: cfg.ServerID,
AgentToken: cfg.AgentToken,
WorkloadLogsResult: res,
})
}
+105
View File
@@ -0,0 +1,105 @@
package workloads
import (
"context"
"errors"
"fmt"
"os"
"os/exec"
"regexp"
"strings"
"time"
)
// ErrProtected is returned for a workload the agent will not act on.
var ErrProtected = errors.New("workload is protected")
// AgentUnit is the systemd unit this agent runs as.
const AgentUnit = "vantage-agent.service"
// controlTimeout bounds a stop that may never finish on its own. `docker stop`
// waits on a container that may ignore SIGTERM, and `systemctl stop` on a unit
// with a long TimeoutStopSec blocks for exactly as long as that says. A
// timeout must return a real error rather than an ack implying success.
const controlTimeout = 90 * time.Second
// ownContainerID is read once: the container this agent runs in, if any.
var ownContainerID = detectOwnContainer()
var cgroupContainerRe = regexp.MustCompile(`[0-9a-f]{64}`)
// detectOwnContainer returns this process's container ID, or "" on a host
// install. The agent is normally a systemd service, so "" is the common case;
// this exists so containerising it later cannot silently remove the guard.
func detectOwnContainer() string {
b, err := os.ReadFile("/proc/self/cgroup")
if err != nil {
return ""
}
if m := cgroupContainerRe.FindString(string(b)); m != "" {
return m
}
return ""
}
// isProtected reports whether the agent refuses to act on this workload.
//
// The refusal lives here, in the agent, and not in the control plane. As with
// the console relay hardcoding 127.0.0.1 agent-side: the control plane may name
// a target, but the agent decides what it will do to itself. A server-side
// denylist alone would be bypassed by the next dispatch path someone adds.
func isProtected(kind, id, name string) bool {
if kind == "unit" {
return id == AgentUnit || name == strings.TrimSuffix(AgentUnit, ".service")
}
if ownContainerID == "" {
return false
}
// Container IDs are commonly abbreviated to 12 characters; compare on the
// shorter of the two so a short id still matches a full one.
return strings.HasPrefix(ownContainerID, id) || strings.HasPrefix(id, ownContainerID)
}
// markProtected stamps the flag onto a collected list so the UI can render the
// action disabled with a reason.
func markProtected(wls []Workload) {
for i := range wls {
wls[i].Protected = isProtected(wls[i].Kind, wls[i].ID, wls[i].Name)
}
}
// Control starts, stops or restarts a workload.
func Control(ctx context.Context, kind, id, action string) error {
switch action {
case "start", "stop", "restart":
default:
return fmt.Errorf("unknown action %q", action)
}
// Checked before anything else happens, and checked here rather than only
// on the server. See isProtected.
if isProtected(kind, id, strings.TrimSuffix(id, ".service")) {
return fmt.Errorf("%w: %s", ErrProtected, id)
}
ctx, cancel := context.WithTimeout(ctx, controlTimeout)
defer cancel()
var cmd *exec.Cmd
switch kind {
case "container":
cmd = exec.CommandContext(ctx, "docker", action, id)
case "unit":
cmd = exec.CommandContext(ctx, "systemctl", action, id)
default:
return fmt.Errorf("unknown workload kind %q", kind)
}
if out, err := cmd.CombinedOutput(); err != nil {
if ctx.Err() == context.DeadlineExceeded {
return fmt.Errorf("%s %s timed out after %s", action, id, controlTimeout)
}
return fmt.Errorf("%s %s: %s", action, id, strings.TrimSpace(string(out)))
}
return nil
}
+141
View File
@@ -0,0 +1,141 @@
package workloads
import (
"context"
"encoding/json"
"os/exec"
"sort"
"strings"
"time"
)
// Workload is one container or one systemd unit, agent-side. It mirrors
// models.Workload on the server.
type Workload struct {
Kind string
ID string
Name string
State string
Health string
Image string
Stack string
Ports []string
Restarts int
StartedAt time.Time
Protected bool
}
const dockerTimeout = 30 * time.Second
// dockerInspect is the subset of `docker inspect` output we read.
//
// We use inspect rather than `docker ps --format '{{json .}}'` because ps
// reports health and uptime inside a human Status string — "Up 2 hours
// (healthy)" — and anything built on that is parsing English that is
// localised, reworded between releases, and silently different for a paused or
// restarting container. inspect gives typed fields instead.
type dockerInspect struct {
ID string `json:"Id"`
Name string `json:"Name"`
State struct {
Status string `json:"Status"`
StartedAt string `json:"StartedAt"`
Restarting bool `json:"Restarting"`
Health *struct {
Status string `json:"Status"`
} `json:"Health"`
} `json:"State"`
Config struct {
Image string `json:"Image"`
Labels map[string]string `json:"Labels"`
} `json:"Config"`
RestartCount int `json:"RestartCount"`
NetworkSettings struct {
Ports map[string][]struct {
HostIP string `json:"HostIp"`
HostPort string `json:"HostPort"`
} `json:"Ports"`
} `json:"NetworkSettings"`
}
// collectDocker enumerates containers. It returns ok=false with an empty error
// string when Docker is simply not installed — the common case on this fleet,
// and not a fault.
func collectDocker(ctx context.Context) ([]Workload, bool, string) {
if _, err := exec.LookPath("docker"); err != nil {
return nil, false, "" // not installed; not an error
}
ctx, cancel := context.WithTimeout(ctx, dockerTimeout)
defer cancel()
idsOut, err := exec.CommandContext(ctx, "docker", "ps", "-aq").Output()
if err != nil {
// Installed but not answering: a different problem with a different
// fix, so it carries a message where "not installed" does not.
return nil, false, "docker ps failed: " + errText(err)
}
ids := strings.Fields(string(idsOut))
if len(ids) == 0 {
return []Workload{}, true, "" // Docker present, nothing running
}
args := append([]string{"inspect", "--format", "{{json .}}"}, ids...)
out, err := exec.CommandContext(ctx, "docker", args...).Output()
if err != nil {
return nil, false, "docker inspect failed: " + errText(err)
}
var wls []Workload
for _, line := range strings.Split(string(out), "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
var di dockerInspect
if err := json.Unmarshal([]byte(line), &di); err != nil {
continue
}
wls = append(wls, dockerToWorkload(di))
}
return wls, true, ""
}
func dockerToWorkload(di dockerInspect) Workload {
w := Workload{
Kind: "container",
ID: di.ID,
Name: strings.TrimPrefix(di.Name, "/"),
State: di.State.Status,
Image: di.Config.Image,
Restarts: di.RestartCount,
}
if di.State.Health != nil {
w.Health = strings.ToLower(di.State.Health.Status)
}
// The compose project label is what Docker itself treats as authoritative.
// No YAML is read from disk: a compose file there may not be what is running.
if v := di.Config.Labels["com.docker.compose.project"]; v != "" {
w.Stack = v
}
if t, err := time.Parse(time.RFC3339Nano, di.State.StartedAt); err == nil {
w.StartedAt = t
}
for container, bindings := range di.NetworkSettings.Ports {
for _, b := range bindings {
w.Ports = append(w.Ports, b.HostIP+":"+b.HostPort+"->"+container)
}
}
// Map iteration order is random; sort so a stored snapshot does not reorder
// its own ports between two otherwise identical reports.
sort.Strings(w.Ports)
return w
}
func errText(err error) string {
if ee, ok := err.(*exec.ExitError); ok && len(ee.Stderr) > 0 {
return strings.TrimSpace(string(ee.Stderr))
}
return err.Error()
}
+83
View File
@@ -0,0 +1,83 @@
package workloads
import (
"context"
"fmt"
"os/exec"
"strconv"
"strings"
"time"
)
const (
// MaxLogLines and MaxLogBytes are BOTH enforced, whichever binds first.
//
// A line count alone does not bound size: 500 lines of a container printing
// 4KB JSON blobs is 2MB travelling over the bus. This is the same reasoning
// that gave workflow logs a per-line cap as well as a per-run one.
MaxLogLines = 500
MaxLogBytes = 256 * 1024
logTimeout = 60 * time.Second
)
// Logs returns a bounded snapshot of a workload's recent output.
//
// There is no follow mode. The browser console already offers a real terminal
// on the same server where `docker logs -f` works properly, with its own
// scrollback and cancellation. A snapshot answers "why did this restart",
// which is the question that sends people to the console in the first place.
func Logs(ctx context.Context, kind, id string, tail int) (string, bool, error) {
if tail <= 0 || tail > MaxLogLines {
tail = MaxLogLines
}
ctx, cancel := context.WithTimeout(ctx, logTimeout)
defer cancel()
var cmd *exec.Cmd
switch kind {
case "container":
cmd = exec.CommandContext(ctx, "docker", "logs",
"--tail", strconv.Itoa(tail), "--timestamps", id)
case "unit":
cmd = exec.CommandContext(ctx, "journalctl", "-u", id,
"-n", strconv.Itoa(tail), "--no-pager", "--output=short-iso")
default:
return "", false, fmt.Errorf("unknown workload kind %q", kind)
}
// docker logs writes container stderr to our stderr, so both streams must
// be captured or half the output silently disappears.
out, err := cmd.CombinedOutput()
if err != nil && len(out) == 0 {
return "", false, fmt.Errorf("read logs for %s: %s", id, errText(err))
}
text, truncated := capLog(string(out))
return text, truncated, nil
}
// capLog enforces both limits, trimming from the FRONT: the most recent lines
// are the ones worth keeping.
func capLog(s string) (string, bool) {
truncated := false
lines := strings.Split(s, "\n")
if len(lines) > MaxLogLines {
lines = lines[len(lines)-MaxLogLines:]
truncated = true
}
s = strings.Join(lines, "\n")
if len(s) > MaxLogBytes {
s = s[len(s)-MaxLogBytes:]
// Drop the leading partial line left by a byte-wise cut.
if i := strings.IndexByte(s, '\n'); i >= 0 {
s = s[i+1:]
}
truncated = true
}
return s, truncated
}
+94
View File
@@ -0,0 +1,94 @@
package workloads
import (
"context"
"os/exec"
"strings"
"time"
)
const systemdTimeout = 30 * time.Second
// excludedPrefixes drops the platform's own units. A typical host carries 300+
// units and systemd accounts for most of them; listing all of them buries the
// ten anyone cares about.
var excludedPrefixes = []string{"systemd-", "user@", "user-", "session-", "init.scope"}
// collectSystemd enumerates services in two passes, because "running or
// failed" and "enabled but stopped" are different questions — and an enabled
// unit that is not running is exactly the one worth seeing.
func collectSystemd(ctx context.Context) ([]Workload, bool, string) {
if _, err := exec.LookPath("systemctl"); err != nil {
return nil, false, ""
}
ctx, cancel := context.WithTimeout(ctx, systemdTimeout)
defer cancel()
// Column output rather than --output=json: the JSON flag needs systemd
// 246+, and this fleet includes older stable distributions. The columns
// have been stable considerably longer than the JSON has existed.
unitsOut, err := exec.CommandContext(ctx, "systemctl",
"list-units", "--type=service", "--state=running,failed",
"--no-legend", "--plain", "--no-pager").Output()
if err != nil {
return nil, false, "systemctl list-units failed: " + errText(err)
}
seen := map[string]bool{}
var wls []Workload
for _, line := range strings.Split(string(unitsOut), "\n") {
f := strings.Fields(line)
// UNIT LOAD ACTIVE SUB DESCRIPTION…
if len(f) < 4 {
continue
}
name := f[0]
if excluded(name) || seen[name] {
continue
}
seen[name] = true
wls = append(wls, Workload{
Kind: "unit",
ID: name,
Name: strings.TrimSuffix(name, ".service"),
State: f[2], // ACTIVE: active | failed | activating | inactive
})
}
filesOut, err := exec.CommandContext(ctx, "systemctl",
"list-unit-files", "--type=service", "--state=enabled",
"--no-legend", "--plain", "--no-pager").Output()
if err == nil {
for _, line := range strings.Split(string(filesOut), "\n") {
f := strings.Fields(line)
// UNIT FILE STATE [PRESET]
if len(f) < 2 {
continue
}
name := f[0]
if excluded(name) || seen[name] {
continue
}
seen[name] = true
wls = append(wls, Workload{
Kind: "unit",
ID: name,
Name: strings.TrimSuffix(name, ".service"),
State: "inactive", // enabled but not currently running
})
}
}
return wls, true, ""
}
func excluded(name string) bool {
for _, p := range excludedPrefixes {
if strings.HasPrefix(name, p) {
return true
}
}
return false
}
+64
View File
@@ -0,0 +1,64 @@
package workloads
import (
"context"
"crypto/sha256"
"encoding/hex"
"runtime"
"sort"
"strconv"
"strings"
)
// Result is one collection pass.
type Result struct {
Workloads []Workload
DockerOK bool
DockerError string
SystemdOK bool
SystemdError string
}
// Collect enumerates every workload on this host. Linux only.
func Collect(ctx context.Context) Result {
if runtime.GOOS != "linux" {
return Result{}
}
var r Result
containers, dockerOK, dockerErr := collectDocker(ctx)
units, systemdOK, systemdErr := collectSystemd(ctx)
r.DockerOK, r.DockerError = dockerOK, dockerErr
r.SystemdOK, r.SystemdError = systemdOK, systemdErr
r.Workloads = append(append([]Workload{}, containers...), units...)
markProtected(r.Workloads)
return r
}
// Hash fingerprints a workload set so an unchanged set never has to be sent.
//
// It sorts first: `docker ps` output ordering is not stable, and an
// ordering-sensitive hash would resend the full list every 60 seconds forever
// — a cost visible only as traffic.
//
// StartedAt is deliberately excluded: it does not change while a container
// runs, and including it would add nothing. Restarts IS included, because a
// container cycling is exactly the change worth reporting.
func Hash(wls []Workload) string {
lines := make([]string, 0, len(wls))
for _, w := range wls {
lines = append(lines, strings.Join([]string{
w.Kind, w.ID, w.Name, w.State, w.Health, w.Image, w.Stack,
strconv.Itoa(w.Restarts),
}, "\x00"))
}
sort.Strings(lines)
h := sha256.New()
for _, l := range lines {
h.Write([]byte(l))
h.Write([]byte("\n"))
}
return hex.EncodeToString(h.Sum(nil))
}
-683
View File
@@ -1,683 +0,0 @@
# Vantage
A self-hosted, multi-tenant infrastructure control plane. It started as SSH key management and has grown into fleet management: SSH key assignment, workflow/script execution, service monitoring, a secrets vault, a browser console (SSH/RDP/VNC), and OS update management.
A central server (Go + Next.js + MongoDB + Redis) drives a lightweight Go agent installed on each managed server. Agents poll over gRPC and also hold a bidirectional command stream for push-style commands.
---
## Architecture Overview
```
┌──────────────────────────────────────────────┐
│ Next.js 16 Frontend (web, :3000) │
│ servers · keys · workflows · monitors │
│ secrets · audit · console · settings │
└───────────────┬──────────────────────────────┘
│ REST + cookie session
┌───────────────▼──────────────────────────────┐
│ Go Backend (server) │
│ :8080 REST (gin) :9090 gRPC (agents) │
│ MongoDB (state) · Redis (sessions) │
│ monitor scheduler · workflow runner │
│ guacd tunnel proxy for browser console │
└───────────────┬──────────────────────────────┘
│ gRPC (TLS) — outbound from agent only
┌───────────────▼──────────────────────────────┐
│ Go Agent (per server, Linux + Windows) │
│ polls SyncKeys · CommandStream │
│ rewrites authorized_keys (Linux only) │
│ runs workflow steps · monitors · inventory │
└──────────────────────────────────────────────┘
```
Multi-tenancy: every domain document carries `org_id`, and every service query is scoped by it. Org is resolved from the session, and optionally cross-checked against the request host (`<slug>.vantage.<tld>`).
---
## Repository Structure
```
vantage/
├── agent/
│ ├── cmd/main.go # flags: -generate-key
│ └── internal/
│ ├── checker/ # monitor check execution
│ ├── config/ # config.yaml load/save
│ ├── exec/ # workflow step execution
│ ├── grpc/ # client + generated pb
│ ├── inventory/ # CPU/mem/disk collection (linux/other)
│ ├── keys/ # authorized_keys read/diff/write
│ ├── monitors/ # agent-run monitor loop
│ ├── sync/ # poll loop + command stream
│ └── updates/ # OS package update check/apply
├── server/
│ ├── cmd/main.go
│ └── internal/
│ ├── api/ # REST handlers
│ ├── auth/ # local, OIDC, session, middleware, orghost
│ ├── checker/ # server-run monitor checks
│ ├── db/ # mongo connect + Col()
│ ├── grpc/ # gRPC server + generated pb
│ ├── models/ # MongoDB documents
│ ├── monitorsched/ # server-side monitor scheduler
│ ├── notify/ # channel dispatch: http, discord, slack, telegram, smtp
│ └── services/ # business logic + migrations
├── web/ # the application UI (authenticated)
│ ├── app/(app)/ # authed routes
│ ├── app/login, app/setup # unauthed routes
│ ├── components/ # ui/, workflows/, monitors/, Sidebar
│ └── lib/ # api client, guac console, query client
├── site/ # public marketing site
│ ├── app/ # one directory per route
│ ├── components/ # Nav, Footer, Logo, InstrumentPanel, forms
│ ├── assets/ # image sources, not served
│ └── Dockerfile # same shape as web/: standalone, node, 3000
├── sitesvc/ # public form: contact mail only
│ ├── cmd/main.go
│ └── internal/
│ ├── api/ # contact
│ └── store/ # Mongo connect helper
├── admin/ # licensing authority: the only signer
│ ├── cmd/main.go # boot: two Mongo connections, reconciler, HTTP
│ ├── cmd/adminctl/ # staff-add; deliberately has no HTTP surface
│ └── internal/
│ ├── api/ # customer + staff handlers, route table
│ ├── auth/ # staff, HQ customer and cloud-owner sessions
│ ├── inject/ # licence write path into the control plane
│ ├── cloudprov/ # instance write path: creates instances + owners
│ ├── licensing/ # Issue, LinkInstance, Relink
│ ├── mail/ # admin's boot-time shared/mail Sender
│ └── models/ # accounts, instances, licences, plans
├── adminsite/ # staff + customer console (vantage-hq)
│ ├── app/(customer)/ # overview, instance, link, billing
│ ├── app/(staff)/staff/ # operations, accounts, licences, plans, audit
│ ├── components/ # AppBar, PageHeader, PageFrame, InstanceRecord
│ └── lib/ # api client, session guards, formatters
├── shared/ # imported by server, sitesvc and admin
│ ├── mail/ # the one email system: transport + tmpl templates
│ ├── license/ # payload, sign, verify, trusted keys, plans
│ ├── models/ # Instance, User, Settings
│ └── cmd/lkctl/ # issue and inspect licences by hand
├── proto/vantage/v1/vantage.proto
├── installer/ # Windows: setup.ps1, nssm.exe, WiX .wxs
├── deploy/ # docker-compose.yml, agent.service
└── .gitea/workflows/ # agent-release.yml, server-deploy.yml
```
---
## Subsystems
### SSH keys
Upload a public key, assign it per server, revoke softly. The agent diffs desired vs on-disk state and rewrites `/root/.ssh/authorized_keys` atomically. Keys can also be generated _on_ a server by the agent; the private half can optionally be uploaded and is stored AES-256-GCM encrypted.
### Workflows
A library of reusable **steps** (bash or PowerShell scripts with declared inputs, outputs, and secret refs) composed into **workflows** targeting a set of servers. Running one snapshots the resolved steps into a `WorkflowRun`, then dispatches `RunStepCmd` over the agent command stream. Step stdout/stderr streams back as `StepOutputChunk` and is written to a log file on disk; the UI streams it live. Steps support `on_failure: stop|continue|retry`, per-run env passed between steps via `output_env`, and a per-run workspace directory the agent cleans up at the end.
Default steps are seeded per org at boot (`SeedDefaultSteps`). Logs are swept by retention (`workflow_log_retention_days`; nil = 30 days, 0 = forever).
### Monitors
HTTP, TCP, ICMP and TLS checks. Each monitor has a `runner`: `"server"` (executed by the server-side scheduler) or a `server_id` (pushed to that agent, which runs it locally and reports results). Consecutive failures beyond `retries` flip state to `down`, open an `Incident`, and notify. Hourly `Rollup` documents back the uptime graphs.
### Notification channels
Per-org outbound destinations: `webhook`, `smtp`, `discord`, `slack`, `telegram`. Monitors reference channels by ID. Channels are testable from the UI.
### Secrets vault
Key/value pairs grouped by name, encrypted at rest with AES-256-GCM. Consumed two ways: referenced by workflow steps via `secret_refs` (injected as env at execution), and read by Kubernetes External Secrets Operator via `GET /api/secrets/:group/values` using a bearer token whose SHA-256 hash is stored in settings.
### Browser console
`POST /api/console/connect` mints a one-time session token; `GET /api/console/tunnel` upgrades to a WebSocket and proxies to **guacd** (Apache Guacamole daemon) using `github.com/wwt/guac`. SSH connections authenticate with a stored private key; RDP/VNC credentials are encrypted, single-use, and consumed when the tunnel opens.
### Inventory and OS updates
Agents report CPU/memory/swap/partitions/kernel — metrics every 30s, full static snapshot every 15 min. They also check for pending OS package updates hourly and can apply them on command (`ApplyUpdatesCmd`).
### Agent self-update
`UpdateAgentCmd` carries a target version and Gitea base URL; the agent downloads and replaces itself.
### Marketing site and sitesvc
`site/` is a separate Next.js app built exactly like `web/``output: "standalone"`, run by Node in a `node:26-alpine` image, listening on `3000` and published as `3003`. The contact form posts to `sitesvc`; account signup posts to `admin` (`NEXT_PUBLIC_ADMIN_API_URL`), which creates an HQ account, not an org — the control plane is not touched until the customer later creates a cloud instance from the portal.
`adminsite/` is built the same way and published as `3004`, served at **`vantage-hq.hostxtra.co.uk`** — deliberately *outside* `*.vantage.hostxtra.co.uk`, because that namespace is per-tenant instance subdomains and `APP_ROOT_LABEL` resolves an org from the label before `vantage`. It shares `site/`'s design tokens verbatim (see Frontend below) and, unlike `web/`, does **not** proxy through a Next rewrite: the browser calls `admin` directly, so `ADMIN_API_URL` must be browser-reachable. Authenticated requests work cross-origin only because both hosts share the registrable domain `hostxtra.co.uk`, which keeps `admin_session`'s `SameSite=Lax` cookie in play.
**`ADMIN_ORIGIN` must list every browser origin that calls admin — currently two**: `https://vantage-hq.hostxtra.co.uk` for the console, and `https://vantage.hostxtra.co.uk` because the marketing site's `/start` form posts account signups to admin directly. It is comma-separated. A missing origin does not produce a 403: `cors()` simply omits the `Access-Control-Allow-Origin` header and still answers the preflight `204`, so the browser blocks the request and **admin logs nothing at all**. Symptom is a CORS preflight failure on an endpoint that works fine under curl.
`sitesvc/` (port `8082`) now owns only the contact flow:
| Form | Endpoint | Effect |
| ------- | --------------------- | ----------------------------------------------------------------------- |
| Contact | `POST /api/contact` | Emails `support@hostxtra.co.uk`, `Reply-To` the sender. Nothing stored. |
Account signup lives in `admin` instead (`POST /auth/signup`, `GET /auth/verify?token=…`) — see Signup and verification below.
`site`, `sitesvc` and `admin` are deliberately **excluded from the self-hosted deployment**: `deploy/docker-compose.yml` mentions none of them, and they live in `deploy/docker-compose.site.yml` instead.
```bash
# self-hosted install — no marketing site, no sitesvc
docker compose up -d
# vantage.hostxtra.co.uk — control plane plus the public site
docker compose -f docker-compose.yml -f docker-compose.site.yml up -d
```
### Signup and verification
Signup is **account-first**: it creates an HQ account and an unverified `customer_user` in admin's own database, nothing in the control plane. Only after a customer later creates a cloud instance from the portal (`POST /api/instances`, see Admin REST API) does an org, or rather an `instance`, come to exist — provisioned by `cloudprov`, with the owner's password hash copied from the HQ user rather than shared. `site_pending_signups` is gone; sitesvc no longer has a signup flow at all.
- The token is 32 random bytes; only its **SHA-256 hash** is stored, so a leaked database yields no working links.
- Links expire after 24 hours (`VerifyWindow`).
- An unverified sign-in gets a distinct "check your email" error rather than the generic auth failure, because the address is already known to be theirs.
- If sending the verification email fails, the freshly inserted `customer_user` (and account, on first signup) is rolled back rather than left stranded holding the unique index on email.
- Rate limited per client IP, plus a honeypot field.
An account is a team, not a person. `customer_users.account_role` is `owner`,
`admin` or `member` — the same three words as the control plane's roles, on
purpose. Owners and admins invite people, create instances and grant instance
access; billing is owner-only.
An invitation creates a `customer_user` with an **empty password hash**, which
cannot authenticate, and the invitee sets their own at `/accept-invite`. An
inviter-chosen password would be a shared credential to every instance that
person is later granted. `GET /auth/verify` therefore peeks before it consumes:
a token belonging to a passwordless row answers `{"needs_password":true}` and is
left unspent.
### Email
`shared/mail` is the only email system. It owns the SMTP conversation, the RFC
5322 envelope and the look of every message; `server`, `admin` and `sitesvc`
each import it and none of them builds a subject line, a MIME part or a colour.
Before this existed the transport was copied three times, and the copies had
already diverged once — the 465-implicit-TLS fix landed in one of them while
the others silently delivered nothing.
`Sender` is a value, not a singleton: `server/internal/notify` builds one per
notification channel from the channel document in Mongo, while `sitesvc` builds
one at boot and `admin` holds one in `admin/internal/mail.Default`, alongside
its other boot-time singletons. Callers only ever see typed methods —
`SendVerification`, `SendExpiring`, `SendMonitorAlert`, `SendEnquiry` and the
rest, grouped by owner into `account.go`, `licence.go`, `billing.go`,
`monitor.go` and `contact.go`.
Every message is `multipart/alternative`, so each one is two templates:
`templates/<name>.html.tmpl` and `.txt.tmpl`, embedded with `go:embed`. They
define `subject`, `title`, `pill` and `body`; `layout.html.tmpl` and
`layout.txt.tmpl` provide the chrome and the helper templates (`p`, `lead`,
`button`, `well`, `note`, `rows`, `chip`) that the bodies compose. One template
set is parsed per message rather than one big set, because every message
defines those same four names. **`subject` is defined in the txt file only** —
`html/template` would escape an ampersand in an instance name and mail clients
show subjects verbatim.
`shared/mail/render_test.go` renders all of them and fails if a template exists
that no case covers, which is the only thing standing between a mistyped field
and a boot-time panic — the templates are parsed in `init()`.
### Shared provisioning
`shared/provision` (`instance.go`, `slug.go`, `user.go`) holds the slug rules, reserved names and instance/user creation logic that both `server` and `admin/internal/cloudprov` need, so there is no longer a second copy to drift: `cloudprov.CreateInstance` calls straight into it to create a control-plane instance and its owner from a customer request.
### Grants project, they do not federate
Granting someone access to a cloud instance writes a real control-plane `users`
row through `cloudprov`, with `auth_source: "hq"` and `hq_user_id` set. The
instance authenticates it exactly as it authenticates anyone else, with **no
runtime dependency on admin**. Revoking deletes that row — the control plane has
no disabled state, and a row that exists is a row that can sign in.
`instance_members` in admin's database is only admin's *index* of those
projections; the control-plane row is the access. That is why a failed
`instance_members` insert unwinds the projection, and why the boot backfill can
rebuild the index from the control plane but never the other way round.
**Self-hosted instances are never projected into.** All three mutating member
endpoints refuse when `deployment != cloud`.
The HQ password is the single source of truth for every `hq`-sourced row.
`PUT /api/account/password` rehashes and has `cloudprov` copy the hash to every
projected row; propagation is best-effort, and `admin/internal/hqsync` compares
and repairs every 15 minutes. It is **its own package rather than a pass inside
`inject`** — `inject` writes three licence fields and nothing else, and that
narrowness is what makes admin's reach into the control plane reviewable.
The control plane refuses to change an `hq`-sourced user's role or delete it
(`services.ErrHQManaged`, 409). `web/` shows those rows read-only with a link to
the portal, but the API is the boundary; the UI is a courtesy. There is no local
password-change endpoint at all, so there is no competing writer for the hash.
---
## Auth and Orgs
- **Bootstrap** — first run has no users. `GET /auth/bootstrap-status` drives `/setup`, `POST /auth/bootstrap` creates the first org plus its owner.
- **Local auth** — email + password (bcrypt), `POST /auth/login`.
- **OIDC** — configured _per org_ (`org_oidc`), issuer + client ID + encrypted client secret. `/auth/oidc/start``/auth/oidc/callback`.
- **Sessions** — opaque 32-byte hex ID in the `km_session` cookie, session body stored in Redis with a 24h TTL.
- **Roles**`owner`, `admin`, `member`. `/api/settings` and `/api/org/*` require owner or admin.
- **Host/org guard**`APP_ROOT_LABEL` (default `vantage`) defines the app root label. A request to `<slug>.vantage.<tld>` resolves that org from the slug and rejects sessions belonging to a different one. Org lookups are cached for 60s.
Unique indexes are a **security property**, not an optimisation. `users` is
unique on `(instance_id, email)` — one address is one user *within* an instance,
and the same address may hold a user in several instances, because an account's
people are projected into each instance they are granted. This is sufficient only
because **every lookup by email is scoped by instance**; there is deliberately no
unscoped lookup anywhere, and adding one would let the login path return an
arbitrary one of several matching users. Instance slug, settings instance and ESO
token hash remain globally unique.
---
## gRPC API
```protobuf
service Vantage {
rpc Register(RegisterRequest) returns (RegisterResponse);
rpc SyncKeys(SyncRequest) returns (SyncResponse);
rpc UploadGeneratedKey(UploadKeyRequest) returns (UploadKeyResponse);
rpc ReportUpdates(ReportUpdatesRequest) returns (ReportUpdatesResponse);
rpc ReportInventory(InventoryReport) returns (InventoryReportResponse);
rpc SyncMonitors(SyncMonitorsRequest) returns (SyncMonitorsResponse);
rpc ReportChecks(ReportChecksRequest) returns (ReportChecksResponse);
rpc CommandStream(stream AgentMessage) returns (stream ServerCommand);
}
```
`CommandStream` is the only streaming RPC: the agent authenticates once with `AgentReady`, then the server pushes `ServerCommand`s and the agent replies with `CommandResult`, `StepResult`, or `StepOutputChunk`.
`ServerCommand` variants: `GenerateKeyCmd`, `DeleteKeyCmd`, `UpdateAgentCmd`, `ApplyUpdatesCmd`, `RunStepCmd`, `CleanupWorkspaceCmd`.
Key-state polling stays on the 30s `SyncKeys` interval. Full message definitions live in `proto/vantage/v1/vantage.proto`.
---
## REST API
Unauthenticated:
```
GET /install /install.ps1 # dynamic agent install scripts
GET /update /update.ps1
GET /auth/bootstrap-status
POST /auth/bootstrap /auth/login /auth/logout
GET /auth/me /auth/oidc/start /auth/oidc/callback
GET /api/secrets/:group/values # bearer token (ESO)
```
Session-authed under `/api`:
```
servers GET,POST /servers · GET,POST /servers/new · GET,DELETE /servers/:id
POST /servers/:id/{generate-key,update-agent,apply-updates}
keys GET,POST /keys · GET,DELETE /keys/:id · GET /keys/:id/private-key
POST /keys/:id/assign · DELETE /keys/:id/assign/:serverId
workflows GET,POST /steps · PUT,DELETE /steps/:id · GET /steps/:id/export
POST /steps/{import,seed-defaults,parse} · GET /steps/usage
GET,POST /workflows · GET,PUT,DELETE /workflows/:id
POST /workflows/:id/run · GET /workflows/:id/runs
GET /runs/:runId · POST /runs/:runId/cancel
GET /runs/:runId/servers/:serverId/logs[/stream]
monitors GET,POST /monitors · GET,PUT,DELETE /monitors/:id
GET /monitors/:id/{incidents,uptime}
channels GET,POST /channels · PUT,DELETE /channels/:id · POST /channels/:id/test
secrets GET,POST /secrets · GET,PUT,DELETE /secrets/:group
POST /secrets/:group/reveal · DELETE /secrets/:group/:key
console POST /console/connect · GET /console/tunnel (websocket)
audit GET /audit
agent GET /agent/latest-version
settings GET,PUT /settings · POST /settings/secrets-token (owner|admin)
licence GET /license · POST /license (POST: self-hosted only)
org GET,POST /org/users · PUT /org/users/:id/role · DELETE /org/users/:id
GET,PUT /org/oidc (owner|admin)
```
`GET /license` reports `deployment`, and **`POST /license` answers 409 `cloud_managed` when it is `cloud`**. A cloud instance's licence is written by `admin/internal/inject` straight into the database and never through this endpoint, so the refusal cannot break injection — it only stops a customer pasting over a licence they do not own. `web/` hides the paste form and points at the HQ portal instead, but as with `hq`-managed users, the API is the boundary and the UI is the courtesy.
`POST /license` is also in `licenceExemptPaths`: pasting a valid licence has to work while the current one is expired, because it is the way out of degraded mode.
Free exists in both deployments, so it is no longer cloud-only by construction. The one-Free-per-account rule is enforced per account **and deployment**, in `licensing.checkFreeLimit` and in `createInstance`'s friendly pre-check — the two must stay scoped identically, because a pre-check stricter than the issuer refuses what would have worked. A self-hosted Free licence is claimed with `POST /api/instances/:id/claim-free` after the install is linked; the metered server count and per-instance feature toggles come from the instance's `entitlement`, which a licence snapshots at issue time (see the `catalogue`/`entitlements` note under MongoDB Collections).
---
## Admin REST API (`admin`, :8083)
A separate service with its own session cookie (`admin_session`) and its own database. Unauthenticated:
```
GET /healthz
GET /auth/me # who am I; 401 drives the UI's redirects
POST /auth/staff/login /auth/login /auth/logout
POST /auth/signup # self-hosted only; honeypot + rate limited
GET /auth/verify?token=…
POST /auth/accept-invite # an invitee sets their own password
POST /api/paddle/webhook # Paddle events; signature-verified, idempotent, no session
```
Customer-session (`/api`), every instance resolved through `ownedInstance`:
```
GET /account # account, instances, max_relinks
POST /instances # create a cloud instance (Free tier, one Free per account per deployment)
POST /instances/:id/renew # Free renewal; refuses outside the renewal window
POST /instances/:id/claim-free # issue Free on a linked self-hosted instance
POST /instances/link · /instances/:id/relink
GET /instances/:id/entitlement
GET /checkout/options # active plans + catalogue prices for the running PADDLE_ENV
POST /instances/self-hosted # create a paid-checkout placeholder (awaiting_link, no licence)
POST /instances/:id/claim-link # bind a paid placeholder to the real UUID and issue
PUT /instances/:id/entitlement # set desired config; pushes line items to Paddle (owner|admin)
POST /billing/portal # mint a Paddle customer-portal URL
GET /instances/:id/license · /instances/:id/license/download
GET /subscriptions
GET,POST /account/users · PUT /account/users/:id/role · DELETE /account/users/:id
PUT /account/password # propagates to every projected user
GET,POST /instances/:id/members # cloud only
PUT /instances/:id/members/:uid/role · DELETE /instances/:id/members/:uid
```
Reading is open to any signed-in member; every mutation above except
`/account/password` (which is your own) sits behind `RequireAccountRole(owner,
admin)`. `:uid` is the **`customer_users.user_id`**, not the projected
control-plane user_id — the portal never has to know that one.
Staff-session (`/api/staff`):
```
GET,POST /accounts · GET /accounts/:id # search by name, email, Paddle ID or instance UUID
GET,POST /instances · GET /instances/:id # instance + account + licence history + injection state
POST /instances/:id/issue · /instances/:id/relink
GET /licenses · /subscriptions · /audit · /plans · PUT /plans/:deployment/:tier
GET,PUT /catalogue
GET,PUT /instances/:id/entitlement
GET /health/injection · /health/billing
```
**Customer endpoints answer 404, never 403, for another account's resource** — a 403 confirms the resource exists. Route-group guards in `adminsite/` mirror this, but the backend is the layer that matters.
### Billing (Paddle)
Paddle is merchant of record; `admin/internal/paddle` is a thin REST client (no vendor SDK) and the only place that talks to it. **Free is entirely outside Paddle** — the shipped self-serve Free flow owns its own renewal, so no £0 subscription exists; an account learns its `paddle_customer_id` from its first paid webhook. Checkout happens in the browser (`@paddle/paddle-js`, token baked into the adminsite build); the server only updates a live subscription (`PUT /instances/:id/entitlement`) and mints a portal session.
`POST /api/paddle/webhook` is the **only** issuing path for paid plans: signature-verified with `PADDLE_WEBHOOK_SECRET` (boot-required), idempotent via `paddle_events`, and a function of the subscription's *current* line items — resolved back to a plan and configuration by `catalogue.ResolveItems`, so out-of-order delivery is correct by construction. A confirmed webhook promotes the entitlement `desired``granted` and signs from `granted` **only**; a checkout is built from `desired`. `subscription.canceled` and `past_due` take **no licence action** — the licence runs to its (grace-padded) expiry, then the existing lifecycle sweep lapses the instance. A renewal (`transaction.completed`, origin `subscription_recurring`) is the only moment a scheduled reduction collapses `desired` into `granted`. Self-hosted purchase creates a placeholder instance before payment (`POST /instances/self-hosted`); the licence is issued only once the customer pastes the install's real UUID (`POST /instances/:id/claim-link`), because a licence binds to that UUID.
## MongoDB Collections
`servers` · `keys` · `assignments` · `orgs` · `users` · `org_oidc` · `settings` · `secrets` · `workflows` · `workflow_steps` · `workflow_runs` · `monitors` · `incidents` · `monitor_rollups` · `notification_channels` · `console_sessions` · `audit_logs` · `migrations`
Every document except `migrations` carries `org_id`. Struct definitions are the source of truth — see `server/internal/models/`.
Notes that are not obvious from the structs:
- `servers.agent_token_hash` stores SHA-256 of the token, never plaintext. `pre_reg_token` is cleared after `Register()`. `status` is `pending``active` on register, `offline` when `last_seen` passes the threshold (swept every 2 min).
- `servers.inventory` holds the latest metrics snapshot with separate `metrics_at` / `static_at` timestamps.
- `keys.private_key_enc` and `passphrase_enc` are AES-256-GCM; the JSON form exposes only `has_private_key` / `has_passphrase`.
- `assignments.revoked_at: null` means active. Revocation is soft, preserving audit history.
- `workflow_runs.steps_snapshot` freezes the resolved steps so editing the library never rewrites history.
- `console_sessions.token_consumed_at` is set atomically to enforce one-time use.
- `users.auth_source` is `local`, `oidc` or `hq`. An `hq` user was projected from a Vantage HQ account and carries `hq_user_id`; HQ owns its role, password and existence.
Admin's own database is separate and holds `accounts` · `admin_instances` · `licenses` · `subscriptions` · `plans` · `catalogue` · `entitlements` · `paddle_events` · `staff_users` · `customer_users` · `instance_members` · `admin_audit`. `paddle_events` is the webhook idempotency log, unique on `event_id`: an event is claimed there before processing, and a duplicate of a handled event is a 200 no-op. `instance_members` is unique on `(instance_id, customer_user_id)` — one person holds at most one user in one instance, which makes a grant idempotent-by-refusal rather than silently doubling a projection. It is an *index* of the control-plane rows, not the authority (see "Grants project, they do not federate"). Admin has no migrations collection; `models.Backfill` runs on every boot and is idempotent by filtering on the absence of what it writes.
`plans` is keyed on `(deployment, tier)` — six rows, two deployments times three tiers — and holds base allowances only. **Every Paddle price ID lives in `catalogue`**, one row per priceable component (`base`, `limit`, `feature`), because a metered plan is priced by several prices and one map on a plan row cannot express that. `entitlements` holds one row per instance with `desired` beside `granted`: the checkout is built from `desired`, a licence is only ever signed from `granted`, and an abandoned checkout therefore leaves a `desired` that reached nothing. The two Free plans have **no catalogue rows at all**, which is what keeps Free outside Paddle.
### Migrations
`services.RunMigrations()` runs at boot, recording markers in `migrations`:
- `0001_default_org_backfill`
- `0002_settings_org_backfill` (must run before 0003 — 0003 can create a `default` org, which pushes 0002 into its ambiguous multi-org branch)
- `0003_missed_org_scopes`
Index builders (`EnsureAuthIndexes`, `EnsureSettingsIndexes`) are fatal on failure; `EnsureSecretIndexes` and `EnsureWorkflowIndexes` only warn.
---
## Agent Lifecycle
### Config file
Linux `/etc/vantage/config.yaml`, Windows `%ProgramData%\vantage\config.yaml`. Directory `0700`, file `0600`.
```yaml
server_url: "vantage.yourdomain.com:9090"
server_id: "<uuid>"
pre_reg_token: "<token>" # removed after first successful Register()
agent_token: "" # written by agent after Register()
poll_interval: 30s
tls: true
```
### Startup
```
1. Load config
2. If pre_reg_token present → Register() → save agent_token, clear pre_reg_token, reconnect
3. Start goroutines: command stream · update check (hourly) · inventory · monitors
4. Enter SyncKeys poll loop (default 30s)
```
### Poll loop
```
1. SyncKeys(server_id, agent_token, agent_version)
2. Non-Linux hosts stop here — Windows agents register and heartbeat only
3. Diff desired keys against /root/.ssh/authorized_keys; unchanged → no write
4. Changed → write .tmp, os.Rename() over the real file, chmod 0600
```
### Install
Linux: systemd unit at `/etc/systemd/system/vantage-agent.service`, `Restart=always`, runs as root.
Windows: MSI built by CI (WiX), or `installer/setup.ps1` registering the agent as a service via NSSM.
---
## Server Registration Flow
1. **Add Server** in the UI calls `POST /api/servers/new`, which generates a `server_id` and a pre-registration token (TTL 1 hour, single-use).
2. The UI shows a one-liner:
```bash
curl -fsSL https://vantage.yourdomain.com/install | \
bash -s -- --server-id=<id> --token=<token>
```
Windows gets the `/install.ps1` equivalent.
3. The script detects arch, downloads the agent from the Gitea release, verifies the SHA-256 checksum, writes the config, installs and starts the service.
4. The server flips to `active` on first sync.
`/install` is served dynamically, injecting the latest agent version from the Gitea API.
---
## Environment Variables (server)
| Name | Required | Notes |
| -------------------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GRPC_HOST` | **yes** | `host:port` agents dial. Boot fails without it — there is no safe default; falling back to the web host would hand agents a port that does not speak gRPC. |
| `MONGO_URI` | no | default `mongodb://localhost:27017` |
| `MONGO_DB` | no | default `vantage` |
| `REDIS_ADDR` | no | default `localhost:6379` |
| `KEY_ENCRYPTION_KEY` | yes in practice | 64-char hex (32 bytes) for AES-256-GCM. Required for private keys, secrets, OIDC secrets, RDP credentials. |
| `GITEA_HOST` | yes | used to build install scripts and agent download URLs |
| `GUACD_ADDR` | no | default `guacd:4822` |
| `APP_ROOT_LABEL` | no | default `vantage`; wrong value disables the host/session org guard |
| `VANTAGE_WORKFLOW_LOG_DIR` | no | where run logs are written |
| `FREE_INSTANCE_REAP_AFTER` | no | duration past a Free licence's expiry before the instance and all its data are deleted. **Empty disables the reaper, and empty is the default.** Set to `336h` in `docker-compose.site.yml` only — a self-hosted deployment must never reap. Must match admin's value, which only names the date in warning emails |
**sitesvc** (`deploy/docker-compose.site.yml` only):
| Name | Required | Notes |
| --------------------------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `MONGO_URI` | yes | **must point at the control plane's database.** sitesvc no longer provisions orgs itself, but it still refuses to start (`RequireMigratedDatabase`) against a database that has not run migration `0004` (the `orgs``instances` rename), and it (re)declares the shared `users.email` / `instances.slug` indexes at boot. The database name is read from the URI path; a URI without one is refused rather than defaulted. Note this differs from the server, which takes `MONGO_DB` separately. |
| `SMTP_HOST` / `SMTP_FROM` | yes | without them the contact form refuses (503) rather than silently dropping |
| `SMTP_TO` | no | default `support@hostxtra.co.uk`; contact enquiries only |
| `SMTP_PORT` | no | default `587`; `465` uses implicit TLS |
| `SMTP_USERNAME` / `SMTP_PASSWORD` | no | auth skipped when username is empty |
| `SITE_ORIGIN` | yes in practice | comma-separated allowed origins; unset refuses every cross-origin browser request |
| `TRUST_PROXY` | no | only `true` behind a proxy that overwrites `X-Forwarded-For`, or clients spoof past the rate limiter |
`deploy/docker-compose.yml` runs four services: `redis`, `guacd`, `server` (8080 + 9090), `web` (3000). MongoDB is external. `deploy/docker-compose.site.yml` adds four more — `site` (3003), `sitesvc` (8082), `admin` (8083) and `adminsite` (3004) — and is only used on vantage.hostxtra.co.uk.
`LICENSE_SIGNING_KEY` appears in **exactly one service in exactly one compose file**: `admin` in `docker-compose.site.yml`. It must never be added to `server`, and the self-hosted `docker-compose.yml` must never mention `admin` or `adminsite` at all. Admin uses an external Redis via `REDIS_ADDR`/`REDIS_USERNAME`/`REDIS_PASSWORD`; the base compose hardcodes `redis:6379` for `server`, so those variables reach admin only.
---
## Security
- gRPC over TLS; agents connect outbound only, no inbound firewall holes on managed servers.
- Per-server agent token stored as SHA-256 on the server, plaintext only in the agent's `0600` config.
- Pre-registration tokens are short-lived (1 hour) and single-use.
- AES-256-GCM at rest for private keys, key passphrases, vault secrets, OIDC client secrets, RDP/VNC credentials.
- Console session tokens are one-time; RDP credentials are consumed on tunnel open.
- ESO read token stored as a SHA-256 hash and rotatable.
- Unique indexes on `(instance_id, email)`, instance slug, settings instance and the ESO token hash are load-bearing for tenant isolation. So is the absence of any unscoped lookup by email.
- `authorized_keys` written `0600`, owned by root. The agent runs as root because it must.
- Every mutating API path writes an audit event.
---
## Frontend
Next.js 16 (App Router) + React 18, Tailwind 3, TanStack Query. Guacamole client bundled locally in `web/lib/guacamole-common.js`.
All three apps are **one visual system**, anchored on the logo navy. What differs between them is which end of it they stand on:
| App | Ground | Accent | Themes |
| --- | --- | --- | --- |
| `web/` | `--ground` dark, `#071628` | `#5b9be8` | dark only, locked |
| `site/` | token-based | `#0b2a58` light / `#5b9be8` dark | light + dark |
| `adminsite/` | token-based | `#0b2a58` light / `#5b9be8` dark | light + dark, light default |
`adminsite/app/globals.css` holds `site/app/globals.css`'s token blocks **copied verbatim** — same names, same values. `web/app/globals.css` holds the same tokens too, but only the **dark** values, since it does not switch. **Change a token in all three files in the same commit; nothing enforces the match automatically**, the same shape of hazard as sitesvc's mirrored slug rules.
There is a **fourth** copy, and it is the one people forget: `shared/mail/templates/layout.html.tmpl` carries web/'s dark values as literal hex. Email clients support neither `var()` nor a reliable `prefers-color-scheme`, so the token indirection is simply not available there — an email is read before the recipient clicks through to the control plane, and the two should not look like different products. Every colour in the email system is in that one file, in the same way no component in the three web apps carries a hex.
Tailwind in all three maps `var(--…)` references only, so **no component in any of them may carry a hex value**. The names differ per app on purpose, because each app has its own subject: `site/` calls the semantic three `--up`/`--pend`/`--down` for monitor state, `adminsite/` aliases them to `valid`/`warn`/`expired` for licence state, and `web/` to `success`/`warning`/`danger`. Same colours, honest names on each side.
`web/` stores its tokens as **RGB channel triplets** with the hex in a trailing comment, and derives `--token: rgb(var(--token-rgb))` from them. That is not a style preference: the console leans on Tailwind's opacity modifiers (`bg-danger/10`, `border-accent/50`, `ring-accent/30`) in a way the other two do not, and `<alpha-value>` only compiles against channels. Keep the hex comments — they are what lets the three token blocks still be diffed by eye. `web/` also adds three tokens site/ has no use for: `--accent-hover` and `--down-hover` (site/ brightens with a CSS `filter`, which a Tailwind colour token cannot do) and `--well`, the floor beneath the ground for install one-liners, key blobs and run logs — surfaces showing machine output rather than interface.
`web/` is locked to dark and `adminsite/` defaults to **light**, and that pairing is the point: an operator with both open should never mistake one for the other before clicking Reissue. Now that both are drawn from the same palette the distinction rests **entirely** on the ground, so do not make dark the adminsite default and do not give web/ a light theme. State never reads by colour alone in either: every pill carries a distinct shape and a text label. The same argument applies one level in: the **staff** masthead sits on `--panel-2` with a `STAFF` chip, so staff and customer screens are not identical either.
`web/` collapses Tailwind's radius scale — `md`, `lg` and `xl` all resolve to site/'s 4px — rather than rewriting the ~140 `rounded-lg` classes across its pages. Every one of them meant "a panel corner", and `tailwind.config.ts` is now where that decision lives. `rounded-full` is untouched: status dots and pills still need it.
**The `adminsite/` shell.** `AppBar` is the single masthead — identity, nav, environment, account menu — and it belongs to the two authenticated layouts, never to `app/layout.tsx`, so `/login` and `/accept-invite` do not render navigation they cannot use. Nav active state is derived from `usePathname`; do not hardcode it. `PageHeader` gives every screen the same back link, title, actions and **record line** (the reference number in mono, click-to-copy) — the reference is what people paste into support tickets, so it has a fixed slot rather than a per-page treatment. `PageFrame` is the main-plus-320px-rail split; the rail carries only what is true account-wide, which is why there is no plan card in it — **tier, limits and expiry belong to a licence, and a licence belongs to one instance**, so an account holding a Free cloud instance and a Professional self-hosted one has no single plan.
Customer nav is three destinations — Overview, People, Billing. Settings is in the account menu because it is your password, not a place, and appearance lives there too: `AccountMenu` is the only thing that sets `data-theme`, which the token blocks have always supported in both directions.
`InstanceRecord` is one component open or closed, and it **replaced** `InstanceCard`. Closed it is a row; open it adds licence contents, members and actions. It defaults open when the instance is the only one or needs attention, and a manual toggle is remembered per instance in `localStorage`. Do not reintroduce a second summary component — the split is what left a one-instance account showing a third of a row and nothing else.
| Route | Purpose |
| --------------------------------------------------------------- | ----------------------------------------------------------------------- |
| `/setup` | First-run bootstrap: create the first org and owner |
| `/login` | Local or OIDC sign-in |
| `/` | Fleet dashboard |
| `/servers`, `/servers/new`, `/servers/[id]` | Fleet list, install one-liner, server detail (keys, inventory, updates) |
| `/servers/[id]/console` | Browser SSH/RDP/VNC session |
| `/keys`, `/keys/[id]` | Key library; assign and revoke per server |
| `/workflows`, `/workflows/[id]`, `/workflows/[id]/runs[/runId]` | Compose, run, and follow live logs |
| `/steps` | Reusable step library |
| `/monitors`, `/monitors/new`, `/monitors/[id][/edit]` | Checks, uptime, incidents |
| `/secrets`, `/secrets/[group]` | Vault |
| `/audit` | Audit log |
| `/settings`, `/settings/notifications`, `/settings/license` | Members, OIDC, alerts, retention, ESO token · channels · licence |
**`/settings` is one page, not a section.** Members and single sign-on used to
live at `/settings/instance` with their own sidebar entry; they are now the
**Access** group at the top of `/settings`, above **Monitoring** and
**Integrations**. Splitting "who can sign in" away from "how this instance
behaves" made two half-pages and a nav entry called Instance that no one could
distinguish from Settings. `next.config.ts` keeps a permanent redirect from the
old path. The cards live in `web/components/settings/` rather than in the page,
which is also where the `Field`/`inputClass` pair the three of them share now
lives — one copy instead of the three that existed while they were apart.
---
## CI/CD — Gitea Actions
### `agent-release.yml` — triggered by `agent/v*` tags
Builds `linux/amd64`, `linux/arm64`, `windows/amd64`, writes `checksums.txt`, creates a Gitea release. A second `msi` job on `windows-2022` packages the WiX installer.
```bash
GOOS=linux GOARCH=amd64 go build \
-ldflags="-s -w -X main.Version=${VERSION}" \
-o dist/vantage-agent-linux-amd64 ./cmd
```
### `server-deploy.yml` — triggered on every push to `main`
Builds and pushes six images to the Gitea container registry: `server`, `web`, `site`, `sitesvc`, `admin` and `adminsite`.
Note that despite the name, **this workflow does not deploy** — it only builds and pushes. There is no SSH step. Rolling images out is a separate manual step on the host:
```bash
cd /opt/vantage && docker compose -f docker-compose.yml -f docker-compose.site.yml pull && \
docker compose -f docker-compose.yml -f docker-compose.site.yml up -d --remove-orphans
```
**Each image only rebuilds when its own inputs changed.** A `git diff` against `github.event.before` decides, which is why the checkout uses `fetch-depth: 0` — the default shallow clone has one commit and nothing to diff — and why `git` is installed in the `docker:dind` container. The mapping follows the build contexts exactly:
| Image | Rebuilds when |
| --- | --- |
| `server` | `server/`, `shared/`, `proto/`, `go.work` |
| `admin` | `admin/`, `shared/`, `go.work` |
| `sitesvc` | `sitesvc/`, `shared/`, `go.work` |
| `web` · `site` · `adminsite` | their own directory only |
`shared/` fans out to all three Go images because each of their Dockerfiles copies `shared/` from a root context — **if a fourth service ever imports `shared/`, add it to that list or it will ship stale**. A change to the workflow file rebuilds everything, since a build arg is baked into the image. So does anything that leaves no trustworthy base commit: a manual `workflow_dispatch`, a new branch, or a force-push whose old head is gone.
The gap this leaves: **changing a repo variable pushes no commit, so nothing rebuilds.** After editing `API_URL`, `ADMIN_API_URL`, `HQ_URL` or `ADMIN_ENV`, run the workflow manually — that is what `workflow_dispatch` is there for. Base images also stop being refreshed on a service nobody touches; a periodic manual run covers that.
### Tagging
```bash
git tag agent/v1.0.0 && git push origin agent/v1.0.0 # agent release
git push origin main # server + web deploy
```
### Secrets / variables
| Name | Type | Value |
| -------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `RELEASE_TOKEN` | Secret | Gitea API token, `write:release` |
| `REGISTRY_USER` | Secret | Gitea username |
| `REGISTRY_PASSWORD` | Secret | Gitea token, `write:packages` |
| `GITEA_HOST` | Variable | `gitea.hostxtra.co.uk` |
| `DOCKER_HOST` | Variable | registry host used for image tags |
| `API_URL` | Variable | baked into the `web` image at build time |
| `SITE_API_URL` | Variable | **browser-reachable** sitesvc URL, baked into the `site` image. Required — if empty, both forms report "not connected" and submit nowhere. Must also be in sitesvc's `SITE_ORIGIN`. |
| `SITE_CONTACT_EMAIL` | Variable | optional; address shown when a form is misconfigured |
| `ADMIN_API_URL` | Variable | **browser-reachable** admin URL, baked into **both** the `adminsite` and `site` images — `site/start` posts account signups straight to admin. Same footgun as `SITE_API_URL`: wrong here and every request fails at runtime with the not-connected panel. |
| `ADMIN_ENV` | Variable | `production` or `sandbox`; drives the persistent environment badge. Anything but `sandbox` reads as production. |
| `HQ_URL` | Variable | optional; browser URL of the HQ portal, baked into `web` so an `hq`-sourced member links to where they are managed. Empty on self-hosted, which renders a plain label instead. |
| `PADDLE_CLIENT_TOKEN`| Variable | **browser** Paddle token, baked into the `adminsite` image for checkout. A repo-variable change pushes no commit, so rebuild `adminsite` manually via `workflow_dispatch` after editing it. |
| `PADDLE_ENV` | Variable | `sandbox` or `production`; baked into `adminsite` AND read by `admin` at runtime. Selects which `catalogue` price IDs are served, and must match on both sides. |
| `PADDLE_API_KEY` | Secret | server-side Paddle key, read by `admin` at runtime. Boot-required. |
| `PADDLE_WEBHOOK_SECRET` | Secret | webhook signature verification, read by `admin`. Boot-required — an unverified endpoint is one anyone can issue licences through. |
---
## Design Decisions
- **gRPC for agent traffic** — strong typing and cheap versioning; polling for state, one bidirectional stream for commands.
- **Outbound-only agents** — no inbound ports on managed servers, works behind NAT.
- **Poll for keys, push for commands** — a 30s key poll is fine, but running a workflow step should not wait up to 30s.
- **Atomic `authorized_keys` rewrite** — temp file plus `os.Rename()`; a machine that dies mid-write keeps the old file.
- **Fingerprint diffing before write** — no disk churn on unchanged state.
- **Soft revocation**`revoked_at` rather than deletes; preserves audit history.
- **Run snapshots** — workflow runs freeze their resolved steps so editing a step never rewrites past runs.
- **Monitors run in two places** — server-side for external endpoints, agent-side for anything only reachable from inside the target network.
- **Redis for sessions only** — all durable state stays in MongoDB; losing Redis logs everyone out and nothing else.
- **guacd for console** — protocol handling is Guacamole's problem, not ours; we proxy the WebSocket and manage credentials.
- **`org_id` on every document** — isolation enforced at the query layer, not by separate databases.
- **root only** — manages `/root/.ssh/authorized_keys`; no per-user key management.
- **Windows agents are second-class by design** — register, heartbeat, run steps, report inventory; no `authorized_keys` management.
- **Deletion lives in the control plane** — admin sends the warnings because it knows the billing address; the control plane performs the delete because it is the only service that knows which collections carry `instance_id`. Mirroring that list into admin would drift, and a drift there deletes the wrong rows.
@@ -0,0 +1,12 @@
{
"kind": "vantage.step/v1",
"name": "Apply Package Updates",
"description": "Apply all pending OS package updates. Supports apt, dnf, yum, zypper, apk and pacman.",
"interpreter": "bash",
"script": "set -u\nif command -v apt-get >/dev/null 2>&1; then\n export DEBIAN_FRONTEND=noninteractive\n apt-get update -qq && apt-get -y -qq upgrade\nelif command -v dnf >/dev/null 2>&1; then\n dnf -y upgrade\nelif command -v yum >/dev/null 2>&1; then\n yum -y update\nelif command -v zypper >/dev/null 2>&1; then\n zypper --non-interactive update\nelif command -v apk >/dev/null 2>&1; then\n apk update && apk upgrade\nelif command -v pacman >/dev/null 2>&1; then\n pacman -Syu --noconfirm\nelse\n echo \"no supported package manager found\"\n exit 1\nfi\nretVal=$?\nif [ $retVal -ne 0 ]; then\n echo \"package update failed\"\n exit 1\nfi\necho \"packages up to date\"\n# Debian and Ubuntu drop this file when a new kernel or libc needs a restart.\n# Reported rather than acted on: rebooting a fleet is a decision, not a detail.\nif [ -f /var/run/reboot-required ]; then\n echo \"REBOOT_REQUIRED=true\" >> $WORKFLOW_ENV\n echo \"a reboot is required to finish applying updates\"\nelse\n echo \"REBOOT_REQUIRED=false\" >> $WORKFLOW_ENV\nfi",
"declared_outputs": [
"REBOOT_REQUIRED"
],
"declared_inputs": [],
"secret_refs": []
}
+23
View File
@@ -0,0 +1,23 @@
{
"kind": "vantage.step/v1",
"name": "Check Port Is Listening",
"description": "Fail unless something is listening on a TCP port.",
"interpreter": "bash",
"script": "set -u\nhost=\"${host:-127.0.0.1}\"\nif command -v nc >/dev/null 2>&1; then\n nc -z -w 5 \"$host\" \"$port\" >/dev/null 2>&1\n ok=$?\nelse\n # bash builds /dev/tcp in, so this needs nothing installed.\n timeout 5 bash -c \"cat < /dev/null > /dev/tcp/$host/$port\" >/dev/null 2>&1\n ok=$?\nfi\nif [ $ok -ne 0 ]; then\n echo \"PORT_OPEN=false\" >> $WORKFLOW_ENV\n echo \"nothing listening on $host:$port\"\n exit 1\nfi\necho \"PORT_OPEN=true\" >> $WORKFLOW_ENV\necho \"$host:$port is open\"",
"declared_outputs": [
"PORT_OPEN"
],
"declared_inputs": [
{
"name": "host",
"default": "127.0.0.1",
"description": "host to test"
},
{
"name": "port",
"default": "",
"description": "TCP port to test"
}
],
"secret_refs": []
}
+23
View File
@@ -0,0 +1,23 @@
{
"kind": "vantage.step/v1",
"name": "Copy File/Directory",
"description": "Copy a file or directory, preserving mode, ownership and timestamps.",
"interpreter": "bash",
"script": "set -u\nif [ ! -e \"$source\" ]; then\n echo \"source $source does not exist\"\n exit 1\nfi\ncp -a \"$source\" \"$destination\" || { echo \"failed to copy $source to $destination\"; exit 1; }\necho \"copied $source to $destination\"\necho \"DEST_PATH=$destination\" >> $WORKFLOW_ENV",
"declared_outputs": [
"DEST_PATH"
],
"declared_inputs": [
{
"name": "source",
"default": "",
"description": "path to copy from"
},
{
"name": "destination",
"default": "",
"description": "path to copy to"
}
],
"secret_refs": []
}
@@ -0,0 +1,23 @@
{
"kind": "vantage.step/v1",
"name": "Create Directory",
"description": "Create a directory, including any missing parents.",
"interpreter": "bash",
"script": "set -u\nmkdir -p \"$path\" || { echo \"failed to create $path\"; exit 1; }\nif [ -n \"${mode:-}\" ]; then\n chmod \"$mode\" \"$path\" || { echo \"failed to set mode $mode on $path\"; exit 1; }\nfi\necho \"created $path\"\necho \"DIR_PATH=$path\" >> $WORKFLOW_ENV",
"declared_outputs": [
"DIR_PATH"
],
"declared_inputs": [
{
"name": "path",
"default": "",
"description": "directory to create"
},
{
"name": "mode",
"default": "",
"description": "optional octal mode, e.g. 0750"
}
],
"secret_refs": []
}
+16
View File
@@ -0,0 +1,16 @@
{
"kind": "vantage.step/v1",
"name": "Delete File/Directory",
"description": "Delete a path. Refuses the root filesystem and an empty value.",
"interpreter": "bash",
"script": "set -u\n# A step that runs as root on every server in a selector has to refuse the\n# one input that would wipe the fleet. An unset variable expands to empty,\n# so the empty case is the accident this actually guards against.\ncase \"$path\" in\n \"\"|\"/\"|\"/.\"|\"/..\")\n echo \"refusing to delete '$path'\"\n exit 1\n ;;\nesac\nif [ ! -e \"$path\" ]; then\n echo \"$path does not exist, nothing to do\"\n exit 0\nfi\nrm -rf \"$path\" || { echo \"failed to delete $path\"; exit 1; }\necho \"deleted $path\"",
"declared_outputs": [],
"declared_inputs": [
{
"name": "path",
"default": "",
"description": "path to delete"
}
],
"secret_refs": []
}
+24
View File
@@ -0,0 +1,24 @@
{
"kind": "vantage.step/v1",
"name": "Disk Usage Report",
"description": "Report usage for a mount point and fail past a threshold.",
"interpreter": "bash",
"script": "set -u\nmount=\"${mountPoint:-/}\"\nlimit=\"${maxPercent:-90}\"\ndf -h \"$mount\"\nused=$(df --output=pcent \"$mount\" | tail -1 | tr -dc \"0-9\")\navail=$(df -h --output=avail \"$mount\" | tail -1 | tr -d \" \")\necho \"DISK_USED_PERCENT=$used\" >> $WORKFLOW_ENV\necho \"DISK_AVAILABLE=$avail\" >> $WORKFLOW_ENV\nif [ \"$used\" -ge \"$limit\" ]; then\n echo \"$mount is ${used}% full, at or over the ${limit}% limit\"\n exit 1\nfi\necho \"$mount is ${used}% full, ${avail} available\"",
"declared_outputs": [
"DISK_USED_PERCENT",
"DISK_AVAILABLE"
],
"declared_inputs": [
{
"name": "mountPoint",
"default": "/",
"description": "mount point to measure"
},
{
"name": "maxPercent",
"default": "90",
"description": "fail at or above this percentage"
}
],
"secret_refs": []
}
@@ -0,0 +1,16 @@
{
"kind": "vantage.step/v1",
"name": "Docker Compose Pull and Up",
"description": "Pull the latest images for a compose project and recreate its containers.",
"interpreter": "bash",
"script": "set -u\ncd \"$projectDir\" || { echo \"no such directory: $projectDir\"; exit 1; }\nif docker compose version >/dev/null 2>&1; then\n dc=\"docker compose\"\nelif command -v docker-compose >/dev/null 2>&1; then\n dc=\"docker-compose\"\nelse\n echo \"docker compose is not installed\"\n exit 1\nfi\n$dc pull || { echo \"pull failed\"; exit 1; }\n$dc up -d --remove-orphans || { echo \"up failed\"; exit 1; }\n$dc ps",
"declared_outputs": [],
"declared_inputs": [
{
"name": "projectDir",
"default": "",
"description": "directory holding docker-compose.yml"
}
],
"secret_refs": []
}
@@ -0,0 +1,16 @@
{
"kind": "vantage.step/v1",
"name": "Download File (Linux)",
"description": "Download a file over HTTP to a local path",
"interpreter": "bash",
"script": "out=$(mktemp -p ./)\necho \"Downloading file from $url\"\nwget -q $url -O $out\nretVal=$?\nif [ $retVal -ne 0 ]; then\n echo \"failed to download file from url\"\n exit 1\nfi\necho \"FILE_PATH=$out\" \u003e\u003e $WORKFLOW_ENV",
"declared_outputs": ["FILE_PATH"],
"declared_inputs": [
{
"name": "url",
"default": "",
"description": ""
}
],
"secret_refs": []
}
@@ -0,0 +1,16 @@
{
"kind": "vantage.step/v1",
"name": "Enable Linux Service",
"description": "Enable a systemd unit so it starts on boot.",
"interpreter": "bash",
"script": "set -u\necho \"enabling service $serviceName\"\nsystemctl enable \"$serviceName\" || { echo \"failed to enable $serviceName\"; exit 1; }\necho \"$serviceName enabled\"",
"declared_outputs": [],
"declared_inputs": [
{
"name": "serviceName",
"default": "",
"description": "systemd unit to enable"
}
],
"secret_refs": []
}
@@ -0,0 +1,23 @@
{
"kind": "vantage.step/v1",
"name": "Extract Archive",
"description": "Extract a tar, tar.gz, tar.bz2, tar.xz or zip archive into a directory.",
"interpreter": "bash",
"script": "set -u\ndest=\"${destination:-.}\"\nif [ ! -f \"$archive\" ]; then\n echo \"archive $archive does not exist\"\n exit 1\nfi\nmkdir -p \"$dest\"\ncase \"$archive\" in\n *.tar.gz|*.tgz) tar -xzf \"$archive\" -C \"$dest\" ;;\n *.tar.bz2|*.tbz2) tar -xjf \"$archive\" -C \"$dest\" ;;\n *.tar.xz|*.txz) tar -xJf \"$archive\" -C \"$dest\" ;;\n *.tar) tar -xf \"$archive\" -C \"$dest\" ;;\n *.zip)\n command -v unzip >/dev/null 2>&1 || { echo \"unzip is not installed\"; exit 1; }\n unzip -oq \"$archive\" -d \"$dest\"\n ;;\n *)\n echo \"unsupported archive type: $archive\"\n exit 1\n ;;\nesac\nretVal=$?\nif [ $retVal -ne 0 ]; then\n echo \"failed to extract $archive\"\n exit 1\nfi\necho \"extracted $archive into $dest\"\necho \"EXTRACT_DIR=$dest\" >> $WORKFLOW_ENV",
"declared_outputs": [
"EXTRACT_DIR"
],
"declared_inputs": [
{
"name": "archive",
"default": "",
"description": "archive file to extract"
},
{
"name": "destination",
"default": ".",
"description": "directory to extract into"
}
],
"secret_refs": []
}
+10
View File
@@ -0,0 +1,10 @@
{
"kind": "vantage.step/v1",
"name": "Get Host Name",
"description": "Gets the agents hostname",
"interpreter": "bash",
"script": "HOSTNAME=$(hostname)\necho $HOSTNAME\necho \"HOSTNAME=$HOSTNAME\" \u003e\u003e $WORKFLOW_ENV",
"declared_outputs": ["HOSTNAME"],
"declared_inputs": [],
"secret_refs": []
}
@@ -0,0 +1,33 @@
{
"kind": "vantage.step/v1",
"name": "HTTP Health Check",
"description": "Request a URL and fail unless it answers with the expected status.",
"interpreter": "bash",
"script": "set -u\nexpected=\"${expectedStatus:-200}\"\nattempts=\"${retries:-3}\"\ndelay=\"${retryDelay:-5}\"\nstatus=\"\"\ni=1\n# Retries live in the script rather than in on_failure: a service coming up\n# after a restart wants a few seconds, not a whole step re-dispatched.\nwhile [ \"$i\" -le \"$attempts\" ]; do\n status=$(curl -s -o /dev/null -w \"%{http_code}\" --max-time 10 \"$url\" || echo \"000\")\n echo \"attempt $i: $url returned $status\"\n if [ \"$status\" = \"$expected\" ]; then\n break\n fi\n i=$(( i + 1 ))\n if [ \"$i\" -le \"$attempts\" ]; then sleep \"$delay\"; fi\ndone\necho \"HTTP_STATUS=$status\" >> $WORKFLOW_ENV\nif [ \"$status\" != \"$expected\" ]; then\n echo \"$url returned $status, expected $expected\"\n exit 1\nfi\necho \"$url is healthy\"",
"declared_outputs": [
"HTTP_STATUS"
],
"declared_inputs": [
{
"name": "url",
"default": "",
"description": "URL to request"
},
{
"name": "expectedStatus",
"default": "200",
"description": "HTTP status that counts as healthy"
},
{
"name": "retries",
"default": "3",
"description": "how many attempts before failing"
},
{
"name": "retryDelay",
"default": "5",
"description": "seconds between attempts"
}
],
"secret_refs": []
}
@@ -0,0 +1,16 @@
{
"kind": "vantage.step/v1",
"name": "List Directory",
"description": "Lists the files in the specified path",
"interpreter": "bash",
"script": "if [ ! -e $path ]; then\n echo \"file or directory doesn't exist: $path\"\n exit 1\nfi\nls -l $path",
"declared_outputs": [],
"declared_inputs": [
{
"name": "path",
"default": "./",
"description": ""
}
],
"secret_refs": []
}
+13
View File
@@ -0,0 +1,13 @@
{
"kind": "vantage.step/v1",
"name": "Memory Usage Report",
"description": "Report memory usage as a percentage of total.",
"interpreter": "bash",
"script": "set -u\nfree -h\ntotal=$(free -m | awk \"/^Mem:/ {print \\$2}\")\nused=$(free -m | awk \"/^Mem:/ {print \\$3}\")\npct=$(( used * 100 / total ))\necho \"MEM_USED_PERCENT=$pct\" >> $WORKFLOW_ENV\necho \"MEM_USED_MB=$used\" >> $WORKFLOW_ENV\necho \"memory ${pct}% used (${used}MB of ${total}MB)\"",
"declared_outputs": [
"MEM_USED_PERCENT",
"MEM_USED_MB"
],
"declared_inputs": [],
"secret_refs": []
}
@@ -0,0 +1,16 @@
{
"kind": "vantage.step/v1",
"name": "Reboot Server",
"description": "Schedule a reboot a minute out, so the step reports success before the machine goes down.",
"interpreter": "bash",
"script": "set -u\ndelay=\"${delayMinutes:-1}\"\n# Scheduled rather than immediate on purpose: `shutdown -r now` kills the\n# agent before it can report, and the run records a failure on a server\n# that did exactly what it was told.\necho \"rebooting in $delay minute(s)\"\nshutdown -r \"+$delay\" \"Reboot requested by Vantage\" || { echo \"failed to schedule a reboot\"; exit 1; }",
"declared_outputs": [],
"declared_inputs": [
{
"name": "delayMinutes",
"default": "1",
"description": "minutes to wait before rebooting"
}
],
"secret_refs": []
}
@@ -0,0 +1,16 @@
{
"kind": "vantage.step/v1",
"name": "Restart Linux Service",
"description": "Restart a systemd unit and fail if it does not come back up.",
"interpreter": "bash",
"script": "set -u\necho \"restarting service $serviceName\"\nsystemctl restart \"$serviceName\" || { echo \"failed to restart $serviceName\"; exit 1; }\nsystemctl is-active --quiet \"$serviceName\" || {\n echo \"$serviceName did not come back up\"\n systemctl status \"$serviceName\" --no-pager --lines=20 || true\n exit 1\n}\necho \"$serviceName is active\"",
"declared_outputs": [],
"declared_inputs": [
{
"name": "serviceName",
"default": "",
"description": "systemd unit to restart, e.g. nginx"
}
],
"secret_refs": []
}
@@ -0,0 +1,19 @@
{
"kind": "vantage.step/v1",
"name": "Linux Service Status",
"description": "Report whether a systemd unit is active and enabled. Does not fail on a stopped unit.",
"interpreter": "bash",
"script": "set -u\nstate=$(systemctl is-active \"$serviceName\" 2>/dev/null || true)\nenabled=$(systemctl is-enabled \"$serviceName\" 2>/dev/null || true)\necho \"$serviceName: state=$state enabled=$enabled\"\necho \"SERVICE_STATE=$state\" >> $WORKFLOW_ENV\necho \"SERVICE_ENABLED=$enabled\" >> $WORKFLOW_ENV",
"declared_outputs": [
"SERVICE_STATE",
"SERVICE_ENABLED"
],
"declared_inputs": [
{
"name": "serviceName",
"default": "",
"description": "systemd unit to inspect"
}
],
"secret_refs": []
}
@@ -0,0 +1,31 @@
{
"kind": "vantage.step/v1",
"name": "Set Permissions and Ownership",
"description": "Set the mode and optionally the owner of a path.",
"interpreter": "bash",
"script": "set -u\nif [ ! -e \"$path\" ]; then\n echo \"$path does not exist\"\n exit 1\nfi\nrecurse=\"\"\nif [ \"${recursive:-false}\" = \"true\" ]; then\n recurse=\"-R\"\nfi\nif [ -n \"${mode:-}\" ]; then\n chmod $recurse \"$mode\" \"$path\" || { echo \"failed to set mode\"; exit 1; }\n echo \"set mode $mode on $path\"\nfi\nif [ -n \"${owner:-}\" ]; then\n chown $recurse \"$owner\" \"$path\" || { echo \"failed to set owner\"; exit 1; }\n echo \"set owner $owner on $path\"\nfi",
"declared_outputs": [],
"declared_inputs": [
{
"name": "path",
"default": "",
"description": "path to change"
},
{
"name": "mode",
"default": "",
"description": "octal mode, e.g. 0640"
},
{
"name": "owner",
"default": "",
"description": "owner, e.g. root:root"
},
{
"name": "recursive",
"default": "false",
"description": "true to apply recursively"
}
],
"secret_refs": []
}
@@ -0,0 +1,16 @@
{
"kind": "vantage.step/v1",
"name": "Start Linux Service",
"description": "Start a linux systemd service",
"interpreter": "bash",
"script": "echo \"starting service $serviceName\"\nsystemctl start $serviceName\nretVal=$?\nif [ $retVal -ne 0 ]; then\n echo \"failed to start service\"\n exit 1\nfi",
"declared_outputs": [],
"declared_inputs": [
{
"name": "serviceName",
"default": "",
"description": ""
}
],
"secret_refs": []
}
@@ -0,0 +1,16 @@
{
"kind": "vantage.step/v1",
"name": "Stop Linux Service",
"description": "Stops a linux systemd service",
"interpreter": "bash",
"script": "echo \"stopping service $serviceName\"\nsystemctl stop $serviceName\nretVal=$?\nif [ $retVal -ne 0 ]; then\n echo \"failed to stop service\"\n exit 1\nfi",
"declared_outputs": [],
"declared_inputs": [
{
"name": "serviceName",
"default": "",
"description": ""
}
],
"secret_refs": []
}
+21
View File
@@ -0,0 +1,21 @@
{
"kind": "vantage.step/v1",
"name": "Tail Log File",
"description": "Print the last N lines of a file, for reading a log after a deployment step.",
"interpreter": "bash",
"script": "set -u\nlines=\"${lines:-50}\"\nif [ ! -f \"$path\" ]; then\n echo \"$path does not exist\"\n exit 1\nfi\necho \"last $lines lines of $path:\"\ntail -n \"$lines\" \"$path\"",
"declared_outputs": [],
"declared_inputs": [
{
"name": "path",
"default": "",
"description": "log file to read"
},
{
"name": "lines",
"default": "50",
"description": "how many lines to print"
}
],
"secret_refs": []
}
+29
View File
@@ -0,0 +1,29 @@
{
"kind": "vantage.step/v1",
"name": "TLS Certificate Expiry",
"description": "Report days remaining on a TLS certificate and fail under a threshold.",
"interpreter": "bash",
"script": "set -u\nport=\"${port:-443}\"\nmin=\"${minDays:-14}\"\ncommand -v openssl >/dev/null 2>&1 || { echo \"openssl is not installed\"; exit 1; }\n# -servername sends SNI, without which a shared host returns the wrong\n# certificate and the expiry reported here belongs to someone else.\nend=$(echo | openssl s_client -servername \"$host\" -connect \"$host:$port\" 2>/dev/null \\\n | openssl x509 -noout -enddate | cut -d= -f2)\nif [ -z \"$end\" ]; then\n echo \"could not read a certificate from $host:$port\"\n exit 1\nfi\nendEpoch=$(date -d \"$end\" +%s)\nnowEpoch=$(date +%s)\ndays=$(( (endEpoch - nowEpoch) / 86400 ))\necho \"CERT_DAYS_REMAINING=$days\" >> $WORKFLOW_ENV\necho \"CERT_EXPIRES=$end\" >> $WORKFLOW_ENV\necho \"$host:$port expires in $days days ($end)\"\nif [ \"$days\" -lt \"$min\" ]; then\n echo \"fewer than $min days remaining\"\n exit 1\nfi",
"declared_outputs": [
"CERT_DAYS_REMAINING",
"CERT_EXPIRES"
],
"declared_inputs": [
{
"name": "host",
"default": "",
"description": "hostname to check"
},
{
"name": "port",
"default": "443",
"description": "TLS port"
},
{
"name": "minDays",
"default": "14",
"description": "fail below this many days remaining"
}
],
"secret_refs": []
}
@@ -0,0 +1,23 @@
{
"kind": "vantage.step/v1",
"name": "Download File (Windows)",
"description": "Download a file over HTTP to a local path.",
"interpreter": "powershell",
"script": "$ErrorActionPreference = \"Stop\"\n$url = $env:url\n$dest = if ($env:destination) { $env:destination } else { Join-Path $env:TEMP ([System.IO.Path]::GetFileName($url)) }\nWrite-Output \"downloading $url\"\ntry {\n # -UseBasicParsing keeps this working on Server Core, where the IE\n # engine Invoke-WebRequest otherwise reaches for is not installed.\n Invoke-WebRequest -Uri $url -OutFile $dest -UseBasicParsing\n} catch {\n Write-Output \"failed to download: $_\"\n exit 1\n}\nWrite-Output \"saved to $dest\"\nAdd-Content -Path $env:WORKFLOW_ENV -Value \"FILE_PATH=$dest\"",
"declared_outputs": [
"FILE_PATH"
],
"declared_inputs": [
{
"name": "url",
"default": "",
"description": "URL to download"
},
{
"name": "destination",
"default": "",
"description": "where to save it; defaults to a file in TEMP"
}
],
"secret_refs": []
}
@@ -0,0 +1,16 @@
{
"kind": "vantage.step/v1",
"name": "Reboot Windows Server",
"description": "Schedule a reboot a minute out, so the step reports success before the machine goes down.",
"interpreter": "powershell",
"script": "$ErrorActionPreference = \"Stop\"\n$delay = if ($env:delaySeconds) { [int]$env:delaySeconds } else { 60 }\nWrite-Output \"rebooting in $delay second(s)\"\n& shutdown.exe /r /t $delay /c \"Reboot requested by Vantage\"\nif ($LASTEXITCODE -ne 0) {\n Write-Output \"failed to schedule a reboot\"\n exit 1\n}",
"declared_outputs": [],
"declared_inputs": [
{
"name": "delaySeconds",
"default": "60",
"description": "seconds to wait before rebooting"
}
],
"secret_refs": []
}
@@ -0,0 +1,16 @@
{
"kind": "vantage.step/v1",
"name": "Restart Windows Service",
"description": "Restart a Windows service and fail if it does not come back up.",
"interpreter": "powershell",
"script": "$ErrorActionPreference = \"Stop\"\n$name = $env:serviceName\nWrite-Output \"restarting service $name\"\ntry {\n Restart-Service -Name $name -Force\n} catch {\n Write-Output \"failed to restart ${name}: $_\"\n exit 1\n}\n$svc = Get-Service -Name $name\nif ($svc.Status -ne \"Running\") {\n Write-Output \"$name is $($svc.Status), not Running\"\n exit 1\n}\nWrite-Output \"$name is running\"",
"declared_outputs": [],
"declared_inputs": [
{
"name": "serviceName",
"default": "",
"description": "Windows service name to restart"
}
],
"secret_refs": []
}
@@ -0,0 +1,24 @@
{
"kind": "vantage.step/v1",
"name": "Windows Disk Report",
"description": "Report free space on a drive and fail past a usage threshold.",
"interpreter": "powershell",
"script": "$ErrorActionPreference = \"Stop\"\n$letter = if ($env:driveLetter) { $env:driveLetter } else { \"C\" }\n$limit = if ($env:maxPercent) { [int]$env:maxPercent } else { 90 }\n$d = Get-PSDrive -Name $letter -ErrorAction SilentlyContinue\nif ($null -eq $d) {\n Write-Output \"drive $letter not found\"\n exit 1\n}\n$total = $d.Used + $d.Free\n$pct = [math]::Round(($d.Used / $total) * 100)\n$freeGb = [math]::Round($d.Free / 1GB, 1)\nWrite-Output \"${letter}: is $pct% full, $freeGb GB free\"\nAdd-Content -Path $env:WORKFLOW_ENV -Value \"DISK_USED_PERCENT=$pct\"\nAdd-Content -Path $env:WORKFLOW_ENV -Value \"DISK_FREE_GB=$freeGb\"\nif ($pct -ge $limit) {\n Write-Output \"at or over the $limit% limit\"\n exit 1\n}",
"declared_outputs": [
"DISK_USED_PERCENT",
"DISK_FREE_GB"
],
"declared_inputs": [
{
"name": "driveLetter",
"default": "C",
"description": "drive letter, without a colon"
},
{
"name": "maxPercent",
"default": "90",
"description": "fail at or above this percentage"
}
],
"secret_refs": []
}
@@ -0,0 +1,19 @@
{
"kind": "vantage.step/v1",
"name": "Windows Service Status",
"description": "Report a Windows service's status and start type. Does not fail on a stopped service.",
"interpreter": "powershell",
"script": "$ErrorActionPreference = \"Stop\"\n$name = $env:serviceName\n$svc = Get-Service -Name $name -ErrorAction SilentlyContinue\nif ($null -eq $svc) {\n Write-Output \"$name is not installed\"\n Add-Content -Path $env:WORKFLOW_ENV -Value \"SERVICE_STATE=missing\"\n Add-Content -Path $env:WORKFLOW_ENV -Value \"SERVICE_START_TYPE=none\"\n exit 0\n}\nWrite-Output \"${name}: $($svc.Status), start type $($svc.StartType)\"\nAdd-Content -Path $env:WORKFLOW_ENV -Value \"SERVICE_STATE=$($svc.Status)\"\nAdd-Content -Path $env:WORKFLOW_ENV -Value \"SERVICE_START_TYPE=$($svc.StartType)\"",
"declared_outputs": [
"SERVICE_STATE",
"SERVICE_START_TYPE"
],
"declared_inputs": [
{
"name": "serviceName",
"default": "",
"description": "Windows service name to inspect"
}
],
"secret_refs": []
}
-23
View File
@@ -1,23 +0,0 @@
[Unit]
Description=Vantage Agent
Documentation=https://github.com/your-org/vantage
After=network.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/local/bin/vantage-agent
Restart=always
RestartSec=10
User=root
StandardOutput=journal
StandardError=journal
SyslogIdentifier=vantage-agent
# Security hardening
NoNewPrivileges=true
ProtectSystem=false
ProtectHome=false
[Install]
WantedBy=multi-user.target
+6
View File
@@ -0,0 +1,6 @@
apiVersion: v2
name: vantage
description: Helm chart for the Vantage stack (Redis, MongoDB, guacd, server, web)
type: application
version: 1.0.7
appVersion: "1.0.7"
+72
View File
@@ -0,0 +1,72 @@
Vantage has been deployed as release "{{ .Release.Name }}" in namespace "{{ .Release.Namespace }}".
Services created:
{{- if .Values.redis.enabled }}
- {{ .Release.Name }}-redis (ClusterIP {{ .Values.redis.port }})
{{- else }}
- Redis: not deployed, using external {{ .Values.redis.addr }}
{{- end }}
{{- if .Values.mongo.enabled }}
- {{ .Release.Name }}-mongo (ClusterIP {{ .Values.mongo.port }})
{{- else }}
- MongoDB: not deployed, using the external server.env.mongoUri
{{- end }}
- {{ .Release.Name }}-guacd ({{ .Values.guacd.service.type }} {{ .Values.guacd.service.port }})
- {{ .Release.Name }}-server ({{ .Values.server.service.type }} http:{{ .Values.server.service.httpPort }} grpc:{{ .Values.server.service.grpcPort }})
- {{ .Release.Name }}-web ({{ .Values.web.service.type }} {{ .Values.web.service.port }})
Scaling (server.replicaCount / web.replicaCount):
- Both scale. Pin the image tags first — replicas on different builds serve
mismatched web asset hashes, and mixed server versions share one bus.
- server replicas route agent commands, step results and console relays to
each other over Redis, so every replica must use the SAME Redis. Workflow
logs are in MongoDB, not on a volume.
- Background work (monitor scheduler, Free reaper, log and audit retention,
the offline sweep) runs on one replica at a time under a Redis leader lock.
- server.persistence must be off to scale past one replica on a ReadWriteOnce
volume. Nothing writes to it any more.
{{- if gt (int .Values.server.replicaCount) 1 }}
- Console relays are reached by pod IP; guacd must be able to dial pod IPs
directly (it can, inside the cluster network).
{{- end }}
{{- if .Values.server.migrationJob.enabled }}
- Migrations run in the {{ .Release.Name }}-migrate Job before each upgrade;
the pods skip them. Its logs are kept: kubectl logs job/{{ .Release.Name }}-migrate
{{- end }}
{{- if .Values.ingress.enabled }}
Ingress (Traefik):
- Browsers: https://{{ .Values.ingress.web.host }}
{{- range .Values.ingress.web.extraHosts }}
https://{{ . }}
{{- end }}
{{- if .Values.ingress.api.enabled }}
{{ join ", " .Values.ingress.api.paths }} go straight to the server; everything else to web.
{{- else }}
Everything goes to web, which proxies /api and /auth onward. Set
ingress.api.enabled=true to route them at the edge instead.
{{- end }}
{{- if .Values.ingress.grpc.enabled }}
- Agents: {{ .Values.ingress.grpc.host }} (gRPC, h2c behind TLS)
Agents dial server.env.grpcHost, currently {{ tpl .Values.server.env.grpcHost . }}.
Point DNS for both hostnames at the Traefik load balancer.
{{- if not .Values.ingress.tls.enabled }}
- WARNING: ingress.tls.enabled is false. Agent tokens and session cookies
would cross the network in clear.
{{- else if and (not .Values.ingress.tls.certResolver) (not .Values.ingress.tls.secretName) }}
- WARNING: TLS is on but neither ingress.tls.secretName nor
ingress.tls.certResolver is set, so Traefik will serve its self-signed
default certificate, which no agent and no browser will trust.
{{- end }}
{{- end }}
{{- end }}
By default the server/web/guacd services are ClusterIP only (no host port publishing,
unlike the original docker-compose file). To expose them externally, set
server.service.type / web.service.type / guacd.service.type to NodePort or LoadBalancer,
or add an Ingress on top of the -web and -server services.
Quick access via port-forward, e.g.:
kubectl port-forward svc/{{ .Release.Name }}-web {{ .Values.web.service.port }}:{{ .Values.web.service.port }}
kubectl port-forward svc/{{ .Release.Name }}-server {{ .Values.server.service.httpPort }}:{{ .Values.server.service.httpPort }}
@@ -0,0 +1,85 @@
{{/*
Common name helpers
*/}}
{{- define "vantage.fullname" -}}
{{ .Release.Name }}
{{- end -}}
{{- define "vantage.labels" -}}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}
{{/*
vantage.server.env renders the server container's environment.
It lives here because two workloads need it identically: the Deployment and the
pre-upgrade migration Job. A Job that connected to a different database than the
pods it migrates for would be worse than no Job at all, so there is one copy and
both read it.
*/}}
{{- define "vantage.server.env" -}}
- name: MONGO_URI
{{- $mongoUri := tpl .Values.server.env.mongoUri . }}
{{- if and (not .Values.mongo.enabled) (contains (printf "%s-mongo" .Release.Name) $mongoUri) }}
{{- fail "mongo.enabled is false, so server.env.mongoUri must point at an external MongoDB rather than the in-chart one" }}
{{- end }}
value: {{ $mongoUri | quote }}
- name: REDIS_ADDR
{{- if .Values.redis.enabled }}
value: "{{ .Release.Name }}-redis:{{ .Values.redis.port }}"
{{- else }}
{{- if not .Values.redis.addr }}
{{- fail "redis.enabled is false, so redis.addr must be set to an external Redis host:port" }}
{{- end }}
value: {{ .Values.redis.addr | quote }}
{{- end }}
{{- if .Values.redis.auth.existingSecret }}
- name: REDIS_USERNAME
valueFrom:
secretKeyRef:
name: {{ .Values.redis.auth.existingSecret }}
key: {{ .Values.redis.auth.usernameKey }}
optional: true
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Values.redis.auth.existingSecret }}
key: {{ .Values.redis.auth.passwordKey }}
{{- else }}
{{- if .Values.redis.auth.username }}
- name: REDIS_USERNAME
value: {{ .Values.redis.auth.username | quote }}
{{- end }}
{{- if .Values.redis.auth.password }}
- name: REDIS_PASSWORD
value: {{ .Values.redis.auth.password | quote }}
{{- end }}
{{- end }}
- name: GRPC_HOST
value: {{ .Values.server.env.grpcHost | quote }}
- name: GRPC_PORT
value: {{ .Values.server.service.grpcPort | quote }}
- name: HTTP_PORT
value: {{ .Values.server.service.httpPort | quote }}
- name: KEY_ENCRYPTION_KEY
value: {{ .Values.server.env.keyEncryptionKey | quote }}
- name: GUACD_ADDR
value: "{{ .Release.Name }}-guacd:{{ .Values.guacd.service.port }}"
- name: APP_ROOT_LABEL
value: {{ .Values.server.env.appRootLabel | quote }}
- name: PROXY_ADVERTISE_HOST
value: {{ .Values.server.env.proxyAdvertiseHost | quote }}
- name: PROXY_LISTEN_HOST
value: {{ .Values.server.env.proxyListenHost | quote }}
{{- if eq .Values.server.env.deploymentType "cloud" }}
- name: VANTAGE_DEPLOYMENT
value: "cloud"
- name: FREE_INSTANCE_REAP_AFTER
value: {{ .Values.server.env.freeInstanceReapAfter | quote }}
{{- end }}
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
{{- end -}}
+47
View File
@@ -0,0 +1,47 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}-guacd
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: guacd
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: guacd
template:
metadata:
labels:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: guacd
spec:
{{- if .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml .Values.imagePullSecrets | nindent 8 }}
{{- end }}
containers:
- name: guacd
image: "{{ .Values.guacd.image.repository }}:{{ .Values.guacd.image.tag }}"
ports:
- containerPort: 4822
---
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-guacd
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: guacd
spec:
type: {{ .Values.guacd.service.type }}
{{- if eq .Values.guacd.service.type "ClusterIP" }}
clusterIP: None
{{- end }}
selector:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: guacd
ports:
- port: {{ .Values.guacd.service.port }}
targetPort: 4822
+173
View File
@@ -0,0 +1,173 @@
{{- if .Values.ingress.enabled }}
{{/*
Two hostnames, because the two audiences arrive over different protocols.
Browsers reach the web host. What answers there depends on the path: with
ingress.api.enabled, /api and /auth go straight to the server and everything
else goes to `web`. Without it, everything goes to `web`, which proxies those
prefixes onward itself (web/next.config.ts).
Both work. Routing at the edge is one hop shorter and is what the Nginx Proxy
Manager deployment in front of the Docker install already does, so leaving it
off changes the shape of the request path between the two deployments. It is
still off by default, because turning it on where `web` is the only thing with
a public certificate would strand /api behind a route nobody can reach.
The web host is normally a wildcard — `*.vantage.example.com` — because that is
the per-tenant instance namespace; APP_ROOT_LABEL resolves the instance from the
label. Kubernetes wildcard hosts match exactly one label, so this does not match
the apex, and on the deployment this chart was written for it must not: the apex
is the marketing site, a separate application (see site/ and
docker-compose.site.yml). extraHosts exists for a genuine second name, not for
reclaiming the apex.
Agents reach the server's gRPC port, which is plain h2c — the server holds no
certificates of its own, TLS has always been terminated by whatever sits in
front. Traefik will not speak h2c to a backend unless told to, and it is told
per Service, which is why the gRPC route gets a Service of its own below rather
than reusing the two-port one. Annotating the shared Service would force h2c on
its HTTP port too.
*/}}
{{- $tls := .Values.ingress.tls }}
{{- $webHost := required "ingress.enabled requires ingress.web.host" .Values.ingress.web.host }}
{{- $hosts := prepend .Values.ingress.web.extraHosts $webHost }}
{{- $apiPaths := .Values.ingress.api.paths }}
{{- if and .Values.ingress.api.enabled (not $apiPaths) }}
{{- fail "ingress.api.enabled requires at least one path in ingress.api.paths" }}
{{- end }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ .Release.Name }}-web
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: web
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: {{ .Values.ingress.entrypoint | quote }}
{{- if and $tls.enabled $tls.certResolver }}
traefik.ingress.kubernetes.io/router.tls: "true"
traefik.ingress.kubernetes.io/router.tls.certresolver: {{ $tls.certResolver | quote }}
{{- else if $tls.enabled }}
traefik.ingress.kubernetes.io/router.tls: "true"
{{- end }}
{{- with .Values.ingress.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
{{- if .Values.ingress.className }}
ingressClassName: {{ .Values.ingress.className }}
{{- end }}
{{- if and $tls.enabled $tls.secretName }}
tls:
- hosts:
{{- range $hosts }}
- {{ . | quote }}
{{- end }}
secretName: {{ $tls.secretName }}
{{- end }}
rules:
{{- range $host := $hosts }}
- host: {{ $host | quote }}
http:
paths:
{{- /*
The API paths come first and, more importantly, are longer. Traefik
derives router priority from rule length, so Host(x) &&
PathPrefix(/api) outranks Host(x) && PathPrefix(/) without anyone
having to assign priorities by hand. Order within the list is for
the reader; the length is what decides.
*/}}
{{- if $.Values.ingress.api.enabled }}
{{- range $apiPaths }}
- path: {{ . | quote }}
pathType: Prefix
backend:
service:
name: {{ $.Release.Name }}-server
port:
number: {{ $.Values.server.service.httpPort }}
{{- end }}
{{- end }}
- path: /
pathType: Prefix
backend:
service:
name: {{ $.Release.Name }}-web
port:
number: {{ $.Values.web.service.port }}
{{- end }}
{{- if .Values.ingress.grpc.enabled }}
{{- $grpcHost := required "ingress.grpc.enabled requires ingress.grpc.host" .Values.ingress.grpc.host }}
{{/*
GRPC_HOST is what an agent is told to dial, and it is baked into every install
one-liner. Left pointing at the in-cluster Service while agents are expected to
arrive through the ingress, every install would succeed and every agent would
fail to connect — with nothing in the control plane saying why.
*/}}
{{- $grpcEnv := tpl .Values.server.env.grpcHost . }}
{{- if contains (printf "%s-server" .Release.Name) $grpcEnv }}
{{- fail (printf "ingress.grpc.enabled routes agents through %s, but server.env.grpcHost is still the in-cluster address %q. Agents dial the value of grpcHost, so set it to the public gRPC address (for example %q)." $grpcHost $grpcEnv (printf "%s:443" $grpcHost)) }}
{{- end }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-server-grpc
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: server
annotations:
# The server speaks h2c: it terminates no TLS itself. Without this Traefik
# dials the backend as HTTP/1.1 and every agent handshake fails.
traefik.ingress.kubernetes.io/service.serversscheme: h2c
spec:
type: ClusterIP
selector:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: server
ports:
- name: grpc
port: {{ .Values.server.service.grpcPort }}
targetPort: {{ .Values.server.service.grpcPort }}
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ .Release.Name }}-grpc
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: server
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: {{ .Values.ingress.entrypoint | quote }}
{{- if and $tls.enabled $tls.certResolver }}
traefik.ingress.kubernetes.io/router.tls: "true"
traefik.ingress.kubernetes.io/router.tls.certresolver: {{ $tls.certResolver | quote }}
{{- else if $tls.enabled }}
traefik.ingress.kubernetes.io/router.tls: "true"
{{- end }}
{{- with .Values.ingress.grpc.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
{{- if .Values.ingress.className }}
ingressClassName: {{ .Values.ingress.className }}
{{- end }}
{{- if and $tls.enabled $tls.grpcSecretName }}
tls:
- hosts:
- {{ $grpcHost | quote }}
secretName: {{ $tls.grpcSecretName }}
{{- end }}
rules:
- host: {{ $grpcHost | quote }}
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: {{ .Release.Name }}-server-grpc
port:
number: {{ .Values.server.service.grpcPort }}
{{- end }}
{{- end }}
@@ -0,0 +1,68 @@
{{- if .Values.server.migrationJob.enabled }}
{{/*
Schema setup, lifted out of the serving pods.
Every server process used to run migrations, index builders and default-step
seeding at boot. With one replica that is fine. With two it is not: 0004 renames
the orgs collection to instances, and a sibling reading it mid-rename is a
corruption, not a retry.
A Helm hook Job runs it once, before any pod of the new version starts. The
Deployment then sets VANTAGE_SKIP_MIGRATIONS, which is what makes the Job's
existence load-bearing rather than decorative — if you disable the Job, the
pods go back to migrating themselves and you must go back to one replica.
hook-weight orders this after the dependency waits; before-hook-creation deletes
the previous Job so a repeat upgrade is not blocked by an immutable object. The
Job is deliberately NOT deleted on success: its logs are the record of what the
upgrade did to the database.
*/}}
apiVersion: batch/v1
kind: Job
metadata:
name: {{ .Release.Name }}-migrate
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: migrate
annotations:
"helm.sh/hook": pre-install,pre-upgrade
"helm.sh/hook-weight": "0"
"helm.sh/hook-delete-policy": before-hook-creation
spec:
backoffLimit: {{ .Values.server.migrationJob.backoffLimit }}
# A migration that has not finished in this long is stuck, and a stuck
# migration should fail the upgrade rather than hold it open forever.
activeDeadlineSeconds: {{ .Values.server.migrationJob.activeDeadlineSeconds }}
template:
metadata:
labels:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: migrate
spec:
restartPolicy: Never
{{- if .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml .Values.imagePullSecrets | nindent 8 }}
{{- end }}
{{- if .Values.mongo.enabled }}
# Only Mongo. The Job never opens Redis, and waiting on a Redis this
# chart may not even deploy would block an upgrade for no reason.
initContainers:
- name: wait-for-mongo
image: busybox:1.36
command:
- sh
- -c
- |
until nc -z {{ .Release.Name }}-mongo {{ .Values.mongo.port }}; do
echo "waiting for mongo..."; sleep 2;
done
{{- end }}
containers:
- name: migrate
image: "{{ .Values.server.image.repository }}:{{ .Values.server.image.tag }}"
env:
{{- include "vantage.server.env" . | nindent 12 }}
- name: VANTAGE_MIGRATE_ONLY
value: "true"
{{- end }}
+92
View File
@@ -0,0 +1,92 @@
{{- if .Values.mongo.enabled }}
{{- if .Values.mongo.persistence.enabled }}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Release.Name }}-mongo-data
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: mongo
spec:
accessModes:
- {{ .Values.mongo.persistence.accessMode }}
{{- if .Values.mongo.persistence.storageClass }}
storageClassName: {{ .Values.mongo.persistence.storageClass }}
{{- end }}
resources:
requests:
storage: {{ .Values.mongo.persistence.size }}
---
{{- end }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}-mongo
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: mongo
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: mongo
template:
metadata:
labels:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: mongo
spec:
{{- if .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml .Values.imagePullSecrets | nindent 8 }}
{{- end }}
containers:
- name: mongo
image: "{{ .Values.mongo.image.repository }}:{{ .Values.mongo.image.tag }}"
ports:
- containerPort: {{ .Values.mongo.port }}
volumeMounts:
- name: mongo-data
mountPath: /data/db
livenessProbe:
exec:
command: ["mongosh", "--quiet", "--eval", "db.adminCommand('ping')"]
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
exec:
command: ["mongosh", "--quiet", "--eval", "db.adminCommand('ping')"]
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
volumes:
- name: mongo-data
{{- if .Values.mongo.persistence.enabled }}
persistentVolumeClaim:
claimName: {{ .Release.Name }}-mongo-data
{{- else }}
emptyDir: {}
{{- end }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-mongo
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: mongo
spec:
type: ClusterIP
selector:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: mongo
ports:
- port: {{ .Values.mongo.port }}
targetPort: {{ .Values.mongo.port }}
{{- end }}
+92
View File
@@ -0,0 +1,92 @@
{{- if .Values.redis.enabled }}
{{- if .Values.redis.persistence.enabled }}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Release.Name }}-redis-data
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: redis
spec:
accessModes:
- {{ .Values.redis.persistence.accessMode }}
{{- if .Values.redis.persistence.storageClass }}
storageClassName: {{ .Values.redis.persistence.storageClass }}
{{- end }}
resources:
requests:
storage: {{ .Values.redis.persistence.size }}
---
{{- end }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}-redis
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: redis
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: redis
template:
metadata:
labels:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: redis
spec:
{{- if .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml .Values.imagePullSecrets | nindent 8 }}
{{- end }}
containers:
- name: redis
image: "{{ .Values.redis.image.repository }}:{{ .Values.redis.image.tag }}"
ports:
- containerPort: {{ .Values.redis.port }}
volumeMounts:
- name: redis-data
mountPath: /data
livenessProbe:
exec:
command: ["redis-cli", "ping"]
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
exec:
command: ["redis-cli", "ping"]
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
volumes:
- name: redis-data
{{- if .Values.redis.persistence.enabled }}
persistentVolumeClaim:
claimName: {{ .Release.Name }}-redis-data
{{- else }}
emptyDir: {}
{{- end }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-redis
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: redis
spec:
type: ClusterIP
selector:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: redis
ports:
- port: {{ .Values.redis.port }}
targetPort: {{ .Values.redis.port }}
{{- end }}
+181
View File
@@ -0,0 +1,181 @@
{{- if and .Values.server.persistence.enabled (not .Values.server.persistence.useHostPath) }}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Release.Name }}-server-data
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: server
spec:
accessModes:
- {{ .Values.server.persistence.accessMode }}
{{- if .Values.server.persistence.storageClass }}
storageClassName: {{ .Values.server.persistence.storageClass }}
{{- end }}
resources:
requests:
storage: {{ .Values.server.persistence.size }}
---
{{- end }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}-server
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: server
spec:
{{- $replicas := int .Values.server.replicaCount }}
replicas: {{ $replicas }}
{{- if and .Values.server.persistence.enabled (eq .Values.server.persistence.accessMode "ReadWriteOnce") }}
# A ReadWriteOnce volume cannot be mounted by a second pod at all, and cannot
# be handed to a new pod while the old one still holds it. Persistence is off
# by default now that nothing writes to it; if it is on, replicas are capped
# at one and updates go through Recreate.
{{- if gt $replicas 1 }}
{{- fail "server.persistence.enabled with a ReadWriteOnce volume cannot be combined with server.replicaCount > 1. Nothing in the server writes to that volume any more (workflow logs live in MongoDB); set server.persistence.enabled=false, or use a ReadWriteMany accessMode if you are keeping it for another reason." }}
{{- end }}
strategy:
type: Recreate
{{- end }}
selector:
matchLabels:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: server
template:
metadata:
labels:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: server
spec:
# The server stops gRPC before draining HTTP, so that every CommandStream
# handler returns and releases its agent's presence claim. A claim left
# behind outlives the pod for its 30s TTL, and during that window other
# replicas dispatch commands to a process that has exited — surfacing to
# the operator as "agent offline" on an agent that is perfectly healthy.
# 10s for gRPC plus 10s for the HTTP drain, with headroom.
terminationGracePeriodSeconds: {{ .Values.server.terminationGracePeriodSeconds }}
{{- if .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml .Values.imagePullSecrets | nindent 8 }}
{{- end }}
{{- if or .Values.redis.enabled .Values.mongo.enabled }}
# Wait for the dependencies this chart deploys to be reachable,
# approximating compose's `depends_on: condition: service_healthy`. An
# external Redis or Mongo is assumed to be up already — waiting on one
# would only turn someone else's outage into a stuck pod.
initContainers:
{{- if .Values.redis.enabled }}
- name: wait-for-redis
image: busybox:1.36
command:
- sh
- -c
- |
until nc -z {{ .Release.Name }}-redis {{ .Values.redis.port }}; do
echo "waiting for redis..."; sleep 2;
done
{{- end }}
{{- if .Values.mongo.enabled }}
- name: wait-for-mongo
image: busybox:1.36
command:
- sh
- -c
- |
until nc -z {{ .Release.Name }}-mongo {{ .Values.mongo.port }}; do
echo "waiting for mongo..."; sleep 2;
done
{{- end }}
{{- end }}
containers:
- name: server
image: "{{ .Values.server.image.repository }}:{{ .Values.server.image.tag }}"
ports:
- containerPort: {{ .Values.server.service.httpPort }}
- containerPort: {{ .Values.server.service.grpcPort }}
env:
{{- include "vantage.server.env" . | nindent 12 }}
{{- if .Values.server.migrationJob.enabled }}
# Schema setup ran in the pre-upgrade Job. Pods that repeated it
# would race each other, and the rename migration is not a race
# that tolerates a loser.
- name: VANTAGE_SKIP_MIGRATIONS
value: "true"
{{- end }}
# Liveness never touches Mongo or Redis: restarting every pod cannot
# fix a database outage, and each restart drops every agent command
# stream and console session it was carrying. Readiness does check
# both, so a pod that cannot serve leaves the Service and stays up.
startupProbe:
httpGet:
path: /healthz
port: {{ .Values.server.service.httpPort }}
periodSeconds: 5
# Generous: without the migration Job this pod runs every migration
# before it listens, and the rename has a ten-minute budget.
failureThreshold: 150
livenessProbe:
httpGet:
path: /healthz
port: {{ .Values.server.service.httpPort }}
periodSeconds: 20
failureThreshold: 3
readinessProbe:
httpGet:
path: /readyz
port: {{ .Values.server.service.httpPort }}
periodSeconds: 10
failureThreshold: 3
{{- if .Values.server.persistence.enabled }}
# Nothing in the server writes here any more — workflow logs moved to
# MongoDB so that every replica can read and write them. The mount
# remains only so an operator upgrading from a file-log release can
# still reach the old files before turning persistence off.
volumeMounts:
- name: server-data
mountPath: /data
volumes:
- name: server-data
persistentVolumeClaim:
claimName: {{ .Release.Name }}-server-data
{{- end }}
---
{{- if gt (int .Values.server.replicaCount) 1 }}
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: {{ .Release.Name }}-server
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: server
spec:
# Agents reconnect on their own, but a drain that took every replica at once
# would disconnect every agent in the fleet simultaneously and stall every
# workflow run in flight.
minAvailable: 1
selector:
matchLabels:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: server
---
{{- end }}
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-server
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: server
spec:
type: {{ .Values.server.service.type }}
selector:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: server
ports:
- name: http
port: {{ .Values.server.service.httpPort }}
targetPort: {{ .Values.server.service.httpPort }}
- name: grpc
port: {{ .Values.server.service.grpcPort }}
targetPort: {{ .Values.server.service.grpcPort }}
+80
View File
@@ -0,0 +1,80 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}-web
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: web
spec:
# web holds no per-process state: sessions live in Redis and every request is
# proxied to the server. It is the one component here that scales freely.
replicas: {{ .Values.web.replicaCount }}
selector:
matchLabels:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: web
template:
metadata:
labels:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: web
spec:
{{- if .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml .Values.imagePullSecrets | nindent 8 }}
{{- end }}
initContainers:
- name: wait-for-server
image: busybox:1.36
command:
- sh
- -c
- |
until nc -z {{ .Release.Name }}-server {{ .Values.server.service.httpPort }}; do
echo "waiting for server..."; sleep 2;
done
containers:
- name: web
image: "{{ .Values.web.image.repository }}:{{ .Values.web.image.tag }}"
ports:
- containerPort: {{ .Values.web.service.port }}
env:
- name: API_URL
value: {{ tpl .Values.web.env.apiUrl . | quote }}
# /healthz is served by this Next process; /api is rewritten to the
# server, so a probe there would report the backend's health and keep
# passing while this pod was wedged.
startupProbe:
httpGet:
path: /healthz
port: {{ .Values.web.service.port }}
periodSeconds: 3
failureThreshold: 20
livenessProbe:
httpGet:
path: /healthz
port: {{ .Values.web.service.port }}
periodSeconds: 20
failureThreshold: 3
readinessProbe:
httpGet:
path: /healthz
port: {{ .Values.web.service.port }}
periodSeconds: 10
failureThreshold: 3
---
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-web
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: web
spec:
type: {{ .Values.web.service.type }}
selector:
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: web
ports:
- port: {{ .Values.web.service.port }}
targetPort: {{ .Values.web.service.port }}
+111
View File
@@ -0,0 +1,111 @@
# Default values for the vantage chart.
redis:
enabled: true
addr: ""
image:
repository: redis
tag: "8"
persistence:
enabled: true
size: 1Gi
storageClass: ""
accessMode: ReadWriteOnce
port: 6379
auth:
username: ""
password: ""
existingSecret: ""
usernameKey: username
passwordKey: password
mongo:
enabled: true
image:
repository: mongo
tag: "7"
persistence:
enabled: true
size: 5Gi
storageClass: ""
accessMode: ReadWriteOnce
port: 27017
guacd:
image:
repository: docker.io/guacamole/guacd
tag: "1.6.0"
service:
type: ClusterIP
port: 4822
server:
replicaCount: 1
# Must exceed the server's own stop sequence (10s gRPC GracefulStop + 10s
# HTTP drain) or the kubelet SIGKILLs mid-shutdown, which is exactly the
# abrupt exit that leaves agent presence claims stranded in Redis.
terminationGracePeriodSeconds: 30
migrationJob:
enabled: true
backoffLimit: 0
activeDeadlineSeconds: 900
image:
repository: gitea.hostxtra.co.uk/mrhid6/vantage/server
tag: latest
service:
type: ClusterIP
httpPort: 8080
grpcPort: 9090
env:
mongoUri: "mongodb://{{ .Release.Name }}-mongo:27017/vantage"
grpcHost: "{{ .Release.Name }}-server:9090"
keyEncryptionKey: ""
appRootLabel: vantage
proxyAdvertiseHost: "{{ .Release.Name }}-server"
proxyListenHost: "0.0.0.0"
persistence:
enabled: false
size: 1Gi
storageClass: ""
accessMode: ReadWriteOnce
hostPath: /data
web:
replicaCount: 1
image:
repository: gitea.hostxtra.co.uk/mrhid6/vantage/web
tag: latest
service:
type: ClusterIP
port: 3000
env:
apiUrl: "http://{{ .Release.Name }}-server:8080"
ingress:
enabled: false
className: traefik
entrypoint: websecure
annotations: {}
web:
host: ""
extraHosts: []
api:
enabled: false
paths:
- /api
- /auth
- /update
- /install
- /update.ps1
- /install.ps1
grpc:
enabled: true
host: ""
annotations: {}
tls:
enabled: true
secretName: ""
grpcSecretName: ""
certResolver: ""
imagePullSecrets: []
@@ -5,10 +5,7 @@
# host:port agents dial for gRPC. No default; boot fails without it.
# Must be reachable from managed servers. Use the public host, port 9090.
GRPC_HOST=192.168.1.250:9090
# Gitea host used to build agent install scripts and download URLs.
GITEA_HOST=gitea.hostxtra.co.uk
GRPC_HOST=vantage.yourdomain.com:9090
# 64-char hex (32 bytes) for AES-256-GCM. Required for private keys,
# secrets, OIDC secrets, RDP/VNC credentials.
@@ -21,4 +18,3 @@ KEY_ENCRYPTION_KEY=
MONGO_URI=mongodb://mongo:27017/vantage
# Where workflow run logs are written inside the server container.
# VANTAGE_WORKFLOW_LOG_DIR=/data/workflow-logs
@@ -55,4 +55,13 @@ services:
- 3004:3000
depends_on:
- admin
# Static docs, served by nginx at vantage.hostxtra.co.uk/docs through its own
# proxy location. That location must sort ABOVE the catch-all forwarding to
# site:3003, or Next serves its own 404 for /docs. The container serves from
# /usr/share/nginx/html/docs because the proxy forwards the full path.
docsite:
image: gitea.hostxtra.co.uk/mrhid6/vantage/docsite:latest
restart: unless-stopped
ports:
- 3005:80
networks: {}
@@ -41,14 +41,12 @@ services:
environment:
MONGO_URI: ${MONGO_URI:-mongodb://mongo:27017/vantage}
REDIS_ADDR: redis:6379
GITEA_HOST: ${GITEA_HOST}
GRPC_HOST: ${GRPC_HOST}
GRPC_PORT: "9090"
HTTP_PORT: "8080"
KEY_ENCRYPTION_KEY: ${KEY_ENCRYPTION_KEY:-}
VANTAGE_WORKFLOW_LOG_DIR: ${VANTAGE_WORKFLOW_LOG_DIR:-}
GUACD_ADDR: guacd:4822
APP_ROOT_LABEL: vantage
PROXY_ADVERTISE_HOST: server
depends_on:
redis:
condition: service_healthy
@@ -61,6 +59,8 @@ services:
restart: unless-stopped
ports:
- 3000:3000
environment:
API_URL: ${API_URL:-http://server:8080}
depends_on:
- server
volumes:
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -1,990 +0,0 @@
# Cloud Instance Creation — Phase 1: Identity
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace the control plane's global unique index on `users.email` with a per-instance one, and scope every lookup that relied on the global index, so one address can belong to several instances.
**Architecture:** The index change is safe only because the two unscoped `FindOne({email})` lookups are scoped in the same binary that performs the swap. The new compound index is created **before** the old one is dropped, so a failure at any point leaves a working constraint in place. The unscoped helper is deleted rather than left unused, and admin's one unscoped control-plane lookup — which has no instance to scope by — is removed entirely.
**Tech Stack:** Go 1.26, gin, MongoDB driver v2.8.0, `shared/indexes`, `shared/models`, `shared/provision`.
## Global Constraints
- **No automated Go tests.** Verification is by compiler, `grep`, and running built images against scratch databases. Every "confirm" step below is a command with expected output. This matches plans 0a through 4.
- **Never run `go` or `npm` on the host.** Everything runs in a container. The wrapper from earlier plans:
```sh
# /tmp/gorun.sh <module-dir> <command...>
DIR="$1"; shift
MSYS_NO_PATHCONV=1 docker run --rm -v "$(pwd)":/src -v vantage-gomod:/go/pkg/mod \
-v vantage-gocache:/root/.cache/go-build -w "/src/$DIR" \
golang:1.26 "$@"
```
- **`MSYS_NO_PATHCONV=1` on every `docker` call.** Git Bash rewrites container paths otherwise.
- **Run `go mod tidy` with `GOWORK=off`.** In workspace mode it drops `require` lines and the Docker build then fails with "missing go.sum entry".
- **`shared/` is consumed through `replace` directives** in `server`, `admin` and `sitesvc`. A change to `shared/` reaches all three on their next build; there is no version to bump.
- **All three service images must ship together.** An older image booting after this change would recreate `email_1`. `.gitea/workflows/server-deploy.yml` rebuilds every image on every push to `main`, so this is automatic — the hazard is only a partial manual rollout on the host.
- **This migration is one-way.** Once two users share an address across instances, `email_1` cannot be recreated. There is no rollback; fixes go forward.
- Nothing in this phase projects users, creates instances, or adds UI. Those are phases 2 and 3.
## Context this plan inherits
`CLAUDE.md` currently states that the unique index on user email is "a security property, not an optimisation", because `GetUserByEmail` does an unscoped `FindOne`. That statement is true today and stops being true in Task 1. Task 7 updates it in the same series of commits, and the replacement property is stronger: a scoped query cannot be ambiguous, whereas an index merely prevents the ambiguity from arising.
Spec: [`docs/superpowers/specs/2026-07-26-cloud-instance-creation-design.md`](../specs/2026-07-26-cloud-instance-creation-design.md), phase 1.
---
## File Structure
**Modified:**
| Path | Change |
| ----------------------------------- | -------------------------------------------------------------------------- |
| `shared/indexes/indexes.go` | compound `(instance_id, email)` unique index; idempotent drop of `email_1` |
| `shared/models/user.go` | `HQUserID` field, `AuthLocal`/`AuthOIDC`/`AuthHQ` constants |
| `server/internal/services/users.go` | `GetUserByEmail` deleted, `GetUserInInstanceByEmail` added |
| `server/internal/auth/local.go` | `resolveLoginInstance`, scoped sign-in |
| `server/internal/auth/oidc.go` | scoped lookup, cross-instance guard deleted |
| `admin/internal/auth/cloud.go` | **deleted** |
| `admin/internal/api/routes.go` | `/auth/login` points at `HandleCustomerLogin`; new staff route |
| `admin/internal/api/staff.go` | `staffCreateAccountUser` |
| `CLAUDE.md` | the index security-property paragraph, and the auth section |
**Created:** none.
---
### Task 1: Compound index and the drop
**Files:**
- Modify: `shared/indexes/indexes.go`
**Interfaces:**
- Consumes: nothing new.
- Produces: `indexes.EnsureCoreIndexes(ctx context.Context, db *mongo.Database) error` — unchanged signature, new behaviour. Called at boot by `server`, `sitesvc` and `admin`.
- [ ] **Step 1: Replace the body of `EnsureCoreIndexes` and add the drop helper**
Replace the whole file with:
```go
// Package indexes declares the MongoDB indexes more than one Vantage service
// depends on.
package indexes
import (
"context"
"errors"
"fmt"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
// legacyUserEmailIndex is the global unique index on users.email that this
// package used to declare. It is dropped on sight.
const legacyUserEmailIndex = "email_1"
// indexNotFound is MongoDB's IndexNotFound error code. Two services booting at
// once can both decide to drop the legacy index; the loser must not treat that
// as a failure.
const indexNotFound = 27
// EnsureCoreIndexes declares the unique indexes on users and instances.
//
// users is unique on (instance_id, email), NOT on email alone. One address is
// one user WITHIN an instance; the same address may hold a user in several
// instances, because an account's people are projected into each instance they
// are granted access to.
//
// This is a security property, not an optimisation, and it is only sufficient
// because every lookup by email is scoped by instance. There is deliberately no
// unscoped lookup by email anywhere in the codebase: an unscoped FindOne would
// return an arbitrary one of several matching users, which on the login path
// means signing someone into a tenant that is not theirs. If you are about to
// add one, you are about to reintroduce that bug.
//
// Creating an index that already exists with the same specification is a no-op,
// so this is safe to call at every boot from every service.
func EnsureCoreIndexes(ctx context.Context, db *mongo.Database) error {
// Create the replacement BEFORE dropping the legacy index. A failure here
// leaves the old constraint in place, which is safe; a failure after the
// drop would leave the collection unconstrained, which is not.
if _, err := db.Collection("users").Indexes().CreateOne(ctx, mongo.IndexModel{
Keys: bson.D{{Key: "instance_id", Value: 1}, {Key: "email", Value: 1}},
Options: options.Index().SetUnique(true).SetName("instance_email_unique"),
}); err != nil {
return fmt.Errorf("users.instance_id+email index: %w", err)
}
if err := dropIndexIfExists(ctx, db.Collection("users"), legacyUserEmailIndex); err != nil {
return fmt.Errorf("drop users.%s: %w", legacyUserEmailIndex, err)
}
if _, err := db.Collection("instances").Indexes().CreateOne(ctx, mongo.IndexModel{
Keys: bson.D{{Key: "slug", Value: 1}},
Options: options.Index().SetUnique(true),
}); err != nil {
return fmt.Errorf("instances.slug index: %w", err)
}
return nil
}
// dropIndexIfExists drops name, treating "it was not there" as success whether
// that is discovered by listing or by racing another service to the drop.
func dropIndexIfExists(ctx context.Context, col *mongo.Collection, name string) error {
cur, err := col.Indexes().List(ctx)
if err != nil {
return err
}
var existing []struct {
Name string `bson:"name"`
}
if err := cur.All(ctx, &existing); err != nil {
return err
}
found := false
for _, i := range existing {
if i.Name == name {
found = true
break
}
}
if !found {
return nil
}
err = col.Indexes().DropOne(ctx, name)
if err == nil {
return nil
}
var srvErr mongo.ServerError
if errors.As(err, &srvErr) && srvErr.HasErrorCode(indexNotFound) {
return nil
}
return err
}
```
- [ ] **Step 2: Confirm it compiles**
Run:
```sh
sh /tmp/gorun.sh shared go build ./...
```
Expected: no output.
- [ ] **Step 3: Confirm the legacy index is not declared anywhere else**
Run:
```sh
grep -rn '"email"' --include=*.go shared/ server/ sitesvc/ admin/ | grep -i index
```
Expected: no matches. If sitesvc or the server declares its own `users.email` index, it would recreate what Task 1 drops.
- [ ] **Step 4: Commit**
```bash
git add shared/indexes/indexes.go
git commit -m "feat(shared): unique users index is (instance_id, email)
One address is one user within an instance, not globally, so an account's
people can be projected into every instance they are granted.
The replacement index is created before email_1 is dropped, so a failure
at any point leaves a working constraint. The drop is idempotent and
tolerates two services racing it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>"
```
---
### Task 2: `hq` fields on the user document
**Files:**
- Modify: `shared/models/user.go`
- Modify: `server/internal/models/user.go`
**Interfaces:**
- Consumes: nothing.
- Produces:
- `shared/models.AuthLocal = "local"`, `AuthOIDC = "oidc"`, `AuthHQ = "hq"`
- `shared/models.User.HQUserID string` — bson `hq_user_id,omitempty`
- the same three constants re-exported from `server/internal/models`, which is a thin alias file over `shared/models` and is what server code imports
Nothing writes `AuthHQ` or `HQUserID` in this phase. They land now so phases 2 and 3 do not have to change the shared module and rebuild every service again.
- [ ] **Step 1: Add the constants and the field**
In `shared/models/user.go`, after the `ValidRole` function, add:
```go
// Auth sources. A user's auth_source says who owns the row.
const (
AuthLocal = "local"
AuthOIDC = "oidc"
// AuthHQ marks a user projected from a Vantage HQ account. Its role,
// password and existence are owned by HQ, and the instance API refuses to
// change any of them locally — a role editable in two places is a role with
// two answers.
AuthHQ = "hq"
)
```
And in the `User` struct, add `HQUserID` immediately after `AuthSource`:
```go
type User struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
UserID string `bson:"user_id" json:"user_id"`
InstanceID string `bson:"instance_id" json:"instance_id"`
Email string `bson:"email" json:"email"`
PasswordHash string `bson:"password_hash,omitempty" json:"-"`
Role string `bson:"role" json:"role"`
AuthSource string `bson:"auth_source" json:"auth_source"`
// HQUserID is the customer_users.user_id this row was projected from,
// absent on locally-created users.
HQUserID string `bson:"hq_user_id,omitempty" json:"hq_user_id,omitempty"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
LastLogin *time.Time `bson:"last_login,omitempty" json:"last_login,omitempty"`
}
```
- [ ] **Step 2: Re-export the constants from the server's alias file**
`server/internal/models/user.go` is a thin alias over `shared/models`, and server code imports that rather than the shared package directly. Add the auth sources alongside the roles it already re-exports:
```go
package models
import shared "gitea.hostxtra.co.uk/mrhid6/vantage/shared/models"
type User = shared.User
const (
RoleOwner = shared.RoleOwner
RoleAdmin = shared.RoleAdmin
RoleMember = shared.RoleMember
)
const (
AuthLocal = shared.AuthLocal
AuthOIDC = shared.AuthOIDC
AuthHQ = shared.AuthHQ
)
func ValidRole(role string) bool { return shared.ValidRole(role) }
```
- [ ] **Step 3: Confirm both compile**
Run:
```sh
sh /tmp/gorun.sh shared go build ./...
sh /tmp/gorun.sh server go build ./...
```
Expected: no output from either.
- [ ] **Step 4: Commit**
```bash
git add shared/models/user.go server/internal/models/user.go
git commit -m "feat(shared): auth_source constants and hq_user_id on User
Nothing writes them yet. They land now so phases 2 and 3 do not require a
second rebuild of every service that consumes the shared module.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>"
```
---
### Task 3: Scoped lookup in the user service
**Files:**
- Modify: `server/internal/services/users.go:65-75`
**Interfaces:**
- Consumes: `shared/indexes` from Task 1.
- Produces: `services.GetUserInInstanceByEmail(instanceID, email string) (*models.User, error)`.
- Removes: `services.GetUserByEmail`. Tasks 4 and 5 fix its two callers; the build will be red between this task and Task 5, which is expected and is why they are adjacent.
- [ ] **Step 1: Replace `GetUserByEmail`**
In `server/internal/services/users.go`, delete the whole `GetUserByEmail` function and put this in its place:
```go
// GetUserInInstanceByEmail finds a user by address WITHIN one instance.
//
// There is deliberately no unscoped lookup by email. users is unique on
// (instance_id, email), not on email alone, so an unscoped FindOne would return
// an arbitrary one of several matching users — which on the login path means
// signing someone into a tenant that is not theirs.
func GetUserInInstanceByEmail(instanceID, email string) (*models.User, error) {
email = strings.ToLower(strings.TrimSpace(email))
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
var u models.User
err := db.Col("users").FindOne(ctx, bson.M{
"instance_id": instanceID,
"email": email,
}).Decode(&u)
if err != nil {
return nil, err
}
return &u, nil
}
```
- [ ] **Step 2: Confirm the unscoped helper is gone and the build is red for the expected reason**
Run:
```sh
grep -rn "GetUserByEmail" --include=*.go .
```
Expected: exactly two matches, both call sites — `server/internal/auth/local.go` and `server/internal/auth/oidc.go`. No definition.
Run:
```sh
sh /tmp/gorun.sh server go build ./...
```
Expected: FAIL with `undefined: services.GetUserByEmail` at those two call sites. Any other error means something else was broken.
- [ ] **Step 3: Do not commit yet**
The build is red. Commit at the end of Task 5, when both callers are fixed. A commit that does not build is a commit nobody can bisect through.
---
### Task 4: Scoped local login
**Files:**
- Modify: `server/internal/auth/local.go:25-49`
**Interfaces:**
- Consumes: `services.GetUserInInstanceByEmail` from Task 3, `services.CountInstances` and `services.FirstInstance` from `server/internal/services/instances.go:57` and `:63`, `auth.InstanceFromHost` from `server/internal/auth/instancehost.go:53`.
- Produces: `resolveLoginInstance(c *gin.Context) (string, error)`, unexported, used only by this file.
**Behaviour change worth knowing:** signing in at the bare apex host stops working when more than one instance exists. Cloud sign-in is always on `<slug>.vantage.<tld>``APP_LOGIN_URL` fills `{slug}` in, so every link already points there — and self-hosted has exactly one instance, so both supported paths keep working. A bookmark to the apex login page on a multi-instance deployment will now get a 400 that names the cause.
- [ ] **Step 1: Add `resolveLoginInstance` and rewrite `HandleLocalLogin`**
In `server/internal/auth/local.go`, add `"fmt"` to the imports if it is not already there, then add above `HandleLocalLogin`:
```go
// resolveLoginInstance decides which instance a sign-in attempt belongs to.
//
// Cloud always answers from the host: every instance has its own subdomain, and
// APP_LOGIN_URL fills the slug in, so every sign-in link already points at one.
// Self-hosted has no subdomain and exactly one instance, because a licence
// binds one instance UUID.
//
// Anything else is refused rather than guessed. Picking an instance on someone's
// behalf is how you sign them into the wrong tenant.
func resolveLoginInstance(c *gin.Context) (string, error) {
if inst, ok := InstanceFromHost(c); ok {
return inst.InstanceID, nil
}
n, err := services.CountInstances()
if err != nil {
return "", err
}
if n != 1 {
return "", fmt.Errorf(
"cannot tell which instance this sign-in is for: %d instances exist and the host %q names none of them; sign in at your instance's own address",
n, c.Request.Host)
}
inst, err := services.FirstInstance()
if err != nil {
return "", err
}
return inst.InstanceID, nil
}
```
Then replace the body of `HandleLocalLogin` between the JSON bind and `SaveSession` with:
```go
instanceID, err := resolveLoginInstance(c)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
u, err := services.GetUserInInstanceByEmail(instanceID, body.Email)
if err != nil || !services.VerifyPassword(u, body.Password) {
c.JSON(http.StatusUnauthorized, gin.H{"error": "invalid credentials"})
return
}
```
The `SaveSession` call below it is unchanged: it already reads `u.InstanceID`.
- [ ] **Step 2: Confirm only the OIDC caller is left broken**
Run:
```sh
sh /tmp/gorun.sh server go build ./...
```
Expected: FAIL with `undefined: services.GetUserByEmail` at `internal/auth/oidc.go:130` only.
---
### Task 5: Scoped OIDC callback
**Files:**
- Modify: `server/internal/auth/oidc.go:129-141`
**Interfaces:**
- Consumes: `services.GetUserInInstanceByEmail` from Task 3.
- Produces: nothing new.
The cross-instance guard is deleted because it becomes unreachable: the lookup is now scoped to `instanceID`, so a user belonging to another instance is simply not found, and the OIDC callback provisions a new member — which is correct. OIDC is configured per instance, so only that instance's identity provider can reach this code with that instance's state.
- [ ] **Step 1: Replace the lookup and delete the guard**
In `server/internal/auth/oidc.go`, replace:
```go
email := strings.ToLower(claims.Email)
u, err := services.GetUserByEmail(email)
if err != nil {
u, err = services.CreateUser(instanceID, email, "", "member", "oidc")
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "provisioning failed"})
return
}
} else if u.InstanceID != instanceID {
c.JSON(http.StatusForbidden, gin.H{"error": "email belongs to a different organization"})
return
}
```
with:
```go
email := strings.ToLower(claims.Email)
// Scoped to the instance the callback state names, so an address that also
// exists in another instance is invisible here. That scoping replaces the
// cross-instance guard this code used to need: there is no longer a way for
// the lookup to return a user belonging to somebody else.
u, err := services.GetUserInInstanceByEmail(instanceID, email)
if err != nil {
u, err = services.CreateUser(instanceID, email, "", models.RoleMember, models.AuthOIDC)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "provisioning failed"})
return
}
}
```
`services.CreateUser`'s signature is `CreateUser(instanceID, email, password, role, authSource string)` — the argument order above matches it, with the two string literals the old code passed replaced by the constants Task 2 added.
`oidc.go` already imports `gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models`; confirm it before relying on the constants:
```sh
grep -n "server/internal/models" server/internal/auth/oidc.go
```
If that returns nothing, add the import rather than reverting to string literals — Task 2 exists so these two values have one spelling.
- [ ] **Step 2: Confirm the build is green**
Run:
```sh
sh /tmp/gorun.sh server go build ./...
```
Expected: no output.
- [ ] **Step 3: Confirm no unscoped email lookup survives anywhere in the server**
Run:
```sh
grep -rn "GetUserByEmail" --include=*.go .
```
Expected: no matches at all.
Run:
```sh
grep -rn 'FindOne(ctx, bson.M{"email"' --include=*.go server/
```
Expected: no matches.
**Coverage note.** The spec's phase-1 test 6 exercises this path end to end, which needs a working identity provider and is not reproducible in the container harness Task 7 uses. It is verified here by inspection and by the greps in Step 3 instead: the lookup is scoped by `instanceID`, which comes from `ConsumeStateInstance` and not from user input, and the deleted guard was the only other consumer of the unscoped helper. The first real OIDC sign-in after deployment is the confirming evidence — check that an existing SSO user still lands in their own instance before considering this closed.
- [ ] **Step 4: Commit Tasks 3, 4 and 5 together**
```bash
git add server/internal/services/users.go server/internal/auth/local.go server/internal/auth/oidc.go
git commit -m "feat(server): scope every user lookup by instance
users is unique on (instance_id, email) now, so an unscoped FindOne could
return an arbitrary one of several matching users. On the login path that
means signing someone into a tenant that is not theirs.
GetUserByEmail is deleted rather than left unused. Local sign-in resolves
its instance from the host, falling back to the single instance a
self-hosted deployment has, and refuses to guess otherwise. The OIDC
cross-instance guard goes: a scoped lookup cannot return another
instance's user, which is a stronger guarantee than the check it replaces.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>"
```
---
### Task 6: Remove admin's unscoped control-plane login
**Files:**
- Delete: `admin/internal/auth/cloud.go`
- Modify: `admin/internal/api/routes.go:30`, `admin/internal/api/routes.go:50-52`
- Modify: `admin/internal/api/staff.go`
**Interfaces:**
- Consumes: `auth.CreateCustomerUser(ctx, accountID, email, password string) error` from `admin/internal/auth/customer.go:32`.
- Produces: `POST /api/staff/accounts/:id/users`.
`HandleCloudLogin` authenticates against control-plane `users` with an unscoped `FindOne({email})`, and unlike the server's two lookups there is no instance in context to scope it by — HQ sign-in is not per-instance. It already falls through to `HandleCustomerLogin` whenever a `customer_users` row exists, which after phase 2 is every customer. Legacy cloud customers get an HQ login from staff, which is what the new endpoint is for; staff already attach those instances by hand per the spec README.
- [ ] **Step 1: Delete the file**
```sh
git rm admin/internal/auth/cloud.go
```
- [ ] **Step 2: Point `/auth/login` at the customer handler**
In `admin/internal/api/routes.go`, replace:
```go
r.POST("/auth/login", auth.HandleCloudLogin) // falls through to customer login
```
with:
```go
// Every customer authenticates against admin's own customer_users. There is
// deliberately no path that looks a customer up in the control plane by
// email alone: HQ sign-in names no instance, so such a lookup could not be
// scoped, and users.email is no longer globally unique.
r.POST("/auth/login", auth.HandleCustomerLogin)
```
- [ ] **Step 3: Add the staff route**
In `admin/internal/api/routes.go`, inside the `staff` group, immediately after the `staff.GET("/accounts/:id", staffGetAccount)` line, add:
```go
staff.POST("/accounts/:id/users", staffCreateAccountUser)
```
- [ ] **Step 4: Add the handler**
At the end of `admin/internal/api/staff.go`, add:
```go
// staffCreateAccountUser gives an account an HQ login.
//
// This is how a legacy cloud customer — one whose instance predates HQ accounts
// — gets into the portal, alongside the manual instance attach the spec README
// describes. It reuses CreateCustomerUser, so the row is unverified until the
// emailed link is opened and is rolled back if that email cannot be sent.
func staffCreateAccountUser(c *gin.Context) {
var body struct {
Email string `json:"email"`
Password string `json:"password"`
}
if err := c.ShouldBindJSON(&body); err != nil || body.Email == "" || len(body.Password) < 12 {
c.JSON(http.StatusBadRequest, gin.H{
"error": "email and a password of at least 12 characters are required"})
return
}
ctx := c.Request.Context()
accountID := c.Param("id")
if n, err := db.Admin("accounts").CountDocuments(ctx,
bson.M{"account_id": accountID}); err != nil || n == 0 {
c.JSON(http.StatusNotFound, gin.H{"error": "no such account"})
return
}
email := strings.ToLower(strings.TrimSpace(body.Email))
if err := auth.CreateCustomerUser(ctx, accountID, email, body.Password); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
s := auth.Current(c)
audit.Write(ctx, models.AuditEntry{
Actor: s.Email, Action: "customer_user.created", AccountID: accountID, Target: email})
c.JSON(http.StatusCreated, gin.H{"pending": true})
}
```
Confirm `strings` is imported in `staff.go`; add it if not:
```sh
grep -n '"strings"' admin/internal/api/staff.go
```
- [ ] **Step 5: Confirm the build is green and nothing still references the deleted handler**
Run:
```sh
grep -rn "HandleCloudLogin" --include=*.go .
```
Expected: no matches.
Run:
```sh
sh /tmp/gorun.sh admin go build ./...
```
Expected: no output. If `sharedmodels` is now an unused import in some file, remove that import line.
- [ ] **Step 6: Confirm admin has no unscoped control-plane user lookup left**
Run:
```sh
grep -rn 'db.Control("users")' --include=*.go admin/
```
Expected: no matches.
- [ ] **Step 7: Commit**
```bash
git add -A admin/
git commit -m "feat(admin): drop the unscoped control-plane login branch
HQ sign-in names no instance, so a lookup of control-plane users by email
alone cannot be scoped — and users.email is no longer globally unique, so
it would return an arbitrary match. Every customer authenticates against
customer_users instead.
Legacy cloud customers get an HQ login from staff via the new
POST /api/staff/accounts/:id/users, alongside the manual instance attach
the spec README already describes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>"
```
---
### Task 7: Documentation and end-to-end verification
**Files:**
- Modify: `CLAUDE.md`
**Interfaces:**
- Consumes: everything above.
- Produces: nothing.
This is the task that proves the change. With no test suite, this transcript is the only evidence, so run it in full rather than skimming it.
- [ ] **Step 1: Update `CLAUDE.md`**
In the **Auth and Orgs** section, replace the paragraph beginning "Unique indexes on user email and org slug are a **security property**" with:
```markdown
Unique indexes are a **security property**, not an optimisation. `users` is
unique on `(instance_id, email)` — one address is one user _within_ an instance,
and the same address may hold a user in several instances, because an account's
people are projected into each instance they are granted. This is sufficient only
because **every lookup by email is scoped by instance**; there is deliberately no
unscoped lookup anywhere, and adding one would let the login path return an
arbitrary one of several matching users. Instance slug, settings instance and ESO
token hash remain globally unique.
```
In the **Security** section, replace the "Unique indexes on user email, org slug…" bullet with:
```markdown
- Unique indexes on `(instance_id, email)`, instance slug, settings instance and the ESO token hash are load-bearing for tenant isolation. So is the absence of any unscoped lookup by email.
```
In the **MongoDB Collections** notes, add:
```markdown
- `users.auth_source` is `local`, `oidc` or `hq`. An `hq` user was projected from a Vantage HQ account and carries `hq_user_id`; HQ owns its role, password and existence.
```
- [ ] **Step 2: Build both images**
```sh
MSYS_NO_PATHCONV=1 docker build -q -f server/Dockerfile -t vantage-server:test .
MSYS_NO_PATHCONV=1 docker build -q -f admin/Dockerfile -t vantage-admin:test .
```
Expected: two image IDs. A "missing go.sum entry" failure here means `go mod tidy` was run in workspace mode.
- [ ] **Step 3: Start a scratch Mongo and Redis, and seed the OLD index**
Redis is not optional here: the server stores sessions in it, so every sign-in below fails without it.
```sh
MSYS_NO_PATHCONV=1 docker run -d --name vantage-idx-redis -p 6389:6379 redis:7
MSYS_NO_PATHCONV=1 docker run -d --name vantage-idx-mongo -p 27023:27017 mongo:7
MSYS_NO_PATHCONV=1 docker run --rm --add-host host.docker.internal:host-gateway mongo:7 \
mongosh "mongodb://host.docker.internal:27023/vantage_idx" --quiet --eval \
'db.users.createIndex({email:1},{unique:true}); db.getCollection("users").getIndexes().map(i=>i.name)'
```
Expected: output includes `email_1`. This reproduces a database that predates the change.
- [ ] **Step 4: Boot the server and confirm the swap**
```sh
MSYS_NO_PATHCONV=1 docker run -d --name vantage-idx-server -p 8091:8080 \
-e MONGO_URI=mongodb://host.docker.internal:27023 -e MONGO_DB=vantage_idx \
-e GRPC_HOST=localhost:9090 -e REDIS_ADDR=host.docker.internal:6389 \
-e GITEA_HOST=example.invalid \
--add-host host.docker.internal:host-gateway vantage-server:test
MSYS_NO_PATHCONV=1 docker run --rm --add-host host.docker.internal:host-gateway mongo:7 \
mongosh "mongodb://host.docker.internal:27023/vantage_idx" --quiet --eval \
'db.getCollection("users").getIndexes().map(i=>({name:i.name,key:i.key,unique:i.unique}))'
```
Expected: `instance_email_unique` present with key `{instance_id:1, email:1}` and `unique:true`; **no `email_1`**.
- [ ] **Step 5: Confirm a second boot is a no-op**
```sh
MSYS_NO_PATHCONV=1 docker restart vantage-idx-server
sleep 5
MSYS_NO_PATHCONV=1 docker logs vantage-idx-server 2>&1 | grep -i "index\|fatal" | tail -5
```
Expected: no index error and no fatal. The drop must tolerate the index already being gone.
- [ ] **Step 6: Bootstrap instance A and capture its user's password hash**
```sh
curl -s -X POST http://localhost:8091/auth/bootstrap \
-H 'Content-Type: application/json' \
-d '{"instance_name":"Alpha","email":"shared@example.com","password":"hunter2hunter2"}'
```
Expected: JSON with `instance_id` and `"slug":"alpha"`.
```sh
MSYS_NO_PATHCONV=1 docker run --rm --add-host host.docker.internal:host-gateway mongo:7 \
mongosh "mongodb://host.docker.internal:27023/vantage_idx" --quiet --eval \
'const u=db.users.findOne({email:"shared@example.com"}); print(u.user_id); print(u.password_hash)'
```
Expected: a UUID and a bcrypt hash. Keep both.
- [ ] **Step 7: Create instance B with the SAME address — the case that was impossible before**
```sh
MSYS_NO_PATHCONV=1 docker run --rm --add-host host.docker.internal:host-gateway mongo:7 \
mongosh "mongodb://host.docker.internal:27023/vantage_idx" --quiet --eval '
const a = db.users.findOne({email:"shared@example.com"});
const bId = UUID().toString().replace(/[{}]/g,"");
db.instances.insertOne({instance_id:bId, name:"Beta", slug:"beta", created_at:new Date()});
db.users.insertOne({
user_id: UUID().toString().replace(/[{}]/g,""),
instance_id: bId,
email: "shared@example.com",
password_hash: a.password_hash,
role: "owner",
auth_source: "local",
created_at: new Date()
});
print("beta instance " + bId);
print("users with that address: " + db.users.countDocuments({email:"shared@example.com"}));
'
```
Expected: `users with that address: 2`. Under the old global index this insert would have failed with E11000 — that failure is exactly what this phase removes.
- [ ] **Step 8: Confirm the compound index still refuses a duplicate WITHIN one instance**
```sh
MSYS_NO_PATHCONV=1 docker run --rm --add-host host.docker.internal:host-gateway mongo:7 \
mongosh "mongodb://host.docker.internal:27023/vantage_idx" --quiet --eval '
const a = db.users.findOne({email:"shared@example.com"});
try {
db.users.insertOne({user_id:"dup", instance_id:a.instance_id,
email:"shared@example.com", role:"member", auth_source:"local", created_at:new Date()});
print("FAIL: duplicate accepted");
} catch (e) { print("refused as expected: " + (e.code === 11000)); }
'
```
Expected: `refused as expected: true`. A `FAIL` line means the compound index is missing or not unique.
- [ ] **Step 9: Confirm each host signs in to its own instance — the whole point of the phase**
```sh
curl -s -X POST http://localhost:8091/auth/login -H 'Host: alpha.vantage.test' \
-H 'Content-Type: application/json' -c /tmp/alpha.jar \
-d '{"email":"shared@example.com","password":"hunter2hunter2"}'
curl -s http://localhost:8091/auth/me -H 'Host: alpha.vantage.test' -b /tmp/alpha.jar
```
Expected: `{"ok":true}`, then a body whose `instance` is **Alpha**.
```sh
curl -s -X POST http://localhost:8091/auth/login -H 'Host: beta.vantage.test' \
-H 'Content-Type: application/json' -c /tmp/beta.jar \
-d '{"email":"shared@example.com","password":"hunter2hunter2"}'
curl -s http://localhost:8091/auth/me -H 'Host: beta.vantage.test' -b /tmp/beta.jar
```
Expected: `{"ok":true}`, then a body whose `instance` is **Beta**, with a different `instance_id` from the Alpha response.
Two sign-ins, one address, one password, two different tenants. If both responses name the same instance, the lookup is not scoped.
- [ ] **Step 10: Confirm the apex host refuses rather than guesses**
```sh
curl -s -o /dev/null -w '%{http_code}\n' -X POST http://localhost:8091/auth/login \
-H 'Host: vantage.test' -H 'Content-Type: application/json' \
-d '{"email":"shared@example.com","password":"hunter2hunter2"}'
```
Expected: `400`. Then read the message:
```sh
curl -s -X POST http://localhost:8091/auth/login -H 'Host: vantage.test' \
-H 'Content-Type: application/json' \
-d '{"email":"shared@example.com","password":"hunter2hunter2"}'
```
Expected: an error naming both the instance count and the host. A `200` here would mean an arbitrary tenant was chosen.
- [ ] **Step 11: Confirm a wrong password still fails, on the right host**
```sh
curl -s -o /dev/null -w '%{http_code}\n' -X POST http://localhost:8091/auth/login \
-H 'Host: alpha.vantage.test' -H 'Content-Type: application/json' \
-d '{"email":"shared@example.com","password":"wrongwrongwrong"}'
```
Expected: `401`.
- [ ] **Step 12: Confirm a single-instance deployment still signs in on a bare host**
```sh
MSYS_NO_PATHCONV=1 docker run --rm --add-host host.docker.internal:host-gateway mongo:7 \
mongosh "mongodb://host.docker.internal:27023/vantage_idx" --quiet --eval \
'const b=db.instances.findOne({slug:"beta"}); db.users.deleteMany({instance_id:b.instance_id}); db.instances.deleteOne({slug:"beta"}); print(db.instances.countDocuments({}))'
```
Expected: `1`.
```sh
curl -s -X POST http://localhost:8091/auth/login -H 'Host: vantage.test' \
-H 'Content-Type: application/json' \
-d '{"email":"shared@example.com","password":"hunter2hunter2"}'
```
Expected: `{"ok":true}`. This is the self-hosted path, and it must keep working.
- [ ] **Step 13: Confirm admin boots and its login route still works**
```sh
MSYS_NO_PATHCONV=1 docker run -d --name vantage-idx-admin -p 8093:8083 \
-e ADMIN_MONGO_URI=mongodb://host.docker.internal:27023/vantage_idx_admin \
-e CONTROL_MONGO_URI=mongodb://host.docker.internal:27023/vantage_idx \
-e REDIS_ADDR=host.docker.internal:6389 \
-e LICENSE_SIGNING_KEY="$LICENSE_SIGNING_KEY" \
-e PUBLIC_URL=http://localhost:8093 -e ADMIN_ORIGIN=http://localhost:3004 \
--add-host host.docker.internal:host-gateway vantage-admin:test
sleep 5
curl -s http://localhost:8093/healthz
```
Expected: `{"ok":true}`. A boot failure here most likely means an unused-import error that `go build` caught but the image build did not, or a missing env var.
```sh
curl -s -o /dev/null -w '%{http_code}\n' -X POST http://localhost:8093/auth/login \
-H 'Content-Type: application/json' \
-d '{"email":"nobody@example.com","password":"hunter2hunter2"}'
```
Expected: `401`, not `500`. This proves `/auth/login` is wired to a live handler after `HandleCloudLogin` was deleted.
- [ ] **Step 14: Tear the scratch environment down**
```sh
MSYS_NO_PATHCONV=1 docker rm -f vantage-idx-server vantage-idx-admin vantage-idx-mongo vantage-idx-redis
```
- [ ] **Step 15: Commit**
```bash
git add CLAUDE.md
git commit -m "docs: users is unique per instance, not globally
The old index was load-bearing because two lookups were unscoped. Both
are scoped now and the unscoped helper is gone, so the property that
matters is the absence of any unscoped lookup by email. Says so, and
documents auth_source hq.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>"
```
---
## Done when
- `instance_email_unique` exists on `users`, `email_1` does not, and a second boot is a no-op.
- Two users share one address across two instances, and each signs in to their own.
- A duplicate address within one instance is still refused.
- The apex host refuses to guess when several instances exist, and still works when only one does.
- `grep -rn "GetUserByEmail"` and `grep -rn "HandleCloudLogin"` both return nothing.
- `admin` boots and `/auth/login` answers `401` rather than `500`.
- `CLAUDE.md` no longer claims `users.email` is globally unique.
**Not proven by this plan:** the OIDC sign-in path, which needs a real identity provider. Verify it manually on the first SSO sign-in after deployment — an existing SSO user must still land in their own instance.
## Not in this phase
`POST /api/instances`, the Free lifecycle, renewal, the notices, the reaper, the sitesvc cutover, account roles, invitations, instance membership, password propagation, and every UI change. Phases 2 and 3 get their own plans once this one lands.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -1,928 +0,0 @@
# Control plane mobile responsiveness — Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make `web/` (the Vantage control plane UI) usable on a phone — the sidebar becomes a hamburger-driven offcanvas below 1024px, tables become card stacks below 640px, and every fixed desktop layout collapses.
**Architecture:** A new client component `AppShell` owns the responsive chrome so `app/(app)/layout.tsx` stays a server component. `Sidebar.tsx` splits into a shared `SidebarContent` plus two containers (permanent aside, offcanvas drawer) so the nav exists in exactly one copy. The table card-stack lives in the `ui/Table.tsx` primitives via Tailwind `max-sm:` variants, so pages keep one markup tree and opt in with a `label` prop per cell.
**Tech Stack:** Next.js 16 (App Router), React 18, Tailwind 3.4, `clsx`. **No new dependencies.**
## Global Constraints
- **Scope is `web/` only.** Do not touch `site/`, `adminsite/`, `server/`, `admin/` or any Go code.
- **No hex colours anywhere.** Tailwind maps `var(--…)` tokens only. Use `bg-surface`, `border-border`, `text-text-secondary` etc. A literal `#` in a class is a defect. (`bg-black/60` is the one existing exception, already used by `Modal.tsx` for its backdrop — reuse it, do not introduce others.)
- **Breakpoints:** sidebar collapses below `lg` (1024px). Tables card-stack below `sm` (640px). Do not invent other breakpoints.
- **No new dependencies.** No headless-ui, no framer-motion.
- **Presentation only.** No API, route, query-key or data-shape changes.
- **Radius:** `rounded`, `rounded-lg`, `rounded-md` and `rounded-xl` all resolve to 46px via `tailwind.config.ts`. Prefer `rounded` in new code.
- Use `dvh`, not `vh`, for any new viewport-height value — mobile browser chrome makes `vh` overshoot.
- Indentation follows the file you are editing. `web/` is mixed: some files use 4 spaces (`Sidebar.tsx`, `keys/page.tsx`), others 2 (`servers/page.tsx`, `ui/*`). Match the file, do not reformat it.
- **There is no test framework in this repo.** No jest, no vitest, no playwright. Verification is `npx next lint`, `npx next build`, and targeted `grep` audits. Do not add a test framework.
- Run all commands from `d:\Development\Websites\vantage\web`.
---
### Task 1: Responsive table primitives
The card stack goes in the primitives, not the pages. Six pages render tables; giving each one a second markup tree would double the markup and drift on the first edit.
**Files:**
- Modify: `web/components/ui/Table.tsx` (whole file)
**Interfaces:**
- Consumes: nothing.
- Produces: `Td` gains an optional prop `label?: string`. Below `sm`, a `Td` with a `label` renders `<span>{label}</span>` before its children; a `Td` without one renders children alone, right-aligned. `Table`, `Thead`, `Tbody`, `Tr`, `Th` keep their existing signatures. Task 4 consumes `label`.
- [ ] **Step 1: Rewrite `web/components/ui/Table.tsx`**
Replace the entire file with:
```tsx
import { clsx } from "clsx";
import { HTMLAttributes, TdHTMLAttributes, ThHTMLAttributes } from "react";
/*
* Below sm the table stops being a table: the head is hidden, each row becomes
* a bordered card and each cell becomes a label/value pair. That lives here
* rather than in the six pages that render tables — two markup trees per page
* would drift apart on the first edit, and every one of those trees would mean
* the same thing.
*
* The mobile label uses Th's exact keyed-label idiom (mono, small, widely
* tracked, dimmed) because a key beside a value on a phone is the same device
* as a column head above it on a desktop.
*/
export function Table({ className, children, ...props }: HTMLAttributes<HTMLTableElement>) {
return (
<div className="overflow-x-auto">
<table
className={clsx("w-full border-collapse text-sm max-sm:block", className)}
{...props}
>
{children}
</table>
</div>
);
}
export function Thead({ className, children, ...props }: HTMLAttributes<HTMLTableSectionElement>) {
return (
<thead className={clsx("border-b border-border max-sm:hidden", className)} {...props}>
{children}
</thead>
);
}
export function Tbody({ className, children, ...props }: HTMLAttributes<HTMLTableSectionElement>) {
return (
<tbody
className={clsx(
"divide-y divide-border",
"max-sm:block max-sm:space-y-3 max-sm:divide-y-0 max-sm:p-3",
className
)}
{...props}
>
{children}
</tbody>
);
}
export function Tr({ className, children, ...props }: HTMLAttributes<HTMLTableRowElement>) {
return (
<tr
className={clsx(
"transition-colors hover:bg-surface-2/50",
"max-sm:block max-sm:rounded max-sm:border max-sm:border-border max-sm:bg-surface-2/40 max-sm:p-3",
className
)}
{...props}
>
{children}
</tr>
);
}
export function Th({ className, children, ...props }: ThHTMLAttributes<HTMLTableCellElement>) {
return (
<th
className={clsx(
// site/'s keyed-label idiom: mono, small, widely tracked, dimmed.
// A column head is a key, not prose.
// text-secondary, not tertiary: a column head is how you navigate the
// table, and tertiary lands under 4.5:1 at this size.
"px-4 py-3 text-left font-mono text-[0.68rem] uppercase tracking-[0.13em] text-text-secondary",
className
)}
{...props}
>
{children}
</th>
);
}
interface TdProps extends TdHTMLAttributes<HTMLTableCellElement> {
/**
* The column head this cell belongs to, shown beside the value below sm
* where the real head is hidden. Omit on a trailing action cell — an action
* needs no key, and the button then sits alone on its own row in the card.
*/
label?: string;
}
export function Td({ className, label, children, ...props }: TdProps) {
return (
<td
className={clsx(
"px-4 py-3 text-text-primary",
"max-sm:flex max-sm:items-start max-sm:gap-4 max-sm:px-0 max-sm:py-1.5",
// Exactly one justify class — clsx picks it. Emitting both and relying
// on string order would not work: Tailwind's output order decides which
// of two same-property utilities wins, not the order in this array.
label ? "max-sm:justify-between" : "max-sm:justify-end max-sm:pt-2.5",
className
)}
{...props}
>
{label && (
<span className="hidden font-mono text-[0.68rem] uppercase leading-5 tracking-[0.13em] text-text-secondary max-sm:inline">
{label}
</span>
)}
{children}
</td>
);
}
```
- [ ] **Step 2: Verify it compiles and lints**
```bash
npx tsc --noEmit
npx next lint
```
Expected: both clean. `tsc` may take ~30s. If `tsc --noEmit` errors on pre-existing issues unrelated to `Table.tsx`, note them and move on — only new errors matter.
- [ ] **Step 3: Commit**
```bash
git add web/components/ui/Table.tsx
git commit -m "feat(web): card-stack tables below sm"
```
---
### Task 2: Offcanvas sidebar
**Files:**
- Modify: `web/components/Sidebar.tsx` (whole file)
- Create: `web/components/AppShell.tsx`
- Modify: `web/app/(app)/layout.tsx` (whole file)
**Interfaces:**
- Consumes: `useAuth()` from `@/components/AuthProvider` returning `{ user, instance, isAdmin }`; `auth.logout()` from `@/lib/api`; `Logo` from `@/components/Logo`.
- Produces:
- `Sidebar.tsx` exports `SidebarContent({ onNavigate }: { onNavigate?: () => void })`, `Sidebar()` (permanent aside) and `SidebarDrawer({ open, onClose }: { open: boolean; onClose: () => void })`.
- `AppShell.tsx` exports `AppShell({ children }: { children: React.ReactNode })`.
- No later task depends on these names.
- [ ] **Step 1: Rewrite `web/components/Sidebar.tsx`**
Keep every icon component and the `navItems` array **exactly as they are** — do not retype the SVG path data, it is long and easy to corrupt. Change only from `export function Sidebar()` (line 135) to the end of the file, replacing it with the following. The file uses 4-space indentation.
```tsx
/** Shared by the permanent aside and the offcanvas drawer — one copy of the nav. */
export function SidebarContent({ onNavigate }: { onNavigate?: () => void }) {
const pathname = usePathname();
const { user, instance, isAdmin } = useAuth();
const visibleItems = navItems.filter((item) => !item.adminOnly || isAdmin);
const activeHref = visibleItems.reduce<string | null>((best, item) => {
const matches = pathname === item.href || pathname.startsWith(item.href + "/");
if (!matches) return best;
return best === null || item.href.length > best.length ? item.href : best;
}, null);
async function handleLogout() {
try {
await auth.logout();
} catch {}
window.location.href = "/login";
}
return (
<>
<div className="flex h-16 shrink-0 items-center gap-3 border-b border-border px-5">
<Logo className="h-8 w-8 text-logo" />
<div className="min-w-0">
<span className="block text-base font-extrabold leading-tight tracking-[-0.035em] text-text-primary">Vantage</span>
{instance && (
<span className="block truncate font-mono text-[0.68rem] uppercase tracking-[0.1em] text-text-secondary">{instance.name}</span>
)}
</div>
</div>
<nav className="flex-1 overflow-y-auto px-3 py-4">
<ul className="space-y-1">
{visibleItems.map((item) => {
const isActive = activeHref === item.href;
return (
<li key={item.href}>
<Link
href={item.href}
onClick={onNavigate}
// The active marker is an accent bar, the same device
// site/ uses to mark the chosen plan. A filled pill
// reads as a button you can press again.
className={clsx(
"relative flex items-center gap-3 rounded px-3 py-2.5 text-sm transition-colors",
isActive
? "bg-surface-2 font-semibold text-text-primary before:absolute before:inset-y-1 before:left-0 before:w-[2px] before:rounded-full before:bg-accent before:content-['']"
: "font-medium text-text-secondary hover:bg-surface-2 hover:text-text-primary",
)}
>
{item.icon}
{item.label}
</Link>
</li>
);
})}
</ul>
</nav>
<div className="shrink-0 border-t border-border px-4 py-3">
{user && (
<div className="mb-3">
<p className="truncate text-sm font-medium text-text-primary">{user.name || user.email}</p>
<p className="truncate text-xs text-text-secondary">
{user.email}
{user.role && <span className="ml-1 text-text-tertiary">· {user.role}</span>}
</p>
</div>
)}
<div className="flex items-center justify-between">
<p className="font-mono text-[0.68rem] uppercase tracking-[0.1em] text-text-secondary">Vantage v1.0</p>
{user && (
<button type="button" onClick={handleLogout} className="text-xs text-text-secondary transition-colors hover:text-danger">
Logout
</button>
)}
</div>
</div>
</>
);
}
/** The permanent sidebar. Below lg the drawer takes over. */
export function Sidebar() {
return (
<aside className="hidden h-screen w-60 shrink-0 flex-col border-r border-border bg-surface lg:flex">
<SidebarContent />
</aside>
);
}
/**
* The offcanvas below lg. Always mounted so the slide runs in both directions;
* closed it is inert (invisible + pointer-events-none) rather than unmounted.
*/
export function SidebarDrawer({ open, onClose }: { open: boolean; onClose: () => void }) {
const panelRef = useRef<HTMLDivElement>(null);
useEffect(() => {
if (!open) return;
const onKey = (e: KeyboardEvent) => {
if (e.key === "Escape") onClose();
};
window.addEventListener("keydown", onKey);
const previousOverflow = document.body.style.overflow;
document.body.style.overflow = "hidden";
panelRef.current?.focus();
return () => {
window.removeEventListener("keydown", onKey);
document.body.style.overflow = previousOverflow;
};
}, [open, onClose]);
return (
<div
className={clsx(
"fixed inset-0 z-50 lg:hidden",
open ? "visible" : "invisible pointer-events-none",
)}
>
<div
aria-hidden="true"
onClick={onClose}
className={clsx(
"absolute inset-0 bg-black/60 transition-opacity duration-200",
open ? "opacity-100" : "opacity-0",
)}
/>
<div
ref={panelRef}
id="app-sidebar-drawer"
role="dialog"
aria-modal="true"
aria-label="Navigation"
tabIndex={-1}
className={clsx(
"absolute inset-y-0 left-0 flex w-72 max-w-[85%] flex-col border-r border-border bg-surface outline-none transition-transform duration-200 ease-out",
open ? "translate-x-0" : "-translate-x-full",
)}
>
<SidebarContent onNavigate={onClose} />
</div>
</div>
);
}
```
Then update the import line at the top of the file (currently line 4) so `useEffect` and `useRef` are available:
```tsx
import { usePathname } from "next/navigation";
import { useEffect, useRef } from "react";
```
- [ ] **Step 2: Create `web/components/AppShell.tsx`**
```tsx
"use client";
import { useEffect, useRef, useState } from "react";
import { usePathname } from "next/navigation";
import { LicenseBanner } from "@/components/LicenseBanner";
import { Logo } from "@/components/Logo";
import { Sidebar, SidebarDrawer } from "@/components/Sidebar";
import { useAuth } from "@/components/AuthProvider";
function MenuIcon() {
return (
<svg className="h-6 w-6" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={1.5}>
<path strokeLinecap="round" strokeLinejoin="round" d="M3.75 6.75h16.5M3.75 12h16.5m-16.5 5.25h16.5" />
</svg>
);
}
/**
* Owns the responsive chrome so app/(app)/layout.tsx can stay a server
* component. Above lg this is the layout it always was; below lg the sidebar
* becomes an offcanvas behind the top bar's hamburger.
*/
export function AppShell({ children }: { children: React.ReactNode }) {
const [open, setOpen] = useState(false);
const pathname = usePathname();
const buttonRef = useRef<HTMLButtonElement>(null);
const { instance } = useAuth();
// A drawer that survives navigation would cover the page you just asked for.
useEffect(() => {
setOpen(false);
}, [pathname]);
function close() {
setOpen(false);
buttonRef.current?.focus();
}
return (
<div className="flex h-screen overflow-hidden">
<Sidebar />
<SidebarDrawer open={open} onClose={close} />
<div className="flex min-w-0 flex-1 flex-col overflow-y-auto">
<header className="sticky top-0 z-40 flex h-14 shrink-0 items-center gap-3 border-b border-border bg-surface px-3 lg:hidden">
<button
ref={buttonRef}
type="button"
onClick={() => setOpen(true)}
aria-label="Open navigation"
aria-expanded={open}
aria-controls="app-sidebar-drawer"
className="-ml-1 rounded p-2 text-text-secondary transition-colors hover:bg-surface-2 hover:text-text-primary"
>
<MenuIcon />
</button>
<Logo className="h-7 w-7 shrink-0 text-logo" />
<div className="min-w-0">
<span className="block text-sm font-extrabold leading-tight tracking-[-0.035em] text-text-primary">Vantage</span>
{instance && (
<span className="block truncate font-mono text-[0.62rem] uppercase tracking-[0.1em] text-text-secondary">{instance.name}</span>
)}
</div>
</header>
<main className="flex min-w-0 flex-1 flex-col">
<LicenseBanner />
{children}
</main>
</div>
</div>
);
}
```
- [ ] **Step 3: Rewrite `web/app/(app)/layout.tsx`**
```tsx
import { AuthProvider } from "@/components/AuthProvider";
import { AppShell } from "@/components/AppShell";
export default function AppLayout({
children,
}: {
children: React.ReactNode;
}) {
return (
<AuthProvider>
<AppShell>{children}</AppShell>
</AuthProvider>
);
}
```
`LicenseBanner` and `Sidebar` are no longer imported here — `AppShell` renders both.
- [ ] **Step 4: Verify**
```bash
npx tsc --noEmit
npx next lint
npx next build
```
Expected: all three succeed. `next build` is the one that matters — it catches a client component imported into a server component boundary.
- [ ] **Step 5: Sanity-check the scroll container**
Read `web/app/(app)/servers/[id]/console/page.tsx` around line 153 and 168. It uses `h-full`, which now resolves against `<main class="flex min-w-0 flex-1 flex-col">` rather than the old `<main class="flex-1 overflow-y-auto">`. Confirm the console page still has a height to fill; if `h-full` no longer resolves, change those two wrappers to `flex-1` instead. Task 7 revisits this file, so a note is acceptable here if you prefer to fix it there — but write the note down.
- [ ] **Step 6: Commit**
```bash
git add web/components/Sidebar.tsx web/components/AppShell.tsx "web/app/(app)/layout.tsx"
git commit -m "feat(web): offcanvas sidebar with hamburger below lg"
```
---
### Task 3: Page padding and header rows
**Files:**
- Modify: all 21 files under `web/app` and `web/components` containing `p-8`
- Modify: the title-plus-action header rows listed below
**Interfaces:**
- Consumes: nothing. Produces: nothing. Pure class edits.
- [ ] **Step 1: List every occurrence**
```bash
cd web && grep -rn "p-8" app components
```
Expected: 30 occurrences across 21 files.
- [ ] **Step 2: Replace each page-level `p-8` with `p-4 sm:p-6 lg:p-8`**
Apply to every occurrence **except** these two, which Task 6 and Task 7 handle and which need different values:
- `app/(app)/workflows/[id]/page.tsx:331` (the canvas `<main>`) — leave for Task 6.
- `app/(app)/servers/[id]/console/page.tsx:168` — leave for Task 7.
The inline loading states (`<div className="p-8 text-text-secondary">Loading…</div>`) get the same treatment: `className="p-4 text-text-secondary sm:p-6 lg:p-8"`.
Do this file by file with `Edit`. A blind `sed` would also hit `p-8` inside strings or unrelated contexts — check each match.
- [ ] **Step 3: Make title-plus-action header rows stack**
In each of these, change `className="mb-6 flex items-center justify-between"` to
`className="mb-6 flex flex-col gap-3 sm:flex-row sm:items-center sm:justify-between"`:
- `app/(app)/servers/page.tsx:75`
- `app/(app)/keys/page.tsx:112`
- `app/(app)/monitors/page.tsx:35`
- `app/(app)/workflows/page.tsx:32`
- `app/(app)/secrets/page.tsx:105`
- `app/(app)/secrets/[group]/page.tsx:251`
- `app/(app)/settings/notifications/page.tsx:153`
Leave `flex items-center justify-between` rows that are *inside* a card header or a table cell — those hold two small items and are fine at 390px. Only the page-top title/action rows change.
- [ ] **Step 4: Verify no unprefixed `p-8` survives**
```bash
cd web && grep -rn 'className="[^"]*\bp-8\b' app components | grep -v "sm:p-8\|lg:p-8"
```
Expected: exactly two lines — the two deferred to Tasks 6 and 7.
- [ ] **Step 5: Verify**
```bash
npx next lint && npx next build
```
Expected: both succeed.
- [ ] **Step 6: Commit**
```bash
git add web/app web/components
git commit -m "feat(web): responsive page padding and stacking page headers"
```
---
### Task 4: Label every table cell
**Files:**
- Modify: `web/app/(app)/servers/page.tsx:116-145`
- Modify: `web/app/(app)/keys/page.tsx:149-169`
- Modify: `web/app/(app)/monitors/page.tsx:79-98`
- Modify: `web/app/(app)/secrets/page.tsx:142-157`
- Modify: `web/app/(app)/secrets/[group]/page.tsx:141-150`
- Modify: `web/app/(app)/workflows/page.tsx:73-86`
- Modify: `web/app/(app)/workflows/[id]/runs/page.tsx:54-68`
- Modify: `web/app/(app)/audit/page.tsx:80-93`
- Modify: `web/app/(app)/keys/[id]/page.tsx:388-420`
- Modify: `web/app/(app)/servers/[id]/page.tsx:272-280` and `:588-605`
- Modify: `web/app/(app)/monitors/[id]/page.tsx:183-195`
- Modify: `web/components/settings/MembersCard.tsx:118-145`
**Interfaces:**
- Consumes: `Td`'s `label?: string` prop from Task 1.
- Produces: nothing.
- [ ] **Step 1: Add `label` to each `Td`, matching its `Th`**
For every table, the Nth `<Td>` in a `<Tr>` takes the text of the Nth `<Th>`. Where the `Th` is empty (`<Th />` — the trailing action column), the matching `Td` gets **no** `label`.
The mapping, `Th` order per file:
| File | Column labels, in order |
| --- | --- |
| `servers/page.tsx` | Hostname · IP Address · OS · Status · Last Seen · *(none)* |
| `keys/page.tsx` | Label · Fingerprint · Source · Assignments · Created · *(none)* |
| `monitors/page.tsx` | Name · Type · Target · Status · Latency · Last check |
| `secrets/page.tsx` | Group · Keys · Last Updated · *(none)* |
| `secrets/[group]/page.tsx` | Key · Value · Updated · *(none)* |
| `workflows/page.tsx` | Name · Targets · Steps · *(none)* |
| `workflows/[id]/runs/page.tsx` | Run · Status · Started · By · Servers |
| `audit/page.tsx` | Time · Event · Actor · Details |
| `keys/[id]/page.tsx` | Server · IP Address · Status · Assigned · Revoked · *(none)* |
| `servers/[id]/page.tsx` (updates table) | Package · Current · Available |
| `servers/[id]/page.tsx` (keys table) | Label · Fingerprint · Source · Status · Assigned · *(none)* |
| `monitors/[id]/page.tsx` | Started · Resolved · Cause |
| `MembersCard.tsx` | Email · Role · Sign-in · Last login · Actions |
Worked example — `servers/page.tsx` lines 116145 become:
```tsx
<Td label="Hostname">
<span className="font-medium text-text-primary">
{server.hostname}
</span>
</Td>
<Td label="IP Address">
<span className="font-mono text-text-secondary">
{server.ip_address}
</span>
</Td>
<Td label="OS">
<span className="text-text-secondary">{server.os_info}</span>
</Td>
<Td label="Status">
<StatusDot status={resolveStatus(server, latestVersion)} />
</Td>
<Td label="Last Seen">
<span className="text-text-secondary">
{server.last_seen
? formatLastSeen(server.last_seen)
: "Never"}
</span>
</Td>
<Td>
<Link href={`/servers/${server.server_id}`}>
<Button variant="ghost" size="sm">
View →
</Button>
</Link>
</Td>
```
Note the last `Td` is unchanged — no `label`, so the "View →" button sits alone on its own row at the bottom of the card.
Second worked example — `MembersCard.tsx` line 142143, where `Td` already carries a `className`. Both props coexist:
```tsx
<Td label="Last login" className="text-text-secondary">{u.last_login ? new Date(u.last_login).toLocaleString() : "Never"}</Td>
<Td label="Actions" className="text-right">
```
`MembersCard`'s last column has a real `Th` ("Actions"), so unlike the others it **does** take a label.
- [ ] **Step 2: Verify no `Td` was missed**
```bash
cd web && grep -rn "<Td" app components | grep -v "label="
```
Expected: only the trailing action cells listed as *(none)* above — 7 of them (`servers`, `keys`, `secrets`, `secrets/[group]`, `workflows`, `keys/[id]`, `servers/[id]` keys table). Any other bare `<Td` is a miss.
- [ ] **Step 3: Verify**
```bash
npx next lint && npx next build
```
Expected: both succeed.
- [ ] **Step 4: Commit**
```bash
git add web/app web/components
git commit -m "feat(web): label table cells for the mobile card stack"
```
---
### Task 5: Modal bottom sheet and shared-component grids
**Files:**
- Modify: `web/components/ui/Modal.tsx:28-31`
- Modify: `web/components/monitors/MonitorForm.tsx:76,98,123,144`
- Modify: `web/components/workflows/StepPickerModal.tsx:132,168`
- Modify: `web/components/ui/Card.tsx:27`
**Interfaces:**
- Consumes: nothing. Produces: nothing.
- [ ] **Step 1: Make `Modal` a bottom sheet below `sm`**
In `web/components/ui/Modal.tsx`, replace lines 2834:
```tsx
<div className="fixed inset-0 z-50 flex items-end justify-center p-0 sm:items-center sm:p-4">
<div className="absolute inset-0 bg-black/60" onClick={onClose} />
<div
className={`relative z-10 w-full ${wide ? "sm:max-w-2xl" : "sm:max-w-md"} max-h-[85dvh] overflow-auto rounded rounded-b-none border border-b-0 border-border bg-surface shadow-panel sm:rounded sm:border-b`}
role="dialog"
aria-modal="true"
>
```
The `max-w-*` gains an `sm:` prefix so the sheet is full-width on a phone. `dvh` rather than `vh` because mobile browser chrome makes `vh` overshoot.
- [ ] **Step 2: Collapse the grids in `MonitorForm.tsx`**
- Line 76: `grid grid-cols-4 gap-2``grid grid-cols-2 gap-2 sm:grid-cols-4`
- Lines 98, 123, 144: `grid grid-cols-2 gap-4``grid grid-cols-1 gap-4 sm:grid-cols-2`
- [ ] **Step 3: Collapse the grids in `StepPickerModal.tsx`**
Lines 132 and 168: `grid grid-cols-2 gap-2.5``grid grid-cols-1 gap-2.5 sm:grid-cols-2`
- [ ] **Step 4: Let `CardHeader` wrap**
`web/components/ui/Card.tsx` line 27: `"mb-4 flex items-center justify-between"``"mb-4 flex flex-wrap items-center justify-between gap-2"`. Card headers hold a title and an action; at 390px they need to be allowed to wrap rather than crush the title.
- [ ] **Step 5: Verify**
```bash
npx next lint && npx next build
```
Expected: both succeed.
- [ ] **Step 6: Commit**
```bash
git add web/components/ui/Modal.tsx web/components/ui/Card.tsx web/components/monitors/MonitorForm.tsx web/components/workflows/StepPickerModal.tsx
git commit -m "feat(web): bottom-sheet modals and collapsing component grids"
```
---
### Task 6: Workflow builder
**Files:**
- Modify: `web/app/(app)/workflows/[id]/page.tsx:305-324` (header), `:329` (grid), `:331` (canvas), `:340` (column), `:372` (node), `:403` (inspector)
**Interfaces:**
- Consumes: nothing. Produces: nothing.
Below `lg` the fixed-height two-column grid is dropped entirely: single column, natural page flow. The `100dvh` arithmetic only makes sense at `lg`, where there is no mobile top bar above it.
- [ ] **Step 1: Let the header wrap (line 305)**
```tsx
<div className="flex flex-wrap items-center gap-3 border-b border-border bg-surface px-4 py-3">
```
and on line 312 change `className="ml-auto flex items-center gap-2"` to
`className="ml-auto flex flex-wrap items-center gap-2"`.
- [ ] **Step 2: Make the shell single-column below lg (line 329)**
```tsx
<div className="flex flex-1 flex-col lg:grid lg:h-[calc(100dvh-53px)] lg:grid-cols-[1fr_320px]">
```
`h-[calc(100vh-53px)]` becomes `lg:h-[calc(100dvh-53px)]``lg:` because the mobile top bar changes the arithmetic, and `dvh` because `vh` overshoots on mobile.
- [ ] **Step 3: Canvas padding (line 331)**
```tsx
<main className="overflow-auto bg-background bg-[radial-gradient(circle_at_1px_1px,theme(colors.border)_1px,transparent_0)] bg-[length:22px_22px] p-4 sm:p-6 lg:p-8">
```
- [ ] **Step 4: Let the node column and nodes be fluid (lines 340 and 372)**
Line 340:
```tsx
<div className="mx-auto flex w-full max-w-[340px] flex-col items-center">
```
Line 372 — the node itself. The wrapping `<div key={wfIdx} className="w-full">` on line 349 already constrains it, so the node just fills:
```tsx
className={`w-full cursor-pointer rounded border bg-surface p-3 ${isSelected ? "border-signal ring-2 ring-signal/40" : "border-border"}`}
```
- [ ] **Step 5: Turn the inspector into a bottom panel below lg (line 403)**
```tsx
<aside
className={`overflow-auto border-border bg-surface p-4 lg:block lg:border-l ${
selected === null || !selectedRef ? "hidden" : "block border-t max-lg:max-h-[60dvh]"
}`}
>
```
Below `lg` the inspector is hidden until a step is selected — an empty "Select a step to configure it" panel is noise on a phone — and when shown it sits under the canvas with a top border and a capped height. Above `lg` it is the left-bordered right rail it always was, always visible.
- [ ] **Step 6: Verify**
```bash
npx next lint && npx next build
```
Expected: both succeed.
- [ ] **Step 7: Commit**
```bash
git add "web/app/(app)/workflows/[id]/page.tsx"
git commit -m "feat(web): single-column workflow builder below lg"
```
---
### Task 7: Remaining fixed layouts
**Files:**
- Modify: `web/app/(app)/servers/[id]/page.tsx:164,495`
- Modify: `web/app/(app)/secrets/page.tsx:53`
- Modify: `web/app/(app)/workflows/[id]/runs/[runId]/page.tsx:~250`
- Modify: `web/app/(app)/servers/[id]/console/page.tsx:168` and its header rows
**Interfaces:**
- Consumes: nothing. Produces: nothing.
- [ ] **Step 1: `servers/[id]/page.tsx` line 164 — inventory grid**
`className="grid grid-cols-3 gap-2"``className="grid grid-cols-2 gap-2 sm:grid-cols-3"`
- [ ] **Step 2: `servers/[id]/page.tsx` line 495 — install one-liner**
`className="relative flex-1 min-w-64 rounded-lg border border-border bg-well px-4 py-2.5 font-mono text-sm"` → replace `min-w-64` with `min-w-0 overflow-x-auto`.
`min-w-64` is 256px of floor on a flex child; combined with a sibling copy button it pushes the row past a 390px viewport and scrolls the whole page sideways. `min-w-0` lets the box shrink and scroll its own content instead. Also check the parent flex row a few lines above and give it `flex-wrap` if the copy button ends up crushed.
- [ ] **Step 3: `secrets/page.tsx` line 53**
`className="grid grid-cols-2 gap-3"``className="grid grid-cols-1 gap-3 sm:grid-cols-2"`
- [ ] **Step 4: `workflows/[id]/runs/[runId]/page.tsx` — the step matrix**
Read the file around lines 240290. The matrix `<table>` has a `<th className="min-w-[240px] …">`. It is a genuine two-dimensional matrix (steps × servers) and must keep scrolling horizontally rather than stacking — stacking would destroy the information.
Confirm the `<table>` sits inside a wrapper with `overflow-x-auto`. If it does not, wrap it:
```tsx
<div className="overflow-x-auto">
<table …>
</table>
</div>
```
If a wrapper already exists, leave it alone and note that in the commit body.
- [ ] **Step 5: `servers/[id]/console/page.tsx`**
Line 168: `className="flex h-full flex-col p-8"``className="flex h-full min-h-0 flex-1 flex-col p-4 sm:p-6 lg:p-8"`.
`flex-1` is added because Task 2 changed the parent `<main>` from `flex-1 overflow-y-auto` to `flex min-w-0 flex-1 flex-col`, so `h-full` alone may no longer resolve to anything. If Task 2 Step 5 recorded a note about this file, resolve it here.
Line 161's error state also has a bare `p-8` — Task 3 should already have handled it. Confirm it reads `p-4 sm:p-6 lg:p-8`.
Then read the connected-state toolbar below line 220 and add `flex-wrap` to any `flex items-center` row that holds three or more controls, so the console's chrome wraps instead of overflowing.
- [ ] **Step 6: Verify**
```bash
npx next lint && npx next build
```
Expected: both succeed.
- [ ] **Step 7: Commit**
```bash
git add web/app
git commit -m "feat(web): collapse remaining fixed layouts on small screens"
```
---
### Task 8: Final audit
**Files:** none modified unless the audit finds a miss.
- [ ] **Step 1: No unprefixed `p-8` remains**
```bash
cd web && grep -rn 'className="[^"]*\bp-8\b' app components | grep -v "sm:p-8\|lg:p-8"
```
Expected: no output.
- [ ] **Step 2: No unprefixed multi-column grid remains**
```bash
cd web && grep -rnoE '(class|className)="[^"]*(^|[" ])grid-cols-[2-9]' app components
```
Every hit must be a `grid-cols-2` that is genuinely fine at 390px (two short items side by side). Check each one and note the justification. Anything holding form controls or long text must gain a `grid-cols-1 sm:` prefix.
- [ ] **Step 3: No fixed pixel width escapes a breakpoint prefix**
```bash
cd web && grep -rnoE '(^|[" ])(w|min-w|max-w)-\[[0-9]{3,}px\]' app components
```
Expected: only `lg:`-prefixed hits, plus `max-w-[340px]` and `max-w-[1180px]` and `max-w-[300px]`, which are all *maximums* and shrink freely. A bare `w-[NNNpx]` or `min-w-[NNNpx]` without a prefix is a defect — except `min-w-[240px]` in the run-detail matrix, which is deliberate (Task 7 Step 4).
- [ ] **Step 4: No hex colours were introduced**
```bash
cd web && git diff main --stat && git diff main -- app components | grep -nE '^\+.*#[0-9a-fA-F]{3,8}\b'
```
Expected: no output from the grep. Tailwind in this app maps `var(--…)` tokens only.
- [ ] **Step 5: Full build and lint**
```bash
npx next lint
npx next build
```
Expected: both succeed with no new warnings.
- [ ] **Step 6: Read the diff end to end**
```bash
git diff main -- web/
```
Check for: an accidentally deleted SVG path, a `Td` whose `label` does not match its `Th`, indentation reformatted in a file that used the other convention.
- [ ] **Step 7: Commit any fixes**
```bash
git add web
git commit -m "fix(web): mobile audit corrections"
```
If the audit found nothing, skip this step — do not create an empty commit.
---
## Self-review notes
**Spec coverage:** shell → Task 2; tables → Tasks 1 and 4; padding and headers → Task 3; modal → Task 5; workflow builder → Task 6; remaining fixed layouts → Task 7; verification → Task 8 plus a verify step in every task.
**Known limitation:** there is no test framework and no running backend in this environment, so no task can prove a page *looks* right — only that it compiles, lints, and contains no pattern known to break at 390px. The first person to open this on a phone should expect to find something. That is a property of the verification approach chosen in the spec, not a gap in the plan.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More