Compare commits

..
151 Commits
Author SHA1 Message Date
mrhid6 c4f1684304 docs(plan): drop backfill, add staff instance attach
Server Deploy / deploy (push) Failing after 37s
Existing cloud instances get licensed by hand through the admin UI instead of
an automated backfill. That needs POST /api/staff/instances, which nothing else
provided — without it there is no way to attach an existing cloud instance to
an account.
2026-07-24 16:08:29 +01:00
mrhid6 8ed3bec511 docs: mark plans 1 and 2 shipped 2026-07-24 16:00:45 +01:00
mrhid6 58faf2e57f docs: add the admin-backend implementation plan 2026-07-24 16:00:37 +01:00
mrhid6 8e5f35f40c revert(server): drop the licence grandfather migration
Instances stay read-only until a licence is set. No licence is generated by
the control plane, which keeps the signing key out of it entirely.
2026-07-24 15:49:46 +01:00
mrhid6 eb8b85ccfe fix(server): add the lk go.sum entry the workspace was masking
Server Deploy / deploy (push) Successful in 2m21s
go build inside the Docker image runs outside the workspace, so server/go.sum
needed the hyperboloide/lk entry that GOWORK resolution was supplying locally.
Caught by the image build, not by go build.
2026-07-24 15:35:01 +01:00
mrhid6 09a39090c6 feat(server): grandfather existing cloud instances onto Professional 2026-07-24 15:26:26 +01:00
mrhid6 8626898e5e feat(web): licence banner, settings page and feature gating 2026-07-24 15:25:29 +01:00
mrhid6 ee4dff09c9 feat: show the instance ID after setup 2026-07-24 15:20:00 +01:00
mrhid6 8d88b16f20 feat(server): keep monitors and in-flight runs going when a licence lapses
Monitor execution and the workflow runner are deliberately unguarded: billing
state must not take away a customer's ability to know their infrastructure is
on fire, and killing a run midway leaves a half-configured server.

The plan called for a server-limit check in gRPC Register. Left out: a server
row only comes from CreateServer, which already checks the cap, so counting in
Register counts the caller itself and would reject a legitimate agent at
exactly the cap.
2026-07-24 15:15:06 +01:00
mrhid6 120c9c6735 feat(server): licence API, mutation gate and feature gates
RequireActiveLicense is mounted on the /api group so new routes are gated by
where they live. GET /api/servers/new is named explicitly: it mints a
pre-registration token, so it mutates despite the method.
2026-07-24 15:13:35 +01:00
mrhid6 855537c535 feat(server): enforce licence limits on servers, secret groups and channels 2026-07-24 15:11:03 +01:00
mrhid6 1d3fcebb28 feat(server): resolve licence state per instance 2026-07-24 15:09:47 +01:00
mrhid6 f9f382049d feat(shared): add licence fields to Instance 2026-07-24 15:08:48 +01:00
mrhid6 e4d7569a1c refactor(server): correct stale org wording in messages 2026-07-24 15:07:54 +01:00
mrhid6 c3e363eccc refactor(server): finish the Org to Instance rename
Private identifiers plan 0b's naming map missed, plus the OrgOIDC model type.
No wire format, database field or route changes.
2026-07-24 15:07:06 +01:00
mrhid6 6fde319b2f feat(license): trust the production signing key 2026-07-24 15:01:03 +01:00
mrhid6 4f1fce32c1 feat(license): add lkctl for issuing licences by hand 2026-07-24 14:59:13 +01:00
mrhid6 968408b955 feat(license): add offline verification 2026-07-24 14:58:25 +01:00
mrhid6 95c5d531ae feat(license): add signing and the trusted key list 2026-07-24 14:57:52 +01:00
mrhid6 b0d8edf9b6 feat(license): add the licence payload and tier seed table 2026-07-24 14:57:21 +01:00
mrhid6 a50219c0d9 docs: correct licence scheme to ECDSA P-384 with SHA-256
hyperboloide/lk signs with ECDSA P-384 and SHA-256, not ed25519, and encodes
keys as base32 rather than hex. Probed in task 1 of the licensing-core plan.
Design is unaffected — only the prose was wrong.
2026-07-24 14:56:36 +01:00
mrhid6 f10fe61916 chore(shared): add hyperboloide/lk for licence signing
Probed the library before building against it. Two corrections to plan 1:

- PublicKey.ToB32String() returns one value, not (string, error)
- The scheme is ECDSA P-384 with SHA-256, not ed25519 as the spec and plan
  claim. Design is unaffected; the prose needs fixing.
2026-07-24 14:53:26 +01:00
mrhid6 bc1cda26f8 docs: add implementation plans for licensing-core and instance-licensing
- 2026-07-24-licensing-core.md: 7 tasks. lk payload, offline verify, the
  trusted key slice, the noSign build tag, and lkctl for issuing by hand.
- 2026-07-24-instance-licensing.md: 10 tasks. Licence on the instance
  document, cached runtime state, deny-by-default mutation gate, feature
  gates, service-layer limits, settings UI, and migration 0005 to
  grandfather existing cloud instances.

Plan 2 opens by finishing the Org to Instance rename: 18 private identifiers
survived plan 0b's sweep. Nothing functional, but the file that gains the
licence cache is one of the two still carrying the old names.

Spec index updated with plan links and shipped status.
2026-07-24 14:48:45 +01:00
mrhid6 f646ce5c47 chore(server): remove rename-rollback
Migration 0004 is complete and verified on live, so the inverse rename has
served its purpose. Reverting the release now means restoring a backup.
2026-07-24 14:40:35 +01:00
mrhid6 1eb8c79623 Merge branch 'feat/shared-module'
Server Deploy / deploy (push) Successful in 2m43s
Shared Go module extraction (spec 0a) and the Org to Instance rename (spec 0b).

- shared/ module holds the documents and provisioning rules the control plane
  and sitesvc both write; sitesvc's hand-copied duplicates are deleted
- Org becomes Instance everywhere, including the org_id field on all 17
  tenant-scoped collections, via migration 0004_org_to_instance
- cmd/rename-rollback reverses the migration
- Go images now build from the repo root so the replace directive resolves

Migration verified against a seeded legacy fixture: lossless, tenant-isolated,
idempotent, resumable and reversible. NOT yet rehearsed against a production
snapshot, which plan 0b requires before deploying.
2026-07-24 14:21:56 +01:00
mrhid6 539403cccf fix(server): drop stale indexes before renaming the tenant key
Two defects found by running migration 0004 against a seeded legacy database.

A unique index on org_id treats a missing org_id as null. Renaming the field
strips it, so the second document collided and the whole update failed:

  E11000 duplicate key error collection: instance_oidc index: org_id_1
  dup key: { org_id: null }

The index cleanup therefore has to run BEFORE the field rename, not after.
rename-rollback needs the symmetric step for instance_id, or reverting hits
the same wall.

The detection also silently matched nothing: the driver decodes an index key
document as bson.D, not bson.M, so the type assertion always failed and no
index was ever dropped. IndexKeyedOn now handles both.
2026-07-24 14:15:05 +01:00
mrhid6 50a06dfdc0 refactor(site): rename Organisation to Instance
The signup form now posts instance_name, matching sitesvc.

Also restores site/next-env.d.ts. Its /// <reference> directives had been
stripped, which removed the Next.js type environment and failed the build
with "Cannot find name 'Promise'". Same cause as the agent's missing build
constraint.
2026-07-24 14:04:52 +01:00
mrhid6 b70ccc97d4 refactor(web): rename Organisation to Instance 2026-07-24 14:01:19 +01:00
mrhid6 3f0f12b111 refactor(sitesvc): rename Org to Instance, refuse an unmigrated database
The signup form's JSON field becomes instance_name, and the pending-signup
document field with it. That collection is sitesvc-private and expires after
24 hours, so no migration is needed, but in-flight signups written before the
deploy will fail verification.
2026-07-24 14:00:11 +01:00
mrhid6 3891a2c239 feat(server): add rename-rollback command for migration 0004 2026-07-24 13:59:04 +01:00
mrhid6 4f041d2f4b refactor(server): rename Org to Instance
Adds migration 0004_org_to_instance, the ScopedCollections list, the
AssertNoScopedCollectionMissed boot check, and moves EnsureAuthIndexes into
its own file.

Two ordering constraints the rename exposed, both now enforced and commented:

- 0004 must run BEFORE EnsureAuthIndexes. The index builder creates
  instances.slug, which would create an empty instances collection and make
  0004 refuse to rename orgs onto an existing target.
- Migrations 0001 to 0003 run BEFORE 0004 and still read and write org_id, so
  they use a private legacyOrg struct rather than shared/models.
2026-07-24 13:58:41 +01:00
mrhid6 43a2fdb3a0 refactor(shared): rename Org to Instance 2026-07-24 13:54:31 +01:00
mrhid6 de0e7b7cae chore: commit go.work.sum 2026-07-24 13:53:35 +01:00
mrhid6 0226936758 fix(agent): restore the !linux build constraint on collect_other.go
Unrelated to the shared module work; pre-existing on main.

"_other" is not a GOOS suffix, so without the constraint this file compiled
on Linux alongside collect_linux.go and redeclared collect. Linux agent
builds failed, which agent-release.yml depends on.
2026-07-24 13:48:23 +01:00
mrhid6 42cd73c5b8 build: build Go images from the repo root for the shared module
Also pins the explicit shared require in both go.mod files. `go mod tidy`
in workspace mode drops it, which breaks the Docker build where no
workspace exists.
2026-07-24 13:45:45 +01:00
mrhid6 51622a922b refactor(sitesvc): use shared models and provision
Deletes internal/provision and the hand-mirrored Org and User structs. The
control plane and sitesvc now share one definition of both.
2026-07-24 13:43:58 +01:00
mrhid6 76d3111a72 refactor(server): use shared models, provision and indexes
Also removes the duplicate Slugify in stepscan.go; workflow step slugs now
use the shared definition too.
2026-07-24 13:42:44 +01:00
mrhid6 b335dc77e3 feat(shared): add EnsureCoreIndexes 2026-07-24 13:39:46 +01:00
mrhid6 51db5ab2e9 feat(shared): add CreateOrg, RollbackOrg and CreateUser
Adopts sitesvc's retry-on-duplicate-key slug loop. The control plane
previously returned an error when it lost the slug race.
2026-07-24 13:39:20 +01:00
mrhid6 d715d2150c feat(shared): add slug rules and shared document models 2026-07-24 13:38:52 +01:00
mrhid6 98687043f7 chore: scaffold shared module 2026-07-24 13:38:05 +01:00
mrhid6 bdc26e4799 docs: add implementation plans for specs 0a and 0b
- 2026-07-24-shared-module.md: 8 tasks, extract the shared Go module
- 2026-07-24-instance-rename.md: 9 tasks, Org -> Instance including the
  database field, with migration 0004 and a rollback command

No automated tests per instruction; verification is by compiler, grep, and
manual end-to-end plus production-snapshot rehearsal. Spec testing sections
updated to match.
2026-07-24 13:31:14 +01:00
mrhid6 b027ad3f7f docs: add licensing programme specs
Seven specs covering the licensing and billing programme:

- 0a shared-module: extract shared Go module, remove sitesvc duplication
- 0b instance-rename: Org -> Instance, including the database field
- 1  licensing-core: lk-signed licence payload, offline verify, CLI issuer
- 2  instance-licensing: storage, enforcement, degraded mode, settings UI
- 3  admin-backend: accounts, instances, licences, subscriptions, injection
- 4  admin-site: staff and customer portal
- 5  paddle-billing: catalog, checkout, webhooks, signup migration

Design only. No implementation.

Un-ignores docs/superpowers/ so specs are versioned.
2026-07-24 13:13:47 +01:00
mrhid6 fd32c96d86 feat: Updated site
Server Deploy / deploy (push) Successful in 2m23s
2026-07-24 11:50:11 +01:00
mrhid6 b74cd10dfc fix: Fixed bg zoffset
Server Deploy / deploy (push) Successful in 51s
2026-07-24 11:23:36 +01:00
mrhid6 460b4afb22 fix: FIxed compile error
Server Deploy / deploy (push) Successful in 1m45s
2026-07-24 11:18:14 +01:00
mrhid6 9cc9e68d90 fix: FIxed compile error
Server Deploy / deploy (push) Failing after 1m3s
2026-07-24 11:06:57 +01:00
mrhid6 accf7493e9 feat: Updated login bg
Server Deploy / deploy (push) Failing after 1m0s
2026-07-24 11:02:44 +01:00
mrhid6 f0547317f7 feat: Updated login screen text
Server Deploy / deploy (push) Successful in 49s
2026-07-24 10:52:25 +01:00
mrhid6 60f0ddecd6 fix: Fixed sidebar logo
Server Deploy / deploy (push) Successful in 1m44s
2026-07-24 10:37:53 +01:00
mrhid6 8f5aebdd63 fix: Fixed sidebar logo
Server Deploy / deploy (push) Successful in 1m33s
2026-07-24 10:36:55 +01:00
mrhid6 a6030ef62f feat: Updated logo
Server Deploy / deploy (push) Successful in 3m51s
2026-07-24 10:31:11 +01:00
mrhid6 7df79d05f1 fix: Fixes to platform page
Server Deploy / deploy (push) Successful in 1m32s
2026-07-24 10:13:33 +01:00
mrhid6 f002eab1f6 fix: Fixed compile error
Server Deploy / deploy (push) Failing after 2m51s
2026-07-24 10:01:06 +01:00
mrhid6 e798365be2 feat: Removed comments
Server Deploy / deploy (push) Failing after 1m13s
2026-07-24 09:56:54 +01:00
mrhid6 1a6cf03c03 feat: Removed comments
Server Deploy / deploy (push) Failing after 1m59s
2026-07-24 09:51:30 +01:00
mrhid6 3b52bcbeb8 updates
Server Deploy / deploy (push) Successful in 2m50s
2026-07-24 09:24:03 +01:00
mrhid6 693d59a3e2 feat: Marketing site
Server Deploy / deploy (push) Successful in 5m51s
2026-07-22 16:50:13 +01:00
mrhid6 7a3b8cb700 feat: Updated claude.md 2026-07-22 13:22:26 +01:00
mrhid6 97bc766afb feat: Remove plans 2026-07-22 13:03:19 +01:00
mrhid6 9ffae221ac fix: Fixed server install scripts
Server Deploy / deploy (push) Successful in 59s
2026-07-22 12:59:05 +01:00
mrhid6 cbb66c63f6 feat: multi-tenant SaaS — orgs, auth, per-org isolation
Server Deploy / deploy (push) Successful in 1m31s
Converts Vantage from a single-admin self-hosted app into a multi-tenant
SaaS. Org isolation is org_id row-scoping in one shared deployment and
database; each org gets a subdomain via wildcard DNS, but the hostname is
a routing hint and never an authorization boundary.

- Orgs, users and roles (owner|admin|member) with local password auth,
  first-run bootstrap, and per-org OIDC replacing the global provider
- Redis sessions carrying org and role; host/session mismatch guard
- Every tenant-scoped collection carries org_id; handlers take org from
  the session only, never from client input
- Agent gRPC path resolves org from the servers record, so agent configs
  and the shared gRPC host are unchanged
- Per-org settings and ESO read token, replacing global singletons
- Frontend: login, setup and org settings pages; org-aware AuthProvider
- Migrations 0001-0003 backfill an existing single-tenant instance

Reviewed per task plus a whole-branch and a migration-focused pass.
2026-07-22 11:13:28 +01:00
mrhid6 0d4fb896bb docs(spec): correct collection names and per-org caveats
The spec named the audit collection "audit" and the channels collection
"channels"; the code uses audit_logs and notification_channels. The
migration followed the spec, which is how it came to backfill two
collections that do not exist.
2026-07-22 11:13:19 +01:00
mrhid6 0b7e55d301 fix(server): adopt the migrated org on bootstrap; survive fresh install
Two failures found by tracing the migration path against a real upgrade.

Bootstrap orphaned the entire dataset. On upgrade, 0001 creates the
Default org and stamps every legacy document with it, but the instance
has no users, so the operator must run /auth/bootstrap to get in — and
that unconditionally created a SECOND org and put the owner in it. Every
org-scoped read then filtered on the new org, so the operator would log
into an empty Vantage while all their data sat under "default". Nothing
errored and agents kept syncing, so it presented as total data loss.
Bootstrap now adopts the sole existing org, renaming and re-slugging it,
and only creates one when no org exists. More than one org with no users
is refused rather than guessed.

Fresh installs crash-looped. Nothing creates the settings collection
before EnsureSettingsIndexes, so DropOne returned NamespaceNotFound (26),
isIndexNotFound matched only IndexNotFound (27), and that check is fatal.
The same early return also skipped index creation in the secrets and
workflow ensures.

Also: only insert the backfill org on ErrNoDocuments, so a transient read
error can't race the fatal unique slug index; run 0002 before 0003 so the
settings migration can't be pushed into its ambiguous branch; fail 0002's
ambiguous case with a remedy instead of continuing into a fatal index
build; and skip non-string ids in the owner backfill rather than aborting.
2026-07-22 10:51:42 +01:00
mrhid6 77a92787fb fix: repair migration collection names and cross-cutting scoping gaps
Findings from the final whole-branch review.

- scopedCollections named "audit" and "channels", but the code writes to
  audit_logs and notification_channels. On upgrade from single-tenant,
  legacy audit events and channels would never get org_id, becoming
  invisible to org-filtered reads while channels silently stopped firing
  — and the detection loop counted the wrong names, so the 0001 marker
  could be written having migrated nothing. Names fixed, plus migration
  0003 so an incorrectly-migrated instance converges with a fresh one.
- EnsureAuthIndexes failure is now fatal. GetUserByEmail is unscoped and
  the OIDC cross-org guard compares against whichever duplicate Mongo
  returns first, so users.email uniqueness is a security invariant, and a
  legacy collection with duplicate emails is the realistic upgrade case.
- Evict the per-org OIDC provider cache on save; rotating away from a
  compromised IdP previously had no effect until restart.
- Build the oauth2 config per request instead of mutating a shared cached
  pointer outside the mutex, which raced on RedirectURL between
  concurrent logins for the same org.
- Stamp org_id on console_sessions, incidents and monitor_rollups, the
  last collections with no tenant column. 0003 derives their org from the
  owning server/monitor rather than defaulting, so one org's console
  history and incident timeline cannot merge into another's.
- Seed default steps when an org is created, not only at boot.
- Reject an empty session OrgID at the middleware.
- Derive the app root label from APP_ROOT_LABEL instead of hardcoding
  "vantage", which silently disabled the host guard off that domain.
- Stop caching negative slug lookups, so a new org's subdomain resolves
  immediately.
2026-07-22 10:44:17 +01:00
mrhid6 aa31cd8a10 fix: validate roles, guard last owner, scope bootstrap status
Security review of e70b2f0. The UI gating was correctly backed by
RequireRole everywhere; these are the missing validation gaps behind it.

- UpdateUserRole and createOrgUser accepted any role string verbatim, so
  an admin could self-promote to owner, create an owner outright, or set
  a junk role that silently stripped a user's access. Roles are now
  whitelisted, only an owner may grant or remove the owner role, and an
  actor cannot change their own.
- Neither demote nor delete guarded the last owner, so an org could reach
  zero owners. Both now refuse when no owner would remain, returning 409.
  Self-delete rejected.
- CountUsers counted across all orgs, so a locked-out org could never
  re-bootstrap once another tenant existed, and the unauthenticated
  bootstrap-status endpoint reported instance-wide state. It now answers
  per-org on an org host, falling back to global only on the apex.
- HandleMe repeats the middleware's host/org check; it sits outside the
  middleware so it can still return its own 401.
- Post-bootstrap now sends the new owner to their org host's login page.
  The session cookie is deliberately scoped to the exact host, so the old
  redirect landed them unauthenticated.
- AuthProvider renders an error state instead of mounting the shell with
  a null user when /auth/me fails for a reason other than 401.
- api.ts unwraps {"error": ...} so these messages render as text.
2026-07-22 10:26:21 +01:00
mrhid6 e70b2f0e67 feat(web): login, first-run setup, org settings; org-aware AuthProvider
- Route group (app) holds AuthProvider + Sidebar, so /login and /setup
  render without app chrome and never mount the provider.
- AuthProvider drops the removed auth_enabled flag and exposes
  {user, org, isAdmin}.
- New login page (password + SSO), first-run setup page, and org settings
  page with a members table and the OIDC provider form.
- Settings page and the Organization nav entry are gated on role, since
  /api/settings now 403s for members.
- GET /api/org/oidc gains client_secret_set so the UI can show whether a
  secret is stored; the secret itself is still never serialized, and an
  empty submitted value still means "keep the stored one".
- Fix logout: the sidebar linked to /auth/logout with a GET, but the route
  is POST-only, so logout was 404ing.
2026-07-22 10:17:12 +01:00
mrhid6 156c5354de fix(server): close fail-open paths in monitor org checks
Review hardening on e2f5f1f. None of these were live bypasses; all were
one bad row or one new caller away from becoming one.

- Replace the empty-orgID sentinel with explicit scheduler entry points.
  The sentinel meant "skip the org check" and was keyed on a value read
  from a DB record on the agent path, so a server doc with a blank org_id
  would silently disable isolation. The exported agent-facing functions
  now reject an empty orgID outright.
- ValidateAgentToken errors when the resolved server has no org.
- UpdateMonitor's runner and channel_ids type assertions were fail-open:
  a wrong-typed value skipped validation while the $set still ran. Now a
  hard error.
- Normalise an empty runner on update to the server runner, matching
  create. Previously it matched no runner at all, so the monitor silently
  stopped being checked and stopped alerting.
- IngestResult returns an error for an unknown monitor, so probing an
  unknown ID looks the same as probing a foreign one.
2026-07-22 10:05:37 +01:00
mrhid6 e2f5f1fa8c feat(grpc): resolve org from server record for agent RPCs
Two instances of the branch's recurring bug class remained in the agent
path: a client-supplied ID accepted as data, then consumed by an
unscoped query.

- ListMonitorsForRunner filtered on `runner` alone, and `runner` is set
  by the client on monitor create/update. Org A could point a monitor at
  org B's server_id and org B's agent would fetch and execute the check.
  Now org-filtered, and `runner` is validated against the caller's org on
  create and update.
- IngestResult resolved the monitor via the unfiltered getMonitorByID
  using a monitor_id from the agent's request body, letting org A's agent
  write state and incidents into org B's monitor and fire its channels.
  Now rejects on org mismatch and on a monitor not assigned to the
  reporting agent.

The in-process scheduler passes an empty orgID as a documented sentinel
for the cross-org server-run sweep.

Install script still emits a single shared GRPC_HOST; the agent path
resolves org from the server record, never from a hostname.
2026-07-22 10:00:30 +01:00
mrhid6 4f512d01f1 fix(server): harden per-org settings migration and sweeps
Review follow-ups on e5363a6:

- MigrateSettingsOrg no longer guesses via the "default" slug. One org
  means stamp that org; zero orgs means synthesise Default; more than
  one means leave it alone and log, since guessing would hand one org
  another's SMTP config and ESO token.
- EnsureSettingsIndexes failure is now fatal. Without the unique index
  on org_id, GetSettings returns an arbitrary duplicate; without the one
  on the token hash, ResolveSecretsReadToken picks an arbitrary org.
- Name the token-hash index explicitly so it stops colliding with the
  legacy name DropOne targets, and exclude the empty string from the
  partial filter.
- Log retention: distinguish a missing run doc from a Mongo error, so a
  transient failure skips the directory rather than purging it at the
  30-day default.
- Offline sweep: fresh context per org, log-and-continue on a per-org
  error, plus a final pass for servers whose org no longer exists.
- ESO handler 401s on an empty token-derived org rather than querying
  org_id "".
2026-07-22 09:41:51 +01:00
mrhid6 e5363a64ee fix(server): per-org settings and ESO read token
The settings collection was a single global document, so every org
shared one SMTP config, alert config, retention policy and ESO read
token. GetSecretGroupDecryptedAny then flattened every org's secrets
for a group into one map, meaning any tenant's token read every
tenant's secrets.

- settings gains org_id; GetSettings/SaveSettings/RotateSecretsReadToken/
  GetWorkflowLogRetentionDays all take orgID
- VerifySecretsReadToken replaced by ResolveSecretsReadToken, which
  resolves the org from the presented token's hash; the ESO endpoint
  derives its org from the token rather than a session, since it is
  called machine-to-machine
- GetSecretGroupDecryptedAny deleted in favour of the org-scoped variant
- settings and token-rotation routes now require owner/admin
- offline sweep and log retention resolve org per server / per run
- migration 0002 stamps the legacy settings doc with the default org

Note: /api/settings now 403s for members; the web settings page needs a
matching role check.
2026-07-22 09:35:47 +01:00
mrhid6 5a701acc82 fix: Removed tests 2026-07-22 09:31:29 +01:00
mrhid6 dff3668a25 fix(server): validate cross-org resource ownership
Review of the org-scoping pass found that org_id on a query filter
protects the row you look up, but does nothing when a handler accepts a
foreign resource ID as data and a downstream unscoped query consumes it.

- AssignKey: verify key and server both belong to the org
- BuildAuthorizedKeys: resolve server first, scope assignments and keys
  to that server's org (was honouring foreign assignment rows)
- Workflows: validate TargetServerIDs on create/update and re-check at
  trigger time
- Monitor incidents/uptime handlers: gate on org-scoped GetMonitor
- GetChannels: take orgID; validate channel_ids on monitor create/update
- Secret and default-step unique indexes: scope to org_id so a second
  org no longer hits E11000
- DeleteServer/DeleteMonitor: scope cascading deletes
2026-07-22 09:28:43 +01:00
mrhid6 850aa0ed05 feat(server): org-scope service layer + handlers + org admin API
Threads org_id through every admin-facing service function (servers, keys,
assignments, secrets, workflows/steps/runs, monitors, channels, audit),
adds RequireRole middleware, and wires /api/org user + OIDC management
routes. Agent/scheduler paths keep unique-key signatures and resolve org
from the loaded record; internal-only helpers (getServerByID,
getRunByID, getMonitorByID) preserve those call sites.
2026-07-21 16:56:39 +01:00
mrhid6 d0ed9885e7 feat(auth): per-org OIDC resolver replaces global provider 2026-07-21 16:41:50 +01:00
mrhid6 ff22340561 feat(auth): host-based org resolution + session/host match guard 2026-07-21 16:38:24 +01:00
mrhid6 2038e86b53 feat(auth): local email/password login + first-run bootstrap 2026-07-21 16:36:13 +01:00
mrhid6 404214d82e fix(services): keep original step Slugify; org reuses it 2026-07-21 16:33:31 +01:00
mrhid6 01fd2e201e feat(services): org create+slug and user service with bcrypt 2026-07-21 16:30:42 +01:00
mrhid6 066095ffca feat(auth): session carries org_id/role; add context helpers 2026-07-21 16:28:06 +01:00
mrhid6 022b1ef8ec feat(server): auth indexes + default-org backfill migration 2026-07-21 16:25:37 +01:00
mrhid6 a8771a6e4d style(models): gofmt org_id field alignment 2026-07-21 16:24:16 +01:00
mrhid6 502045d3af feat(models): add org/user/org_oidc models and org_id on scoped collections 2026-07-21 16:23:14 +01:00
mrhid6 ea73fc3a18 docs(plan): saas auth + orgs implementation plan 2026-07-21 16:20:20 +01:00
mrhid6 75b86e3843 docs(spec): add org slug, host-based resolution, single gRPC endpoint 2026-07-21 16:15:29 +01:00
mrhid6 9767e18123 chore: removed old plans 2026-07-21 15:54:11 +01:00
mrhid6 19b4aef95b feat(server): dark-themed HTML email notification template
Server Deploy / deploy (push) Successful in 1m22s
2026-07-21 15:14:57 +01:00
mrhid6 0464c540b2 feat: edit monitors + notification channels; HTTP monitor insecure-TLS option
Server Deploy / deploy (push) Successful in 2m9s
Agent Release / build (push) Successful in 10m39s
Agent Release / msi (push) Successful in 35s
2026-07-21 14:55:41 +01:00
mrhid6 45178d455e fix(server): SMTP dispatch dial timeout + implicit/STARTTLS handling
Server Deploy / deploy (push) Successful in 2m19s
2026-07-21 14:46:15 +01:00
mrhid6 f28ab1a741 feat(web): redesign settings page, drop legacy webhook/email alerting UI
Server Deploy / deploy (push) Successful in 1m24s
2026-07-21 14:40:36 +01:00
mrhid6 df6f8b6f62 feat(web): notification channel settings UI
Server Deploy / deploy (push) Successful in 1m24s
2026-07-21 14:24:38 +01:00
mrhid6 8f3a27100f feat(server): multi-channel monitor notifications
Server Deploy / deploy (push) Successful in 1m16s
2026-07-21 14:22:55 +01:00
mrhid6 69c7a352f6 feat(agent): agent-run monitor scheduler
Server Deploy / deploy (push) Successful in 1m21s
Agent Release / build (push) Successful in 10m43s
Agent Release / msi (push) Successful in 1m5s
2026-07-21 14:19:59 +01:00
mrhid6 a2bfa98a2d feat(proto): SyncMonitors + ReportChecks RPCs
Server Deploy / deploy (push) Successful in 1m27s
2026-07-21 14:18:49 +01:00
mrhid6 57151826ae feat(web): monitors list/detail/form UI
Server Deploy / deploy (push) Successful in 2m1s
2026-07-21 14:16:08 +01:00
mrhid6 e413009faa feat(server): server-run monitor scheduler + REST API
Server Deploy / deploy (push) Successful in 58s
2026-07-21 14:13:26 +01:00
mrhid6 3ec9f1b35f feat(server): monitor ingest pipeline, incidents, rollups
Server Deploy / deploy (push) Successful in 2m0s
2026-07-21 14:12:03 +01:00
mrhid6 e019493087 feat(server): monitor model + checker package
Server Deploy / deploy (push) Successful in 53s
2026-07-21 14:10:46 +01:00
mrhid6 ca2c05db14 feat(web): inventory panel on server detail
Server Deploy / deploy (push) Successful in 41s
2026-07-21 14:08:36 +01:00
mrhid6 1850f352a2 feat(agent): schedule inventory reporting (30s metrics, 15m static)
Server Deploy / deploy (push) Successful in 1m15s
2026-07-21 14:07:09 +01:00
mrhid6 850ffbafe1 feat(agent): /proc-based inventory collectors
Server Deploy / deploy (push) Successful in 16s
2026-07-21 14:06:11 +01:00
mrhid6 a28157dcf8 feat(server): store inventory and handle ReportInventory RPC
Server Deploy / deploy (push) Successful in 1m4s
2026-07-21 14:05:19 +01:00
mrhid6 03e2c3c50d feat(proto): add ReportInventory RPC and inventory model
Server Deploy / deploy (push) Successful in 2m38s
2026-07-21 14:04:28 +01:00
mrhid6 fb1a1292ec docs: add service monitoring phases to fleet inventory plan
Server Deploy / deploy (push) Successful in 21s
2026-07-21 14:01:50 +01:00
mrhid6 ec201a23a2 fix: Fixed draft workflow status
Server Deploy / deploy (push) Successful in 41s
2026-07-21 12:09:19 +01:00
mrhid6 d9a33b0672 fix(web): stop autosave loop by ignoring volatile server-echo fields
Server Deploy / deploy (push) Successful in 39s
2026-07-21 12:01:57 +01:00
mrhid6 2b7ef98dff fix(web): move autosave hooks above early return (React #310)
Server Deploy / deploy (push) Successful in 1m37s
2026-07-21 11:55:20 +01:00
mrhid6 b5f30bc7c8 Merge feat/step-picker-modal: autosave, step-picker modal, Steps page
Server Deploy / deploy (push) Successful in 1m22s
2026-07-21 11:51:03 +01:00
mrhid6 3a0116248e fix(web): guard autosave against in-flight lost-update race 2026-07-21 11:50:53 +01:00
mrhid6 6af0a88841 feat(web): add Steps to main nav 2026-07-21 11:45:43 +01:00
mrhid6 e4c3fc24d3 feat(web): standalone Steps management page 2026-07-21 11:43:42 +01:00
mrhid6 e46d0edbf2 feat(web): replace builder step sidebar with Add-step modal 2026-07-21 11:41:37 +01:00
mrhid6 a4c4a72dbc feat(web): StepPickerModal step picker component 2026-07-21 11:37:47 +01:00
mrhid6 67d729b360 feat(web): api.stepUsage binding 2026-07-21 11:36:03 +01:00
mrhid6 da6d825f45 fix(server): TestMain must not exit(0) before m.Run(); skip DB test individually 2026-07-21 11:34:56 +01:00
mrhid6 93423e32e6 feat(server): step usage counts endpoint 2026-07-21 11:31:05 +01:00
mrhid6 d0442291f5 feat(web): workflow builder autosave with last-saved status 2026-07-21 11:27:55 +01:00
mrhid6 6c5472760b Merge branch 'feat/adhoc-steps-import-export': ad-hoc steps, step import/export, default steps, auto-derived outputs
Server Deploy / deploy (push) Successful in 1m26s
2026-07-21 10:42:02 +01:00
mrhid6 7c4a676742 fix: inline step secrets + inline input/output display + import body limit 2026-07-21 10:40:45 +01:00
mrhid6 fbda26a188 feat(web): ad-hoc inline steps + import-to-inline in workflow editor 2026-07-21 10:34:51 +01:00
mrhid6 90ce7af769 feat(web): step export/import, sync defaults, auto outputs, default badge 2026-07-21 10:31:28 +01:00
mrhid6 b543cd1b3d feat(web): api client for step import/export/inline/defaults 2026-07-21 10:28:41 +01:00
mrhid6 15c9da1b01 feat(server): default steps seed-on-boot + admin re-sync 2026-07-21 10:25:41 +01:00
mrhid6 baa7bb239d feat(server): step import/export/parse endpoints 2026-07-21 10:22:35 +01:00
mrhid6 7342c46d99 feat(server): auto-derive outputs on save + resolve inline steps 2026-07-21 10:20:12 +01:00
mrhid6 813f9e6fef feat(server): derive declared_outputs from script + slugify 2026-07-21 10:17:50 +01:00
mrhid6 434f14ae3a feat(server): inline step ref + workflow validation 2026-07-21 10:15:52 +01:00
mrhid6 8398fd2279 docs: implementation plan for adhoc steps, import/export, defaults, auto-outputs 2026-07-21 10:11:16 +01:00
mrhid6 56f06b9eaf docs: auto-derive declared_outputs from script scan 2026-07-21 10:05:45 +01:00
mrhid6 d9d241f83b docs: design for adhoc steps, step import/export, default steps 2026-07-21 09:59:34 +01:00
mrhid6 aee910c1f8 fix: fixed variable inputs
Server Deploy / deploy (push) Successful in 1m22s
2026-07-20 18:00:47 +01:00
mrhid6 bea545e873 feat: More verbose logging on workflow logs
Agent Release / build (push) Successful in 45s
Agent Release / msi (push) Successful in 42s
Server Deploy / deploy (push) Successful in 1m52s
2026-07-20 17:35:58 +01:00
mrhid6 82d7dde5f8 fix: Fixed style on workflow run
Server Deploy / deploy (push) Successful in 42s
2026-07-20 16:44:53 +01:00
mrhid6 397016ad68 feat: Updated workflow runs page
Server Deploy / deploy (push) Successful in 1m20s
2026-07-20 16:00:41 +01:00
mrhid6 39348c9491 feat(web): live SSE log tail and log retention setting
Server Deploy / deploy (push) Successful in 1m35s
2026-07-20 15:18:36 +01:00
mrhid6 63dadf6239 feat(api): server-run log fetch and SSE stream endpoints 2026-07-20 15:18:35 +01:00
mrhid6 d905c99d32 feat(server): stream step logs to files, drop log bodies from run docs 2026-07-20 15:18:35 +01:00
mrhid6 85e1baf59a feat(server): workflow log-writer registry, retention setting, sweeper 2026-07-20 15:18:35 +01:00
mrhid6 351ad59dd8 fix(web): reseed edit-workflow modal state on open
Server Deploy / deploy (push) Successful in 1m23s
2026-07-20 14:56:14 +01:00
mrhid6 dcc901b0d2 feat(web): workflow runs list page and navigation links
Server Deploy / deploy (push) Successful in 1m29s
2026-07-20 14:52:13 +01:00
mrhid6 99bf093f00 feat(web): rebuild workflow builder — mockup styling, drag-and-drop, inputs inspector 2026-07-20 14:48:29 +01:00
mrhid6 619ccd28cb feat(web): edit-workflow modal (name/targets/delete) 2026-07-20 14:44:59 +01:00
mrhid6 f22f0a4729 feat(web): edit-base-step modal with inputs/outputs editor 2026-07-20 14:42:13 +01:00
mrhid6 78194daf5f feat(web): builder tokens, Modal primitive, input-param types 2026-07-20 14:40:07 +01:00
mrhid6 f141767fc2 feat(workflows): inject step inputs into env; update returns workflow 2026-07-20 14:37:36 +01:00
mrhid6 05cd8e154b feat(workflows): step input params model + cascade step delete 2026-07-20 14:34:50 +01:00
mrhid6 004cc03ba6 docs: add workflow builder v2 plan 2026-07-20 14:33:46 +01:00
mrhid6 236e89989f docs: add workflow builder v2 spec 2026-07-20 14:30:55 +01:00
mrhid6 b0a2de8ca1 fix: Fixed workflow style topbar
Server Deploy / deploy (push) Successful in 1m20s
2026-07-20 13:55:17 +01:00
mrhid6 e9ac7be8c3 feat: updates to workflow page
Server Deploy / deploy (push) Successful in 36s
2026-07-20 13:35:35 +01:00
mrhid6 47690c58d9 fix: Fixed workflow id
Server Deploy / deploy (push) Successful in 1m14s
2026-07-20 12:49:43 +01:00
225 changed files with 34075 additions and 9922 deletions
+19 -1
View File
@@ -24,7 +24,8 @@ jobs:
- name: Build and push server image
run: |
IMAGE="${{ vars.DOCKER_HOST }}/${{ github.repository_owner }}/vantage/server:latest"
docker build -t "$IMAGE" -f server/Dockerfile server/
# Root context: server depends on the shared module.
docker build -t "$IMAGE" -f server/Dockerfile .
docker push "$IMAGE"
- name: Build and push web image
@@ -35,3 +36,20 @@ jobs:
-t "$IMAGE" \
-f web/Dockerfile web/
docker push "$IMAGE"
- name: Build and push site image
run: |
IMAGE="${{ vars.DOCKER_HOST }}/${{ github.repository_owner }}/vantage/site:latest"
docker build \
--build-arg NEXT_PUBLIC_SITE_API="${{ vars.SITE_API_URL }}" \
--build-arg NEXT_PUBLIC_CONTACT_EMAIL="support@hostxtra.co.uk" \
-t "$IMAGE" \
-f site/Dockerfile site/
docker push "$IMAGE"
- name: Build and push sitesvc image
run: |
IMAGE="${{ vars.DOCKER_HOST }}/${{ github.repository_owner }}/vantage/sitesvc:latest"
# Root context: sitesvc depends on the shared module.
docker build -t "$IMAGE" -f sitesvc/Dockerfile .
docker push "$IMAGE"
+4 -2
View File
@@ -2,10 +2,12 @@ node_modules
dist
build
.env
docs
docs/*
!docs/superpowers/
.superpowers
installer/vantage-agent-windows-amd64.exe
installer/*.msi
installer/nssm.zip
installer/checksums-msi.txt
.next
.next
*.tsbuildinfo
+222
View File
@@ -0,0 +1,222 @@
package checker
import (
"context"
"crypto/tls"
"fmt"
"io"
"net"
"net/http"
"os"
"strings"
"time"
)
const (
TypeHTTP = "http"
TypeTCP = "tcp"
TypeICMP = "icmp"
TypeTLS = "tls"
)
type Spec struct {
Type string
URL string
Host string
Port int
Method string
ExpectedStatus int
Keyword string
TLSWarnDays int
Insecure bool
TimeoutSec int
}
type Result struct {
Up bool
LatencyMs int
Message string
CertExpiry *time.Time
}
func (s Spec) timeout() time.Duration {
t := s.TimeoutSec
if t <= 0 || t > 10 {
t = 10
}
return time.Duration(t) * time.Second
}
func Run(ctx context.Context, s Spec) Result {
switch s.Type {
case TypeHTTP:
return runHTTP(ctx, s)
case TypeTCP:
return runTCP(ctx, s)
case TypeICMP:
return runICMP(ctx, s)
case TypeTLS:
return runTLS(ctx, s)
default:
return Result{Message: "unknown check type: " + s.Type}
}
}
func runHTTP(ctx context.Context, s Spec) Result {
method := s.Method
if method == "" {
method = http.MethodGet
}
expect := s.ExpectedStatus
if expect == 0 {
expect = 200
}
client := &http.Client{Timeout: s.timeout()}
if s.Insecure {
client.Transport = &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}}
}
start := time.Now()
req, err := http.NewRequestWithContext(ctx, method, s.URL, nil)
if err != nil {
return Result{Message: err.Error()}
}
resp, err := client.Do(req)
if err != nil {
return Result{LatencyMs: msSince(start), Message: err.Error()}
}
defer resp.Body.Close()
res := Result{LatencyMs: msSince(start), Up: true}
if resp.TLS != nil && len(resp.TLS.PeerCertificates) > 0 {
exp := resp.TLS.PeerCertificates[0].NotAfter
res.CertExpiry = &exp
}
if resp.StatusCode != expect {
return Result{LatencyMs: res.LatencyMs, CertExpiry: res.CertExpiry, Message: fmt.Sprintf("status %d (want %d)", resp.StatusCode, expect)}
}
if s.Keyword != "" {
body, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if !strings.Contains(string(body), s.Keyword) {
return Result{LatencyMs: res.LatencyMs, CertExpiry: res.CertExpiry, Message: "keyword not found"}
}
}
return res
}
func runTCP(ctx context.Context, s Spec) Result {
addr := net.JoinHostPort(s.Host, fmt.Sprint(s.Port))
start := time.Now()
d := net.Dialer{Timeout: s.timeout()}
conn, err := d.DialContext(ctx, "tcp", addr)
if err != nil {
return Result{LatencyMs: msSince(start), Message: err.Error()}
}
conn.Close()
return Result{Up: true, LatencyMs: msSince(start)}
}
func runTLS(ctx context.Context, s Spec) Result {
port := s.Port
if port == 0 {
port = 443
}
addr := net.JoinHostPort(s.Host, fmt.Sprint(port))
start := time.Now()
d := net.Dialer{Timeout: s.timeout()}
conn, err := tls.DialWithDialer(&d, "tcp", addr, &tls.Config{ServerName: s.Host})
if err != nil {
return Result{LatencyMs: msSince(start), Message: err.Error()}
}
defer conn.Close()
certs := conn.ConnectionState().PeerCertificates
if len(certs) == 0 {
return Result{LatencyMs: msSince(start), Message: "no peer certificate"}
}
exp := certs[0].NotAfter
res := Result{LatencyMs: msSince(start), CertExpiry: &exp}
warn := s.TLSWarnDays
if warn <= 0 {
warn = 14
}
remaining := time.Until(exp)
if remaining <= 0 {
res.Message = "certificate expired"
return res
}
if remaining <= time.Duration(warn)*24*time.Hour {
res.Message = fmt.Sprintf("certificate expires in %d days", int(remaining.Hours()/24))
return res
}
res.Up = true
return res
}
func msSince(t time.Time) int { return int(time.Since(t).Milliseconds()) }
func runICMP(ctx context.Context, s Spec) Result {
dst, err := net.ResolveIPAddr("ip4", s.Host)
if err != nil {
return Result{Message: err.Error()}
}
conn, err := net.ListenPacket("ip4:icmp", "0.0.0.0")
if err != nil {
return Result{Message: "icmp socket: " + err.Error()}
}
defer conn.Close()
id := os.Getpid() & 0xffff
pkt := icmpEcho(id, 1)
deadline := time.Now().Add(s.timeout())
if d, ok := ctx.Deadline(); ok && d.Before(deadline) {
deadline = d
}
_ = conn.SetDeadline(deadline)
start := time.Now()
if _, err := conn.WriteTo(pkt, dst); err != nil {
return Result{Message: err.Error()}
}
reply := make([]byte, 1500)
for {
n, peer, err := conn.ReadFrom(reply)
if err != nil {
return Result{LatencyMs: msSince(start), Message: "no reply"}
}
if n < 28 || peer.String() != dst.String() {
continue
}
if reply[20] == 0 {
return Result{Up: true, LatencyMs: msSince(start)}
}
}
}
func icmpEcho(id, seq int) []byte {
b := []byte{8, 0, 0, 0, byte(id >> 8), byte(id), byte(seq >> 8), byte(seq)}
cs := icmpChecksum(b)
b[2] = byte(cs >> 8)
b[3] = byte(cs)
return b
}
func icmpChecksum(b []byte) uint16 {
var sum uint32
for i := 0; i < len(b)-1; i += 2 {
sum += uint32(b[i])<<8 | uint32(b[i+1])
}
if len(b)%2 == 1 {
sum += uint32(b[len(b)-1]) << 8
}
for sum>>16 != 0 {
sum = (sum & 0xffff) + (sum >> 16)
}
return ^uint16(sum)
}
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"gopkg.in/yaml.v3"
)
// ConfigDir returns the platform-specific config directory.
func ConfigDir() string {
if runtime.GOOS == "windows" {
base := os.Getenv("ProgramData")
+36 -11
View File
@@ -14,9 +14,9 @@ import (
"github.com/mrhid6/vantage/agent/internal/grpc/pb"
)
// streamWriter forwards every write to emit() as an ordered chunk. Used as both
// Stdout and Stderr so output interleaves in real execution order. The mutex
// ensures a single stdout/stderr write is not interleaved mid-slice with another.
type streamWriter struct {
mu sync.Mutex
seq uint64
@@ -35,10 +35,21 @@ func (w *streamWriter) Write(p []byte) (int, error) {
return len(p), nil
}
// RunStep writes the script to a temp file, provides a WORKFLOW_ENV file for
// the script to append KEY=value output to, executes it under the requested
// interpreter, and streams output via emit, returning the terminal result
// with empty stdout/stderr but populated exit_code/output_env.
func WorkspacePath(workspaceID string) string {
return filepath.Join(os.TempDir(), "vantage-run-"+workspaceID)
}
func RunStep(cmd *pb.RunStepCmd, emit func(seq uint64, data []byte)) *pb.StepResult {
res := &pb.StepResult{CommandId: "", OutputEnv: map[string]string{}}
@@ -50,6 +61,16 @@ func RunStep(cmd *pb.RunStepCmd, emit func(seq uint64, data []byte)) *pb.StepRes
}
defer os.RemoveAll(dir)
workDir := ""
if cmd.WorkspaceId != "" {
workDir = WorkspacePath(cmd.WorkspaceId)
if err := os.MkdirAll(workDir, 0700); err != nil {
res.ExitCode = 1
res.Stderr = "create workspace: " + err.Error()
return res
}
}
envFile := filepath.Join(dir, "workflow_env")
if err := os.WriteFile(envFile, nil, 0600); err != nil {
res.ExitCode = 1
@@ -81,7 +102,7 @@ func RunStep(cmd *pb.RunStepCmd, emit func(seq uint64, data []byte)) *pb.StepRes
}
}
c = exec.CommandContext(ctx, shell, "-NoProfile", "-NonInteractive", "-File", scriptPath)
default: // "bash"
default:
scriptPath = filepath.Join(dir, "step.sh")
if err := os.WriteFile(scriptPath, []byte(cmd.Script), 0700); err != nil {
res.ExitCode = 1
@@ -91,6 +112,10 @@ func RunStep(cmd *pb.RunStepCmd, emit func(seq uint64, data []byte)) *pb.StepRes
c = exec.CommandContext(ctx, "bash", scriptPath)
}
if workDir != "" {
c.Dir = workDir
}
c.Env = append(os.Environ(), "WORKFLOW_ENV="+envFile)
for k, v := range cmd.Env {
c.Env = append(c.Env, k+"="+v)
@@ -101,7 +126,7 @@ func RunStep(cmd *pb.RunStepCmd, emit func(seq uint64, data []byte)) *pb.StepRes
c.Stderr = sw
runErr := c.Run()
// stdout/stderr are streamed via emit, not returned in the result.
if ctx.Err() == context.DeadlineExceeded {
res.ExitCode = 124
res.Stderr = "[vantage] step timed out"
@@ -116,8 +141,8 @@ func RunStep(cmd *pb.RunStepCmd, emit func(seq uint64, data []byte)) *pb.StepRes
return res
}
// parseEnvFile reads KEY=value lines (last write wins). Blank lines and lines
// without '=' are ignored.
func parseEnvFile(path string) map[string]string {
out := map[string]string{}
f, err := os.Open(path)
+28 -4
View File
@@ -27,8 +27,8 @@ func New(serverURL string, useTLS bool) (*Client, error) {
serverURL = strings.TrimPrefix(serverURL, "https://")
serverURL = strings.TrimPrefix(serverURL, "http://")
// Send a ping every 30s so proxies with a 60s idle timeout don't kill the
// long-lived CommandStream when no commands are flowing.
dialOpts := []grpc.DialOption{
grpc.WithKeepaliveParams(keepalive.ClientParameters{
Time: 30 * time.Second,
@@ -126,8 +126,32 @@ func (c *Client) ReportUpdates(serverID, agentToken string, updates []pb.Package
return err
}
// CommandStream opens a long-lived bidirectional stream for server-pushed commands.
// The caller controls the stream lifetime via ctx.
func (c *Client) ReportInventory(report *pb.InventoryReport) error {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
_, err := c.client.ReportInventory(ctx, report)
return err
}
func (c *Client) SyncMonitors(serverID, agentToken string) ([]pb.MonitorSpec, error) {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
resp, err := c.client.SyncMonitors(ctx, &pb.SyncMonitorsRequest{ServerId: serverID, AgentToken: agentToken})
if err != nil {
return nil, err
}
return resp.Monitors, nil
}
func (c *Client) ReportChecks(serverID, agentToken string, results []pb.CheckResult) error {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
_, err := c.client.ReportChecks(ctx, &pb.ReportChecksRequest{ServerId: serverID, AgentToken: agentToken, Results: results})
return err
}
func (c *Client) CommandStream(ctx context.Context) (pb.Vantage_CommandStreamClient, error) {
return c.client.CommandStream(ctx)
}
+116 -10
View File
@@ -1,4 +1,4 @@
// Hand-written gRPC bindings for vantage.proto (agent side, JSON codec).
package pb
@@ -44,7 +44,7 @@ type UploadKeyResponse struct {
KeyId string `json:"key_id"`
}
// CommandStream message types
type PackageUpdate struct {
Name string `json:"name"`
@@ -60,15 +60,91 @@ type ReportUpdatesRequest struct {
type ReportUpdatesResponse struct{}
type CPUReport struct {
Model string `json:"model,omitempty"`
Cores int `json:"cores,omitempty"`
UsagePct float64 `json:"usage_pct"`
Load1 float64 `json:"load1,omitempty"`
}
type MemReport struct {
TotalBytes uint64 `json:"total_bytes"`
UsedBytes uint64 `json:"used_bytes"`
}
type PartitionReport struct {
Device string `json:"device"`
Mountpoint string `json:"mountpoint"`
Fstype string `json:"fstype,omitempty"`
TotalBytes uint64 `json:"total_bytes"`
UsedBytes uint64 `json:"used_bytes"`
}
type InventoryReport struct {
ServerId string `json:"server_id"`
AgentToken string `json:"agent_token"`
IncludeStatic bool `json:"include_static"`
CPU *CPUReport `json:"cpu,omitempty"`
Memory *MemReport `json:"memory,omitempty"`
SwapTotal uint64 `json:"swap_total"`
SwapUsed uint64 `json:"swap_used"`
Partitions []PartitionReport `json:"partitions,omitempty"`
Kernel string `json:"kernel,omitempty"`
}
type InventoryReportResponse struct{}
type MonitorSpec struct {
MonitorId string `json:"monitor_id"`
Type string `json:"type"`
URL string `json:"url,omitempty"`
Host string `json:"host,omitempty"`
Port int `json:"port,omitempty"`
Method string `json:"method,omitempty"`
ExpectedStatus int `json:"expected_status,omitempty"`
Keyword string `json:"keyword,omitempty"`
TLSWarnDays int `json:"tls_warn_days,omitempty"`
Insecure bool `json:"insecure,omitempty"`
IntervalSec int `json:"interval_sec"`
Retries int `json:"retries"`
}
type SyncMonitorsRequest struct {
ServerId string `json:"server_id"`
AgentToken string `json:"agent_token"`
}
type SyncMonitorsResponse struct {
Monitors []MonitorSpec `json:"monitors,omitempty"`
}
type CheckResult struct {
MonitorId string `json:"monitor_id"`
Up bool `json:"up"`
LatencyMs int `json:"latency_ms"`
Message string `json:"message,omitempty"`
CertExpiryUnix int64 `json:"cert_expiry_unix,omitempty"`
}
type ReportChecksRequest struct {
ServerId string `json:"server_id"`
AgentToken string `json:"agent_token"`
Results []CheckResult `json:"results,omitempty"`
}
type ReportChecksResponse struct{}
type ApplyUpdatesCmd struct{}
type ServerCommand struct {
CommandId string `json:"command_id"`
GenerateKey *GenerateKeyCmd `json:"generate_key,omitempty"`
DeleteKey *DeleteKeyCmd `json:"delete_key,omitempty"`
UpdateAgent *UpdateAgentCmd `json:"update_agent,omitempty"`
ApplyUpdates *ApplyUpdatesCmd `json:"apply_updates,omitempty"`
RunStep *RunStepCmd `json:"run_step,omitempty"`
CommandId string `json:"command_id"`
GenerateKey *GenerateKeyCmd `json:"generate_key,omitempty"`
DeleteKey *DeleteKeyCmd `json:"delete_key,omitempty"`
UpdateAgent *UpdateAgentCmd `json:"update_agent,omitempty"`
ApplyUpdates *ApplyUpdatesCmd `json:"apply_updates,omitempty"`
RunStep *RunStepCmd `json:"run_step,omitempty"`
CleanupWorkspace *CleanupWorkspaceCmd `json:"cleanup_workspace,omitempty"`
}
type CleanupWorkspaceCmd struct {
WorkspaceId string `json:"workspace_id"`
}
type DeleteKeyCmd struct {
@@ -110,6 +186,9 @@ type RunStepCmd struct {
Script string `json:"script"`
Env map[string]string `json:"env,omitempty"`
TimeoutSeconds int `json:"timeout_seconds,omitempty"`
WorkspaceId string `json:"workspace_id,omitempty"`
}
type StepResult struct {
@@ -127,7 +206,7 @@ type StepOutputChunk struct {
Eof bool `json:"eof,omitempty"`
}
// CommandStream client-side interface
type Vantage_CommandStreamClient interface {
Send(*AgentMessage) error
@@ -151,7 +230,7 @@ func (c *vantageCommandStreamClient) Recv() (*ServerCommand, error) {
return m, nil
}
// CommandStream server-side interface (included for completeness)
type Vantage_CommandStreamServer interface {
Send(*ServerCommand) error
@@ -180,6 +259,9 @@ type VantageClient interface {
SyncKeys(ctx context.Context, in *SyncRequest, opts ...grpc.CallOption) (*SyncResponse, error)
UploadGeneratedKey(ctx context.Context, in *UploadKeyRequest, opts ...grpc.CallOption) (*UploadKeyResponse, error)
ReportUpdates(ctx context.Context, in *ReportUpdatesRequest, opts ...grpc.CallOption) (*ReportUpdatesResponse, error)
ReportInventory(ctx context.Context, in *InventoryReport, opts ...grpc.CallOption) (*InventoryReportResponse, error)
SyncMonitors(ctx context.Context, in *SyncMonitorsRequest, opts ...grpc.CallOption) (*SyncMonitorsResponse, error)
ReportChecks(ctx context.Context, in *ReportChecksRequest, opts ...grpc.CallOption) (*ReportChecksResponse, error)
CommandStream(ctx context.Context, opts ...grpc.CallOption) (Vantage_CommandStreamClient, error)
}
@@ -235,6 +317,30 @@ func (c *keyManagerClient) ReportUpdates(ctx context.Context, in *ReportUpdatesR
return out, nil
}
func (c *keyManagerClient) ReportInventory(ctx context.Context, in *InventoryReport, opts ...grpc.CallOption) (*InventoryReportResponse, error) {
out := new(InventoryReportResponse)
if err := c.cc.Invoke(ctx, "/vantage.v1.Vantage/ReportInventory", in, out, opts...); err != nil {
return nil, err
}
return out, nil
}
func (c *keyManagerClient) SyncMonitors(ctx context.Context, in *SyncMonitorsRequest, opts ...grpc.CallOption) (*SyncMonitorsResponse, error) {
out := new(SyncMonitorsResponse)
if err := c.cc.Invoke(ctx, "/vantage.v1.Vantage/SyncMonitors", in, out, opts...); err != nil {
return nil, err
}
return out, nil
}
func (c *keyManagerClient) ReportChecks(ctx context.Context, in *ReportChecksRequest, opts ...grpc.CallOption) (*ReportChecksResponse, error) {
out := new(ReportChecksResponse)
if err := c.cc.Invoke(ctx, "/vantage.v1.Vantage/ReportChecks", in, out, opts...); err != nil {
return nil, err
}
return out, nil
}
func (c *keyManagerClient) CommandStream(ctx context.Context, opts ...grpc.CallOption) (Vantage_CommandStreamClient, error) {
desc := &grpc.StreamDesc{StreamName: "CommandStream", ServerStreams: true, ClientStreams: true}
stream, err := c.cc.NewStream(ctx, desc, "/vantage.v1.Vantage/CommandStream", opts...)
+157
View File
@@ -0,0 +1,157 @@
package inventory
import (
"bufio"
"os"
"strconv"
"strings"
"syscall"
"time"
"github.com/mrhid6/vantage/agent/internal/grpc/pb"
)
func collect(r *pb.InventoryReport, includeStatic bool) {
r.CPU.UsagePct = cpuUsage()
r.CPU.Load1 = load1()
memTotal, memAvail, swapTotal, swapFree := meminfo()
if memTotal > memAvail {
r.Memory.UsedBytes = memTotal - memAvail
}
if swapTotal > swapFree {
r.SwapUsed = swapTotal - swapFree
}
if includeStatic {
r.Memory.TotalBytes = memTotal
r.SwapTotal = swapTotal
r.CPU.Model, r.CPU.Cores = cpuStatic()
r.Kernel = kernel()
r.Partitions = partitions()
}
}
func readProc(path string) string { b, _ := os.ReadFile(path); return string(b) }
func cpuSample() (idle, total uint64) {
f, err := os.Open("/proc/stat")
if err != nil {
return
}
defer f.Close()
sc := bufio.NewScanner(f)
if sc.Scan() {
fields := strings.Fields(sc.Text())
for i, v := range fields[1:] {
n, _ := strconv.ParseUint(v, 10, 64)
total += n
if i == 3 {
idle = n
}
}
}
return
}
func cpuUsage() float64 {
i1, t1 := cpuSample()
time.Sleep(100 * time.Millisecond)
i2, t2 := cpuSample()
dt := float64(t2 - t1)
if dt <= 0 {
return 0
}
return (1 - float64(i2-i1)/dt) * 100
}
func load1() float64 {
fields := strings.Fields(readProc("/proc/loadavg"))
if len(fields) > 0 {
v, _ := strconv.ParseFloat(fields[0], 64)
return v
}
return 0
}
func meminfo() (total, avail, swapTotal, swapFree uint64) {
f, err := os.Open("/proc/meminfo")
if err != nil {
return
}
defer f.Close()
sc := bufio.NewScanner(f)
for sc.Scan() {
fields := strings.Fields(sc.Text())
if len(fields) < 2 {
continue
}
kb, _ := strconv.ParseUint(fields[1], 10, 64)
b := kb * 1024
switch strings.TrimSuffix(fields[0], ":") {
case "MemTotal":
total = b
case "MemAvailable":
avail = b
case "SwapTotal":
swapTotal = b
case "SwapFree":
swapFree = b
}
}
return
}
func cpuStatic() (model string, cores int) {
f, err := os.Open("/proc/cpuinfo")
if err != nil {
return
}
defer f.Close()
sc := bufio.NewScanner(f)
for sc.Scan() {
line := sc.Text()
if strings.HasPrefix(line, "processor") {
cores++
} else if strings.HasPrefix(line, "model name") && model == "" {
if i := strings.Index(line, ":"); i >= 0 {
model = strings.TrimSpace(line[i+1:])
}
}
}
return
}
func kernel() string {
return strings.TrimSpace(readProc("/proc/sys/kernel/osrelease"))
}
func partitions() []pb.PartitionReport {
allowed := map[string]bool{"ext4": true, "xfs": true, "btrfs": true, "zfs": true, "vfat": true, "ntfs": true, "ext3": true}
f, err := os.Open("/proc/mounts")
if err != nil {
return nil
}
defer f.Close()
var out []pb.PartitionReport
seen := map[string]bool{}
sc := bufio.NewScanner(f)
for sc.Scan() {
fields := strings.Fields(sc.Text())
if len(fields) < 3 || !allowed[fields[2]] || seen[fields[1]] {
continue
}
seen[fields[1]] = true
var st syscall.Statfs_t
if syscall.Statfs(fields[1], &st) != nil {
continue
}
total := st.Blocks * uint64(st.Bsize)
free := st.Bavail * uint64(st.Bsize)
out = append(out, pb.PartitionReport{
Device: fields[0], Mountpoint: fields[1], Fstype: fields[2],
TotalBytes: total, UsedBytes: total - free,
})
}
return out
}
+12
View File
@@ -0,0 +1,12 @@
//go:build !linux
// Inventory collection is Linux-only. This no-op stands in everywhere else.
//
// The build constraint above is load-bearing: "_other" is not a GOOS suffix, so
// without it this file compiles on Linux too and collides with collect_linux.go.
package inventory
import "github.com/mrhid6/vantage/agent/internal/grpc/pb"
func collect(r *pb.InventoryReport, includeStatic bool) {}
+11
View File
@@ -0,0 +1,11 @@
package inventory
import "github.com/mrhid6/vantage/agent/internal/grpc/pb"
func Collect(includeStatic bool) *pb.InventoryReport {
r := &pb.InventoryReport{IncludeStatic: includeStatic, CPU: &pb.CPUReport{}, Memory: &pb.MemReport{}}
collect(r, includeStatic)
return r
}
+17 -17
View File
@@ -96,16 +96,16 @@ func fingerprint(pubKey string) string {
return "MD5:" + strings.Join(pairs, ":")
}
// KeyGenOptions controls how ssh-keygen is invoked.
type KeyGenOptions struct {
KeyType string // ed25519 (default), rsa, ecdsa
KeySize int // bits; used for rsa and ecdsa
Passphrase string // empty = no passphrase
Comment string // embedded in the public key
KeyType string
KeySize int
Passphrase string
Comment string
}
// GenerateKeyPair generates an SSH keypair and returns the public key.
// The private key is written to keyPath; keyPath+".pub" holds the public key.
func GenerateKeyPair(keyPath string, opts KeyGenOptions) (string, error) {
if err := os.MkdirAll(filepath.Dir(keyPath), 0700); err != nil {
return "", err
@@ -139,8 +139,8 @@ func GenerateKeyPair(keyPath string, opts KeyGenOptions) (string, error) {
return strings.TrimSpace(string(pubData)), nil
}
// AddSSHIdentity writes an IdentityFile entry for keyPath into the managed
// vantage.conf include file, and ensures ~/.ssh/config includes it.
func AddSSHIdentity(keyPath string) error {
if err := os.MkdirAll(filepath.Dir(sshConfigPath), 0700); err != nil {
return fmt.Errorf("mkdir .ssh: %w", err)
@@ -150,7 +150,7 @@ func AddSSHIdentity(keyPath string) error {
return err
}
// Read existing managed config (it may not exist yet).
var existing string
data, err := os.ReadFile(managedConfigPath)
if err != nil && !os.IsNotExist(err) {
@@ -161,7 +161,7 @@ func AddSSHIdentity(keyPath string) error {
line := "IdentityFile " + keyPath
for _, l := range strings.Split(existing, "\n") {
if strings.TrimSpace(l) == line {
return nil // already present
return nil
}
}
@@ -176,7 +176,7 @@ func AddSSHIdentity(keyPath string) error {
return nil
}
// RemoveSSHIdentity removes the IdentityFile entry for keyPath from the managed config.
func RemoveSSHIdentity(keyPath string) error {
data, err := os.ReadFile(managedConfigPath)
if os.IsNotExist(err) {
@@ -204,9 +204,9 @@ func RemoveSSHIdentity(keyPath string) error {
return nil
}
// ensureIncludeDirective adds "Include /root/.ssh/vantage.conf" to the top
// of ~/.ssh/config if it is not already present. The Include must appear before
// any Host stanzas to be effective for all connections.
func ensureIncludeDirective() error {
data, err := os.ReadFile(sshConfigPath)
if err != nil && !os.IsNotExist(err) {
@@ -215,11 +215,11 @@ func ensureIncludeDirective() error {
for _, l := range strings.Split(string(data), "\n") {
if strings.TrimSpace(l) == includeDirective {
return nil // already present
return nil
}
}
// Prepend the Include directive so it takes effect before any Host blocks.
updated := includeDirective + "\n" + string(data)
if err := os.WriteFile(sshConfigPath, []byte(updated), 0600); err != nil {
return fmt.Errorf("write %s: %w", sshConfigPath, err)
+166
View File
@@ -0,0 +1,166 @@
package monitors
import (
"context"
"log"
"sync"
"time"
"github.com/mrhid6/vantage/agent/internal/checker"
"github.com/mrhid6/vantage/agent/internal/config"
grpcclient "github.com/mrhid6/vantage/agent/internal/grpc"
"github.com/mrhid6/vantage/agent/internal/grpc/pb"
)
const syncInterval = 30 * time.Second
type runner struct {
intervalSec int
cancel context.CancelFunc
}
func Run(ctx context.Context, cfg *config.Config) {
active := map[string]*runner{}
var mu sync.Mutex
results := make(chan pb.CheckResult, 64)
go reporter(ctx, cfg, results)
syncOnce := func() {
specs, err := fetchSpecs(cfg)
if err != nil {
log.Printf("monitors: sync: %v", err)
return
}
want := map[string]pb.MonitorSpec{}
for _, s := range specs {
want[s.MonitorId] = s
}
mu.Lock()
defer mu.Unlock()
for id, r := range active {
s, ok := want[id]
if !ok || s.IntervalSec != r.intervalSec {
r.cancel()
delete(active, id)
}
}
for id, s := range want {
if _, ok := active[id]; ok {
continue
}
rctx, cancel := context.WithCancel(ctx)
active[id] = &runner{intervalSec: s.IntervalSec, cancel: cancel}
go runSpec(rctx, s, results)
}
}
syncOnce()
t := time.NewTicker(syncInterval)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
syncOnce()
}
}
}
func fetchSpecs(cfg *config.Config) ([]pb.MonitorSpec, error) {
client, err := grpcclient.New(cfg.ServerURL, cfg.TLS)
if err != nil {
return nil, err
}
defer client.Close()
return client.SyncMonitors(cfg.ServerID, cfg.AgentToken)
}
func runSpec(ctx context.Context, s pb.MonitorSpec, out chan<- pb.CheckResult) {
interval := time.Duration(s.IntervalSec) * time.Second
if interval <= 0 {
interval = 60 * time.Second
}
spec := checker.Spec{
Type: s.Type,
URL: s.URL,
Host: s.Host,
Port: s.Port,
Method: s.Method,
ExpectedStatus: s.ExpectedStatus,
Keyword: s.Keyword,
TLSWarnDays: s.TLSWarnDays,
Insecure: s.Insecure,
TimeoutSec: s.IntervalSec,
}
run := func() {
res := checker.Run(ctx, spec)
cr := pb.CheckResult{MonitorId: s.MonitorId, Up: res.Up, LatencyMs: res.LatencyMs, Message: res.Message}
if res.CertExpiry != nil {
cr.CertExpiryUnix = res.CertExpiry.Unix()
}
select {
case out <- cr:
case <-ctx.Done():
}
}
run()
t := time.NewTicker(interval)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
run()
}
}
}
func reporter(ctx context.Context, cfg *config.Config, in <-chan pb.CheckResult) {
t := time.NewTicker(5 * time.Second)
defer t.Stop()
var batch []pb.CheckResult
flush := func() {
if len(batch) == 0 {
return
}
client, err := grpcclient.New(cfg.ServerURL, cfg.TLS)
if err != nil {
log.Printf("monitors: report dial: %v", err)
batch = nil
return
}
if err := client.ReportChecks(cfg.ServerID, cfg.AgentToken, batch); err != nil {
log.Printf("monitors: report: %v", err)
}
client.Close()
batch = nil
}
for {
select {
case <-ctx.Done():
flush()
return
case r := <-in:
batch = append(batch, r)
if len(batch) >= 32 {
flush()
}
case <-t.C:
flush()
}
}
}
+80 -26
View File
@@ -21,7 +21,9 @@ import (
agentexec "github.com/mrhid6/vantage/agent/internal/exec"
grpcclient "github.com/mrhid6/vantage/agent/internal/grpc"
"github.com/mrhid6/vantage/agent/internal/grpc/pb"
"github.com/mrhid6/vantage/agent/internal/inventory"
"github.com/mrhid6/vantage/agent/internal/keys"
"github.com/mrhid6/vantage/agent/internal/monitors"
"github.com/mrhid6/vantage/agent/internal/updates"
)
@@ -32,7 +34,7 @@ func Run(ctx context.Context, cfg *config.Config, version string) error {
}
defer client.Close()
// Register if we have a pre-reg token
if cfg.PreRegToken != "" {
log.Println("registering with server...")
hostname, _ := os.Hostname()
@@ -59,19 +61,25 @@ func Run(ctx context.Context, cfg *config.Config, version string) error {
}
if cfg.AgentToken == "" {
return fmt.Errorf("no agent token available registration required")
return fmt.Errorf("no agent token available registration required")
}
// Start the command stream alongside the poll loop.
go runCommandStream(ctx, cfg)
// Check for OS updates on startup and then hourly.
go runUpdateCheck(ctx, cfg)
go runInventory(ctx, cfg)
go monitors.Run(ctx, cfg)
ticker := time.NewTicker(cfg.PollInterval)
defer ticker.Stop()
// Run immediately on startup
if err := poll(client, cfg, version); err != nil {
log.Printf("poll error: %v", err)
}
@@ -94,7 +102,6 @@ func poll(client *grpcclient.Client, cfg *config.Config, version string) error {
return fmt.Errorf("SyncKeys: %w", err)
}
// Windows agents register and heartbeat only — no authorized_keys management.
if runtime.GOOS != "linux" {
return nil
}
@@ -116,8 +123,8 @@ func poll(client *grpcclient.Client, cfg *config.Config, version string) error {
return nil
}
// runCommandStream maintains a persistent bidirectional stream with the server
// for instant command delivery. Reconnects with exponential backoff on failure.
func runCommandStream(ctx context.Context, cfg *config.Config) {
backoff := time.Second
const maxBackoff = 2 * time.Minute
@@ -170,9 +177,9 @@ func connectAndHandleStream(ctx context.Context, cfg *config.Config) error {
log.Println("command stream connected")
// grpc streams are not safe for concurrent Send; RunStep results are sent
// from per-command goroutines, so all sends on this stream must go through
// this mutex-protected helper.
var sendMu sync.Mutex
send := func(msg *pb.AgentMessage) error {
sendMu.Lock()
@@ -198,6 +205,9 @@ func connectAndHandleStream(ctx context.Context, cfg *config.Config) error {
if cmd.ApplyUpdates != nil {
go handleApplyUpdates(cfg, cmd)
}
if cmd.CleanupWorkspace != nil {
go handleCleanupWorkspace(cmd)
}
if cmd.RunStep != nil {
go func(rc *pb.RunStepCmd, cid string) {
emit := func(seq uint64, data []byte) {
@@ -209,7 +219,7 @@ func connectAndHandleStream(ctx context.Context, cfg *config.Config) error {
}
res := agentexec.RunStep(rc, emit)
res.CommandId = cid
// Final eof marker so the server closes the log file.
_ = send(&pb.AgentMessage{
ServerId: cfg.ServerID,
AgentToken: cfg.AgentToken,
@@ -269,6 +279,40 @@ func runUpdateCheck(ctx context.Context, cfg *config.Config) {
}
}
func runInventory(ctx context.Context, cfg *config.Config) {
client, err := grpcclient.New(cfg.ServerURL, cfg.TLS)
if err != nil {
log.Printf("inventory dial error: %v", err)
return
}
defer client.Close()
report := func(static bool) {
r := inventory.Collect(static)
r.ServerId = cfg.ServerID
r.AgentToken = cfg.AgentToken
if err := client.ReportInventory(r); err != nil {
log.Printf("report inventory: %v", err)
}
}
report(true)
ticker := time.NewTicker(30 * time.Second)
defer ticker.Stop()
tick := 0
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
tick++
report(tick%30 == 0)
}
}
}
func handleApplyUpdates(cfg *config.Config, cmd *pb.ServerCommand) {
log.Printf("applying OS updates (cmd=%s)…", cmd.CommandId)
if err := updates.ApplyAll(); err != nil {
@@ -277,7 +321,7 @@ func handleApplyUpdates(cfg *config.Config, cmd *pb.ServerCommand) {
}
log.Printf("OS updates applied successfully (cmd=%s)", cmd.CommandId)
// Re-report the (now empty) update list so the server reflects the new state.
client, err := grpcclient.New(cfg.ServerURL, cfg.TLS)
if err != nil {
return
@@ -286,6 +330,16 @@ func handleApplyUpdates(cfg *config.Config, cmd *pb.ServerCommand) {
_ = client.ReportUpdates(cfg.ServerID, cfg.AgentToken, nil)
}
func handleCleanupWorkspace(cmd *pb.ServerCommand) {
id := cmd.CleanupWorkspace.WorkspaceId
dir := agentexec.WorkspacePath(id)
if err := os.RemoveAll(dir); err != nil {
log.Printf("cleanup workspace %s failed (cmd=%s): %v", dir, cmd.CommandId, err)
return
}
log.Printf("removed run workspace %s (cmd=%s)", dir, cmd.CommandId)
}
func handleDeleteKey(cmd *pb.ServerCommand) {
label := cmd.DeleteKey.Label
keyPath := fmt.Sprintf("/root/.ssh/vantage_%s", strings.ReplaceAll(label, " ", "_"))
@@ -309,21 +363,21 @@ func handleUpdateAgent(cmd *pb.ServerCommand) {
}
u := cmd.UpdateAgent
arch := runtime.GOARCH // "amd64" or "arm64"
arch := runtime.GOARCH
tag := "agent%2Fv" + u.Version
binaryURL := fmt.Sprintf("%s/mrhid6/vantage/releases/download/%s/vantage-agent-linux-%s", u.GiteaBaseURL, tag, arch)
checksumURL := fmt.Sprintf("%s/mrhid6/vantage/releases/download/%s/checksums.txt", u.GiteaBaseURL, tag)
log.Printf("updating agent to v%s from %s (cmd=%s)", u.Version, u.GiteaBaseURL, cmd.CommandId)
// Download binary
tmpBin := "/tmp/vantage-agent-update"
if err := downloadFile(binaryURL, tmpBin); err != nil {
log.Printf("update download failed (cmd=%s): %v", cmd.CommandId, err)
return
}
// Download and verify checksum
checksumData, err := httpGetBytes(checksumURL)
if err != nil {
log.Printf("update checksum fetch failed (cmd=%s): %v", cmd.CommandId, err)
@@ -348,11 +402,11 @@ func handleUpdateAgent(cmd *pb.ServerCommand) {
exec.Command("systemctl", "restart", "vantage-agent").Run()
}
// handleUpdateAgentWindows downloads the latest MSI and launches msiexec to
// perform a MajorUpgrade. msiexec is started DETACHED (via "cmd /c start") so
// that when the upgrade stops the VantageAgent service, nssm's process-tree
// kill of this agent does not also kill the installer mid-flight. Config
// (server_id, agent_token) is preserved by setup.ps1 on upgrade.
func handleUpdateAgentWindows(cmd *pb.ServerCommand) {
u := cmd.UpdateAgent
tag := "agent%2Fv" + u.Version
@@ -380,8 +434,8 @@ func handleUpdateAgentWindows(cmd *pb.ServerCommand) {
logPath := filepath.Join(os.TempDir(), "vantage-agent-msi.log")
log.Printf("launching msiexec for upgrade to v%s (cmd=%s)", u.Version, cmd.CommandId)
// "start" detaches msiexec from this process tree so the service stop
// during the upgrade does not terminate the installer.
up := exec.Command("cmd", "/c", "start", "", "/wait", "msiexec", "/i", msiPath, "/qn", "/norestart", "/l*v", logPath)
if err := up.Start(); err != nil {
log.Printf("failed to launch msiexec (cmd=%s): %v", cmd.CommandId, err)
@@ -390,7 +444,7 @@ func handleUpdateAgentWindows(cmd *pb.ServerCommand) {
}
func downloadFile(url, dest string) error {
resp, err := http.Get(url) //nolint:gosec
resp, err := http.Get(url)
if err != nil {
return err
}
@@ -408,7 +462,7 @@ func downloadFile(url, dest string) error {
}
func httpGetBytes(url string) ([]byte, error) {
resp, err := http.Get(url) //nolint:gosec
resp, err := http.Get(url)
if err != nil {
return nil, err
}
@@ -500,7 +554,7 @@ func localIP() string {
return ""
}
// GenerateAndUpload generates an SSH keypair and uploads the public key to the server.
func GenerateAndUpload(cfg *config.Config, label string) error {
client, err := grpcclient.New(cfg.ServerURL, cfg.TLS)
if err != nil {
+10 -11
View File
@@ -27,8 +27,8 @@ func detectPM() string {
return ""
}
// CheckAvailable returns the list of packages with available upgrades.
// Returns nil, nil when no supported package manager is found.
func CheckAvailable() ([]PackageUpdate, error) {
switch detectPM() {
case "apt":
@@ -48,11 +48,11 @@ func CheckAvailable() ([]PackageUpdate, error) {
}
}
// ApplyAll runs a full non-interactive upgrade using the detected package manager.
func ApplyAll() error {
switch detectPM() {
case "apt":
// Refresh lists first, then upgrade.
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
defer cancel()
if err := exec.CommandContext(ctx, "apt-get", "update", "-qq").Run(); err != nil {
@@ -77,8 +77,8 @@ func ApplyAll() error {
func checkApt() ([]PackageUpdate, error) {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
// Best-effort refresh; ignore errors (cached data is fine).
exec.CommandContext(ctx, "apt-get", "update", "-qq").Run() //nolint:errcheck
exec.CommandContext(ctx, "apt-get", "update", "-qq").Run()
out, err := exec.Command("apt", "list", "--upgradable").Output()
if err != nil {
@@ -88,7 +88,7 @@ func checkApt() ([]PackageUpdate, error) {
scanner := bufio.NewScanner(bytes.NewReader(out))
for scanner.Scan() {
line := scanner.Text()
// Format: package/suite version arch [upgradable from: old-ver]
if !strings.Contains(line, "[upgradable from:") {
continue
}
@@ -111,7 +111,6 @@ func checkApt() ([]PackageUpdate, error) {
func checkDnfYum(pm string) ([]PackageUpdate, error) {
cmd := exec.Command(pm, "check-update")
out, err := cmd.Output()
// Exit code 100 means updates are available — not an error.
if exitErr, ok := err.(*exec.ExitError); ok && exitErr.ExitCode() == 100 {
err = nil
}
@@ -133,7 +132,7 @@ func checkDnfYum(pm string) ([]PackageUpdate, error) {
if len(parts) < 2 {
continue
}
// name.arch new-version repo
name := strings.SplitN(parts[0], ".", 2)[0]
updates = append(updates, PackageUpdate{Name: name, NewVersion: parts[1]})
}
@@ -146,7 +145,7 @@ func checkPacman() ([]PackageUpdate, error) {
scanner := bufio.NewScanner(bytes.NewReader(out))
for scanner.Scan() {
parts := strings.Fields(scanner.Text())
// Format: package old-version -> new-version
if len(parts) < 4 {
continue
}
@@ -164,7 +163,7 @@ func checkZypper() ([]PackageUpdate, error) {
scanner := bufio.NewScanner(bytes.NewReader(out))
for scanner.Scan() {
line := scanner.Text()
// Data rows start with "v |" (available) or "i |" (installed but updatable).
if !strings.HasPrefix(line, "v |") && !strings.HasPrefix(line, "i |") {
continue
}
+342 -234
View File
@@ -1,34 +1,38 @@
# Vantage
A self-hosted SSH key management system. A central server (Go + Next.js + MongoDB) manages public key assignments across servers. A lightweight Go agent runs on each managed server, polls the central server via gRPC, and atomically rewrites `/root/.ssh/authorized_keys` to match the desired state.
A self-hosted, multi-tenant infrastructure control plane. It started as SSH key management and has grown into fleet management: SSH key assignment, workflow/script execution, service monitoring, a secrets vault, a browser console (SSH/RDP/VNC), and OS update management.
A central server (Go + Next.js + MongoDB + Redis) drives a lightweight Go agent installed on each managed server. Agents poll over gRPC and also hold a bidirectional command stream for push-style commands.
---
## Architecture Overview
```
┌─────────────────────────────────┐
│ Next.js Frontend
- Upload/manage keys
- Add servers (install script)
│ - Assign/revoke per server │
└────────────┬────────────────────┘
│ REST
┌────────────▼────────────────────┐
Go Backend
- REST API for frontend
- gRPC server for agents
- MongoDB
└────────────────────────────────┘
│ gRPC (TLS)
┌────────────────────────────────┐
│ Go Agent (per server)
- Polls every 30s
- Rewrites authorized_keys │
- Can generate SSH keypairs
└─────────────────────────────────┘
┌──────────────────────────────────────────────
│ Next.js 16 Frontend (web, :3000)
servers · keys · workflows · monitors
secrets · audit · console · settings
└───────────────┬──────────────────────────────┘
│ REST + cookie session
┌───────────────▼──────────────────────────────┐
│ Go Backend (server) │
:8080 REST (gin) :9090 gRPC (agents)
MongoDB (state) · Redis (sessions)
monitor scheduler · workflow runner
guacd tunnel proxy for browser console
└───────────────┬──────────────────────────────┘
│ gRPC (TLS) — outbound from agent only
┌───────────────▼──────────────────────────────┐
│ Go Agent (per server, Linux + Windows)
polls SyncKeys · CommandStream
rewrites authorized_keys (Linux only)
runs workflow steps · monitors · inventory
└──────────────────────────────────────────────
```
Multi-tenancy: every domain document carries `org_id`, and every service query is scoped by it. Org is resolved from the session, and optionally cross-checked against the request host (`<slug>.vantage.<tld>`).
---
## Repository Structure
@@ -36,143 +40,245 @@ A self-hosted SSH key management system. A central server (Go + Next.js + MongoD
```
vantage/
├── agent/
│ ├── cmd/main.go
│ ├── cmd/main.go # flags: -generate-key
│ └── internal/
│ ├── config/
│ ├── grpc/
│ ├── keys/
── sync/
│ ├── checker/ # monitor check execution
│ ├── config/ # config.yaml load/save
│ ├── exec/ # workflow step execution
── grpc/ # client + generated pb
│ ├── inventory/ # CPU/mem/disk collection (linux/other)
│ ├── keys/ # authorized_keys read/diff/write
│ ├── monitors/ # agent-run monitor loop
│ ├── sync/ # poll loop + command stream
│ └── updates/ # OS package update check/apply
├── server/
│ ├── cmd/main.go
│ └── internal/
│ ├── api/ # REST handlers for Next.js
│ ├── grpc/ # gRPC server implementation
│ ├── models/ # MongoDB models
── services/
├── keys.go
├── servers.go
└── sync.go # builds desired state per server
├── web/
├── app/
│ └── components/
├── proto/
── vantage/v1/vantage.proto
├── deploy/
── docker-compose.yml
│ └── agent.service
└── .gitea/
── workflows/
├── agent-release.yml
└── server-deploy.yml
│ ├── api/ # REST handlers
│ ├── auth/ # local, OIDC, session, middleware, orghost
│ ├── checker/ # server-run monitor checks
── db/ # mongo connect + Col()
├── grpc/ # gRPC server + generated pb
├── models/ # MongoDB documents
├── monitorsched/ # server-side monitor scheduler
│ ├── notify/ # smtp, http, templating, dispatch
└── services/ # business logic + migrations
├── web/ # the application UI (authenticated)
│ ├── app/(app)/ # authed routes
── app/login, app/setup # unauthed routes
│ ├── components/ # ui/, workflows/, monitors/, Sidebar
── lib/ # api client, guac console, query client
├── site/ # public marketing site
│ ├── app/ # one directory per route
── components/ # Nav, Footer, Logo, InstrumentPanel, forms
│ ├── assets/ # image sources, not served
│ └── Dockerfile # same shape as web/: standalone, node, 3000
├── sitesvc/ # public forms: contact mail + signup
│ ├── cmd/main.go
│ └── internal/
│ ├── api/ # contact, signup, verify
│ ├── mail/ # SMTP
│ ├── models/ # mirrors server org/user + pending signup
│ ├── provision/ # slug rules mirrored from the control plane
│ └── store/ # mongo: pending signups, org/user creation
├── proto/vantage/v1/vantage.proto
├── installer/ # Windows: setup.ps1, nssm.exe, WiX .wxs
├── deploy/ # docker-compose.yml, agent.service
└── .gitea/workflows/ # agent-release.yml, server-deploy.yml
```
---
## Subsystems
### SSH keys
Upload a public key, assign it per server, revoke softly. The agent diffs desired vs on-disk state and rewrites `/root/.ssh/authorized_keys` atomically. Keys can also be generated _on_ a server by the agent; the private half can optionally be uploaded and is stored AES-256-GCM encrypted.
### Workflows
A library of reusable **steps** (bash or PowerShell scripts with declared inputs, outputs, and secret refs) composed into **workflows** targeting a set of servers. Running one snapshots the resolved steps into a `WorkflowRun`, then dispatches `RunStepCmd` over the agent command stream. Step stdout/stderr streams back as `StepOutputChunk` and is written to a log file on disk; the UI streams it live. Steps support `on_failure: stop|continue|retry`, per-run env passed between steps via `output_env`, and a per-run workspace directory the agent cleans up at the end.
Default steps are seeded per org at boot (`SeedDefaultSteps`). Logs are swept by retention (`workflow_log_retention_days`; nil = 30 days, 0 = forever).
### Monitors
HTTP, TCP, ICMP and TLS checks. Each monitor has a `runner`: `"server"` (executed by the server-side scheduler) or a `server_id` (pushed to that agent, which runs it locally and reports results). Consecutive failures beyond `retries` flip state to `down`, open an `Incident`, and notify. Hourly `Rollup` documents back the uptime graphs.
### Notification channels
Per-org outbound destinations: `webhook`, `smtp`, `discord`, `slack`, `telegram`. Monitors reference channels by ID. Channels are testable from the UI.
### Secrets vault
Key/value pairs grouped by name, encrypted at rest with AES-256-GCM. Consumed two ways: referenced by workflow steps via `secret_refs` (injected as env at execution), and read by Kubernetes External Secrets Operator via `GET /api/secrets/:group/values` using a bearer token whose SHA-256 hash is stored in settings.
### Browser console
`POST /api/console/connect` mints a one-time session token; `GET /api/console/tunnel` upgrades to a WebSocket and proxies to **guacd** (Apache Guacamole daemon) using `github.com/wwt/guac`. SSH connections authenticate with a stored private key; RDP/VNC credentials are encrypted, single-use, and consumed when the tunnel opens.
### Inventory and OS updates
Agents report CPU/memory/swap/partitions/kernel — metrics every 30s, full static snapshot every 15 min. They also check for pending OS package updates hourly and can apply them on command (`ApplyUpdatesCmd`).
### Agent self-update
`UpdateAgentCmd` carries a target version and Gitea base URL; the agent downloads and replaces itself.
### Marketing site and sitesvc
`site/` is a separate Next.js app built exactly like `web/``output: "standalone"`, run by Node in a `node:26-alpine` image, listening on `3000` and published as `3001`. Both of its forms post to `sitesvc`; the control plane is not involved and has no public signup endpoint.
`sitesvc/` (port `8082`) owns both flows end to end:
| Form | Endpoint | Effect |
| ------------------- | ------------------------- | ----------------------------------------------------------------------- |
| Contact | `POST /api/contact` | Emails `support@hostxtra.co.uk`, `Reply-To` the sender. Nothing stored. |
| Create organisation | `POST /api/signup` | Records a pending signup and emails a verification link. |
| Verification link | `GET /api/verify?token=…` | Creates the org and its owner, then redirects to the org's sign-in page (`APP_LOGIN_URL` with `{slug}` filled in). |
All three are deliberately **excluded from the self-hosted deployment**: `deploy/docker-compose.yml` mentions none of them, and they live in `deploy/docker-compose.site.yml` instead.
```bash
# self-hosted install — no marketing site, no sitesvc
docker compose up -d
# vantage.hostxtra.co.uk — control plane plus the public site
docker compose -f docker-compose.yml -f docker-compose.site.yml up -d
```
### Signup and verification
**Nothing is written to `orgs` or `users` until the emailed link is opened.** A signup lands in sitesvc's own `site_pending_signups` collection holding the org name, the address, and the password already bcrypt-hashed at cost 12. The consequence is worth stating: an address nobody controls can never occupy an email, hold an organisation slug, or produce an account that can sign in. It also means the control plane's login path needs no concept of "unverified".
- The token is 32 random bytes; only its **SHA-256 hash** is stored, so a leaked database yields no working links.
- `Verify` deletes the pending record **atomically before provisioning** (`FindOneAndDelete`), so a double-clicked link cannot create two organisations — the second delete matches nothing.
- Links expire after 24 hours, and a **TTL index** lets Mongo drop abandoned signups so password hashes do not linger.
- Re-submitting the form for the same address replaces the previous pending record, so only the newest link works.
- If the owner insert fails after the org is created, the org is rolled back rather than stranded holding a slug. The rollback refuses to touch an org that has users.
- Rate limited to 3 signups per client IP per hour, plus a honeypot field.
### The one piece of duplicated logic
`sitesvc/internal/provision` and `sitesvc/internal/models` mirror the control plane's slug rules, reserved names, bcrypt cost and document shapes. They are duplicated rather than imported because sitesvc is a separate module that deliberately does not depend on the server.
**Nothing enforces the match automatically.** If the control plane's `Slugify`, `reservedSlugs`, `CreateOrg` or `CreateUser` change, update `sitesvc/internal/provision` in the same commit — a divergence would provision tenants under rules the app does not agree with.
sitesvc also (re)declares the unique indexes on `users.email` and `orgs.slug` at boot so it does not depend on the server having started first. Creating an existing index is a no-op.
---
## Auth and Orgs
- **Bootstrap** — first run has no users. `GET /auth/bootstrap-status` drives `/setup`, `POST /auth/bootstrap` creates the first org plus its owner.
- **Local auth** — email + password (bcrypt), `POST /auth/login`.
- **OIDC** — configured _per org_ (`org_oidc`), issuer + client ID + encrypted client secret. `/auth/oidc/start``/auth/oidc/callback`.
- **Sessions** — opaque 32-byte hex ID in the `km_session` cookie, session body stored in Redis with a 24h TTL.
- **Roles** — `owner`, `admin`, `member`. `/api/settings` and `/api/org/*` require owner or admin.
- **Host/org guard** — `APP_ROOT_LABEL` (default `vantage`) defines the app root label. A request to `<slug>.vantage.<tld>` resolves that org from the slug and rejects sessions belonging to a different one. Org lookups are cached for 60s.
Unique indexes on user email and org slug are a **security property**, not an optimisation: `GetUserByEmail` does an unscoped `FindOne`, so duplicates would let the OIDC cross-org guard compare against an arbitrary user. Same for duplicate settings docs and duplicate ESO token hashes.
---
## gRPC API
```protobuf
syntax = "proto3";
package vantage.v1;
service Vantage {
rpc Register(RegisterRequest) returns (RegisterResponse);
rpc SyncKeys(SyncRequest) returns (SyncResponse);
rpc UploadGeneratedKey(UploadKeyRequest) returns (UploadKeyResponse);
}
message RegisterRequest {
string server_id = 1;
string pre_reg_token = 2;
string hostname = 3;
string ip_address = 4;
string os_info = 5;
}
message RegisterResponse {
string agent_token = 1;
}
message SyncRequest {
string server_id = 1;
string agent_token = 2;
}
message SyncResponse {
repeated string public_keys = 1; // full authorized_keys lines
}
message UploadKeyRequest {
string server_id = 1;
string agent_token = 2;
string public_key = 3;
string label = 4;
}
message UploadKeyResponse {
string key_id = 1;
rpc Register(RegisterRequest) returns (RegisterResponse);
rpc SyncKeys(SyncRequest) returns (SyncResponse);
rpc UploadGeneratedKey(UploadKeyRequest) returns (UploadKeyResponse);
rpc ReportUpdates(ReportUpdatesRequest) returns (ReportUpdatesResponse);
rpc ReportInventory(InventoryReport) returns (InventoryReportResponse);
rpc SyncMonitors(SyncMonitorsRequest) returns (SyncMonitorsResponse);
rpc ReportChecks(ReportChecksRequest) returns (ReportChecksResponse);
rpc CommandStream(stream AgentMessage) returns (stream ServerCommand);
}
```
No streaming — polling only. Poll interval: **30 seconds**.
`CommandStream` is the only streaming RPC: the agent authenticates once with `AgentReady`, then the server pushes `ServerCommand`s and the agent replies with `CommandResult`, `StepResult`, or `StepOutputChunk`.
`ServerCommand` variants: `GenerateKeyCmd`, `DeleteKeyCmd`, `UpdateAgentCmd`, `ApplyUpdatesCmd`, `RunStepCmd`, `CleanupWorkspaceCmd`.
Key-state polling stays on the 30s `SyncKeys` interval. Full message definitions live in `proto/vantage/v1/vantage.proto`.
---
## REST API
Unauthenticated:
```
GET /install /install.ps1 # dynamic agent install scripts
GET /update /update.ps1
GET /auth/bootstrap-status
POST /auth/bootstrap /auth/login /auth/logout
GET /auth/me /auth/oidc/start /auth/oidc/callback
GET /api/secrets/:group/values # bearer token (ESO)
```
Session-authed under `/api`:
```
servers GET,POST /servers · GET,POST /servers/new · GET,DELETE /servers/:id
POST /servers/:id/{generate-key,update-agent,apply-updates}
keys GET,POST /keys · GET,DELETE /keys/:id · GET /keys/:id/private-key
POST /keys/:id/assign · DELETE /keys/:id/assign/:serverId
workflows GET,POST /steps · PUT,DELETE /steps/:id · GET /steps/:id/export
POST /steps/{import,seed-defaults,parse} · GET /steps/usage
GET,POST /workflows · GET,PUT,DELETE /workflows/:id
POST /workflows/:id/run · GET /workflows/:id/runs
GET /runs/:runId · POST /runs/:runId/cancel
GET /runs/:runId/servers/:serverId/logs[/stream]
monitors GET,POST /monitors · GET,PUT,DELETE /monitors/:id
GET /monitors/:id/{incidents,uptime}
channels GET,POST /channels · PUT,DELETE /channels/:id · POST /channels/:id/test
secrets GET,POST /secrets · GET,PUT,DELETE /secrets/:group
POST /secrets/:group/reveal · DELETE /secrets/:group/:key
console POST /console/connect · GET /console/tunnel (websocket)
audit GET /audit
agent GET /agent/latest-version
settings GET,PUT /settings · POST /settings/secrets-token (owner|admin)
org GET,POST /org/users · PUT /org/users/:id/role · DELETE /org/users/:id
GET,PUT /org/oidc (owner|admin)
```
---
## MongoDB Collections
### `servers`
`servers` · `keys` · `assignments` · `orgs` · `users` · `org_oidc` · `settings` · `secrets` · `workflows` · `workflow_steps` · `workflow_runs` · `monitors` · `incidents` · `monitor_rollups` · `notification_channels` · `console_sessions` · `audit_logs` · `migrations`
```json
{
"_id": "ObjectId",
"server_id": "uuid",
"hostname": "proxmox-node-1",
"ip_address": "10.10.10.5",
"os_info": "Ubuntu 24.04",
"pre_reg_token": "abc123",
"pre_reg_expires": "ISODate",
"agent_token_hash": "sha256...",
"status": "pending|active|offline",
"last_seen": "ISODate",
"created_at": "ISODate"
}
```
Every document except `migrations` carries `org_id`. Struct definitions are the source of truth — see `server/internal/models/`.
- `pre_reg_token` is cleared after the agent successfully calls `Register()`
- `agent_token_hash` stores SHA-256 of the token — never plaintext
- `status` transitions: `pending``active` on first `Register()`, `offline` if last_seen exceeds threshold
`site_pending_signups` is written only by sitesvc and holds unverified signups; the control plane neither reads nor knows about it.
### `keys`
Notes that are not obvious from the structs:
```json
{
"_id": "ObjectId",
"key_id": "uuid",
"label": "dom-macbook",
"public_key": "ssh-ed25519 AAAA...",
"fingerprint": "SHA256:...",
"source": "uploaded|generated",
"generated_by_server_id": "uuid",
"created_at": "ISODate"
}
```
- `servers.agent_token_hash` stores SHA-256 of the token, never plaintext. `pre_reg_token` is cleared after `Register()`. `status` is `pending``active` on register, `offline` when `last_seen` passes the threshold (swept every 2 min).
- `servers.inventory` holds the latest metrics snapshot with separate `metrics_at` / `static_at` timestamps.
- `keys.private_key_enc` and `passphrase_enc` are AES-256-GCM; the JSON form exposes only `has_private_key` / `has_passphrase`.
- `assignments.revoked_at: null` means active. Revocation is soft, preserving audit history.
- `workflow_runs.steps_snapshot` freezes the resolved steps so editing the library never rewrites history.
- `console_sessions.token_consumed_at` is set atomically to enforce one-time use.
### `assignments`
### Migrations
```json
{
"_id": "ObjectId",
"key_id": "uuid",
"server_id": "uuid",
"assigned_at": "ISODate",
"revoked_at": "ISODate | null"
}
```
`services.RunMigrations()` runs at boot, recording markers in `migrations`:
- `revoked_at: null` = key is active on that server
- Revocation is soft — set `revoked_at`, agent picks it up on next poll
- `0001_default_org_backfill`
- `0002_settings_org_backfill` (must run before 0003 — 0003 can create a `default` org, which pushes 0002 into its ambiguous multi-org branch)
- `0003_missed_org_scopes`
Index builders (`EnsureAuthIndexes`, `EnsureSettingsIndexes`) are fatal on failure; `EnsureSecretIndexes` and `EnsureWorkflowIndexes` only warn.
---
## Agent Lifecycle
### Config file — `/etc/vantage/config.yaml`
### Config file
Linux `/etc/vantage/config.yaml`, Windows `%ProgramData%\vantage\config.yaml`. Directory `0700`, file `0600`.
```yaml
server_url: "vantage.yourdomain.com:9090"
@@ -183,117 +289,119 @@ poll_interval: 30s
tls: true
```
Config file permissions: `0600`. Config directory: `0700`.
### Startup flow
### Startup
```
1. Load config
2. If pre_reg_token present:
→ call Register(server_id, pre_reg_token, hostname, ip, os_info)
→ save returned agent_token to config
→ delete pre_reg_token from config
3. Enter poll loop
2. If pre_reg_token present → Register() → save agent_token, clear pre_reg_token, reconnect
3. Start goroutines: command stream · update check (hourly) · inventory · monitors
4. Enter SyncKeys poll loop (default 30s)
```
### Poll loop (every 30s)
### Poll loop
```
1. Call SyncKeys(server_id, agent_token)
2. Receive []public_keys
3. Compute fingerprints of current /root/.ssh/authorized_keys
4. If state unchanged → skip write
5. If changed:
→ write to /root/.ssh/authorized_keys.tmp
→ os.Rename() to /root/.ssh/authorized_keys (atomic)
→ chmod 0600
1. SyncKeys(server_id, agent_token, agent_version)
2. Non-Linux hosts stop here — Windows agents register and heartbeat only
3. Diff desired keys against /root/.ssh/authorized_keys; unchanged → no write
4. Changed → write .tmp, os.Rename() over the real file, chmod 0600
```
### Key generation (on demand)
### Install
- Triggered by a flag or API call from the server
- Runs `ssh-keygen` via `exec.Command`
- Uploads public key via `UploadGeneratedKey()`
- Private key stays local on the machine
### Systemd unit — `/etc/systemd/system/vantage-agent.service`
```ini
[Unit]
Description=Vantage Agent
After=network.target
[Service]
ExecStart=/usr/local/bin/vantage-agent
Restart=always
RestartSec=10
User=root
[Install]
WantedBy=multi-user.target
```
Linux: systemd unit at `/etc/systemd/system/vantage-agent.service`, `Restart=always`, runs as root.
Windows: MSI built by CI (WiX), or `installer/setup.ps1` registering the agent as a service via NSSM.
---
## Server Registration Flow
1. Click **Add Server** in the UI
2. Backend generates a short-lived pre-registration token (TTL: 1 hour) and a `server_id`
3. UI displays a one-liner install command with copy button:
1. **Add Server** in the UI calls `POST /api/servers/new`, which generates a `server_id` and a pre-registration token (TTL 1 hour, single-use).
2. The UI shows a one-liner:
```bash
curl -fsSL https://vantage.yourdomain.com/install | \
bash -s -- --server-id=<id> --token=<token>
```
4. Install script:
- Detects arch (`amd64` / `arm64`)
- Downloads agent binary from Gitea release
- Verifies SHA-256 checksum
- Writes `/etc/vantage/config.yaml`
- Installs and starts systemd unit
5. On first `SyncKeys` call, server marks status as `active`
Windows gets the `/install.ps1` equivalent.
3. The script detects arch, downloads the agent from the Gitea release, verifies the SHA-256 checksum, writes the config, installs and starts the service.
4. The server flips to `active` on first sync.
The backend serves `/install` dynamically, injecting the latest agent version by querying the Gitea API for the most recent `agent/v*` release tag.
`/install` is served dynamically, injecting the latest agent version from the Gitea API.
---
## Environment Variables (server)
| Name | Required | Notes |
| -------------------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GRPC_HOST` | **yes** | `host:port` agents dial. Boot fails without it — there is no safe default; falling back to the web host would hand agents a port that does not speak gRPC. |
| `MONGO_URI` | no | default `mongodb://localhost:27017` |
| `MONGO_DB` | no | default `vantage` |
| `REDIS_ADDR` | no | default `localhost:6379` |
| `KEY_ENCRYPTION_KEY` | yes in practice | 64-char hex (32 bytes) for AES-256-GCM. Required for private keys, secrets, OIDC secrets, RDP credentials. |
| `GITEA_HOST` | yes | used to build install scripts and agent download URLs |
| `GUACD_ADDR` | no | default `guacd:4822` |
| `APP_ROOT_LABEL` | no | default `vantage`; wrong value disables the host/session org guard |
| `VANTAGE_WORKFLOW_LOG_DIR` | no | where run logs are written |
**sitesvc** (`deploy/docker-compose.site.yml` only):
| Name | Required | Notes |
| --------------------------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `MONGO_URI` | yes | **must point at the control plane's database**, or the app will not see organisations created here. The database name is read from the URI path (`mongodb://user:pass@host:27017/vantage?authSource=vantage`); a URI without one is refused at boot rather than defaulted. Note this differs from the server, which takes `MONGO_DB` separately. |
| `PUBLIC_URL` | yes | sitesvc's own public base URL; verification links are built from it |
| `APP_LOGIN_URL` | no | template for the org sign-in URL a verified owner is redirected to. `{slug}` is replaced with the new org's slug (each org has its own subdomain), e.g. `https://{slug}.vantage.hostxtra.co.uk/login`. A value without `{slug}` is used verbatim; empty means a plain confirmation page. |
| `SMTP_HOST` / `SMTP_FROM` | yes | without them both forms refuse (503) rather than silently dropping |
| `SMTP_TO` | no | default `support@hostxtra.co.uk`; contact enquiries only |
| `SMTP_PORT` | no | default `587`; `465` uses implicit TLS |
| `SMTP_USERNAME` / `SMTP_PASSWORD` | no | auth skipped when username is empty |
| `SITE_ORIGIN` | yes in practice | comma-separated allowed origins; unset refuses every cross-origin browser request |
| `TRUST_PROXY` | no | only `true` behind a proxy that overwrites `X-Forwarded-For`, or clients spoof past the rate limiter |
`deploy/docker-compose.yml` runs four services: `redis`, `guacd`, `server` (8080 + 9090), `web` (3000). MongoDB is external. `deploy/docker-compose.site.yml` adds the public marketing site on `3001` and is only used on vantage.hostxtra.co.uk.
---
## Security
- gRPC over TLS (Let's Encrypt or self-signed with cert pinning on the agent)
- Agent authenticates with a per-server token stored at `/etc/vantage/config.yaml` (`0600`)
- Server stores `SHA-256(agent_token)` — never the plaintext token
- Private keys generated by agents are encrypted at rest in MongoDB (AES-256)
- `authorized_keys` written as `0600`, owned by root
- Pre-registration tokens are short-lived (1 hour) and single-use
- Agent runs as `root` (required for `/root/.ssh/authorized_keys` writes)
- gRPC over TLS; agents connect outbound only, no inbound firewall holes on managed servers.
- Per-server agent token stored as SHA-256 on the server, plaintext only in the agent's `0600` config.
- Pre-registration tokens are short-lived (1 hour) and single-use.
- AES-256-GCM at rest for private keys, key passphrases, vault secrets, OIDC client secrets, RDP/VNC credentials.
- Console session tokens are one-time; RDP credentials are consumed on tunnel open.
- ESO read token stored as a SHA-256 hash and rotatable.
- Unique indexes on user email, org slug, settings org, and ESO token hash are load-bearing for tenant isolation.
- `authorized_keys` written `0600`, owned by root. The agent runs as root because it must.
- Every mutating API path writes an audit event.
---
## Frontend Routes
## Frontend
| Route | Purpose |
| --------------- | -------------------------------------------------------------------- |
| `/servers` | List all servers, online/offline status badge, last seen timestamp |
| `/servers/new` | Displays the one-liner install script with copy button |
| `/servers/[id]` | Keys installed on this server, trigger key generation, remove server |
| `/keys` | All keys — label, fingerprint, source, assigned count |
| `/keys/[id]` | Assign key to servers, revoke per server |
Next.js 16 (App Router) + React 18, Tailwind 3, TanStack Query. Guacamole client bundled locally in `web/lib/guacamole-common.js`.
| Route | Purpose |
| --------------------------------------------------------------- | ----------------------------------------------------------------------- |
| `/setup` | First-run bootstrap: create the first org and owner |
| `/login` | Local or OIDC sign-in |
| `/` | Fleet dashboard |
| `/servers`, `/servers/new`, `/servers/[id]` | Fleet list, install one-liner, server detail (keys, inventory, updates) |
| `/servers/[id]/console` | Browser SSH/RDP/VNC session |
| `/keys`, `/keys/[id]` | Key library; assign and revoke per server |
| `/workflows`, `/workflows/[id]`, `/workflows/[id]/runs[/runId]` | Compose, run, and follow live logs |
| `/steps` | Reusable step library |
| `/monitors`, `/monitors/new`, `/monitors/[id][/edit]` | Checks, uptime, incidents |
| `/secrets`, `/secrets/[group]` | Vault |
| `/audit` | Audit log |
| `/settings`, `/settings/org`, `/settings/notifications` | Alerts, members, OIDC, channels |
---
## CI/CD — Gitea Actions
### Agent release — `.gitea/workflows/agent-release.yml`
### `agent-release.yml` — triggered by `agent/v*` tags
Triggered by a `agent/v*` tag. Cross-compiles for `linux/amd64` and `linux/arm64`, creates a Gitea release with binaries and checksums.
```yaml
on:
push:
tags:
- "agent/v*"
```
Build command:
Builds `linux/amd64`, `linux/arm64`, `windows/amd64`, writes `checksums.txt`, creates a Gitea release. A second `msi` job on `windows-2022` packages the WiX installer.
```bash
GOOS=linux GOARCH=amd64 go build \
@@ -301,51 +409,51 @@ GOOS=linux GOARCH=amd64 go build \
-o dist/vantage-agent-linux-amd64 ./cmd
```
Release assets:
### `server-deploy.yml` — triggered on every push to `main`
- `vantage-agent-linux-amd64`
- `vantage-agent-linux-arm64`
- `checksums.txt`
Builds and pushes four images to the Gitea container registry: `server`, `web`, `site` and `sitesvc`.
### Server deploy — `.gitea/workflows/server-deploy.yml`
Triggered on pushes to `main` touching `server/**`, `web/**`, or `proto/**`. Builds and pushes Docker images to the Gitea container registry, then deploys via SSH:
Note that despite the name, **this workflow does not deploy**it only builds and pushes. There is no SSH step and no path filter; every push to `main` rebuilds all three images. Rolling them out is a separate manual step on the host:
```bash
cd /opt/vantage && docker compose pull && docker compose up -d --remove-orphans
cd /opt/vantage && docker compose -f docker-compose.yml -f docker-compose.site.yml pull && \
docker compose -f docker-compose.yml -f docker-compose.site.yml up -d --remove-orphans
```
### Tagging convention
### Tagging
```bash
# Release a new agent version
git tag agent/v1.0.0 && git push origin agent/v1.0.0
# Server + web deploy automatically on push to main
git push origin main
git tag agent/v1.0.0 && git push origin agent/v1.0.0 # agent release
git push origin main # server + web deploy
```
### Required Gitea secrets / variables
### Secrets / variables
| Name | Type | Value |
| ------------------- | -------- | ------------------------------------------ |
| `RELEASE_TOKEN` | Secret | Gitea API token with `write:release` scope |
| `REGISTRY_USER` | Secret | Gitea username |
| `REGISTRY_PASSWORD` | Secret | Gitea token with `write:packages` scope |
| `DEPLOY_HOST` | Secret | IP/hostname of the server VM |
| `DEPLOY_USER` | Secret | SSH user for deploy |
| `DEPLOY_SSH_KEY` | Secret | Private key for deploy SSH |
| `GITEA_HOST` | Variable | `gitea.hostxtra.co.uk` |
| Name | Type | Value |
| -------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `RELEASE_TOKEN` | Secret | Gitea API token, `write:release` |
| `REGISTRY_USER` | Secret | Gitea username |
| `REGISTRY_PASSWORD` | Secret | Gitea token, `write:packages` |
| `GITEA_HOST` | Variable | `gitea.hostxtra.co.uk` |
| `DOCKER_HOST` | Variable | registry host used for image tags |
| `API_URL` | Variable | baked into the `web` image at build time |
| `SITE_API_URL` | Variable | **browser-reachable** sitesvc URL, baked into the `site` image. Required — if empty, both forms report "not connected" and submit nowhere. Must also be in sitesvc's `SITE_ORIGIN`. |
| `SITE_CONTACT_EMAIL` | Variable | optional; address shown when a form is misconfigured |
---
## Design Decisions
- **gRPC over REST for agent communication** — strong typing, easy versioning, bi-directional streaming available later if push-based updates are needed
- **Poll-only, no streaming** — 30s interval is sufficient for a homelab; simplifies agent implementation
- **Outbound-only agent connections** — no inbound firewall holes required on managed servers
- **Atomic `authorized_keys` rewrite** — write to `.tmp` then `os.Rename()` prevents partial writes
- **Fingerprint diffing before write** — avoids unnecessary disk writes on unchanged state
- **Soft revocation** — `revoked_at` timestamp rather than hard deletes; preserves audit history
- **root only** — manages `/root/.ssh/authorized_keys` only; no per-user key management
- **Gitea releases for agent binaries** — slots into existing act_runner CI pipeline; install script queries Gitea API for latest version at serve time
- **gRPC for agent traffic** — strong typing and cheap versioning; polling for state, one bidirectional stream for commands.
- **Outbound-only agents** — no inbound ports on managed servers, works behind NAT.
- **Poll for keys, push for commands** — a 30s key poll is fine, but running a workflow step should not wait up to 30s.
- **Atomic `authorized_keys` rewrite** — temp file plus `os.Rename()`; a machine that dies mid-write keeps the old file.
- **Fingerprint diffing before write** — no disk churn on unchanged state.
- **Soft revocation** — `revoked_at` rather than deletes; preserves audit history.
- **Run snapshots** — workflow runs freeze their resolved steps so editing a step never rewrites past runs.
- **Monitors run in two places** — server-side for external endpoints, agent-side for anything only reachable from inside the target network.
- **Redis for sessions only** — all durable state stays in MongoDB; losing Redis logs everyone out and nothing else.
- **guacd for console** — protocol handling is Guacamole's problem, not ours; we proxy the WebSocket and manage credentials.
- **`org_id` on every document** — isolation enforced at the query layer, not by separate databases.
- **root only** — manages `/root/.ssh/authorized_keys`; no per-user key management.
- **Windows agents are second-class by design** — register, heartbeat, run steps, report inventory; no `authorized_keys` management.
+27
View File
@@ -0,0 +1,27 @@
services:
site:
image: gitea.hostxtra.co.uk/mrhid6/vantage/site:latest
restart: unless-stopped
ports:
- 3001:3000
depends_on:
- sitesvc
sitesvc:
image: gitea.hostxtra.co.uk/mrhid6/vantage/sitesvc:latest
restart: unless-stopped
ports:
- 8082:8082
environment:
PORT: "8082"
MONGO_URI: ${MONGO_URI:-}
PUBLIC_URL: ${SITE_PUBLIC_URL:-}
APP_LOGIN_URL: ${SITE_APP_LOGIN_URL:-}
SITE_ORIGIN: ${SITE_ORIGIN:-}
TRUST_PROXY: ${SITE_TRUST_PROXY:-false}
SMTP_HOST: ${SITE_SMTP_HOST:-}
SMTP_PORT: ${SITE_SMTP_PORT:-587}
SMTP_USERNAME: ${SITE_SMTP_USERNAME:-}
SMTP_PASSWORD: ${SITE_SMTP_PASSWORD:-}
SMTP_FROM: ${SITE_SMTP_FROM:-}
SMTP_TO: ${SITE_SMTP_TO:-support@hostxtra.co.uk}
+3 -5
View File
@@ -27,19 +27,17 @@ services:
MONGO_URI: ${MONGO_URI:-}
REDIS_ADDR: redis:6379
GITEA_HOST: ${GITEA_HOST}
PUBLIC_HOST: ${PUBLIC_HOST}
GRPC_HOST: ${GRPC_HOST}
GRPC_PORT: "9090"
HTTP_PORT: "8080"
OIDC_ISSUER: ${OIDC_ISSUER:-}
OIDC_CLIENT_ID: ${OIDC_CLIENT_ID:-}
OIDC_CLIENT_SECRET: ${OIDC_CLIENT_SECRET:-}
OIDC_REDIRECT_URL: ${OIDC_REDIRECT_URL:-}
KEY_ENCRYPTION_KEY: ${KEY_ENCRYPTION_KEY:-}
VANTAGE_WORKFLOW_LOG_DIR: ${VANTAGE_WORKFLOW_LOG_DIR:-}
GUACD_ADDR: guacd:4822
depends_on:
redis:
condition: service_healthy
volumes:
- ./data:/data
web:
image: gitea.hostxtra.co.uk/mrhid6/vantage/web:latest
restart: unless-stopped
File diff suppressed because it is too large Load Diff
@@ -1,653 +0,0 @@
# Fleet Inventory Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Agents collect CPU/RAM/swap/disk/partition inventory and report it to the server via a new `ReportInventory` RPC; the server stores the latest snapshot per server and the UI displays it.
**Architecture:** New unary gRPC `ReportInventory` (mirrors existing `ReportUpdates`). Agent runs a 30s metrics ticker (CPU/RAM/swap usage) and, every 15 min, a full static collection (disks, partitions, CPU model, kernel). Server upserts an embedded `inventory` sub-doc on the `servers` document with merge rules that preserve static fields between slow ticks.
**Tech Stack:** Go (gin, mongo-driver v2, hand-written JSON-codec gRPC), `/proc` readers, Next.js 16 + react-query + Tailwind.
## Global Constraints
- **No tests this iteration.** Verify with `go build ./...`, `go vet ./...`, `npm run build`.
- gRPC uses a JSON codec: edit **both** `server/internal/grpc/pb/vantage.pb.go` and `agent/internal/grpc/pb/vantage.pb.go` identically, plus `proto/vantage/v1/vantage.proto` as documentation. No codegen. Mirror the existing `ReportUpdates` RPC wiring exactly (service interface, `_Vantage_*_Handler`, client method, `Vantage_ServiceDesc`).
- Mongo: `db.Col("servers")`, `context.WithTimeout`. Follow `server/internal/services/servers.go`.
- Agent already runs as root; `/proc` is readable. Linux is primary; Windows collectors may return empty.
- Module path `github.com/mrhid6/vantage`.
- Do not add heavy dependencies; implement `/proc` parsing directly.
---
## Task 1: Inventory model + gRPC messages
**Files:**
- Modify: `server/internal/models/server.go`
- Modify: `proto/vantage/v1/vantage.proto`
- Modify: `server/internal/grpc/pb/vantage.pb.go`
- Modify: `agent/internal/grpc/pb/vantage.pb.go`
**Interfaces:**
- Produces: `models.Inventory` (+ `CPUInfo`, `MemInfo`, `Partition`) and `Server.Inventory *Inventory`. pb structs `InventoryReport`, `CPUReport`, `MemReport`, `PartitionReport`, `InventoryReportResponse`. Service method `ReportInventory` on both client and server interfaces.
- [ ] **Step 1: Add model structs**
In `server/internal/models/server.go` add (keep the existing `import "time"`):
```go
type CPUInfo struct {
Model string `bson:"model,omitempty" json:"model,omitempty"`
Cores int `bson:"cores,omitempty" json:"cores,omitempty"`
UsagePct float64 `bson:"usage_pct" json:"usage_pct"`
Load1 float64 `bson:"load1,omitempty" json:"load1,omitempty"`
}
type MemInfo struct {
TotalBytes uint64 `bson:"total_bytes" json:"total_bytes"`
UsedBytes uint64 `bson:"used_bytes" json:"used_bytes"`
}
type Partition struct {
Device string `bson:"device" json:"device"`
Mountpoint string `bson:"mountpoint" json:"mountpoint"`
Fstype string `bson:"fstype,omitempty" json:"fstype,omitempty"`
TotalBytes uint64 `bson:"total_bytes" json:"total_bytes"`
UsedBytes uint64 `bson:"used_bytes" json:"used_bytes"`
}
type Inventory struct {
CPU CPUInfo `bson:"cpu" json:"cpu"`
Memory MemInfo `bson:"memory" json:"memory"`
SwapTotalBytes uint64 `bson:"swap_total_bytes" json:"swap_total_bytes"`
SwapUsedBytes uint64 `bson:"swap_used_bytes" json:"swap_used_bytes"`
Partitions []Partition `bson:"partitions,omitempty" json:"partitions,omitempty"`
Kernel string `bson:"kernel,omitempty" json:"kernel,omitempty"`
MetricsAt *time.Time `bson:"metrics_at,omitempty" json:"metrics_at,omitempty"`
StaticAt *time.Time `bson:"static_at,omitempty" json:"static_at,omitempty"`
}
```
Add to the `Server` struct: `Inventory *Inventory \`bson:"inventory,omitempty" json:"inventory,omitempty"\``.
- [ ] **Step 2: Document RPC in proto**
In `proto/vantage/v1/vantage.proto`, add to the service: `rpc ReportInventory(InventoryReport) returns (InventoryReportResponse);` and the messages `InventoryReport`, `CPUReport`, `MemReport`, `PartitionReport`, `InventoryReportResponse` per spec §4.
- [ ] **Step 3: Add pb structs + RPC wiring (server pb)**
In `server/internal/grpc/pb/vantage.pb.go` add the message structs:
```go
type CPUReport struct {
Model string `json:"model,omitempty"`
Cores int `json:"cores,omitempty"`
UsagePct float64 `json:"usage_pct"`
Load1 float64 `json:"load1,omitempty"`
}
type MemReport struct {
TotalBytes uint64 `json:"total_bytes"`
UsedBytes uint64 `json:"used_bytes"`
}
type PartitionReport struct {
Device string `json:"device"`
Mountpoint string `json:"mountpoint"`
Fstype string `json:"fstype,omitempty"`
TotalBytes uint64 `json:"total_bytes"`
UsedBytes uint64 `json:"used_bytes"`
}
type InventoryReport struct {
ServerId string `json:"server_id"`
AgentToken string `json:"agent_token"`
IncludeStatic bool `json:"include_static"`
CPU *CPUReport `json:"cpu,omitempty"`
Memory *MemReport `json:"memory,omitempty"`
SwapTotal uint64 `json:"swap_total"`
SwapUsed uint64 `json:"swap_used"`
Partitions []PartitionReport `json:"partitions,omitempty"`
Kernel string `json:"kernel,omitempty"`
}
type InventoryReportResponse struct{}
```
Then mirror the `ReportUpdates` RPC plumbing for `ReportInventory`. Locate every `ReportUpdates` reference in this file and add the parallel `ReportInventory`:
- `VantageServer` interface: add `ReportInventory(context.Context, *InventoryReport) (*InventoryReportResponse, error)`.
- `UnimplementedVantageServer`: add the stub returning `Unimplemented`.
- `VantageClient` interface + `keyManagerClient`: add the client method `Invoke`-ing `/vantage.v1.Vantage/ReportInventory`.
- `Vantage_ServiceDesc.Methods`: add `{MethodName: "ReportInventory", Handler: _Vantage_ReportInventory_Handler}`.
- Add `_Vantage_ReportInventory_Handler` copied from `_Vantage_ReportUpdates_Handler` with types swapped.
- [ ] **Step 4: Mirror pb structs + wiring (agent pb)**
Apply the identical additions to `agent/internal/grpc/pb/vantage.pb.go`.
- [ ] **Step 5: Verify build**
Run: `cd server && go build ./... && cd ../agent && go build ./...`
Expected: both succeed.
- [ ] **Step 6: Commit**
```bash
git add server/internal/models/server.go proto/vantage/v1/vantage.proto server/internal/grpc/pb/vantage.pb.go agent/internal/grpc/pb/vantage.pb.go
git commit -m "feat(proto): add ReportInventory RPC and inventory model"
```
---
## Task 2: Server handler + store service
**Files:**
- Create: `server/internal/services/inventory.go`
- Modify: `server/internal/grpc/server.go`
**Interfaces:**
- Consumes: `pb.InventoryReport` (T1), `db.Col("servers")`.
- Produces: `services.StoreInventory(serverID string, r *pb.InventoryReport) error`; gRPC method `(*vantageServer).ReportInventory`.
- [ ] **Step 1: Write the store service**
```go
package services
import (
"context"
"time"
"github.com/mrhid6/vantage/server/internal/db"
"github.com/mrhid6/vantage/server/internal/grpc/pb"
"go.mongodb.org/mongo-driver/v2/bson"
)
// StoreInventory upserts the latest inventory snapshot onto the server document.
// Metrics fields update every call; static fields only when r.IncludeStatic.
func StoreInventory(serverID string, r *pb.InventoryReport) error {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
now := time.Now()
set := bson.M{"inventory.metrics_at": now}
if r.CPU != nil {
set["inventory.cpu.usage_pct"] = r.CPU.UsagePct
set["inventory.cpu.load1"] = r.CPU.Load1
}
if r.Memory != nil {
set["inventory.memory.used_bytes"] = r.Memory.UsedBytes
}
set["inventory.swap_used_bytes"] = r.SwapUsed
if r.IncludeStatic {
set["inventory.static_at"] = now
set["inventory.swap_total_bytes"] = r.SwapTotal
set["inventory.kernel"] = r.Kernel
if r.CPU != nil {
set["inventory.cpu.model"] = r.CPU.Model
set["inventory.cpu.cores"] = r.CPU.Cores
}
if r.Memory != nil {
set["inventory.memory.total_bytes"] = r.Memory.TotalBytes
}
parts := make([]bson.M, 0, len(r.Partitions))
for _, p := range r.Partitions {
parts = append(parts, bson.M{
"device": p.Device, "mountpoint": p.Mountpoint, "fstype": p.Fstype,
"total_bytes": p.TotalBytes, "used_bytes": p.UsedBytes,
})
}
set["inventory.partitions"] = parts
}
_, err := db.Col("servers").UpdateOne(ctx, bson.M{"server_id": serverID}, bson.M{"$set": set})
return err
}
```
- [ ] **Step 2: Add the gRPC handler**
In `server/internal/grpc/server.go`, add (mirroring the existing `ReportUpdates` handler that validates the agent token):
```go
func (s *vantageServer) ReportInventory(ctx context.Context, req *pb.InventoryReport) (*pb.InventoryReportResponse, error) {
srv, err := services.ValidateAgentToken(req.ServerId, req.AgentToken)
if err != nil {
return nil, status.Errorf(codes.Unauthenticated, "invalid agent token")
}
if err := services.StoreInventory(srv.ServerID, req); err != nil {
log.Printf("store inventory for %s: %v", srv.ServerID, err)
}
return &pb.InventoryReportResponse{}, nil
}
```
Confirm `status`, `codes`, `log` are already imported in the file (they are, used by other handlers).
- [ ] **Step 3: Verify build**
Run: `cd server && go build ./... && go vet ./...`
Expected: success.
- [ ] **Step 4: Commit**
```bash
git add server/internal/services/inventory.go server/internal/grpc/server.go
git commit -m "feat(server): store inventory and handle ReportInventory RPC"
```
---
## Task 3: Agent collectors
**Files:**
- Create: `agent/internal/inventory/collect_linux.go`
- Create: `agent/internal/inventory/collect_other.go`
- Create: `agent/internal/inventory/inventory.go`
**Interfaces:**
- Produces: `inventory.Collect(includeStatic bool) *pb.InventoryReport`.
- [ ] **Step 1: Common entry (`inventory.go`)**
```go
package inventory
import "github.com/mrhid6/vantage/agent/internal/grpc/pb"
// Collect gathers metrics always and static hardware info when includeStatic.
// Platform specifics are provided by collect_linux.go / collect_other.go.
func Collect(includeStatic bool) *pb.InventoryReport {
r := &pb.InventoryReport{IncludeStatic: includeStatic, CPU: &pb.CPUReport{}, Memory: &pb.MemReport{}}
collect(r, includeStatic)
return r
}
```
- [ ] **Step 2: Linux collector (`collect_linux.go`)**
Build-tagged `//go:build linux`. Implement `collect(r *pb.InventoryReport, includeStatic bool)`:
- CPU usage: read `/proc/stat` first line twice ~100ms apart, compute `1 - idleDelta/totalDelta` × 100 → `r.CPU.UsagePct`.
- Load: first field of `/proc/loadavg` → `r.CPU.Load1`.
- Mem/swap: parse `/proc/meminfo` (`MemTotal`, `MemAvailable`, `SwapTotal`, `SwapFree`; used = total available; swap used = swaptotal swapfree) → `r.Memory.*`, `r.SwapUsed`, and on static `r.SwapTotal`.
- Static only: `/proc/cpuinfo` (`model name`, count `processor` lines) → `r.CPU.Model/Cores`; `/proc/meminfo MemTotal` → `r.Memory.TotalBytes`; kernel via `syscall.Uname` or read `/proc/sys/kernel/osrelease` → `r.Kernel`; partitions from `/proc/mounts` filtered to fstypes in {ext4,xfs,btrfs,zfs,vfat,ntfs} then `syscall.Statfs` for total/used → `r.Partitions`.
```go
//go:build linux
package inventory
import (
"bufio"
"os"
"strconv"
"strings"
"syscall"
"time"
"github.com/mrhid6/vantage/agent/internal/grpc/pb"
)
func collect(r *pb.InventoryReport, includeStatic bool) {
r.CPU.UsagePct = cpuUsage()
r.CPU.Load1 = load1()
memTotal, memAvail, swapTotal, swapFree := meminfo()
if memTotal > memAvail {
r.Memory.UsedBytes = memTotal - memAvail
}
if swapTotal > swapFree {
r.SwapUsed = swapTotal - swapFree
}
if includeStatic {
r.Memory.TotalBytes = memTotal
r.SwapTotal = swapTotal
r.CPU.Model, r.CPU.Cores = cpuStatic()
r.Kernel = kernel()
r.Partitions = partitions()
}
}
func readProc(path string) string { b, _ := os.ReadFile(path); return string(b) }
func cpuSample() (idle, total uint64) {
f, err := os.Open("/proc/stat")
if err != nil {
return
}
defer f.Close()
sc := bufio.NewScanner(f)
if sc.Scan() {
fields := strings.Fields(sc.Text()) // cpu user nice system idle iowait ...
for i, v := range fields[1:] {
n, _ := strconv.ParseUint(v, 10, 64)
total += n
if i == 3 { // idle
idle = n
}
}
}
return
}
func cpuUsage() float64 {
i1, t1 := cpuSample()
time.Sleep(100 * time.Millisecond)
i2, t2 := cpuSample()
dt := float64(t2 - t1)
if dt <= 0 {
return 0
}
return (1 - float64(i2-i1)/dt) * 100
}
func load1() float64 {
fields := strings.Fields(readProc("/proc/loadavg"))
if len(fields) > 0 {
v, _ := strconv.ParseFloat(fields[0], 64)
return v
}
return 0
}
func meminfo() (total, avail, swapTotal, swapFree uint64) {
f, err := os.Open("/proc/meminfo")
if err != nil {
return
}
defer f.Close()
sc := bufio.NewScanner(f)
for sc.Scan() {
fields := strings.Fields(sc.Text())
if len(fields) < 2 {
continue
}
kb, _ := strconv.ParseUint(fields[1], 10, 64)
b := kb * 1024
switch strings.TrimSuffix(fields[0], ":") {
case "MemTotal":
total = b
case "MemAvailable":
avail = b
case "SwapTotal":
swapTotal = b
case "SwapFree":
swapFree = b
}
}
return
}
func cpuStatic() (model string, cores int) {
f, err := os.Open("/proc/cpuinfo")
if err != nil {
return
}
defer f.Close()
sc := bufio.NewScanner(f)
for sc.Scan() {
line := sc.Text()
if strings.HasPrefix(line, "processor") {
cores++
} else if strings.HasPrefix(line, "model name") && model == "" {
if i := strings.Index(line, ":"); i >= 0 {
model = strings.TrimSpace(line[i+1:])
}
}
}
return
}
func kernel() string {
return strings.TrimSpace(readProc("/proc/sys/kernel/osrelease"))
}
func partitions() []pb.PartitionReport {
allowed := map[string]bool{"ext4": true, "xfs": true, "btrfs": true, "zfs": true, "vfat": true, "ntfs": true, "ext3": true}
f, err := os.Open("/proc/mounts")
if err != nil {
return nil
}
defer f.Close()
var out []pb.PartitionReport
seen := map[string]bool{}
sc := bufio.NewScanner(f)
for sc.Scan() {
fields := strings.Fields(sc.Text())
if len(fields) < 3 || !allowed[fields[2]] || seen[fields[1]] {
continue
}
seen[fields[1]] = true
var st syscall.Statfs_t
if syscall.Statfs(fields[1], &st) != nil {
continue
}
total := st.Blocks * uint64(st.Bsize)
free := st.Bavail * uint64(st.Bsize)
out = append(out, pb.PartitionReport{
Device: fields[0], Mountpoint: fields[1], Fstype: fields[2],
TotalBytes: total, UsedBytes: total - free,
})
}
return out
}
```
- [ ] **Step 3: Non-linux stub (`collect_other.go`)**
```go
//go:build !linux
package inventory
import "github.com/mrhid6/vantage/agent/internal/grpc/pb"
// collect is a no-op best-effort stub on non-Linux platforms.
func collect(r *pb.InventoryReport, includeStatic bool) {}
```
- [ ] **Step 4: Verify build**
Run: `cd agent && go build ./... && go vet ./...`
Expected: success (build both native and, if convenient, `GOOS=windows go build ./...`).
- [ ] **Step 5: Commit**
```bash
git add agent/internal/inventory/
git commit -m "feat(agent): /proc-based inventory collectors"
```
---
## Task 4: Agent client method + scheduler
**Files:**
- Modify: `agent/internal/grpc/client.go`
- Modify: the agent main loop (`agent/cmd/main.go` or `agent/internal/sync/sync.go` — wherever the poll loop/tickers live).
**Interfaces:**
- Consumes: `inventory.Collect` (T3), pb (T1).
- Produces: `(*Client).ReportInventory(report *pb.InventoryReport) error`; a running ticker that reports metrics every 30s and static every 15 min.
- [ ] **Step 1: Add client method**
In `agent/internal/grpc/client.go`, mirroring `ReportUpdates`:
```go
func (c *Client) ReportInventory(report *pb.InventoryReport) error {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
_, err := c.client.ReportInventory(ctx, report)
return err
}
```
The report already carries `ServerId`/`AgentToken`; ensure the caller sets them (see Step 2).
- [ ] **Step 2: Add the scheduler to the agent loop**
Find where the agent starts its poll loop (the goroutine that calls `SyncKeys`/`ReportUpdates`). Add a parallel inventory ticker. `serverID`, `agentToken`, and the `*Client` are in scope there:
```go
go func() {
tick := 0
t := time.NewTicker(30 * time.Second)
defer t.Stop()
report := func(static bool) {
r := inventory.Collect(static)
r.ServerId = serverID
r.AgentToken = agentToken
if err := client.ReportInventory(r); err != nil {
log.Printf("report inventory: %v", err)
}
}
report(true) // send a full snapshot on startup
for range t.C {
tick++
report(tick%30 == 0) // every 30th tick = 15 min → include static
}
}()
```
Add imports `"github.com/mrhid6/vantage/agent/internal/inventory"`, `time`, `log` if missing. Match variable names to the actual loop (e.g. the client may be named `c`).
- [ ] **Step 3: Verify build**
Run: `cd agent && go build ./... && go vet ./...`
Expected: success.
- [ ] **Step 4: Commit**
```bash
git add agent/internal/grpc/client.go agent/
git commit -m "feat(agent): schedule inventory reporting (30s metrics, 15m static)"
```
---
## Task 5: Frontend — inventory panel on server detail
**Files:**
- Modify: `web/lib/api.ts` (extend the `Server`/server-detail type with `inventory`)
- Modify: `web/app/servers/[id]/page.tsx` (add panel; enable polling)
**Interfaces:**
- Consumes: server-detail query.
- [ ] **Step 1: Add the inventory type**
In `web/lib/api.ts`, add and attach to the server type used by the detail page:
```ts
export interface Inventory {
cpu: { model?: string; cores?: number; usage_pct: number; load1?: number };
memory: { total_bytes: number; used_bytes: number };
swap_total_bytes: number;
swap_used_bytes: number;
partitions?: { device: string; mountpoint: string; fstype?: string; total_bytes: number; used_bytes: number }[];
kernel?: string;
metrics_at?: string;
static_at?: string;
}
```
Add `inventory?: Inventory;` to the server detail interface.
- [ ] **Step 2: Add a `formatBytes` helper + Inventory panel**
In `web/app/servers/[id]/page.tsx`, add a helper and a panel component. Enable polling on the server-detail `useQuery` with `refetchInterval: 30000`.
```tsx
function formatBytes(n: number): string {
if (!n) return "0 B";
const u = ["B", "KB", "MB", "GB", "TB"];
const i = Math.floor(Math.log(n) / Math.log(1024));
return `${(n / Math.pow(1024, i)).toFixed(1)} ${u[i]}`;
}
function UsageBar({ used, total }: { used: number; total: number }) {
const pct = total > 0 ? Math.min(100, (used / total) * 100) : 0;
return (
<div className="h-2 w-full overflow-hidden rounded-full bg-surface-2">
<div className={`h-full rounded-full ${pct > 90 ? "bg-danger" : "bg-accent"}`} style={{ width: `${pct}%` }} />
</div>
);
}
function InventoryPanel({ inv }: { inv: Inventory }) {
return (
<Card>
<h2 className="mb-4 text-lg font-semibold text-text-primary">Inventory</h2>
<div className="grid gap-4 sm:grid-cols-2">
<div>
<div className="mb-1 flex justify-between text-sm"><span className="text-text-secondary">CPU</span><span className="text-text-primary">{inv.cpu.usage_pct.toFixed(0)}%</span></div>
<UsageBar used={inv.cpu.usage_pct} total={100} />
<p className="mt-1 text-xs text-text-secondary">{inv.cpu.model} · {inv.cpu.cores} cores · load {inv.cpu.load1?.toFixed(2)}</p>
</div>
<div>
<div className="mb-1 flex justify-between text-sm"><span className="text-text-secondary">Memory</span><span className="text-text-primary">{formatBytes(inv.memory.used_bytes)} / {formatBytes(inv.memory.total_bytes)}</span></div>
<UsageBar used={inv.memory.used_bytes} total={inv.memory.total_bytes} />
<div className="mb-1 mt-3 flex justify-between text-sm"><span className="text-text-secondary">Swap</span><span className="text-text-primary">{formatBytes(inv.swap_used_bytes)} / {formatBytes(inv.swap_total_bytes)}</span></div>
<UsageBar used={inv.swap_used_bytes} total={inv.swap_total_bytes} />
</div>
</div>
{inv.partitions && inv.partitions.length > 0 && (
<div className="mt-5">
<h3 className="mb-2 text-sm font-medium text-text-secondary">Partitions</h3>
<div className="space-y-3">
{inv.partitions.map((p) => (
<div key={p.mountpoint}>
<div className="mb-1 flex justify-between text-xs">
<span className="font-mono text-text-primary">{p.mountpoint}</span>
<span className="text-text-secondary">{formatBytes(p.used_bytes)} / {formatBytes(p.total_bytes)} · {p.fstype}</span>
</div>
<UsageBar used={p.used_bytes} total={p.total_bytes} />
</div>
))}
</div>
</div>
)}
{inv.kernel && <p className="mt-4 text-xs text-text-secondary">Kernel {inv.kernel}</p>}
</Card>
);
}
```
Render `{server.inventory && <InventoryPanel inv={server.inventory} />}` in the page body (ensure `Card`, `Inventory` are imported). Match how the page currently reads the server object.
- [ ] **Step 3: Verify build**
Run: `cd web && npm run build`
Expected: success.
- [ ] **Step 4: Commit**
```bash
git add web/lib/api.ts web/app/servers/[id]/page.tsx
git commit -m "feat(web): inventory panel on server detail"
```
---
## Task 6: End-to-end manual verification
- [ ] **Step 1: Build all**
Run: `cd server && go build ./... && cd ../agent && go build ./... && cd ../web && npm run build`
Expected: all succeed.
- [ ] **Step 2: Smoke (if environment available)**
With server + Mongo + a connected Linux agent: within ~30s the server detail page shows CPU %, RAM/swap bars; within 15 min (or on agent restart, which sends a full snapshot immediately) partitions, CPU model and kernel appear. Confirm metrics update roughly every 30s.
- [ ] **Step 3: Commit any fixes**
```bash
git add -A
git commit -m "fix: fleet inventory verification fixes"
```
---
## Self-Review Notes
- **Spec coverage:** §3 model → T1; §4 RPC → T1; §5 collectors + scheduler → T3, T4; §6 handler/store → T2; §7 frontend → T5. Split cadence (30s metrics / 15m static) in T4 scheduler; merge rules preserving static in T2 `StoreInventory`. Tests omitted per Global Constraints.
- **Startup snapshot:** agent sends `Collect(true)` immediately so static fields populate without waiting 15 min.
- **Types consistent:** `InventoryReport` field names identical across proto, both pb files, store service, and TS interface (`usage_pct`, `used_bytes`, `total_bytes`, `swap_*`).
- **Follow-ups (out of scope):** time-series history, usage alerting, Windows collectors, servers-list CPU/RAM badges.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -1,887 +0,0 @@
# Workflow Log Streaming Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Stream workflow step output live from agents to per-server-run log files on the server, tail them live in the UI over SSE, and auto-expire them on a configurable retention period.
**Architecture:** Agent streams interleaved stdout/stderr chunks over the existing `CommandStream` (`AgentMessage.StepOutput`). Server appends secret-masked chunks to `<logdir>/<run_id>/<server_id>.log` via a per-command log-writer registry, records a per-step byte offset, and stops persisting log bodies in Mongo. UI tails via an SSE endpoint while running and fetches the whole file after. An hourly sweeper deletes run-log dirs older than the retention setting.
**Tech Stack:** Go (gin, mongo-driver v2), hand-written JSON-codec gRPC structs (no protoc), Next.js 16 app-router + react-query + EventSource, MongoDB, local filesystem for logs.
## Global Constraints
- **No tests this iteration** — do not write `*_test.go` or frontend tests. Verify each task with `go build ./...`, `go vet ./...`, and (frontend) `npm run build`.
- gRPC uses a **JSON codec** — proto messages are hand-written Go structs in **two** files that must stay identical: `server/internal/grpc/pb/vantage.pb.go` and `agent/internal/grpc/pb/vantage.pb.go`. No codegen. Also update `proto/vantage/v1/vantage.proto` as documentation.
- Mongo access pattern: `db.Col("collection_name")` with `context.WithTimeout`. Follow `server/internal/services/workflows.go`.
- Secret values must never be written into log files unmasked — mask by literal `***` replacement at write time, boundary-safe via a carry buffer.
- Interpreter values are the literals `"bash"` and `"powershell"`.
- Go module path: `github.com/mrhid6/vantage`.
- Log dir from env `VANTAGE_WORKFLOW_LOG_DIR`, default `<data>/workflow-logs`; files `0600`, dirs `0700`.
- Retention default **30** days, stored `settings.workflow_log_retention_days`; `0`/negative = keep forever.
- The agent's stream `Send` is only safe through the existing per-connection mutex-guarded `send()` closure in `connectAndHandleStream` — all `StepOutput`/`StepResult` sends MUST go through it.
---
## Task 1: Proto/pb — StepOutputChunk
**Files:**
- Modify: `proto/vantage/v1/vantage.proto`
- Modify: `server/internal/grpc/pb/vantage.pb.go`
- Modify: `agent/internal/grpc/pb/vantage.pb.go`
**Interfaces:**
- Produces: `pb.StepOutputChunk{CommandId string, Seq uint64, Data []byte, Eof bool}`; `pb.AgentMessage` gains `StepOutput *StepOutputChunk`.
- [ ] **Step 1: Document in the proto file**
In `proto/vantage/v1/vantage.proto`, add to the `AgentMessage` oneof: `StepOutputChunk step_output = 6;` and add the message:
```protobuf
message StepOutputChunk {
string command_id = 1;
uint64 seq = 2;
bytes data = 3;
bool eof = 4;
}
```
- [ ] **Step 2: Add struct + field to server pb file**
In `server/internal/grpc/pb/vantage.pb.go`, add to `type AgentMessage struct { ... }`:
```go
StepOutput *StepOutputChunk `json:"step_output,omitempty"`
```
and add the new struct:
```go
type StepOutputChunk struct {
CommandId string `json:"command_id"`
Seq uint64 `json:"seq"`
Data []byte `json:"data,omitempty"`
Eof bool `json:"eof,omitempty"`
}
```
- [ ] **Step 3: Mirror identical additions into the agent pb file**
Apply the identical `AgentMessage.StepOutput` field and `StepOutputChunk` struct to `agent/internal/grpc/pb/vantage.pb.go`.
- [ ] **Step 4: Verify build**
Run: `cd server && go build ./... && cd ../agent && go build ./...`
Expected: both succeed.
- [ ] **Step 5: Commit**
```bash
git add proto/vantage/v1/vantage.proto server/internal/grpc/pb/vantage.pb.go agent/internal/grpc/pb/vantage.pb.go
git commit -m "feat(proto): add StepOutputChunk streaming message"
```
---
## Task 2: Agent — stream step output
**Files:**
- Modify: `agent/internal/exec/exec.go`
- Modify: `agent/internal/sync/sync.go` (the `cmd.RunStep != nil` goroutine)
**Interfaces:**
- Consumes: `pb.RunStepCmd`, `pb.StepResult`, `pb.StepOutputChunk` (Task 1).
- Produces: `exec.RunStep(cmd *pb.RunStepCmd, emit func(seq uint64, data []byte)) *pb.StepResult` — streams output via `emit`, returns terminal result with empty stdout/stderr but populated exit_code/output_env.
- [ ] **Step 1: Rework `exec.RunStep` to stream**
In `agent/internal/exec/exec.go`, change the signature and replace the two `bytes.Buffer`s with a single mutex-guarded streaming writer. Full new body of the run/capture section (keep the existing temp-dir, env-file, interpreter-selection, timeout, and `parseEnvFile` logic exactly as-is):
Add this type at package scope:
```go
// streamWriter forwards every write to emit() as an ordered chunk. Used as both
// Stdout and Stderr so output interleaves in real execution order. The mutex
// ensures a single stdout/stderr write is not interleaved mid-slice with another.
type streamWriter struct {
mu sync.Mutex
seq uint64
emit func(seq uint64, data []byte)
}
func (w *streamWriter) Write(p []byte) (int, error) {
w.mu.Lock()
defer w.mu.Unlock()
if w.emit != nil {
buf := make([]byte, len(p))
copy(buf, p)
w.emit(w.seq, buf)
w.seq++
}
return len(p), nil
}
```
Add `"sync"` to the imports. Change the signature to:
```go
func RunStep(cmd *pb.RunStepCmd, emit func(seq uint64, data []byte)) *pb.StepResult {
```
Replace the block that currently declares `var stdout, stderr bytes.Buffer`, assigns `c.Stdout`/`c.Stderr`, and sets `res.Stdout`/`res.Stderr` from them, with:
```go
sw := &streamWriter{emit: emit}
c.Stdout = sw
c.Stderr = sw
runErr := c.Run()
// stdout/stderr are streamed via emit, not returned in the result.
if ctx.Err() == context.DeadlineExceeded {
res.ExitCode = 124
res.Stderr = "[vantage] step timed out"
} else if ee, ok := runErr.(*exec.ExitError); ok {
res.ExitCode = ee.ExitCode()
} else if runErr != nil {
res.ExitCode = 1
res.Stderr = "[vantage] " + runErr.Error()
}
res.OutputEnv = parseEnvFile(envFile)
return res
```
Remove the now-unused `"bytes"` and `"bufio"` imports **only if** they are no longer referenced (`parseEnvFile` uses `bufio` + `os` — keep `bufio`; `bytes` is likely now unused — remove it if so). Verify with `go build`.
- [ ] **Step 2: Wire streaming into the agent loop**
In `agent/internal/sync/sync.go`, the `cmd.RunStep != nil` goroutine currently calls `agentexec.RunStep(rc)` and sends one `StepResult` via `send()`. Change it to pass an `emit` closure that streams chunks, then send an eof chunk, then the terminal result — all through the existing mutex-guarded `send()`:
```go
if cmd.RunStep != nil {
go func(rc *pb.RunStepCmd, cid string) {
emit := func(seq uint64, data []byte) {
_ = send(&pb.AgentMessage{
ServerId: cfg.ServerID,
AgentToken: cfg.AgentToken,
StepOutput: &pb.StepOutputChunk{CommandId: cid, Seq: seq, Data: data},
})
}
res := agentexec.RunStep(rc, emit)
res.CommandId = cid
// Final eof marker so the server closes the log file.
_ = send(&pb.AgentMessage{
ServerId: cfg.ServerID,
AgentToken: cfg.AgentToken,
StepOutput: &pb.StepOutputChunk{CommandId: cid, Eof: true},
})
_ = send(&pb.AgentMessage{
ServerId: cfg.ServerID,
AgentToken: cfg.AgentToken,
StepResult: res,
})
}(cmd.RunStep, cmd.CommandId)
continue
}
```
(Match the exact field names already used by the existing `send()` calls in this function — `cfg.ServerID`, `cfg.AgentToken`, and the `send` closure. If the existing RunStep branch used different local names, keep those.)
- [ ] **Step 3: Verify build**
Run: `cd agent && go build ./... && go vet ./...`
Expected: success. Resolve any leftover unused-import error from Step 1.
- [ ] **Step 4: Commit**
```bash
git add agent/internal/exec/exec.go agent/internal/sync/sync.go
git commit -m "feat(agent): stream step output chunks over CommandStream"
```
---
## Task 3: Server log-writer registry + retention sweeper
**Files:**
- Create: `server/internal/services/steplogs.go`
**Interfaces:**
- Consumes: `settings` service (retention), `db.Col("workflow_runs")` (sweeper), env `VANTAGE_WORKFLOW_LOG_DIR`.
- Produces:
- `WorkflowLogDir() string` — resolved base dir (env or default), created on first call.
- `ServerRunLogPath(runID, serverID string) string``<logdir>/<runID>/<serverID>.log`.
- `AppendMarker(runID, serverID, line string) (int64, error)` — appends a marker line, returns the byte offset **before** the write (the step's `log_offset`).
- `var StepLogs *stepLogRegistry` with `Open(commandID, path string, secrets []string) error`, `Append(commandID string, data []byte)`, `Close(commandID string)`.
- `StartLogSweeper()` — launches the hourly retention goroutine; also sweeps once immediately.
- [ ] **Step 1: Write the registry, paths, and sweeper**
```go
package services
import (
"os"
"path/filepath"
"strings"
"sync"
"time"
"github.com/mrhid6/vantage/server/internal/db"
"go.mongodb.org/mongo-driver/v2/bson"
)
// WorkflowLogDir returns the base directory for workflow step logs, creating it.
func WorkflowLogDir() string {
dir := os.Getenv("VANTAGE_WORKFLOW_LOG_DIR")
if dir == "" {
dir = filepath.Join("data", "workflow-logs")
}
_ = os.MkdirAll(dir, 0700)
return dir
}
// ServerRunLogPath is the per-server-run log file path.
func ServerRunLogPath(runID, serverID string) string {
return filepath.Join(WorkflowLogDir(), runID, serverID+".log")
}
// AppendMarker appends a line to the server-run log and returns the byte offset
// at which the write began (used as a step's log_offset).
func AppendMarker(runID, serverID, line string) (int64, error) {
path := ServerRunLogPath(runID, serverID)
if err := os.MkdirAll(filepath.Dir(path), 0700); err != nil {
return 0, err
}
f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0600)
if err != nil {
return 0, err
}
defer f.Close()
off, _ := f.Seek(0, 2) // current end = offset before write
if _, err := f.WriteString(line); err != nil {
return off, err
}
return off, nil
}
// ---- streamed chunk writer, boundary-safe secret masking ----
type stepLogWriter struct {
mu sync.Mutex
f *os.File
carry []byte
secrets []string
maxSecret int
}
type stepLogRegistry struct {
mu sync.Mutex
writers map[string]*stepLogWriter
}
var StepLogs = &stepLogRegistry{writers: make(map[string]*stepLogWriter)}
// Open opens (append) the server-run file for a step's streamed chunks.
func (r *stepLogRegistry) Open(commandID, path string, secrets []string) error {
if err := os.MkdirAll(filepath.Dir(path), 0700); err != nil {
return err
}
f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0600)
if err != nil {
return err
}
max := 0
for _, s := range secrets {
if len(s) > max {
max = len(s)
}
}
w := &stepLogWriter{f: f, secrets: secrets, maxSecret: max}
r.mu.Lock()
r.writers[commandID] = w
r.mu.Unlock()
return nil
}
func (r *stepLogRegistry) get(commandID string) *stepLogWriter {
r.mu.Lock()
defer r.mu.Unlock()
return r.writers[commandID]
}
// Append masks and writes a chunk, holding back the last maxSecret-1 bytes so a
// secret split across a chunk boundary is still masked on the next append/close.
func (r *stepLogRegistry) Append(commandID string, data []byte) {
w := r.get(commandID)
if w == nil {
return
}
w.mu.Lock()
defer w.mu.Unlock()
if len(w.secrets) == 0 || w.maxSecret <= 1 {
_, _ = w.f.Write(data)
return
}
buf := append(w.carry, data...)
hold := w.maxSecret - 1
if len(buf) <= hold {
w.carry = buf
return
}
flush := buf[:len(buf)-hold]
w.carry = append([]byte{}, buf[len(buf)-hold:]...)
_, _ = w.f.Write(maskBytes(flush, w.secrets))
}
// Close flushes the carry (masked) and closes the file.
func (r *stepLogRegistry) Close(commandID string) {
r.mu.Lock()
w := r.writers[commandID]
delete(r.writers, commandID)
r.mu.Unlock()
if w == nil {
return
}
w.mu.Lock()
defer w.mu.Unlock()
if len(w.carry) > 0 {
_, _ = w.f.Write(maskBytes(w.carry, w.secrets))
w.carry = nil
}
_ = w.f.Close()
}
func maskBytes(b []byte, secrets []string) []byte {
s := string(b)
for _, v := range secrets {
if v == "" {
continue
}
s = strings.ReplaceAll(s, v, "***")
}
return []byte(s)
}
// ---- retention sweeper ----
// StartLogSweeper sweeps expired run-log dirs hourly (and once now).
func StartLogSweeper() {
go func() {
sweepLogs()
t := time.NewTicker(time.Hour)
defer t.Stop()
for range t.C {
sweepLogs()
}
}()
}
func sweepLogs() {
days := retentionDays()
if days <= 0 {
return
}
cutoff := time.Now().AddDate(0, 0, -days)
base := WorkflowLogDir()
entries, err := os.ReadDir(base)
if err != nil {
return
}
for _, e := range entries {
if !e.IsDir() {
continue
}
runID := e.Name()
dir := filepath.Join(base, runID)
if runExpired(runID, dir, cutoff) {
_ = os.RemoveAll(dir)
}
}
}
// runExpired is true when the run finished before cutoff (falling back to dir
// mtime when the run doc is gone).
func runExpired(runID, dir string, cutoff time.Time) bool {
ctx, cancel := wfCtx()
defer cancel()
var run struct {
FinishedAt *time.Time `bson:"finished_at"`
}
err := db.Col("workflow_runs").FindOne(ctx, bson.M{"run_id": runID}).Decode(&run)
if err == nil {
if run.FinishedAt == nil {
return false // still running / never finished — keep
}
return run.FinishedAt.Before(cutoff)
}
// run doc gone: use dir mtime
if fi, e := os.Stat(dir); e == nil {
return fi.ModTime().Before(cutoff)
}
return false
}
func retentionDays() int {
if v, err := GetWorkflowLogRetentionDays(); err == nil {
return v
}
return 30
}
```
Note: `wfCtx` is defined in `workflows.go` (same package) — reuse it. `GetWorkflowLogRetentionDays` is added in Task 4; this file references it (same package, compiles together).
- [ ] **Step 2: Verify build**
Run: `cd server && go build ./... && go vet ./...`
Expected: FAIL — `GetWorkflowLogRetentionDays` undefined until Task 4. This is expected; proceed to commit the file so Task 4 completes it. (If you prefer a green build, do Task 4's settings accessor first, then return — but committing here is fine since Task 4 immediately follows.)
Actually to keep every commit buildable: **temporarily** add a local stub at the bottom of this file and remove it in Task 4:
```go
// TEMP stub, replaced in Task 4.
func GetWorkflowLogRetentionDays() (int, error) { return 30, nil }
```
Then `cd server && go build ./... && go vet ./...` must succeed.
- [ ] **Step 3: Commit**
```bash
git add server/internal/services/steplogs.go
git commit -m "feat(server): workflow log-writer registry, paths, retention sweeper"
```
---
## Task 4: Settings — retention accessor + startup wiring
**Files:**
- Modify: `server/internal/services/settings.go` (or wherever settings get/set lives — search `settings` collection usage)
- Modify: `server/internal/services/steplogs.go` (remove the temp stub)
- Modify: `server/cmd/main.go` (start the sweeper)
**Interfaces:**
- Produces: `GetWorkflowLogRetentionDays() (int, error)` (default 30 when unset), `SetWorkflowLogRetentionDays(int) error`. If settings are exposed as a single document/struct, add the field there and derive these accessors.
- [ ] **Step 1: Inspect the settings service**
Read the existing settings service (search for the `settings` collection: `grep -rn "\"settings\"" server/internal/services`). Determine whether settings are a typed struct document or key/value. Match that pattern.
- [ ] **Step 2: Add the retention accessor**
If settings are a **typed document** (e.g. a `GetSettings()/UpdateSettings()`), add a field `WorkflowLogRetentionDays int `bson:"workflow_log_retention_days" json:"workflow_log_retention_days"`` to the settings struct and implement:
```go
func GetWorkflowLogRetentionDays() (int, error) {
s, err := GetSettings() // use the real accessor name
if err != nil {
return 30, err
}
if s.WorkflowLogRetentionDays == 0 && /* unset sentinel */ !s.WorkflowLogRetentionSet {
return 30, nil
}
return s.WorkflowLogRetentionDays, nil
}
```
Simplify to match reality: if the settings doc uses zero-value-means-unset and you cannot distinguish "0 = keep forever" from "unset", store the retention as a pointer `*int` or default at read: **treat a missing field as 30, an explicit 0 as keep-forever.** Prefer `*int` in the struct so the three states (unset→30, 0→forever, N→N) are representable. Implement `GetWorkflowLogRetentionDays` to return 30 when the pointer is nil, else its value. `SetWorkflowLogRetentionDays(n int)` sets the pointer.
If settings are **key/value**, implement both accessors against that store with the same nil→30 / 0→forever semantics (store empty/absent = 30).
- [ ] **Step 3: Remove the temp stub from `steplogs.go`**
Delete the `// TEMP stub` `GetWorkflowLogRetentionDays` added in Task 3 so the real one is used.
- [ ] **Step 4: Start the sweeper at boot**
In `server/cmd/main.go`, next to `EnsureWorkflowIndexes()`, add `services.StartLogSweeper()`.
- [ ] **Step 5: Verify build**
Run: `cd server && go build ./... && go vet ./...`
Expected: success (real accessor now resolves the reference from Task 3).
- [ ] **Step 6: Commit**
```bash
git add server/internal/services/settings.go server/internal/services/steplogs.go server/cmd/main.go
git commit -m "feat(server): workflow log retention setting + sweeper startup"
```
---
## Task 5: Runner + model — write to files, drop log bodies from Mongo
**Files:**
- Modify: `server/internal/models/workflow.go` (`StepRun`)
- Modify: `server/internal/services/workflow_runner.go`
- Modify: `server/internal/grpc/server.go` (stream delivery of `StepOutput`)
**Interfaces:**
- Consumes: `StepLogs`, `AppendMarker`, `ServerRunLogPath` (Task 3), `pb.StepOutputChunk` (Task 1).
- Produces: runner writes markers + streams chunks to files; `StepRun.LogOffset` persisted; `StepRun.Stdout/Stderr` removed.
- [ ] **Step 1: Update the `StepRun` model**
In `server/internal/models/workflow.go`, in `type StepRun struct`:
- Remove the `Stdout` and `Stderr` fields.
- Add: `LogOffset int64 `bson:"log_offset" json:"log_offset"``
- [ ] **Step 2: Deliver StepOutput chunks in the gRPC receive loop**
In `server/internal/grpc/server.go`, after the existing `if m.StepResult != nil { services.StepResults.Deliver(m.StepResult) }` block, add:
```go
if m.StepOutput != nil {
if m.StepOutput.Eof {
services.StepLogs.Close(m.StepOutput.CommandId)
} else {
services.StepLogs.Append(m.StepOutput.CommandId, m.StepOutput.Data)
}
}
```
- [ ] **Step 3: Rework `runServer` to open logs + write markers, drop persisted bodies**
In `server/internal/services/workflow_runner.go`, `runServer`:
Inside the per-step loop, **before** `dispatchAndWait`, add marker + open (compute `secretVals` first, which already exists in the loop):
```go
// Write the step marker and remember the offset for later slicing.
marker := fmt.Sprintf("\n===== step %d: %s =====\n", step.Order, step.Name)
offset, _ := AppendMarker(runID, serverID, marker)
logPath := ServerRunLogPath(runID, serverID)
_ = StepLogs.Open(commandID_placeholder, logPath, secretsSlice(secretVals))
```
There is a chicken-and-egg with `commandID`: today `dispatchAndWait` generates the `commandID` internally. Refactor so the runner owns the `commandID`:
1. Change `dispatchAndWait(serverID string, cmd *pb.RunStepCmd)` to `dispatchAndWait(serverID, commandID string, cmd *pb.RunStepCmd)` and remove its internal `commandID := uuid.New().String()` (use the passed one).
2. In `runServer`, generate `commandID := uuid.New().String()` at the top of each attempt-group (before the marker/open), open the log with it, then call `dispatchAndWait(serverID, commandID, cmd)`.
3. After the step completes (result received), call `StepLogs.Close(commandID)` defensively (idempotent — the agent's eof usually closed it already; Close on a missing key is a no-op).
Add a helper to convert the `secretVals map[string]string` to a `[]string` of values:
```go
func secretsSlice(m map[string]string) []string {
out := make([]string, 0, len(m))
for _, v := range m {
out = append(out, v)
}
return out
}
```
Update `finishStep(...)` call + signature: **remove** the `stdout, stderr string` params and the `output_env` masking stays. Persist `log_offset` instead. New `finishStep`:
```go
func finishStep(runID, serverID string, order int, status string, attempts, exit int, logOffset int64, outEnv map[string]string) {
now := time.Now()
updateStep(runID, serverID, order, bson.M{
"server_runs.$[s].steps.$[t].status": status,
"server_runs.$[s].steps.$[t].attempts": attempts,
"server_runs.$[s].steps.$[t].exit_code": exit,
"server_runs.$[s].steps.$[t].log_offset": logOffset,
"server_runs.$[s].steps.$[t].output_env": outEnv,
"server_runs.$[s].steps.$[t].finished_at": now,
})
}
```
In the loop, after receiving `res`, drop the `stdout, stderr := ...` masking of `res.Stdout/res.Stderr` (those are now streamed to file). Keep the `outEnv` build **with existing masking** (`maskSecrets(v, allSecrets)` per the merged secret-leak fix) — `output_env`/`run_env` masking is unchanged. Call:
```go
finishStep(runID, serverID, i, status, attempts, exit, offset, outEnv)
```
where `offset` is the marker offset captured before dispatch. If `res == nil`, still write a short note to the file so failures are visible:
```go
if res == nil {
_, _ = AppendMarker(runID, serverID, "[vantage] agent did not return a result\n")
}
```
Remove the initial `StepRun{... Status:"queued"}` `Stdout/Stderr` references if any (the model no longer has them — the queued StepRun in `TriggerWorkflow` set only `Order/Name/Status/OutputEnv`, so no change needed there; verify).
Ensure `fmt` is imported (it already is).
- [ ] **Step 4: Verify build**
Run: `cd server && go build ./... && go vet ./...`
Expected: success. Fix any remaining references to the removed `Stdout`/`Stderr` fields or the old `finishStep`/`dispatchAndWait` signatures.
- [ ] **Step 5: Commit**
```bash
git add server/internal/models/workflow.go server/internal/services/workflow_runner.go server/internal/grpc/server.go
git commit -m "feat(server): stream step logs to files, drop log bodies from run docs"
```
---
## Task 6: REST — log fetch + SSE stream endpoints
**Files:**
- Modify: `server/internal/api/workflows.go`
**Interfaces:**
- Consumes: `ServerRunLogPath`, `GetRun` (existing).
- Produces: `GET /api/runs/:runId/servers/:serverId/logs` and `GET /api/runs/:runId/servers/:serverId/logs/stream` (SSE).
- [ ] **Step 1: Add the two handlers + routes**
In `registerWorkflowRoutes`, add:
```go
g.GET("/runs/:runId/servers/:serverId/logs", getServerRunLog)
g.GET("/runs/:runId/servers/:serverId/logs/stream", streamServerRunLog)
```
Add a UUID-ish validator and the handlers:
```go
var uuidLike = regexp.MustCompile(`^[a-zA-Z0-9-]{1,64}$`)
func getServerRunLog(c *gin.Context) {
runID, serverID := c.Param("runId"), c.Param("serverId")
if !uuidLike.MatchString(runID) || !uuidLike.MatchString(serverID) {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid id"})
return
}
path := services.ServerRunLogPath(runID, serverID)
b, err := os.ReadFile(path)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "no logs"})
return
}
c.Data(http.StatusOK, "text/plain; charset=utf-8", b)
}
func streamServerRunLog(c *gin.Context) {
runID, serverID := c.Param("runId"), c.Param("serverId")
if !uuidLike.MatchString(runID) || !uuidLike.MatchString(serverID) {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid id"})
return
}
path := services.ServerRunLogPath(runID, serverID)
c.Writer.Header().Set("Content-Type", "text/event-stream")
c.Writer.Header().Set("Cache-Control", "no-cache")
c.Writer.Header().Set("Connection", "keep-alive")
c.Writer.Header().Set("X-Accel-Buffering", "no")
flusher, ok := c.Writer.(http.Flusher)
if !ok {
c.JSON(http.StatusInternalServerError, gin.H{"error": "stream unsupported"})
return
}
var offset int64
sendNew := func() bool {
f, err := os.Open(path)
if err != nil {
return true // file may not exist yet; keep waiting
}
defer f.Close()
if _, err := f.Seek(offset, 0); err != nil {
return true
}
buf := make([]byte, 32*1024)
for {
n, _ := f.Read(buf)
if n <= 0 {
break
}
offset += int64(n)
// SSE data frame; split on newlines to keep frames well-formed.
for _, line := range splitSSE(buf[:n]) {
_, _ = c.Writer.WriteString("data: " + line + "\n")
}
_, _ = c.Writer.WriteString("\n")
flusher.Flush()
}
return true
}
ctx := c.Request.Context()
ticker := time.NewTicker(500 * time.Millisecond)
defer ticker.Stop()
for {
sendNew()
if serverRunTerminal(runID, serverID) {
sendNew() // final drain
_, _ = c.Writer.WriteString("event: done\ndata: end\n\n")
flusher.Flush()
return
}
select {
case <-ctx.Done():
return
case <-ticker.C:
}
}
}
// serverRunTerminal reports whether the given server-run has reached a terminal status.
func serverRunTerminal(runID, serverID string) bool {
r, err := services.GetRun(runID)
if err != nil {
return true
}
for _, sr := range r.ServerRuns {
if sr.ServerID == serverID {
switch sr.Status {
case "success", "failed", "skipped", "cancelled":
return true
}
return false
}
}
return true
}
// splitSSE turns a raw byte slice into SSE-safe payload lines (newlines become
// separate data lines; carriage returns stripped).
func splitSSE(b []byte) []string {
s := strings.ReplaceAll(string(b), "\r", "")
return strings.Split(s, "\n")
}
```
Add imports: `"os"`, `"regexp"`, `"strings"`, `"time"`, `"net/http"` (already present). Confirm `services.GetRun` and `ServerRun.Status`/`ServerID` fields exist (they do from the Workflows feature).
- [ ] **Step 2: Verify build**
Run: `cd server && go build ./... && go vet ./...`
Expected: success.
- [ ] **Step 3: Commit**
```bash
git add server/internal/api/workflows.go
git commit -m "feat(api): server-run log fetch and SSE stream endpoints"
```
---
## Task 7: Frontend — live SSE tail + retention setting
**Files:**
- Modify: `web/lib/api.ts`
- Modify: `web/app/workflows/[id]/runs/[runId]/page.tsx`
- Modify: `web/app/settings/page.tsx`
**Interfaces:**
- Consumes: SSE endpoint, logs endpoint, settings mutation.
- [ ] **Step 1: Update API types + helpers**
In `web/lib/api.ts`:
- In `StepRun`, remove `stdout` and `stderr`; add `log_offset: number`.
- Add: `getServerRunLog: (runId: string, serverId: string) => request<string>(...)` — but the logs endpoint returns `text/plain`, so add a dedicated fetch that reads text. If `request<T>` assumes JSON, add a sibling:
```ts
async getServerRunLog(runId: string, serverId: string): Promise<string> {
const res = await fetch(`${API_BASE}/api/runs/${runId}/servers/${serverId}/logs`, { credentials: "include" });
if (!res.ok) throw new Error("no logs");
return res.text();
},
```
(Use the file's real base-URL constant / credentials pattern — inspect how `request` builds URLs and mirror it. If the app is same-origin with a rewrite, a relative `/api/...` fetch is fine.)
- Export a helper to build the SSE URL: `serverRunLogStreamUrl(runId, serverId)` returning the `/api/runs/:runId/servers/:serverId/logs/stream` URL against the same base.
- In the Settings type, add `workflow_log_retention_days?: number | null`.
- [ ] **Step 2: Live tail in the run detail page**
In `web/app/workflows/[id]/runs/[runId]/page.tsx`:
- Remove all use of `st.stdout` / `st.stderr` (fields gone). Step `<details>` now show status/exit/attempts pills only.
- Add a per-server live terminal. For each `server_run`, render a `<pre>` and, while `sr.status === "running"`, subscribe via `EventSource`:
```tsx
function ServerLog({ runId, serverId, status }: { runId: string; serverId: string; status: string }) {
const [text, setText] = useState("");
const preRef = useRef<HTMLPreElement>(null);
const running = status === "running";
useEffect(() => {
if (running) {
const es = new EventSource(api.serverRunLogStreamUrl(runId, serverId), { withCredentials: true });
es.onmessage = (e) => setText((t) => t + e.data + "\n");
es.addEventListener("done", () => es.close());
es.onerror = () => es.close();
return () => es.close();
}
// terminal: fetch the whole file once
api.getServerRunLog(runId, serverId).then(setText).catch(() => setText(""));
}, [running, runId, serverId]);
useEffect(() => { preRef.current?.scrollTo(0, preRef.current.scrollHeight); }, [text]);
return (
<pre ref={preRef} className="mt-2 max-h-80 overflow-auto rounded bg-black/40 p-2 font-mono text-xs text-text-secondary whitespace-pre-wrap">
{text || (running ? "Waiting for output…" : "No output.")}
</pre>
);
}
```
Render `<ServerLog runId={run.run_id} serverId={sr.server_id} status={sr.status} />` inside each server card, below the step pills. Keep the existing react-query `refetchInterval` on the run (drives status pills); the SSE handles live text.
- [ ] **Step 3: Retention field in Settings**
In `web/app/settings/page.tsx`, add a "Workflow log retention (days)" number input bound to `workflow_log_retention_days`, saved through the existing settings save mutation. Add helper text: "0 = keep forever." Match the page's existing input styling.
- [ ] **Step 4: Verify build**
Run: `cd web && npm run build`
Expected: type-checks and builds. Fix any lingering `st.stdout`/`st.stderr` references.
- [ ] **Step 5: Commit**
```bash
git add web/lib/api.ts web/app/workflows/[id]/runs/[runId]/page.tsx web/app/settings/page.tsx
git commit -m "feat(web): live SSE log tail and log retention setting"
```
---
## Task 8: End-to-end verification
**Files:** none (verification only).
- [ ] **Step 1: Build everything**
Run: `cd server && go build ./... && go vet ./... && cd ../agent && go build ./... && go vet ./... && cd ../web && npm run build`
Expected: all succeed.
- [ ] **Step 2: Manual smoke (documented, run if an environment is available)**
With server + MongoDB + a connected agent:
1. Run a workflow with a step that emits output slowly (e.g. `for i in $(seq 1 10); do echo "line $i"; sleep 1; done`). Open the run detail page while running; confirm lines appear live (SSE), not only at the end.
2. Confirm `<logdir>/<run_id>/<server_id>.log` exists on the server with step markers and the output.
3. Confirm `workflow_runs` doc no longer stores stdout/stderr bodies; `steps[].log_offset` is set.
4. Add a secret ref and echo it; confirm the file shows `***`, including when the secret would straddle a chunk boundary.
5. Set retention to 0 in Settings → confirm sweeper keeps files; set to a small value and backdate a run's `finished_at` → confirm the dir is removed within the hour (or call `sweepLogs` path manually).
- [ ] **Step 3: Commit any fixes found**
```bash
git add -A
git commit -m "fix: workflow log streaming e2e fixes"
```
---
## Self-Review Notes
- **Spec coverage:** §3 proto → T1; §4 agent streaming → T2; §5.1 registry + §7 sweeper → T3; §7.1 setting + startup → T4; §5.3/§5.4 runner+model → T5; §6 REST/SSE → T6; §8 frontend → T7. Tests omitted per Global Constraints.
- **Masking** boundary-safe carry buffer in `StepLogs.Append`, flushed in `Close` (T3); `output_env`/`run_env` masking unchanged (T5 keeps the merged fix).
- **commandID ownership** moved to the runner so the log file can be opened before dispatch (T5) — mirrors the `StepResults.Await`-before-dispatch ordering.
- **Buildable commits:** T3 adds a temp stub for `GetWorkflowLogRetentionDays`, removed in T4.
- **Removed fields** `StepRun.Stdout/Stderr` — every reader updated in T5 (runner) and T7 (frontend).
- **Open follow-ups (out of scope):** per-step SSE channels, log download/zip, compression, pre-existing runs have no files.
```
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -1,241 +0,0 @@
# Vantage Web Console (Guacamole Replacement) — Design
**Date:** 2026-07-17
**Status:** Approved design, pre-implementation
## Goal
Add a browser-based remote-access console to Vantage — SSH, RDP, and VNC into
managed servers — as a self-hosted Guacamole replacement. Users select an SSH
key to connect over SSH. RDP targets are reachable from a new Windows agent that
registers the host and reports status. Windows agent ships as an MSI installer
produced by CI.
## Non-Goals (YAGNI)
- Session recording / replay (may be added later).
- Native Go RDP implementation (guacd handles protocol translation).
- Per-user Linux/Windows account management from the agent.
- Tunneling console traffic through the agent (direct network path assumed).
---
## Architecture
```
Browser (guacamole-common-js, vendored — no CDN)
│ Guacamole protocol over WebSocket
Go server: /api/console/tunnel (github.com/wwt/guac)
│ Guacamole protocol over TCP :4822
guacd container (Apache Guacamole daemon)
│ SSH :22 / RDP :3389 / VNC :5900 — direct to target IP
Target host (LAN / VPN line-of-sight from server)
```
- **Browser:** loads vendored `guacamole-common-js`, renders RDP/VNC display and
SSH terminal. No external CDN (matches existing infra rules).
- **Go server:** exposes a WebSocket tunnel endpoint using `github.com/wwt/guac`
(Go Guacamole tunnel library). No Java `guacamole-client` required.
- **guacd:** new container in `deploy/docker-compose.yml`, bound to the internal
docker network only, reachable by the server on `:4822`.
- **Network path:** guacd connects **directly** to the target IP. Requires the
central server to have network line-of-sight to hosts (homelab LAN / VPN). The
agent's outbound-only guarantee is unchanged — the console path is
server→target, not agent-mediated.
---
## Data Model Changes
### `keys` — extend to hold private material
```json
{
"key_id": "uuid",
"label": "dom-macbook",
"public_key": "ssh-ed25519 AAAA...",
"private_key_enc": "<AES-256-GCM ciphertext | null>",
"has_private": true,
"passphrase_enc": "<AES-256-GCM ciphertext | null>",
"fingerprint": "SHA256:...",
"source": "uploaded|generated",
"created_at": "ISODate"
}
```
- A key may be created from an uploaded **private+public** pair, upload of a
public key only, or agent generation.
- Agent key generation now also uploads `private_key_enc` (reuses the existing
AES-256 key used for at-rest encryption). Private key no longer stays local
only — it is stored encrypted so the console can reuse it.
- Optional `passphrase_enc` for passphrase-protected private keys.
- Console lists only keys where `has_private = true`.
### `servers` — extend with console metadata
```json
{
"...": "...existing fields...",
"os_type": "linux|windows",
"console_protocols": ["ssh"],
"ssh_port": 22,
"rdp_port": 3389
}
```
- `os_type` set at registration from the agent.
- `console_protocols` lists enabled protocols per server (`ssh`, `rdp`, `vnc`).
- Port fields default to standard ports, overridable in the UI.
### `console_sessions` — new collection (audit)
```json
{
"session_id": "uuid",
"server_id": "uuid",
"protocol": "ssh|rdp|vnc",
"key_id": "uuid | null",
"user": "who opened it",
"started_at": "ISODate",
"ended_at": "ISODate | null",
"client_ip": "string"
}
```
---
## Session Broker + Connection Flow
New service: `server/internal/services/console.go`.
1. Browser `POST /api/console/connect`
`{ server_id, protocol, key_id?, rdp_username?, rdp_password? }`.
2. Broker validates request, loads the server (host IP, port for protocol),
loads the key and **decrypts `private_key_enc` in memory only**.
3. Builds the guacd connection parameter map:
- **SSH:** `hostname`, `port`, `username`, `private-key` (decrypted),
`passphrase` (if any).
- **RDP:** `hostname`, `port`, `username`, `password`, `security=any`,
`ignore-cert=true`.
- **VNC:** `hostname`, `port`, `password`.
4. Creates a `console_sessions` document, returns a short-lived signed session
token.
5. Browser opens WebSocket `/api/console/tunnel?token=…`. The `wwt/guac` handler
validates the token, dials guacd `:4822`, and pipes bytes in both directions.
6. On socket close, the broker sets `ended_at` on the session doc.
### Security
- Decrypted private keys and RDP passwords are **never persisted, never logged,
never sent to the browser** — passed only to guacd.
- Session token: short TTL (~60s to open the WebSocket), single-use,
HMAC-signed, bound to the authenticated user.
- guacd is bound to the internal docker network only; not exposed publicly.
- At-rest encryption (`private_key_enc`, `passphrase_enc`) reuses the existing
AES-256 key already used for agent-generated private keys.
---
## Windows Agent
Same Go codebase as the Linux agent, with a reduced role: **register +
heartbeat + status only**. No `authorized_keys` management (meaningless on
Windows).
- Build target: `GOOS=windows GOARCH=amd64``vantage-agent-windows-amd64.exe`.
- Agent detects OS at registration and sends `os_type=windows`.
- The key-sync loop is disabled on Windows via a runtime OS check (or build tag)
— no `authorized_keys` writes are ever attempted.
- Config file: `C:\ProgramData\vantage\config.yaml`, locked down via ACL to the
equivalent of `0600`.
- Runs as a Windows service via **nssm**.
---
## Windows Installer (MSI)
Agent ships as a WiX v4 MSI produced in CI.
- **WiX v4** chosen because it is a dotnet tool that builds MSIs
**cross-platform** — runs on the Linux Gitea act_runner. (Inno Setup is
Windows-only and does not fit the runner.)
- MSI bundles `vantage-agent.exe`, installs it to `C:\Program Files\Vantage\`,
and registers the nssm service (ships nssm or uses a CustomAction).
- Accepts install parameters as MSI properties for silent/headless install:
```
msiexec /i vantage-agent.msi /qn SERVERID=<id> TOKEN=<token> SERVERURL=vantage..:9090
```
- GUI install (double-click) prompts for server-id / token / server-url via a
dialog.
### Two install paths
1. **Installer direct** — user downloads `vantage-agent.msi`, double-clicks,
fills the dialog. No script required.
2. **PowerShell one-liner** — served dynamically (like the existing bash
`/install`). Script downloads the `.msi`, verifies SHA-256, then runs
`msiexec /qn` with injected `SERVERID` / `TOKEN` / `SERVERURL`. Used by the
copy-paste "Add Server" flow.
The PowerShell script (`/install.ps1`) steps:
1. Detect arch.
2. Download `vantage-agent.msi` from the latest Gitea `agent/v*` release.
3. Verify SHA-256 against `checksums.txt`.
4. Run `msiexec /i vantage-agent.msi /qn SERVERID=.. TOKEN=.. SERVERURL=..`.
---
## Frontend Routes
| Route | Change |
| ------------------------- | ------------------------------------------------------------- |
| `/servers` | Show `os_type` badge, enabled console protocols |
| `/servers/[id]` | Add **Connect** button(s) per enabled protocol |
| `/servers/[id]/console` | New — full-screen console (guacamole-common-js), key picker |
| `/servers/new` | Offer Windows (MSI) vs Linux (bash) install instructions |
Console page: select protocol + SSH key (SSH) or enter RDP credentials, call
`/api/console/connect`, open the tunnel WebSocket, mount the Guacamole client.
---
## CI/CD Changes
### `agent-release.yml`
- Add `windows/amd64` build: `vantage-agent-windows-amd64.exe`.
- Add WiX v4 MSI build job → `vantage-agent.msi`.
- Add both to `checksums.txt` and release assets.
Release assets become:
- `vantage-agent-linux-amd64`
- `vantage-agent-linux-arm64`
- `vantage-agent-windows-amd64.exe`
- `vantage-agent.msi`
- `checksums.txt`
### `server-deploy.yml`
- Add guacd service to `deploy/docker-compose.yml` (deployed alongside server).
---
## New Dependencies
- **Go:** `github.com/wwt/guac` (Guacamole tunnel/WebSocket in Go).
- **Container:** `guacamole/guacd` official image.
- **Frontend:** vendored `guacamole-common-js` (no CDN).
- **CI:** WiX v4 dotnet tool; nssm binary bundled for the MSI.
---
## Open Implementation Notes
- Confirm `wwt/guac` API surface for connection-parameter passing and token auth
binding during implementation.
- nssm packaging inside MSI: bundle the nssm binary as a payload + CustomAction,
or run `sc.exe`-based service install if nssm proves awkward in WiX.
- ACL hardening of `C:\ProgramData\vantage\config.yaml` in the MSI CustomAction.
@@ -1,142 +0,0 @@
# Fleet Inventory — Design
**Date:** 2026-07-20
**Status:** Approved (design) — ready for implementation planning
**Scope:** Fleet Inventory only. Server Workflows and SaaS/auth are separate sub-projects.
---
## 1. Summary
Each agent collects hardware/OS inventory about its host and reports it to the server, which stores the latest snapshot per server and surfaces it in the UI. Two cadences:
- **Metrics (near-real-time):** CPU load/usage, RAM used/total, swap used/total — every **30s** (aligned with existing poll rhythm).
- **Static inventory (slow):** disks, partitions and their usage, CPU model/cores, total RAM, OS details — every **15 min**.
Transport: a **new unary gRPC `ReportInventory` RPC** (mirrors the existing `ReportUpdates` pattern). No streaming.
---
## 2. Locked decisions
| Topic | Decision |
|-------|----------|
| Transport | New `ReportInventory` unary RPC. |
| Cadence | Metrics every 30s; static inventory every 15 min. One RPC carries both, but static fields are only populated on the 15-min tick (empty/omitted otherwise → server keeps prior static snapshot). |
| Storage | Latest snapshot embedded on the `servers` document (`inventory` sub-doc). No history/time-series in v1. |
| Collection | Pure-Go where practical (`/proc`, `gopsutil`-style). Agent already runs as root. |
| Platform | Linux primary; Windows agent populates what it can, leaves the rest empty. |
---
## 3. Data model
Add an `Inventory` sub-document to the existing `Server` model (`server/internal/models/server.go`):
```go
type CPUInfo struct {
Model string `bson:"model,omitempty" json:"model,omitempty"`
Cores int `bson:"cores,omitempty" json:"cores,omitempty"`
UsagePct float64 `bson:"usage_pct" json:"usage_pct"` // metrics tick
Load1 float64 `bson:"load1,omitempty" json:"load1,omitempty"`
}
type MemInfo struct {
TotalBytes uint64 `bson:"total_bytes" json:"total_bytes"`
UsedBytes uint64 `bson:"used_bytes" json:"used_bytes"` // metrics tick
}
type Partition struct {
Device string `bson:"device" json:"device"`
Mountpoint string `bson:"mountpoint" json:"mountpoint"`
Fstype string `bson:"fstype,omitempty" json:"fstype,omitempty"`
TotalBytes uint64 `bson:"total_bytes" json:"total_bytes"`
UsedBytes uint64 `bson:"used_bytes" json:"used_bytes"`
}
type Inventory struct {
CPU CPUInfo `bson:"cpu" json:"cpu"`
Memory MemInfo `bson:"memory" json:"memory"`
SwapTotalBytes uint64 `bson:"swap_total_bytes" json:"swap_total_bytes"`
SwapUsedBytes uint64 `bson:"swap_used_bytes" json:"swap_used_bytes"`
Partitions []Partition `bson:"partitions,omitempty" json:"partitions,omitempty"`
Kernel string `bson:"kernel,omitempty" json:"kernel,omitempty"`
MetricsAt *time.Time `bson:"metrics_at,omitempty" json:"metrics_at,omitempty"`
StaticAt *time.Time `bson:"static_at,omitempty" json:"static_at,omitempty"`
}
```
Add `Inventory *Inventory` field to `Server`.
Server-side update rules:
- Metrics fields (`cpu.usage_pct`, `cpu.load1`, `memory.used_bytes`, swap used) always updated + `metrics_at`.
- Static fields (`cpu.model/cores`, `memory.total_bytes`, `partitions`, `kernel`, swap total) updated only when the report includes them (non-zero/non-empty) + `static_at`.
---
## 4. gRPC protocol (`proto/vantage/v1/vantage.proto` + both `pb.go` files)
```protobuf
rpc ReportInventory(InventoryReport) returns (InventoryReportResponse);
message InventoryReport {
string server_id = 1;
string agent_token = 2;
bool include_static = 3; // true on the 15-min tick
CPUReport cpu = 4;
MemReport memory = 5;
uint64 swap_total = 6;
uint64 swap_used = 7;
repeated PartitionReport partitions = 8; // only when include_static
string kernel = 9; // only when include_static
}
message CPUReport { string model = 1; int32 cores = 2; double usage_pct = 3; double load1 = 4; }
message MemReport { uint64 total_bytes = 1; uint64 used_bytes = 2; }
message PartitionReport { string device = 1; string mountpoint = 2; string fstype = 3; uint64 total_bytes = 4; uint64 used_bytes = 5; }
message InventoryReportResponse {}
```
Hand-written JSON-codec structs added to `server/internal/grpc/pb/vantage.pb.go` and `agent/internal/grpc/pb/vantage.pb.go`, plus the RPC method wiring (service interface, client method, handler registration) mirroring `ReportUpdates`.
---
## 5. Agent collection (`agent/internal/inventory/`)
- `Collect(includeStatic bool) *pb.InventoryReport` — reads:
- CPU usage: sample `/proc/stat` delta; load from `/proc/loadavg`; model/cores from `/proc/cpuinfo` (static).
- Memory/swap: `/proc/meminfo`.
- Partitions: `/proc/mounts` filtered to real filesystems + `statfs` for total/used (static).
- Kernel: `uname` / `/proc/version` (static).
- Windows: best-effort via `wmic`/PS or leave empty.
- Scheduler in the agent main loop: a 30s ticker calls `Collect(false)` and `ReportInventory`; every 30th tick (15 min) calls `Collect(true)`.
- Reuse existing gRPC client; add `Client.ReportInventory(...)` like `ReportUpdates`.
Prefer implementing the `/proc` readers directly (no new heavy deps) unless a `gopsutil` dependency is already vendored.
---
## 6. Server handler + service
- gRPC handler `ReportInventory` in `server/internal/grpc/server.go`: validate agent token (`ValidateAgentToken`), then call `services.StoreInventory(serverID, report)`.
- `services.StoreInventory` (in `server/internal/services/inventory.go`): builds the `$set` per the update rules in §3 and `UpdateOne` on `servers`.
---
## 7. Frontend
Surface inventory on the existing server detail page (`web/app/servers/[id]/page.tsx`) — add an "Inventory" panel:
- CPU usage gauge + model/cores, load.
- RAM used/total bar, swap bar.
- Partitions table: device, mount, fstype, used/total with a usage bar.
- "Updated Xs ago" from `metrics_at`/`static_at`.
Optionally add compact CPU/RAM badges to the servers list (`web/app/servers/page.tsx`). Reuse `@/components/ui` + Tailwind tokens. Poll the server detail query while the page is open (react-query `refetchInterval` ~30s) so metrics stay fresh.
---
## 8. Out of scope
- Time-series history / graphs (only latest snapshot stored).
- Alerting thresholds on usage (settings/alerts is a separate concern).
- Per-process / network / GPU inventory.
- Tests (skipped, consistent with the Workflows iteration).
@@ -1,142 +0,0 @@
# SaaS: Auth + Organizations — Design
**Date:** 2026-07-20
**Status:** Approved (design) — ready for implementation planning
**Scope:** Local auth + organizations + per-org OIDC, and org-scoping of existing data. Billing/plan-limits explicitly deferred. Fleet Inventory and Server Workflows are separate sub-projects.
---
## 1. Summary
Turn Vantage from a single-admin, single global-OIDC tool into a multi-tenant app:
1. **Replace** the global Authentik/env-based OIDC with **local email/password accounts** as the primary login.
2. **Organizations** — every user belongs to an org; every domain object (servers, keys, secrets, assignments, workflows, steps, runs, audit) carries an `org_id` and all queries are scoped to the caller's org.
3. **Per-org OpenID** — an org admin can configure their own OIDC provider (issuer/client id/secret); users in that org can then sign in through it.
No billing, no seat/server limits this iteration (schema leaves room).
---
## 2. Locked decisions
| Topic | Decision |
|-------|----------|
| Primary auth | Local email + password (bcrypt). Replaces global Authentik. |
| Org SSO | Per-org OIDC provider, configured by org admin, resolved dynamically at login. |
| Isolation | `org_id` on every collection; every service query filtered by org. Enforced in the request layer via session→org. |
| Roles | `owner`, `admin`, `member` (v1: owner/admin can manage users + org OIDC + all resources; member can use resources). Keep minimal. |
| Bootstrapping | First-run creates the initial org + owner account (setup flow) when no users exist. |
| Sessions | Keep existing Redis session store; session now carries `user_id`, `org_id`, `role`, `email`. |
| Agent auth | Unchanged (per-server agent tokens). Servers gain `org_id`; agent RPCs resolve org from the server record. |
---
## 3. Data model
### `orgs`
```json
{ "_id":"ObjectId", "org_id":"uuid", "name":"Acme", "created_at":"ISODate" }
```
### `users`
```json
{
"_id":"ObjectId", "user_id":"uuid", "org_id":"uuid",
"email":"a@b.com", "password_hash":"bcrypt...", "role":"owner|admin|member",
"auth_source":"local|oidc", "created_at":"ISODate", "last_login":"ISODate|null"
}
```
Unique index on `email` (global — email identifies the account and its org).
### `org_oidc` (per-org provider config)
```json
{
"_id":"ObjectId", "org_id":"uuid",
"issuer":"https://id.acme.com", "client_id":"...",
"client_secret_enc":"AES...", // encrypted with existing crypto.go
"redirect_url":"https://vantage.../auth/oidc/callback",
"enabled": true, "updated_at":"ISODate"
}
```
### Existing collections — add `org_id`
`servers`, `keys`, `assignments`, `secrets`, `workflows`, `workflow_steps`, `workflow_runs`, `audit` each gain `org_id string`. A **migration** backfills all existing documents into a default org (see §7).
---
## 4. Auth flows
### Local
- `POST /auth/register` — only allowed during first-run bootstrap (creates org + owner) OR by an org admin inviting a user (see below). Not open self-serve.
- `POST /auth/login` — email + password → verify bcrypt → create session with `{user_id, org_id, role, email}`.
- `POST /auth/logout` — destroy session.
- `GET /auth/me` — returns current user + org.
### Org-admin user management
- `GET /api/org/users` / `POST /api/org/users` (create local user in caller's org) / `PUT /api/org/users/:id/role` / `DELETE /api/org/users/:id`.
### Per-org OIDC
- `GET/PUT /api/org/oidc` — read/save the caller org's provider config (admin only). Secret stored encrypted.
- `GET /auth/oidc/start?org=<org_id or slug>` — look up org's `org_oidc`, build the OIDC provider on demand (cache per org), redirect to authorize.
- `GET /auth/oidc/callback` — exchange code, match/provision the user by email **within that org**, create session.
- If the email exists in the org → log in. If not → provision a `member` with `auth_source=oidc` (org admin can promote). Reject if email belongs to a different org.
### First-run bootstrap
- `GET /auth/bootstrap-status``{ needs_setup: bool }` (true when `users` is empty).
- Setup page collects org name + owner email/password → creates org + owner → session.
---
## 5. Request scoping
- `auth.Middleware` already loads the session; extend `Session` to include `OrgID`, `UserID`, `Role`. Add helper `auth.OrgID(c) string`.
- **Every service function that reads/writes a scoped collection takes an `orgID` argument** and adds `"org_id": orgID` to its filter and on insert. Handlers pass `auth.OrgID(c)`.
- Add a `requireRole(role)` gin middleware for admin-only routes (org user mgmt, org OIDC).
- Agent-facing gRPC: resolve `org_id` from the `servers` record (already tied to `server_id`); inventory/keys/sync operate on that org implicitly.
---
## 6. Removing global Authentik
- Delete/retire env-driven `InitOIDC` global provider (`OIDC_ISSUER` etc.). Keep the `go-oidc`/`oauth2` machinery but move it behind the per-org resolver.
- `authEnabled` global replaced by "auth always on" (there is always local auth). Update `middleware.go` accordingly (no more `if !authEnabled { next }` bypass — except the bootstrap endpoints and login/register which are unauthenticated).
- Login page (`web/app/login` or existing) offers: email/password form + "Sign in with your organization's SSO" (enter org, redirect to `/auth/oidc/start`).
---
## 7. Migration
One-shot migration run at startup (idempotent):
1. If `orgs` is empty AND `servers`/`keys`/etc. contain documents without `org_id`: create a **default org** ("Default").
2. Set `org_id = <default>` on all existing `servers`, `keys`, `assignments`, `secrets`, `workflows`, `workflow_steps`, `workflow_runs`, `audit` documents missing it.
3. If `OIDC_ISSUER` env was set previously and an admin email is known, optionally seed an owner user (documented manual step) — otherwise first-run bootstrap handles owner creation.
Guard with a marker (e.g. a `migrations` collection entry) so it runs once.
---
## 8. Frontend
- **Login/Setup:** `web/app/login/page.tsx` (email/password + org SSO entry) and `web/app/setup/page.tsx` (first-run). Redirect logic based on `bootstrap-status` and `auth/me`.
- **Org settings:** `web/app/settings/org/` — members list + invite/create user + role management; OIDC provider form (issuer/client id/secret/enabled).
- Existing pages unchanged functionally but now implicitly org-scoped by the backend. Show current org + user in the sidebar/header.
---
## 9. Security
- Passwords: bcrypt (cost ≥ 12). Never returned.
- Org OIDC client secret encrypted at rest (reuse `services/crypto.go` AES).
- Cross-org access prevented at the service layer (org_id in every filter) — the primary isolation boundary. Handlers must never accept an `org_id` from the client; always derive from session.
- OIDC callback must bind the returned identity to the org that initiated the flow (state carries org_id) to prevent org-mixing.
- Role checks on all org-admin mutations.
---
## 10. Out of scope
- Billing, plans, seat/server limits.
- Cross-org resource sharing, org switching for a single user (one user = one org in v1).
- SCIM / directory sync, SAML.
- Email delivery for invites (create-user sets a password or invite token; email sending deferred — document as manual/console output).
- Tests (skipped, consistent with prior iterations).
@@ -1,238 +0,0 @@
# Server Workflows — Design
**Date:** 2026-07-20
**Status:** Approved (design) — ready for implementation planning
**Scope:** Server Workflows only. Fleet Inventory and SaaS/local-auth are separate sub-projects with their own specs.
Approved UI mockup: three-pane builder (Step Library · Canvas · Inspector), env vars shown riding the wire between nodes.
---
## 1. Summary
Let operators compose **reusable shell steps** (Bash or PowerShell) into **workflows** and run them across many managed servers in parallel. Steps pass data to later steps through a `$WORKFLOW_ENV` file (GitHub-Actions style). Every run is recorded with full per-step logs. Steps can reference org secrets, injected as environment variables at runtime.
Builds directly on the existing `CommandStream` gRPC infrastructure (`dispatch.go`, `ServerCommand` oneof, agent command loop).
---
## 2. Locked decisions
| Topic | Decision |
|-------|----------|
| Data passing | Implicit. Every step's `$WORKFLOW_ENV` outputs merge into the run's env and are exposed to **all** later steps as `$KEY`. No explicit port wiring. |
| Failure model | Per-step policy: `stop` (default), `continue`, `retry` (with max attempt count). |
| Targets | Fan-out. Same step sequence runs on N target servers **in parallel**. Steps within one server run **sequentially**. |
| History/logs | Every run persisted: status, timing, per-server per-step stdout/stderr/exit code, captured output env. |
| Secrets | Steps declare needed secret keys; resolved from existing `secrets` store and injected as env vars at exec time. Never persisted into run logs. |
| Testing | **Skipped** for this iteration per request. No test files written. |
---
## 3. Data model (MongoDB)
### `workflow_steps` — reusable step library
```json
{
"_id": "ObjectId",
"step_id": "uuid",
"name": "Restart service",
"description": "Restart-Service by name, wait ready",
"interpreter": "bash | powershell",
"script": "Restart-Service vantage-api\n...",
"declared_outputs": ["STARTED_AT"], // documentation/UI hints; not enforced
"secret_refs": ["DEPLOY_TOKEN"], // secret keys this step needs injected
"org_id": "uuid", // for future multi-tenant; single-org for now
"created_at": "ISODate",
"updated_at": "ISODate"
}
```
### `workflows` — ordered composition
```json
{
"_id": "ObjectId",
"workflow_id": "uuid",
"name": "Deploy & Restart API",
"target_server_ids": ["uuid", "uuid"],
"steps": [
{
"step_id": "uuid", // reference to library step
"order": 0,
"on_failure": "stop | continue | retry",
"max_retries": 0, // used when on_failure = retry
"overrides": { // optional local fork of the library step
"script": null,
"secret_refs": null
}
}
],
"created_at": "ISODate",
"updated_at": "ISODate"
}
```
Editing a library step from the Inspector writes an `overrides` block on that workflow step (a local fork) rather than mutating the shared step.
### `workflow_runs` — execution records
```json
{
"_id": "ObjectId",
"run_id": "uuid",
"workflow_id": "uuid",
"workflow_snapshot": { }, // frozen copy of workflow + resolved steps at trigger time
"status": "running | success | failed | cancelled",
"triggered_by": "user-id",
"started_at": "ISODate",
"finished_at": "ISODate | null",
"server_runs": [
{
"server_id": "uuid",
"status": "queued | running | success | failed | skipped",
"started_at": "ISODate | null",
"finished_at": "ISODate | null",
"run_env": { "VERSION": "a1b9f0" }, // accumulated non-secret output env
"steps": [
{
"order": 0,
"name": "Git pull & build",
"status": "success | failed | running | queued | skipped",
"attempts": 1,
"exit_code": 0,
"stdout": "…",
"stderr": "…",
"output_env": { "VERSION": "a1b9f0" },
"started_at": "ISODate",
"finished_at": "ISODate"
}
]
}
]
}
```
Secret values are never written to `stdout`/`stderr`/`run_env` by us; masking of known secret values in captured output is applied before persistence.
---
## 4. gRPC protocol changes (`proto/vantage/v1/vantage.proto`)
### New command in the `ServerCommand` oneof
```protobuf
message RunStepCmd {
string interpreter = 1; // "bash" | "powershell"
string script = 2;
map<string, string> env = 3; // inputs = accumulated run env + injected secrets
int32 timeout_seconds = 4;
}
```
Add `RunStepCmd run_step = 6;` to the `ServerCommand` oneof.
### Richer result — new `AgentMessage` payload
Current `CommandResult{command_id, success, message}` is too thin. Add a dedicated step result:
```protobuf
message StepResult {
string command_id = 1;
int32 exit_code = 2;
string stdout = 3;
string stderr = 4;
map<string, string> output_env = 5; // parsed $WORKFLOW_ENV KEY=value lines
}
```
Add `StepResult step_result = 5;` to the `AgentMessage` oneof (alongside existing `ready` / `result`).
---
## 5. Agent execution (`agent/internal/...`)
New handler for `RunStepCmd` in the agent command loop:
1. Create a temp dir; create empty `WORKFLOW_ENV` file inside it.
2. Write `script` to a temp script file.
3. Build the process environment: inherited env + `cmd.env` (run env + secrets) + `WORKFLOW_ENV=<path to env file>`.
4. Execute:
- `bash``bash <script>`
- `powershell``pwsh -NoProfile -File <script>` (fallback `powershell.exe` on Windows if `pwsh` absent).
5. Capture stdout, stderr, exit code. Enforce `timeout_seconds` (kill on exceed → non-zero exit, stderr note).
6. Parse the `WORKFLOW_ENV` file: each `KEY=value` line becomes an `output_env` entry (last write wins; supports multi-line via simple `KEY<<EOF` heredoc form, optional for v1 — start with single-line `KEY=value`).
7. Reply with `StepResult`. Delete temp dir.
Agent runs as root (existing), so no privilege change. Script content is trusted operator input.
---
## 6. Server orchestration (`server/internal/services/workflows.go`)
Runner responsibilities:
1. On trigger: snapshot the workflow (resolve each library step + overrides), create a `workflow_runs` doc with one `server_run` per target, all `queued`.
2. Spawn one goroutine **per target server** (parallel fan-out). Each goroutine:
- Verifies the agent is connected (`Dispatcher.IsConnected`); if not → `server_run.status = skipped`, reason recorded.
- Maintains a `run_env map[string]string`, seeded empty.
- For each step in order:
- Resolve `secret_refs` from the secrets service → merge into the command env (kept separate from persisted `run_env`).
- Dispatch `RunStepCmd{env: run_env + secrets}` via a **correlated** send — needs a way to await the matching `StepResult` by `command_id` (see §7).
- On result: persist step record (stdout/stderr/exit, masked); merge `output_env` into `run_env`.
- Apply `on_failure` on non-zero exit: `stop` (fail server_run, break), `continue` (mark failed, proceed), `retry` (re-dispatch up to `max_retries`).
3. Aggregate: run `status = success` if all server_runs succeeded, else `failed`. Set `finished_at`.
### Concurrency / queue
- One workflow run per workflow at a time (reject or queue concurrent triggers — v1: reject with clear error).
- Per-server step dispatch is serial; servers are parallel.
---
## 7. Correlated command results
The existing dispatcher is fire-and-forget; workflows need request/response by `command_id`. Add a small **pending-result registry** alongside `Dispatcher`:
- `AwaitResult(commandID) <-chan *pb.StepResult` — registers a channel before dispatch.
- The `CommandStream` receive loop, on a `StepResult`, looks up the pending channel by `command_id` and delivers it (falls back to existing `CommandResult` handling for other command types).
- Timeout guard on the server side (step `timeout_seconds` + grace) so a dead agent can't hang a run.
This is additive; existing `CommandResult` flow for key/update commands is unchanged.
---
## 8. REST API (`server/internal/api/workflows.go`)
| Method + path | Purpose |
|---------------|---------|
| `GET /api/steps` / `POST` / `PUT /:id` / `DELETE /:id` | Reusable step library CRUD |
| `GET /api/workflows` / `POST` / `PUT /:id` / `DELETE /:id` | Workflow CRUD (name, targets, ordered steps) |
| `POST /api/workflows/:id/run` | Trigger a run; returns `run_id` |
| `GET /api/workflows/:id/runs` | Run history (summary list) |
| `GET /api/runs/:run_id` | Full run detail incl. per-server per-step logs |
| `POST /api/runs/:run_id/cancel` | Best-effort cancel |
Secrets are referenced by key only through these APIs; values never returned.
---
## 9. Frontend (`web/app/workflows/`)
- `/workflows` — list workflows, last run status/time, Run button.
- `/workflows/[id]` — the three-pane builder from the approved mockup:
- **Library** (left): reusable steps, `bash`/`pwsh` badges, search, add.
- **Canvas** (center): ordered nodes, env chips on wires, live status pills.
- **Inspector** (right): name, command editor, declared inputs/outputs, `secret_refs` picker, `on_failure` + retry count.
- `/workflows/[id]/runs/[runId]` — run detail: per-server columns, expandable per-step stdout/stderr, exit codes, timing. Live-updating while `running` (poll, consistent with existing 30s-poll ethos — or reuse whatever the console screen uses).
Reuse existing web components/styling patterns (there is already `servers`, `secrets`, `audit`, console UI to match).
---
## 10. Security notes
- Scripts are trusted operator input executed as root — same trust level as the existing console feature. No new sandbox in v1.
- Secret values injected as env only; masked from all persisted logs (`stdout`/`stderr`/`run_env`) by literal replacement before write.
- Run triggering and step/workflow CRUD gated behind existing auth (`server/internal/auth`).
- Audit: emit audit-log entries (existing `audit` service) on workflow create/edit/delete and run trigger.
---
## 11. Out of scope (this iteration)
- Tests (explicitly skipped).
- Branching/conditional steps, matrix per-server conditionals (fan-out only).
- Scheduled/cron triggers (manual run only for v1).
- Multi-org isolation enforcement (schema carries `org_id` for later; single-org behavior now).
- Artifact upload/collection beyond env vars.
@@ -1,193 +0,0 @@
# Workflow Log Streaming — Design
**Date:** 2026-07-20
**Status:** Approved (design) — ready for implementation planning
**Scope:** Stream step stdout/stderr live from agent to server-side log files, tail them live in the UI, and auto-expire them on a retention period. Enhancement to the already-merged Server Workflows feature. No auth/orgs, no inventory.
---
## 1. Summary
Today a workflow step buffers all stdout/stderr in agent RAM, ships it in one terminal `StepResult`, and the server persists the whole body into the `workflow_runs` Mongo document. Long/chatty steps risk: agent memory blow-up, the gRPC 4MB message ceiling, and the Mongo 16MB document cap.
Change to **live streaming**:
1. Agent streams output chunks over the existing `CommandStream` as the process runs.
2. Server appends chunks (secret-masked) to a **per-server-run log file** on disk — not Mongo.
3. UI tails the file live via **SSE** while a server-run is running; slices per-step by byte offset after completion.
4. A **retention sweeper** deletes old run-log directories on a configurable period (default 30 days, set in Settings).
`workflow_runs` documents shrink: they no longer carry `stdout`/`stderr` bodies, only status/exit/attempts/output_env/timestamps plus a per-step `log_offset`.
---
## 2. Locked decisions
| Topic | Decision |
|-------|----------|
| Transport | Reuse bidirectional `CommandStream`. New `AgentMessage.StepOutput` chunk message. |
| Chunk shape | `{command_id, seq, data, eof}`. Interleaved stdout+stderr in execution order. |
| Terminal result | `StepResult` still sent at step end, now carries only `exit_code` + `output_env` (no stdout/stderr). |
| Log granularity | **One file per server-run**: `<logdir>/<run_id>/<server_id>.log`, with a marker line before each step. |
| Streams | **Interleaved** — single synchronized writer on the agent, terminal-order output. |
| Masking | **Server-side** (agent can't tell secret env from normal env). Per-stream carry buffer of `maxSecretLen-1` bytes so a secret split across a chunk boundary still masks; flushed on EOF. |
| Live tail | **SSE** at per-server-run granularity: `GET /api/runs/:runId/servers/:serverId/logs/stream`. Post-run whole-file fetch + per-step offset slice. |
| Retention | `settings.workflow_log_retention_days`, default **30**, editable in `/settings`. Hourly sweeper deletes `<logdir>/<run_id>/` dirs older than retention by run `finished_at`. |
| Log dir | Env `VANTAGE_WORKFLOW_LOG_DIR`, default `<data>/workflow-logs`. Created `0700`. |
| Mongo | No log bodies in `workflow_runs`. Disk is the source of truth for output. |
---
## 3. gRPC protocol (`proto/vantage/v1/vantage.proto` + both `pb.go` files)
Add to the `AgentMessage` oneof: `StepOutputChunk step_output = 6;`
```protobuf
message StepOutputChunk {
string command_id = 1;
uint64 seq = 2; // monotonic per command_id, 0-based
bytes data = 3; // raw interleaved stdout+stderr bytes
bool eof = 4; // true on the final (empty) chunk
}
```
`StepResult` is unchanged in shape but `stdout`/`stderr` are now left empty by the agent (kept in the message for backward-compat / error notes only — server ignores them for log content). The server still reads `exit_code` and `output_env` from `StepResult`.
Hand-written JSON-codec struct added to **both** `server/internal/grpc/pb/vantage.pb.go` and `agent/internal/grpc/pb/vantage.pb.go`, identical. `AgentMessage` gains `StepOutput *StepOutputChunk` in both.
`data` is `[]byte` in the Go structs (JSON-codec base64-encodes it, which is fine).
---
## 4. Agent (`agent/internal/exec/exec.go`)
`RunStep` signature gains a chunk sink:
```go
func RunStep(cmd *pb.RunStepCmd, emit func(seq uint64, data []byte)) *pb.StepResult
```
- Replace the two `bytes.Buffer`s with a single `streamWriter` set as **both** `c.Stdout` and `c.Stderr`. Its `Write` takes a mutex (so stdout+stderr interleave without interleaving *within* a write), assigns the next `seq`, and calls `emit(seq, copyOfBytes)`. Chunks are whatever the OS pipe delivers (typically ≤64KB); no extra buffering/line-assembly.
- `StepResult` returns with `Stdout`/`Stderr` empty; `ExitCode` and `OutputEnv` populated as today (env parsing unchanged).
- On timeout/exec error, put the short note in `StepResult.Stderr` (terminal, not streamed) so the runner can still surface a failure reason even if nothing streamed.
Agent loop (`agent/internal/sync/sync.go`, the `cmd.RunStep != nil` goroutine): pass an `emit` closure that sends `AgentMessage{ServerId, AgentToken, StepOutput: &pb.StepOutputChunk{CommandId, Seq, Data}}` through the existing mutex-guarded `send()`. After `RunStep` returns, send a final `StepOutput{eof:true, seq:last+1}` then the terminal `StepResult` (both via `send()`). Ordering: all chunks, then eof, then StepResult.
---
## 5. Server write path
### 5.1 Log writer registry (`server/internal/services/steplogs.go`)
Parallel to `StepResults`. Keyed by `command_id`:
```go
type stepLogWriter struct {
f *os.File
mu sync.Mutex
carry []byte // held-back tail for boundary-safe masking
secrets []string // secret literals to mask
maxSecret int
}
var StepLogs = &stepLogRegistry{ ... }
func (r *stepLogRegistry) Open(commandID, path string, secrets []string) (*stepLogWriter, error)
func (r *stepLogRegistry) Append(commandID string, data []byte) // masked write
func (r *stepLogRegistry) Close(commandID string) // flush carry, close file
```
- `Append` masking: concatenate `carry+data`, mask all secret literals (`ReplaceAll(v,"***")`), then write everything except the last `maxSecret-1` bytes; keep those as the new `carry`. `Close` masks+writes the remaining carry. If `secrets` empty, write straight through (no carry).
- The file handle is opened append-only (`O_APPEND|O_CREATE|O_WRONLY`, `0600`); dir `0700`.
### 5.2 Stream delivery (`server/internal/grpc/server.go`)
In the receive loop, after the `m.StepResult` block, add:
```go
if m.StepOutput != nil {
if m.StepOutput.Eof {
services.StepLogs.Close(m.StepOutput.CommandId)
} else {
services.StepLogs.Append(m.StepOutput.CommandId, m.StepOutput.Data)
}
}
```
### 5.3 Runner changes (`server/internal/services/workflow_runner.go`)
- Resolve the log dir + run/server file path once per server-run; ensure `<logdir>/<run_id>/` exists.
- Before dispatching each step: write the step marker line to the file (`\n===== step <order>: <name> =====\n`), record the current file byte offset as the step's `log_offset` (persisted on the `StepRun`), and `StepLogs.Open(commandID, path, secretVals)` **before** `DispatchRunStep` (same ordering rule as `StepResults.Await`).
- `dispatchAndWait` no longer expects stdout/stderr in the result. On terminal `StepResult`, `StepLogs.Close(commandID)` is driven by the agent's eof; the runner also calls `Close` defensively on timeout/dispatch-failure (idempotent).
- **Drop** `stdout`/`stderr` from `finishStep` persistence. Masking of the streamed body is done in `Append`; `run_env`/`output_env` masking (existing, from the merged fix) stays.
- The shared server-run file is written by two writers that never overlap in time (steps are serial, and the runner writes each step marker *before* `StepLogs.Open`): (a) the runner writes markers directly to the path, serially between steps; (b) `StepLogs` writes chunks during a step. **Resolved approach:** `Open(commandID, path, secrets)` opens the path fresh with `O_APPEND|O_CREATE|O_WRONLY` for that step and `Close(commandID)` closes it on eof. One handle live at a time per server-run (serial steps guarantee this), so there is no shared-handle race and no ref-counting. The runner's marker write is a separate short `O_APPEND` open/write/close on the same path.
### 5.4 Data model (`server/internal/models/workflow.go`)
`StepRun`:
- **Remove** `Stdout`, `Stderr` string fields.
- **Add** `LogOffset int64 `bson:"log_offset" json:"log_offset"`` — byte offset in the server-run file where this step's marker begins.
`ServerRun` gains nothing structural (its file path is derivable: `<logdir>/<run_id>/<server_id>.log`).
---
## 6. REST API (`server/internal/api/workflows.go`)
- `GET /api/runs/:runId/servers/:serverId/logs` — returns the whole server-run log file (`text/plain`). 404 if absent. Used post-run and as SSE fallback.
- `GET /api/runs/:runId/servers/:serverId/logs/stream`**SSE**. Opens the file, streams existing content as `data:` events, then polls for appends (~500ms) emitting new bytes, until the server-run status is terminal (success/failed/skipped/cancelled) AND no more bytes, then sends a final `event: done` and closes. Sets `Content-Type: text/event-stream`, disables gin's buffering. Guards against path traversal (runId/serverId are used as literal path segments — validate they are UUIDs / contain no separators).
Log content served by these endpoints is already masked (masking happens at write time), so no masking needed on read.
---
## 7. Retention
### 7.1 Setting
`settings` collection gains `workflow_log_retention_days int` (default 30 when unset). Read/write via the existing settings service + surfaced in `/settings` UI as a number input. `0` or negative disables sweeping (keep forever) — document this.
### 7.2 Sweeper (`server/internal/services/steplogs.go` or `logsweeper.go`)
- `StartLogSweeper()` launched at server startup (next to index setup): hourly `time.Ticker`.
- Each tick: read retention setting; if ≤0 skip. Compute cutoff = `now - retentionDays`. For each `<logdir>/<run_id>/` dir, look up the run's `finished_at` (query `workflow_runs` by run_id); if finished and older than cutoff, `os.RemoveAll` the dir. Fallback to dir mtime if the run doc is gone.
- Also run once at startup.
---
## 8. Frontend
### 8.1 API client (`web/lib/api.ts`)
- `StepRun`: remove `stdout`/`stderr`; add `log_offset: number`.
- Add `getServerRunLog(runId, serverId): Promise<string>` (GET .../logs).
- SSE consumed directly via `EventSource` in the component (not through the `request` helper), URL built from the same base.
- Settings type gains `workflow_log_retention_days`.
### 8.2 Run detail (`web/app/workflows/[id]/runs/[runId]/page.tsx`)
- Per-server card: while the server-run is `running`, open an `EventSource` to the stream endpoint and render a live `<pre>` terminal that appends incoming chunks (auto-scroll). Close the source on `event: done`, unmount, or terminal status.
- After completion: fetch the whole file once and render it; step `<details>` still list status/exit/attempts pills. (Per-step slicing by `log_offset` is optional polish — v1 may show the whole server log under the card and keep step pills as the status summary.)
- Remove reliance on `st.stdout`/`st.stderr` (fields gone).
### 8.3 Settings (`web/app/settings/page.tsx`)
- Add a "Workflow log retention (days)" number input bound to `workflow_log_retention_days`, saved via the existing settings mutation. Note that `0` = keep forever.
---
## 9. Security
- Secret masking moves to the streaming write path but remains server-side and boundary-safe (carry buffer). Same `***` replacement.
- Log files `0600`, dirs `0700`, under a dedicated log dir.
- SSE/read endpoints validate `runId`/`serverId` as UUID-shaped path segments to prevent traversal; they are session-authed (same `apiGroup`).
- Terminal `StepResult.Stderr` (error notes only) is still masked before any persistence (it is no longer persisted as log body; if surfaced, mask against secretVals).
---
## 10. Out of scope
- Per-step (rather than per-server) live SSE channels.
- Log compression / rotation within a run, remote log storage (S3), download-as-zip.
- Full-text search over logs.
- Backfilling/migrating already-existing `workflow_runs` stdout/stderr into files (pre-existing runs keep whatever they had; new field just won't be set — acceptable, feature is new).
- Tests (skipped, consistent with the Workflows iteration).
```
@@ -0,0 +1,410 @@
# Spec 3 — Admin Backend
Date: 2026-07-24
Status: Design approved, not implemented
Depends on: spec 0a, spec 0b, spec 1 (`licensing-core`)
Ships: with spec 4 (the site it serves). Blocks specs 4 and 5.
## Context
A fourth Go service, `admin/`, owning customers, instances, licences and
subscriptions. It is the only service that holds the signing key.
Its data model separates two things the control plane deliberately does not know
about:
- **Account** — a paying customer. Holds a Paddle customer, a billing email, and
one or more instances.
- **Instance** — one deployment. Cloud instances mirror a control-plane
`Instance` row; self-hosted instances exist only here, because the customer's
database is theirs and we cannot see it.
## Goals
1. Issue, store and re-issue licences, with full history.
2. Inject licences into cloud instances.
3. Serve both staff and customers, with the right things hidden from each.
4. Never be a runtime dependency of a Vantage instance. If admin is down,
every instance keeps working; only purchasing and renewals stop.
## Non-goals
- The UI. Spec 4.
- Paddle. Spec 5. This spec defines the `subscriptions` table and the issuance
functions that spec 5's webhooks call, and nothing more.
- Rebuilding billing management. Card changes, invoices and cancellation go to
Paddle's own customer portal.
## Design
### Module
```
admin/
├── go.mod # replace => ../shared
├── cmd/main.go
└── internal/
├── api/ # gin handlers
├── auth/ # staff, cloud-customer and local-customer sessions
├── db/ # two connections: admin DB and control-plane DB
├── models/ # admin-owned documents
├── licensing/ # issuance, renewal, relink
├── inject/ # control-plane writes
├── mail/ # licence delivery
└── paddle/ # spec 5 lands here
```
Port `8083`. In `deploy/docker-compose.site.yml` only — like sitesvc, admin is
**excluded from the self-hosted deployment**. A self-hosted customer runs
instances, not the licensing authority.
### Two database connections
The service holds two:
- `ADMIN_MONGO_URI` — its own database, `vantage_admin`. Sole owner.
- `CONTROL_MONGO_URI` — the control plane's database, used to write licence
fields onto cloud instance documents and to authenticate cloud customers.
The control-plane connection uses the `Instance` and `User` structs from
`shared` (spec 0a). This is what makes direct writes safe: there is no admin-side
copy of the document shape to drift, which is the coupling hazard sitesvc used to
carry.
Admin's control-plane access is **narrow by construction**: it reads
`instances` and `users`, and it writes exactly three fields on `instances`. The
Mongo credential it is given should be scoped to that where the deployment allows
it. It must never write to any other collection.
### Data model
```go
type Account struct {
ID bson.ObjectID
AccountID string // uuid
Name string
BillingEmail string
PaddleCustomerID string // empty until first checkout
Status string // active | suspended
CreatedAt time.Time
}
type Instance struct {
ID bson.ObjectID
InstanceID string // for cloud: equals the control-plane instance_id
// for self-hosted: the UUID the customer pasted
AccountID string
Name string
Slug string // cloud only; the subdomain label
Deployment string // cloud | self_hosted
Tier string
Status string // awaiting_link | active | lapsed | cancelled
CurrentLicense string // licence ID
RelinkCount int // reset each term
CreatedAt time.Time
}
type License struct {
ID bson.ObjectID
LicenseID string
InstanceID string
AccountID string
Tier string
Deployment string
Limits license.Limits // snapshot
Features []string // snapshot
IssuedAt time.Time
ExpiresAt time.Time
Blob string
SupersededBy string // licence ID, when replaced
IssuedBy string // staff user, "system", or "paddle:<event id>"
Reason string // new | renewal | tier_change | relink | manual
}
type Subscription struct {
ID bson.ObjectID
SubscriptionID string
AccountID string
InstanceID string
PaddleSubscriptionID string
PaddlePriceID string
Tier string
Term string // monthly | annual
Status string // active | past_due | cancelled | awaiting_link
CurrentPeriodEnd time.Time
}
type Plan struct {
Tier string
Name string
Deployment string
Limits license.Limits
Features []string
PaddleProductID string
PaddlePriceIDs map[string]string // "monthly" | "annual"
Active bool
}
```
Collections: `accounts`, `admin_instances`, `licenses`, `subscriptions`,
`plans`, `staff_users`, `customer_users`, `admin_audit`.
Unique indexes: `accounts.account_id`, `admin_instances.instance_id`,
`licenses.license_id`, `subscriptions.paddle_subscription_id`, `plans.tier`,
`staff_users.email`, `customer_users.email`.
`admin_instances.instance_id` unique is load-bearing: it is what stops the same
self-hosted UUID being linked to two accounts.
**Licences are append-only.** A renewal writes a new row and sets
`SupersededBy` on the old one. Nothing is ever edited or deleted. When a support
question arrives about why a customer's instance stopped working on a given
date, the answer is in the table.
`plans` holds tier contents so they change without a deploy, seeded from the
table in spec 1. Every issued licence snapshots the plan, so editing a plan never
changes an existing licence — the same rule as `workflow_runs.steps_snapshot`.
### Issuance
```go
func Issue(ctx, instanceID, tier, term, reason, issuedBy string) (*models.License, error)
```
1. Load the instance and its account.
2. Load the plan for `tier`; refuse if `plan.Deployment != instance.Deployment`.
**This is the check that makes Free cloud-only** — Free's plan is
`deployment: cloud`, so it can never be issued to a self-hosted instance.
3. Build the payload with the instance's UUID bound in, `ExpiresAt` from the term
plus a **3-day grace** so a renewal webhook arriving slightly late does not
create a gap.
4. Sign with `LICENSE_SIGNING_KEY`.
5. Insert the licence row; set `SupersededBy` on the previous one; update
`instance.CurrentLicense` and `instance.Tier`.
6. If cloud, inject. If self-hosted, email the blob and make it downloadable.
7. Write an `admin_audit` entry.
Steps 5 and 6 are not transactional. Order matters: **record first, deliver
second.** A licence recorded but not delivered is recoverable — the customer
downloads it. A licence delivered but not recorded is a support mystery.
### Free tier rule
One Free instance per account, enforced in `Issue`: refuse a second Free instance
for an account that already has one that is not `cancelled`. Additional
instances must be paid.
### Injection
```go
func InjectCloud(ctx, instanceID string, lic *models.License) error
```
Writes `license_blob`, `license_tier`, `license_expiry` onto the control-plane
`instances` document via a single `UpdateOne`. Idempotent, retryable, and safe to
re-run.
Retries three times with backoff; on final failure the licence stays recorded and
`instance.Status` is set to `active` regardless, with the failure logged and
surfaced as a staff alert. A **reconciliation job runs every 15 minutes**,
comparing each cloud instance's `CurrentLicense` against the blob actually stored
in the control plane, and re-injecting on mismatch. That job, not the webhook, is
what guarantees eventual consistency.
The control-plane instance caches licence state for 60 seconds (spec 2), so an
injection takes effect within a minute without a restart.
### Self-hosted linking
The flow, end to end:
```
Customer runs /setup on their own install → instance UUID generated and shown
Customer buys Self Hosted in the admin site → subscription created,
status awaiting_link
Customer pastes the UUID into the admin site → admin_instances row created,
status active
Admin issues the licence with that UUID bound in
Customer downloads the .lic file or copies the blob
Customer pastes it into /settings/license on their install
```
Validation on link: the UUID must parse as a UUID, must not already exist in
`admin_instances`, and must not collide with a cloud instance ID. A duplicate
returns "That instance ID is already linked to an account" without revealing
which — it is a small enumeration surface but there is no reason to leave it
open.
### Relink
A rebuilt server has a new UUID. `POST /api/instances/:id/relink` with the new
UUID:
- Allowed **3 times per term**, `RelinkCount` reset on renewal.
- Updates `admin_instances.instance_id`, issues a replacement licence for the
**remaining term** with `reason: relink`, supersedes the old one.
- The old licence is not revoked — it cannot be, offline verification has no
revocation. It simply no longer matches any UUID the customer controls, and its
binding stops it being useful on a different machine anyway.
- Beyond 3, the endpoint returns a message directing the customer to support, and
staff can relink without limit.
`RelinkCount` is the abuse signal, not the abuse prevention. Its real job is to
put a human in front of the fourth attempt.
### Authentication
Three identities, three paths, one session store (Redis, `admin_session`
cookie, 24h).
**Staff**`staff_users`, local email plus bcrypt. Full access. Created by CLI
only; there is no staff signup.
**Cloud customers** — authenticate against the control plane's `users`
collection with the credentials they already use. Admin looks the user up
by email, checks bcrypt, resolves their control-plane instance, then resolves the
account that owns it.
Two consequences, stated plainly because they are real:
1. A cloud user's control-plane password now also unlocks billing. Any password
change or compromise has a wider blast radius than before.
2. Only users with control-plane role `owner` may sign in to the admin site.
`admin` and `member` are refused. Billing is an owner concern.
Mitigations: rate-limit to 5 attempts per email per 15 minutes and 20 per IP per
hour; log every attempt to `admin_audit`; return an identical error for unknown
email and wrong password.
**Self-hosted customers**`customer_users`, local email plus bcrypt at cost 12,
created during purchase, scoped to one account. Email verification reuses the
pattern sitesvc already proved: 32 random bytes, only the SHA-256 hash stored,
24-hour expiry, TTL index.
A single email address could in principle be both a cloud user and a
self-hosted customer user. `customer_users` is checked first; if it matches, that
identity wins. Documented so the behaviour is chosen rather than emergent.
### API
Staff:
```
GET /api/staff/accounts list, search
POST /api/staff/accounts
GET /api/staff/accounts/:id
GET /api/staff/instances filter by account, deployment, status, expiry
POST /api/staff/instances/:id/issue manual issue or reissue
POST /api/staff/instances/:id/relink no limit
GET /api/staff/licenses full history, filterable
GET /api/staff/plans
PUT /api/staff/plans/:tier
GET /api/staff/audit
GET /api/staff/health/injection reconciliation status and failures
```
Customer:
```
GET /api/account own account and instances
POST /api/instances/link self-hosted UUID link
POST /api/instances/:id/relink rate-limited
GET /api/instances/:id/license current licence metadata
GET /api/instances/:id/license/download .lic file
GET /api/subscriptions status, next renewal
POST /api/billing/portal Paddle portal redirect (spec 5)
```
Every customer handler resolves the account from the session and scopes by it.
The scoping is enforced by a helper every handler calls, not by each handler
remembering — the same deny-by-default reasoning as spec 2's middleware.
### Configuration
| Variable | Required | Notes |
|---|---|---|
| `ADMIN_MONGO_URI` | yes | admin's own database; name read from the URI path, refused if absent |
| `CONTROL_MONGO_URI` | yes | control-plane database, for injection and cloud auth |
| `REDIS_ADDR` | yes | sessions |
| `LICENSE_SIGNING_KEY` | yes | ECDSA P-384 private key, base32 (lk PrivateKey.ToB32String). **Boot fails without it** — a licensing service that cannot sign is worse than one that is down, because it looks healthy |
| `PUBLIC_URL` | yes | for verification and licence links |
| `SMTP_*` | yes | licence delivery |
| `ADMIN_ORIGIN` | yes | CORS allow-list |
| `TRUST_PROXY` | no | only behind a proxy that overwrites `X-Forwarded-For` |
| Paddle variables | spec 5 | |
### Backfill
Licences issued by `lkctl` during the spec 12 period exist only as blobs.
A one-shot `admin backfill --from=blobs.json` parses each with
`license.Parse`, creates the account, instance and licence rows, and marks them
`reason: manual`. Run once when admin goes live.
## Testing
**Issuance:**
1. `Issue` produces a licence that `license.Verify` accepts for that instance.
2. Deployment mismatch (Free plan, self-hosted instance) is refused.
3. A second Free instance for the same account is refused; a third paid one is
allowed.
4. Renewal supersedes the previous licence and leaves it in the table.
5. The issued licence snapshots the plan; editing the plan afterwards does not
change the issued licence.
6. Grace period: `ExpiresAt` is term end plus 3 days.
**Injection:**
7. `InjectCloud` writes all three fields; the control plane then reports `valid`.
8. Injection is idempotent across two calls.
9. Injection failure leaves the licence recorded and flags the instance.
10. The reconciliation job detects a control-plane blob that does not match
`CurrentLicense` and re-injects.
**Linking and relink:**
11. Linking an unknown UUID succeeds; linking one already linked is refused.
12. Relink issues a licence for the *remaining* term, not a fresh full term.
13. The fourth relink in a term is refused for a customer and allowed for staff.
14. `RelinkCount` resets on renewal.
**Auth:**
15. Cloud owner signs in with control-plane credentials; `admin` and `member`
roles are refused.
16. Unknown email and wrong password return identical errors and timing is not a
meaningful oracle.
17. Rate limits trigger at the documented thresholds.
18. Self-hosted customer cannot sign in before verifying their email.
19. A customer requesting another account's instance gets `404`, not `403`
no existence disclosure.
**Scoping:**
20. Every customer endpoint, called with a session for account A against a
resource of account B, returns `404`. Written as a table-driven test over the
route list so a new endpoint that forgets to scope fails the build.
## Verification before merge
1. Full suite green, including the scoping table test (test 20).
2. End to end, cloud: create account → create instance → issue Professional →
confirm the control-plane instance reports `valid` within 60 seconds with no
restart.
3. End to end, self-hosted: run `/setup` on a scratch install, copy the UUID,
link it, issue, download, paste, confirm `valid`.
4. Confirm admin's control-plane credential cannot write to `servers`, `keys` or
any collection other than `instances`.
5. Kill the admin service and confirm every Vantage instance keeps working
entirely normally.
## Risks
| Risk | Mitigation |
|---|---|
| Admin becomes a runtime dependency | Verification step 5; instances verify offline and never call admin |
| Signing key exposure | Single service, single variable, never in an image; rotation path from spec 1 |
| Cloud password now unlocks billing | Owner-only, rate-limited, audited, and stated in the release notes |
| Injection silently fails | Reconciliation every 15 minutes plus a staff health endpoint |
| Admin writes outside its remit in the control plane | Narrow code path; scoped Mongo credential; reviewed on every change |
| Self-hosted UUID squatted by another account | Unique index plus a non-disclosing error |
@@ -0,0 +1,203 @@
# Spec 4 — Admin Site
Date: 2026-07-24
Status: Design approved, not implemented
Depends on: spec 3 (`admin-backend`)
Ships: with spec 3. Can be developed in parallel with spec 5 once spec 3's API
is stable.
## Context
A fifth Next.js app, `adminsite/`, serving two audiences from one codebase:
- **Staff** — internal operators. Accounts, instances, licence history, plan
editing, injection health, audit.
- **Customers** — their own account, instances, licences and subscription state.
They share auth plumbing and a component library but almost no screens. The
split is by route group, so a customer route can never accidentally render a
staff view.
## Goals
1. A customer can buy, link a self-hosted instance, download a licence, and see
when it expires — without contacting anyone.
2. Staff can answer "why did this customer's instance stop working" in one screen.
3. Nothing about the marketing site or the control-plane UI changes.
## Non-goals
- Rebuilding billing management. Card details, invoices, payment methods and
cancellation all deep-link into Paddle's customer portal.
- Server management. This is not a second control plane; there is exactly one
link out to the instance and no data about servers, keys or workflows.
- Public signup for cloud. That stays on the marketing site (moving to admin's
backend in spec 5, but the *form* stays where customers already find it).
## Design
### App
Built exactly like `web/` and `site/`: Next.js 16 App Router, React 18, Tailwind
3, TanStack Query, `output: "standalone"`, `node:26-alpine`, listening on `3000`,
published as `3002`. In `docker-compose.site.yml` only.
`ADMIN_API_URL` is baked in at build time, as `API_URL` is for `web/`. It must be
**browser-reachable** and must appear in the backend's `ADMIN_ORIGIN`. Getting
this wrong is the single most common deployment failure in this repo's history —
`SITE_API_URL` has the same footgun documented in `CLAUDE.md` — so the app
renders an explicit "not connected" state rather than failing silently.
```
adminsite/
├── app/
│ ├── login/
│ ├── signup/ # self-hosted customer account creation
│ ├── verify/
│ ├── (customer)/
│ │ ├── page.tsx # account overview
│ │ ├── instances/[id]/
│ │ ├── instances/link/
│ │ ├── billing/
│ │ └── layout.tsx # customer nav, account guard
│ └── (staff)/staff/
│ ├── page.tsx # operations dashboard
│ ├── accounts/[id]/
│ ├── instances/[id]/
│ ├── licenses/
│ ├── plans/
│ └── layout.tsx # staff nav, staff guard
├── components/
└── lib/
```
Route-group layouts do the guarding. A customer session hitting `/staff/*` gets
redirected, not a 403 page — there is nothing to tell them about.
### Customer screens
**Overview** — the account, its instances as cards. Each card: name, cloud or
self-hosted, tier, licence state, expiry with days remaining, and a link either
to the instance's subdomain (cloud) or to its licence page (self-hosted).
Licence state is colour-coded and blunt: green valid, amber under 14 days, red
expired. An expired card says what still works — "servers and monitors are still
running; changes are disabled" — because that is the first thing a worried
customer wants to know.
**Instance detail** — tier, limits, features, subscription status, next renewal
date. For self-hosted: the linked UUID, a **Download licence** button, the blob
in a copy-to-clipboard box, and step-by-step paste instructions with the target
route named (`Settings → Licence` on their own install). A **Relink** action
showing the remaining allowance ("2 of 3 relinks remaining this term").
**Link an instance** — the self-hosted activation screen. Explains where to find
the UUID (shown on `/setup`, and permanently on `/settings/license`), takes the
paste, validates the format client-side, and on success issues the licence and
lands the customer directly on the download.
The whole flow — buy, link, download, paste — should be completable without
reading documentation. That is the bar for this screen.
**Billing** — subscription list with status and renewal date, plus a button to
Paddle's portal. Deliberately thin.
### Staff screens
**Dashboard** — the operational answers, not vanity metrics: licences expiring
in the next 14 days, subscriptions `past_due`, instances `awaiting_link` for more
than 48 hours, and **failed injections** from the reconciliation job. Each row
links straight to the thing that needs doing.
**Accounts** — searchable by name, email, Paddle customer ID and instance UUID.
Searching by UUID matters: a support email arrives containing a UUID and nothing
else.
**Account detail** — instances, subscriptions, customer users, audit trail.
**Instance detail** — everything about one instance, with the **full licence
history as a timeline**: issued, superseded, renewed, relinked, each with a
timestamp, reason and who did it. This is the screen that answers "why did this
stop working on the 14th". Actions: issue, reissue, relink without limit, and a
live view of the control-plane injection state for cloud instances.
**Licences** — global history, filterable by tier, deployment, expiry window and
issuance reason.
**Plans** — edit limits and features per tier. Two guard rails, because this
screen changes what every future customer gets:
- A confirmation step naming exactly what changes and stating that existing
licences are unaffected until reissued.
- The deployment field is not editable. Moving Free to `self_hosted` would break
the cloud-only rule that spec 1 leans on; changing it is a code review, not a
form field.
**Audit** — every mutating action, filterable.
### Design language
Visually distinct from `web/`. Staff regularly have both open, and a moment of
"which app am I in" before clicking Reissue is worth designing out. Different
accent colour and a persistent environment badge in the header (sandbox or
production, from a build-time flag) — clicking Issue against the wrong Paddle
environment should be hard.
Shared component patterns with `web/` where they exist; this is not a reason to
invent a second design system.
### Error and empty states
- Backend unreachable: a page-level "not connected" state naming
`ADMIN_API_URL`, matching the pattern the marketing site already uses.
- No instances yet: a customer-facing explanation of the two paths — buy cloud,
or buy self-hosted and link.
- `awaiting_link`: a prominent prompt on the overview, since a customer who has
paid and not linked is a customer who has paid for nothing yet.
- Licence download failure: show the blob inline as a fallback so the customer is
never blocked by a file download.
## Testing
Component and integration tests with mocked API responses. The repo has no
frontend test setup today; this is where one starts, scoped to the flows that
lose money or leak data when broken.
1. Customer session on `/staff/*` redirects; staff session reaches it.
2. Instance card renders correctly for each licence state, including expired,
and the expired copy names what still works.
3. Link flow: valid UUID succeeds and lands on download; malformed UUID is caught
client-side; already-linked UUID surfaces the backend's message.
4. Relink shows the remaining allowance and disables at zero with the support
message.
5. Licence download failure falls back to the inline blob.
6. Not-connected state renders when the API is unreachable.
7. Staff dashboard renders each alert category and links to the right resource.
8. Plan edit requires confirmation and shows the "existing licences unaffected"
wording.
9. Instance search by UUID returns the instance.
10. Licence history timeline renders every reason type in order.
## Verification before merge
1. Test suite green.
2. Full manual pass, self-hosted purchase to working licence, using only the UI
and no documentation — timed, and if it takes more than five minutes the flow
needs work.
3. Full manual pass, cloud: buy, confirm the licence appears in the control plane
within a minute, confirm the instance's own settings page agrees.
4. Staff pass: find an account by instance UUID, read its licence history,
reissue, confirm the control plane picks it up.
5. Responsive check at mobile width — a customer hit by an expiry email will open
this on a phone.
6. `docker build` from the repo root succeeds and the image runs with
`ADMIN_API_URL` baked in.
## Risks
| Risk | Mitigation |
|---|---|
| `ADMIN_API_URL` misconfigured at build | Explicit not-connected state; documented alongside the existing `SITE_API_URL` footgun |
| Staff action taken against the wrong environment | Persistent environment badge; confirmation on destructive actions |
| Customer confused by the self-hosted flow | Step-by-step link screen; five-minute bar in verification |
| Customer session reaching staff data | Route-group guards plus backend scoping (spec 3, test 20). Two layers, because one is not enough for this |
@@ -0,0 +1,353 @@
# Spec 2 — Instance Licensing and Enforcement
Date: 2026-07-24
Status: Design approved, not implemented
Depends on: spec 0a, spec 0b, spec 1 (`licensing-core`)
Ships: independently, with licenses issued by hand via `lkctl`. No admin site
needed.
## Context
Spec 1 defines what a license is. This spec makes the control plane hold one,
act on it, and let a self-hosted operator paste one in.
The guiding rule: **an expired license must never break a running fleet.** Agents
keep their keys, monitors keep watching, alerts keep firing. What stops is
growth and change. A customer whose card fails should be inconvenienced, not
paged at 3am because their monitoring went dark when Vantage decided to sulk.
## Goals
1. A license lives on the instance document and is verified on read.
2. Enforcement is deny-by-default: a new mutating route is gated because of where
it is mounted, not because someone remembered.
3. Degraded mode is obvious in the UI and reversible by pasting a valid license.
4. Self-hosted operators get an instance UUID they can hand to the admin site.
## Non-goals
- Issuing licenses. `lkctl` (spec 1) or the admin backend (spec 3).
- Any outbound network call. Verification is offline, permanently.
- Per-user or per-role licensing. The unit is the instance.
## Design
### Instance identity
Every install already has an `Instance` document with an `InstanceID` UUID. For
cloud instances this is created by signup; for self-hosted it is created by
`/setup`.
Change to `/setup`: after bootstrapping the first instance and its owner, the
setup page **displays the instance UUID** with a copy button and the text that
it is needed to activate a license. It is also shown permanently on
`/settings/license`.
No new identifier is invented. The instance UUID is the licensing identity.
### Storage
`shared/models.Instance` gains:
```go
LicenseBlob string `bson:"license_blob,omitempty" json:"-"`
LicenseTier string `bson:"license_tier,omitempty" json:"license_tier,omitempty"`
LicenseExpiry *time.Time `bson:"license_expiry,omitempty" json:"license_expiry,omitempty"`
```
The blob is authoritative. `LicenseTier` and `LicenseExpiry` are a denormalised
cache for listing and for the admin site's queries, rewritten from the verified
payload every time a blob is accepted. Nothing reads them for enforcement.
`LicenseBlob` is `json:"-"`. It is not a secret in the confidentiality sense —
it is signed public data — but there is no reason to spray it through API
responses.
### Runtime state
```go
type State struct {
Status license.State // valid | expired | invalid
Reason string
Tier string
ExpiresAt *time.Time
Limits license.Limits
Features map[string]bool
}
```
Resolved by `services.LicenseState(instanceID) State`, cached for 60 seconds
alongside the existing instance cache and invalidated immediately when a blob is
stored.
Three inputs, in precedence order:
1. `Instance.LicenseBlob`.
2. `VANTAGE_LICENSE` environment variable, used **only when the instance has no
stored blob**. This lets an automated self-hosted deployment ship a license
without a human pasting one. A blob stored through the UI always wins
afterwards, so an operator is never locked out by a stale environment value.
3. Neither → `Status: invalid`, `Reason: no_license`.
The verifier is called with `InstanceID` from the instance document and
`Deployment` from `VANTAGE_DEPLOYMENT` (`cloud` on our infrastructure,
`self_hosted` everywhere else, defaulting to `self_hosted`). The default matters:
an operator who removes the variable gets the stricter mode, not the looser one.
`invalid` and `expired` degrade identically. They differ only in the message.
### Enforcement
Three layers, deliberately separate because they answer different questions.
**Layer 1 — mutation gate.** A gin middleware `RequireActiveLicense` mounted on
the `/api` group, applying to every request whose method is not `GET` or `HEAD`.
```go
api := r.Group("/api", auth.RequireSession(), services.RequireActiveLicense())
```
Non-`valid``403 {"error":"license_required","state":"expired","reason":"..."}`.
Mounting at the group means **a route added tomorrow is gated by default**. That
is the whole point of putting it here rather than on individual handlers.
Explicit exemptions, allow-listed by path because they must work in degraded
mode:
| Route | Why |
|---|---|
| `POST /api/license` | Pasting a valid license is how you recover |
| `POST /auth/*` | Login and logout are outside `/api` already; listed for clarity |
| `DELETE` on any resource | Deleting is how you get back under a limit |
| `POST /api/servers/:id/apply-updates` | Security patching must never be paywalled |
The `DELETE` exemption deserves emphasis: a customer downgraded to Free with 10
servers must be able to remove 7 of them. Blocking deletes would trap them.
**Layer 2 — feature gate.** `RequireFeature(name)` on the route groups that need
it:
- `console``POST /api/console/connect`, `GET /api/console/tunnel`
- `oidc``GET,PUT /api/instance/oidc`
Missing feature → `403 {"error":"feature_unavailable","feature":"console"}`.
OIDC needs care: `/auth/oidc/start` and `/auth/oidc/callback` are unauthenticated
and outside `/api`. They check the feature directly and, if unavailable, redirect
to `/login?error=oidc_unavailable` rather than returning JSON. **Existing OIDC
sessions are not terminated** — losing the feature stops new SSO logins, it does
not evict people mid-session.
**Layer 3 — limits.** Enforced in the service layer, because a limit needs a
count that middleware does not have:
| Limit | Checked in |
|---|---|
| `max_servers` | `services.CreateServer` / `POST /api/servers/new` |
| `max_secret_groups` | `services.CreateSecretGroup` |
| `max_channels` | `services.CreateChannel` |
`-1` means unlimited. Over limit → `403 {"error":"limit_exceeded","limit":"max_servers","current":3,"max":3}`.
Counts are of live rows: revoked assignments and deleted servers do not count.
**Over-limit instances are never truncated.** A Professional instance with 20
servers that lapses to Free keeps all 20 running; it simply cannot add a 21st.
Deleting resources is always permitted. Silently disabling a customer's servers
because their card expired is not a behaviour this system will have.
### Background work in degraded mode
This is where "read-only" needs to be specific, because these paths do not go
through gin at all.
| Subsystem | Degraded behaviour |
|---|---|
| **Monitor scheduler** | **Keeps running.** Checks execute, incidents open, notifications fire. |
| Monitor create/edit/delete | Blocked by layer 1 (delete exempted). |
| Workflow runner | New runs blocked by layer 1. **In-flight runs finish** rather than being killed mid-step — a half-run workflow is worse than a completed one. |
| Agent `SyncKeys` | Returns the existing desired key set unchanged. Nothing is torn off disk. New assignments cannot be created, so nothing changes anyway. |
| Agent registration | A **new** agent registering against an over-limit instance is refused with a clear message; existing agents re-register freely. |
| Inventory, heartbeat, update reporting | Unaffected. |
| `ApplyUpdatesCmd` | Allowed. Security patching is not gated. |
| ESO secrets read (`GET /api/secrets/:group/values`) | **Allowed.** It is a `GET`, and breaking a Kubernetes cluster's secret sync over a billing state is disproportionate. |
| Log retention sweep, offline sweep | Unaffected. |
Keeping monitors alive is a deliberate reversal of a stricter earlier draft. It
is the single most important line in this spec: **billing state must not take
away a customer's ability to know their infrastructure is on fire.**
### API
```
GET /api/license any authenticated user
POST /api/license owner only
```
`GET` returns:
```json
{
"instance_id": "…",
"state": "valid",
"reason": "",
"tier": "professional",
"expires_at": "2027-07-24T00:00:00Z",
"days_remaining": 365,
"limits": { "max_servers": -1, "max_secret_groups": -1, "max_channels": -1 },
"features": { "console": true, "oidc": true },
"usage": { "servers": 12, "secret_groups": 4, "channels": 2 },
"source": "stored"
}
```
`usage` is included so the UI can render "12 of 3 servers" honestly when an
instance is over its limit, rather than pretending.
`POST` takes `{"blob": "..."}`, verifies with the instance's own ID and
deployment mode, and on success stores the blob, refreshes the cache, and writes
an audit event. On failure it returns `400` with the specific reason:
| Reason | Message |
|---|---|
| `bad_signature` | This licence key is not valid. Check it was copied in full. |
| `deployment_mismatch` | This licence is for Vantage Cloud and cannot be used on a self-hosted install. |
| `instance_mismatch` | This licence was issued for a different instance. Your instance ID is `<uuid>`. |
| `expired` | This licence expired on `<date>`. |
An **expired** blob is still stored if it is otherwise valid, so the UI can show
what expired and when. An **invalid** blob is rejected and the previous one kept.
Rate-limited to 10 attempts per instance per hour. There is no oracle here worth
protecting, but an unbounded verify endpoint is an unbounded CPU endpoint.
### Frontend
`useLicense()` hook over `GET /api/license`, cached by TanStack Query and
invalidated after a successful paste.
- **Banner, persistent, top of every page** when `state != valid`:
- `expired` — "Your Vantage licence expired on `<date>`. Your servers and
monitors are still running, but changes are disabled until it is renewed."
with a link to the admin site.
- `invalid` / `no_license` — "This instance has no valid licence. Add one in
Settings → Licence."
- **Warning banner** in the final 14 days of a valid term, dismissible per
session.
- **Gated features render disabled with an upgrade tooltip, not hidden.** A
customer cannot buy what they cannot see, and a feature that vanishes reads as
a bug.
- **Limit indicators** on the servers, secrets and channels list pages: "3 of 3
servers used" with the create button disabled at the cap.
- `/settings/license`: current state, tier, expiry, limits with live usage, the
instance UUID with a copy button, and a textarea plus file upload for a new
blob. Owner-only; other roles see the state read-only.
### Grandfathering existing tenants
Migration `0005_grandfather_licenses`, cloud only, guarded on
`VANTAGE_DEPLOYMENT == "cloud"`:
For every instance with no `license_blob`, issue a Professional license expiring
**one year** from the migration date and store it.
The migration cannot sign — the server has no private key and, per spec 1, no
signing code. So the blobs are **generated ahead of time with `lkctl`** and
supplied to the migration through `VANTAGE_GRANDFATHER_BLOBS`, a JSON map of
instance ID to blob. The migration stores what it is given, verifies each blob
against its instance before storing, and logs any instance it had no blob for.
Clumsy, and correct. The alternative is putting a signing key in the control
plane, which is the thing this design most wants to avoid.
Self-hosted installs are not grandfathered. On upgrade they land in `no_license`
and read-only until an operator pastes a key — which is the intended behaviour
for a paid product, and is why the release notes must lead with it.
## Testing
**Unit, no database:**
1. `State` resolution precedence: stored blob wins over `VANTAGE_LICENSE`;
environment used when no blob; neither → `no_license`.
2. Feature map construction from the payload's `Features` slice.
3. Limit comparison with `-1`, with zero, and with a count exactly at the cap.
**Middleware, with a stub state:**
4. `GET` passes in every state.
5. `POST`/`PUT`/`DELETE` pass when `valid`, fail `403` when `expired` and when
`invalid` — except `DELETE`, which passes in all states.
6. `POST /api/license` passes when `expired` (the recovery path).
7. `POST /api/servers/:id/apply-updates` passes when `expired`.
8. `RequireFeature("console")` passes with the feature, `403`s without it.
9. **Coverage test:** enumerate every registered route and assert that every
non-`GET` route is either behind `RequireActiveLicense` or on the exemption
allow-list. This test is what stops layer 1 rotting as routes are added.
**Service layer, against MongoDB:**
10. `CreateServer` at the cap → `limit_exceeded`; one below → succeeds.
11. Over-limit instance can still `DELETE` a server, and can create again once
back under the cap.
12. Deleted and revoked rows do not count toward limits.
**Degraded background behaviour:**
13. Monitor scheduler executes checks for an instance with an expired license.
14. An incident opened during degraded mode still dispatches notifications.
15. `SyncKeys` for an expired instance returns the same key set as before expiry.
16. A new agent registering against an over-limit instance is refused; an
existing agent re-registers successfully.
17. A workflow run in flight when the license expires completes its remaining
steps.
**API:**
18. `POST /api/license` with a valid blob stores it and flips state to `valid`.
19. Each rejection reason returns its own message and leaves the stored blob
untouched.
20. An expired-but-well-formed blob is stored and reported as `expired`.
21. Non-owner `POST``403`.
**Migration:**
22. `0005` stores and verifies supplied blobs, skips instances that already have
one, logs instances with no blob supplied, and is a no-op when
`VANTAGE_DEPLOYMENT != "cloud"`.
## Verification before merge
1. Full test suite green, including the route-coverage test (test 9).
2. Manual pass on a scratch instance: issue a Professional license with `lkctl`,
paste it, confirm everything works. Issue one expiring in 60 seconds, wait,
confirm the banner appears, mutations `403`, **monitors keep firing**, and
pasting a fresh license restores normal operation without a restart.
3. Manual pass on the Free tier: confirm the 3-server cap, that console and OIDC
are visibly disabled with upgrade tooltips, and that a 4th server is refused
with a clear message.
4. Confirm a cloud-issued Free license is rejected on a `self_hosted` install
with `deployment_mismatch`.
5. Confirm a license issued for another instance is rejected with
`instance_mismatch` and the message shows the correct local UUID.
## Rollout
1. Generate grandfather blobs with `lkctl` for every existing cloud instance.
2. Deploy with `VANTAGE_GRANDFATHER_BLOBS` set; migration `0005` runs.
3. Verify every cloud instance reports `valid`, Professional, one year out.
4. Unset the variable on the next deploy — it is single-use.
5. Release notes for self-hosted must state plainly that upgrading requires a
licence key, and how to get one.
## Risks
| Risk | Mitigation |
|---|---|
| A mutating route added later without a gate | Route-coverage test (test 9) fails the build |
| Customer locked out and unable to recover | `POST /api/license` and all `DELETE`s exempt from the gate |
| Existing cloud tenants degrade on deploy | Migration 0005, verified before the traffic switch |
| Over-limit customer trapped | Deletes always allowed; existing resources never truncated |
| Clock wrong on a self-hosted host | `Verify` warns on a future `IssuedAt`; documented in the licence settings page |
| Monitoring lost on billing failure | Explicitly designed out — the scheduler ignores licence state |
@@ -0,0 +1,240 @@
# Spec 0b — Org to Instance Rename
Date: 2026-07-24
Status: Design approved, not implemented
Depends on: spec 0a (`shared-module`)
Ships: independently, before any licensing code
## Context
The licensing model separates two concepts that the codebase currently conflates
under one word:
- **Account** — a paying customer. Lives only in the admin control plane
(spec 3). The control plane never learns about it.
- **Instance** — one deployment of Vantage: its own subdomain, its own users,
its own servers, keys, workflows, monitors and secrets. One license attaches
to one instance.
Today's control-plane `Org` **is** an Instance. An Account may hold several,
some cloud and some self-hosted, and the self-hosted ones have no row in the
cloud database at all.
Keeping the name `Org` would leave the control plane using a word that means
something different in the admin site, in Paddle, and in every support
conversation. This spec renames it everywhere, including on disk.
This is the highest-risk change in the programme: `org_id` is the tenant
isolation key on every document in every collection. It is done alone, before
anything else, so that nothing else is in flight when it deploys.
## Goals
1. `Instance` is the only word for a tenant, in code, API, UI and database.
2. No document is lost and no tenant scoping is weakened.
3. The migration is reversible.
## Non-goals
- Any behaviour change. Same routes' semantics, same permissions, same data.
- Introducing Accounts. The control plane never gets them.
- Touching the agent. It talks gRPC and has no concept of a tenant.
## Design
### Naming map
| Today | After |
|---|---|
| collection `orgs` | `instances` |
| collection `org_oidc` | `instance_oidc` |
| field `org_id` (all collections) | `instance_id` |
| `models.Org` | `models.Instance` |
| `Org.OrgID` | `Instance.InstanceID` |
| `User.OrgID`, `Settings.OrgID`, every `OrgID` field | `InstanceID` |
| `services/orgs.go`, `GetOrg`, `CreateOrg`, `ListOrgIDs`, `CountOrgs`, `FirstOrg`, `AdoptOrg`, `GetOrgBySlug` | `services/instances.go`, `GetInstance`, `CreateInstance`, … |
| `services/org_oidc.go` | `services/instance_oidc.go` |
| `auth/orghost.go` | `auth/instancehost.go` |
| `/api/org/users`, `/api/org/oidc` | `/api/instance/users`, `/api/instance/oidc` |
| `shared/provision.CreateOrg`, `RollbackOrg` | `CreateInstance`, `RollbackInstance` |
| session field `org_id` | `instance_id` |
| `GET /auth/me` response `org_id` / `org` | `instance_id` / `instance` |
| UI copy "Organisation" | "Instance" |
Reserved slugs gain no new entries here, but note `admin` is already reserved,
which the admin site relies on later.
### Collections carrying `org_id`
All of: `servers`, `keys`, `assignments`, `users`, `org_oidc`, `settings`,
`secrets`, `workflows`, `workflow_steps`, `workflow_runs`, `monitors`,
`incidents`, `monitor_rollups`, `notification_channels`, `console_sessions`,
`audit_logs`, plus `orgs` itself. `migrations` does not carry one.
`site_pending_signups` does not carry `org_id`, but its `org_name` field becomes
`instance_name` for consistency; it is sitesvc-private so this is free.
The migration must derive this list from a constant in code, not from a
hand-written list in a runbook, so that a collection added between design and
deploy is not silently missed:
```go
var scopedCollections = []string{ /* the list above */ }
```
A boot-time assertion (spec 2 onwards) checks that no collection outside this
list contains an `org_id` field. Cheap insurance against a future collection
being added without being renamed.
### Migration `0004_org_to_instance`
Recorded in `migrations` like the existing three. Runs after
`0003_missed_org_scopes`.
**The migration only renames. It never deletes and never drops.** A bad deploy
is recovered by running the inverse rename, not by restoring a backup.
Steps, in order:
1. **Guard.** If collection `instances` already exists and `orgs` does not, the
migration has already run against this database by an earlier binary; record
the marker and return. Idempotency matters because the marker write and the
data work are not in one transaction.
2. **Rename collections.** `orgs``instances`, `org_oidc``instance_oidc`,
via `adminCommand{renameCollection}`. Fails loudly if the target exists.
3. **Rename the field.** For each collection in `scopedCollections`:
`UpdateMany({org_id: {$exists: true}}, {$rename: {"org_id": "instance_id"}})`.
Record `matched` and `modified` per collection in the log.
4. **Verify.** For each collection, assert
`CountDocuments({org_id: {$exists: true}}) == 0` and
`CountDocuments({instance_id: {$exists: true}}) == totalCount`. Any mismatch
aborts before the marker is written, leaving the migration to retry.
5. **Indexes.** Drop and recreate indexes that name `org_id` in their key spec:
unique `settings.instance_id`, the ESO token-hash index, and any compound
scoping indexes. Unique `instances.slug` and `users.email` are unaffected by
the field rename but are re-declared idempotently.
6. **Write the marker.**
Steps 24 are not atomic across collections. Mongo multi-document transactions
would require a replica set, which is not guaranteed for self-hosted installs.
Instead the migration is written to be **safely re-runnable**: `$rename` on a
document that has already been renamed matches nothing, and the collection
rename is guarded in step 1.
Rollback, if ever needed, is the same code with the rename reversed, shipped as
a one-shot command rather than a migration — deliberately manual, because the
only reason to run it is a decision to revert the release.
### Version skew
`sitesvc` and `server` write the same documents. A skew where one writes
`org_id` and the other reads `instance_id` creates tenants that are invisible to
the application — the exact failure `CLAUDE.md` warns about.
After spec 0a both read the shape from `shared`, so the skew window is a
deployment-ordering problem rather than a code-drift problem:
- Both images are built from the same commit and deployed together.
- The migration runs from the `server` container at boot, as the existing three
do.
- `sitesvc` at boot asserts that collection `instances` exists and refuses to
start otherwise, with the message
`instances collection not found; deploy the control plane first`. Failing to
start is strictly better than provisioning into a collection nobody reads.
The self-hosted deployment runs no sitesvc, so it sees only the server change.
### API and frontend
REST route renames are **breaking**, but every consumer is first-party (`web/`)
and ships in the same release. No compatibility aliases — a permanent dual path
in the tenant-scoping layer is worse than a coordinated release.
`web/` changes: the API client's paths, the `useMe` shape, all UI copy from
"Organisation" to "Instance", and the settings route `/settings/org`
`/settings/instance`.
`site/` marketing copy changes where it says "organisation" about a tenant. Where
it means the customer, it becomes "account" — that word now has a specific
meaning and the marketing site is the first place a customer meets it.
## Testing
**No automated tests.** Decision taken 2026-07-24, consistent with spec 0a.
This is the change where that costs the most: it moves the tenant isolation key
across 17 collections, and a mistake orphans a customer's entire fleet rather
than breaking a build. The compensating controls are therefore not optional, and
the implementation plan makes each a mandatory step:
1. **Dry run against a restored copy** before the code is even committed —
migrate a `mongorestore`d duplicate of production and read the per-collection
rename counts.
2. **Idempotency by hand** — run the dry run twice; the second must complete
with no error and nothing left to rename.
3. **Interrupted-run recovery by hand** — rename `orgs` manually, then run the
migration; it must complete and leave every document carrying `instance_id`.
4. **Count comparison against a production snapshot** — record every
collection's document count before and after; any difference stops the
release.
5. **Per-tenant isolation comparison** — for three real tenants, count rows in
`servers`, `keys`, `workflows`, `monitors`, `secrets` and `audit_logs` by
`org_id` before and by `instance_id` after. Identical, or the release stops.
This is the check that proves tenant isolation survived.
6. **Stale-field sweep** — assert no collection anywhere still holds an
`org_id`.
7. **Rollback rehearsal** — migrate a third copy, run `rename-rollback`, confirm
the counts return to baseline and the pre-release binary boots against it.
Deploying without having done this is not permitted.
8. **Boot guard, both directions** — sitesvc must refuse an unmigrated database
and start normally against a migrated one.
`AssertNoScopedCollectionMissed` runs at every boot and is fatal. With no test
suite it is the standing protection against a future collection being added
without being added to `ScopedCollections`.
## Verification before merge
Run against a **restored production snapshot**, not a synthetic database:
1. Record `db.getCollectionNames()` and per-collection `countDocuments()` before.
2. Run the migration.
3. Assert every count is identical afterwards.
4. Assert `instances.countDocuments()` equals the old `orgs.countDocuments()`.
5. Pick three real tenants; run the same scoped query before (by `org_id`) and
after (by `instance_id`) and confirm identical result sets. This is the test
that proves tenant isolation survived.
6. Boot the server against the migrated snapshot; log in as a real user; confirm
servers, keys, workflows, monitors and secrets all list correctly.
7. Boot sitesvc against the migrated snapshot; complete a signup end to end.
8. Boot sitesvc against an **un**migrated snapshot; confirm it refuses to start
with the expected message.
## Rollout
1. Take a database backup. Not optional — this is the one change where the
inverse rename is the recovery path and the backup is the second.
2. Deploy `server`, `web`, `site` and `sitesvc` from one commit, together.
3. Server boots, migration runs, marker recorded.
4. Watch for the sitesvc guard message; if it appears, sitesvc started first and
will restart cleanly.
Expect a short window during the server restart where the API is unavailable.
Agents are unaffected: they reconnect, and no gRPC message carries a tenant ID.
## Risks
| Risk | Mitigation |
|---|---|
| Partial migration leaves mixed field names | Step 4 verification aborts before the marker; migration is re-runnable |
| A collection missed from the list | List is a code constant plus a completeness test plus a boot-time assertion |
| sitesvc deployed before server | Boot guard refuses to start |
| An index still keyed on `org_id` | Step 5 drops and recreates; verification includes an index listing diff |
| A hard-coded `org_id` string outside the model layer | `grep -rn '"org_id"' server/ sitesvc/ shared/` must return only the migration file after the change |
| Frontend missed a renamed route | Full manual pass over every route in the UI before release |
## Follow-on
With `Instance` established, spec 1 (`licensing-core`) can define a license
payload that binds to `instance_id` without inventing a word the codebase does
not use.
@@ -0,0 +1,294 @@
# Spec 1 — Licensing Core
Date: 2026-07-24
Status: Design approved, not implemented
Depends on: spec 0a (`shared-module`), spec 0b (`instance-rename`)
Ships: independently. Adds a package and a CLI; changes no running behaviour.
## Context
Licenses are **offline-verified signed blobs**. A Vantage server checks a
signature and an expiry date and asks nobody's permission. That choice buys
self-hosted installs that work in air-gapped networks and a control plane with no
licensing availability dependency.
It costs revocation. Once issued, a license is valid until it expires, whatever
Paddle later says. Every other decision in the programme follows from accepting
that: Self Hosted is annual-only so the unenforceable window is bounded, and
cancellation takes effect at term end rather than immediately (spec 5).
This spec defines the payload, the signing and verification, and a CLI to issue
licenses by hand. It deliberately lands before the admin site so that specs 1+2
together give working licensing with no new service to operate.
## Goals
1. One struct, in `shared`, read identically by the verifier and the issuer.
2. Verification that needs no network, no clock sync beyond a rough one, and no
configuration.
3. A hand-issuance path good enough to run production on until spec 3 lands.
## Non-goals
- Storing licenses. Spec 2 owns the instance document; spec 3 owns issuance
history.
- Deciding tier contents. Tiers are data; the values in this spec are the
initial seed, and spec 3's `plans` table becomes their home.
- Any phone-home, revocation list or online check. There is none, anywhere, by
design.
## Design
### Package
`shared/license/`, inside the module created by spec 0a:
```
shared/license/
├── license.go # License, Limits, feature constants
├── sign.go # Sign, build-tagged out of the server binary
├── verify.go # Verify, Parse
├── keys.go # trustedPublicKeys
└── license_test.go
```
Uses `github.com/hyperboloide/lk` (ECDSA P-384 with SHA-256, base32 encoding).
### Payload
```go
package license
type License struct {
ID string `json:"id"` // uuid, for support and audit
InstanceID string `json:"instance_id"` // the instance this license is bound to
AccountID string `json:"account_id"` // admin-side customer, informational
InstanceName string `json:"instance_name"` // display only
Tier string `json:"tier"` // "free" | "professional" | "self_hosted"
Deployment string `json:"deployment"` // "cloud" | "self_hosted"
IssuedAt time.Time `json:"issued_at"`
ExpiresAt time.Time `json:"expires_at"`
Limits Limits `json:"limits"`
Features []string `json:"features"`
}
type Limits struct {
MaxServers int `json:"max_servers"` // -1 means unlimited
MaxSecretGroups int `json:"max_secret_groups"`
MaxChannels int `json:"max_channels"`
}
const (
FeatureConsole = "console" // browser SSH/RDP/VNC
FeatureOIDC = "oidc" // per-instance single sign-on
)
const (
TierFree = "free"
TierProfessional = "professional"
TierSelfHosted = "self_hosted"
DeploymentCloud = "cloud"
DeploymentSelfHosted = "self_hosted"
)
```
`InstanceID` is **always populated**. There is no unbound license: the
self-hosted purchase flow (spec 4) links the instance UUID before the license is
issued, so binding happens at signing time. This removes the claim endpoint, the
best-effort phone-home and the multi-claim reconciliation that an unbound design
would have needed.
**The server never branches on `Tier`.** It reads `Limits` and `Features` only.
`Tier` exists for display, support and analytics. Adding a tier, or changing what
a tier includes, must never require a server release.
### Tier seed values
Recorded here as the initial contents of spec 3's `plans` table. Snapshotted into
each license at issue, so changing the table never rewrites an issued license —
the same principle as `workflow_runs.steps_snapshot`.
| | Free | Professional | Self Hosted |
|---|---|---|---|
| `deployment` | `cloud` | `cloud` | `self_hosted` |
| `max_servers` | 3 | -1 | -1 |
| `max_secret_groups` | 1 | -1 | -1 |
| `max_channels` | 1 | -1 | -1 |
| `console` | no | yes | yes |
| `oidc` | no | yes | yes |
| billing term | monthly, £0 | monthly or annual | **annual only** |
Free is cloud-only. A self-hosted install can never hold a valid Free license
because Free is only ever signed with `deployment: "cloud"`, and verification
rejects a deployment mismatch. There is no server-side flag to edit.
### Signing
```go
//go:build !noSign
func Sign(l License, privateKeyHex string) (string, error)
```
Marshals to canonical JSON, signs with lk, returns the base32 blob.
`Sign` is excluded from the server binary with a build tag. The server has no
reason to hold signing code and there is no reason to ship it into a customer's
data centre.
The private key lives in `LICENSE_SIGNING_KEY` on the issuing side only — the
CLI now, the admin backend from spec 3. It is never in the repo, never in an
image, never in the control plane's environment.
### Verification
```go
type VerifyOpts struct {
InstanceID string // required: the verifier's own instance
Deployment string // required: "cloud" or "self_hosted"
Now time.Time // injectable for tests
}
type Result struct {
License License
State State // Valid, Expired, Invalid
Reason string
}
const (
StateValid State = "valid"
StateExpired State = "expired"
StateInvalid State = "invalid"
)
func Verify(blob string, opts VerifyOpts) Result
```
Checks, in order, stopping at the first failure:
1. Blob decodes and the signature verifies against one of `trustedPublicKeys`.
Failure → `Invalid`, reason `bad_signature`.
2. `l.Deployment == opts.Deployment`. Failure → `Invalid`, reason
`deployment_mismatch`. This is the check that makes Free cloud-only.
3. `l.InstanceID == opts.InstanceID`. Failure → `Invalid`, reason
`instance_mismatch`.
4. `opts.Now.Before(l.ExpiresAt)`. Failure → `Expired`.
5. Otherwise `Valid`.
**`Expired` and `Invalid` are distinct states and the caller treats them
differently in messaging** (spec 2), even though both degrade the instance the
same way. A customer whose card failed and a customer who pasted the wrong blob
need different words.
`Parse(blob) (License, error)` verifies the signature only, ignoring binding and
expiry. Used by the admin site to display a license and by support to inspect a
blob a customer has emailed in. Never used for enforcement.
Clock skew: no tolerance is applied. Terms are a month or a year; a server whose
clock is wrong by enough to matter has bigger problems, and a tolerance window is
a thing to get wrong. `Verify` logs at warn level if `IssuedAt` is in the future,
which is the signal that a clock is badly off.
### Key management
```go
// trustedPublicKeys is ordered. Index 0 is the current signing key.
// To rotate: prepend the new key, ship a server release, then reissue.
// Remove a retired key only after every license signed with it has expired.
var trustedPublicKeys = []string{
"<base32 ECDSA P-384 public key>",
}
```
A slice from day one even though it holds one entry, because retrofitting a
single-key verifier into a multi-key one during an incident is not a thing to
plan for.
Public keys are compiled in. They are not configurable, because a configurable
trust root is a licensing bypass: a self-hosted operator could point it at a
keypair they generated.
Key generation is a documented one-off:
```
go run ./shared/license/cmd/lkgen keypair
```
prints a private key (base32) for the vault and a public key (base32) to paste into
`keys.go`. The private key is stored in a password manager and in the admin
service's environment. **If it is lost, no new licenses can be issued for any
existing customer without a server release.** Back it up in two places.
### CLI issuer
`shared/license/cmd/lkctl`, built only for internal use:
```
lkctl keypair
lkctl issue --instance-id=<uuid> --instance-name="Acme" \
--tier=professional --deployment=cloud \
--term=1y [--account-id=<id>] [--out=acme.lic]
lkctl inspect <file-or-blob>
```
`issue` reads `LICENSE_SIGNING_KEY`, applies the tier seed values from a table
compiled into the CLI, and prints the blob. `--term` accepts `1m`, `1y` or an
explicit `--expires=RFC3339`.
This is the production issuance path until spec 3 ships. It is kept afterwards
for support and disaster recovery — if the admin service is down and a customer's
license expires, a blob can still be cut by hand.
Issued blobs from `lkctl` are not recorded anywhere. Spec 3 backfills its
`licenses` table from `inspect` output when it takes over.
## Testing
`shared/license` is pure and needs no database, so this suite is fast and
thorough. Written test-first.
1. Round trip: `Sign` then `Verify` returns `Valid` with an identical payload.
2. Tampering: flip one character of the blob → `Invalid`, `bad_signature`.
3. Tampering with intent: re-sign a payload with a *different* keypair →
`Invalid`. This is the test that proves an attacker cannot mint licenses.
4. Expiry: `ExpiresAt` one second in the past → `Expired`. One second in the
future → `Valid`.
5. Deployment mismatch: a Free (`cloud`) license verified with
`Deployment: "self_hosted"``Invalid`, `deployment_mismatch`.
6. Instance mismatch: correct signature, different `InstanceID``Invalid`,
`instance_mismatch`.
7. Check order: a blob that is both expired *and* instance-mismatched reports
`instance_mismatch`, not `Expired`. Order is part of the contract because the
reason drives the message.
8. Multi-key: a license signed with `trustedPublicKeys[1]` verifies. One signed
with a key not in the slice does not.
9. `Parse` returns the payload for an expired and for a mismatched license, and
errors for a bad signature.
10. Unicode and long instance names survive the round trip.
11. Golden blob: a fixture blob checked into the repo, signed with a **test-only**
keypair, must keep verifying. This catches an accidental change to the
canonical JSON encoding, which would silently invalidate every issued
license in the field.
Test 11 matters more than it looks. The encoding is part of the wire format.
## Verification before merge
1. `go test ./shared/license/...` passes, including the golden fixture.
2. `lkctl keypair``lkctl issue``lkctl inspect` round trips at the command
line.
3. `go build -tags noSign ./server/...` succeeds and
`go tool nm` on the resulting binary shows no `license.Sign` symbol.
4. The production keypair is generated, the private half stored in two places,
and the public half committed in `keys.go`.
## Risks
| Risk | Mitigation |
|---|---|
| Signing key lost | Documented two-location backup; generation is a one-off with an explicit checklist |
| Signing key leaked | Rotation path exists from day one: prepend key, release, reissue. Retire the old key once its licenses expire |
| Canonical encoding changes | Golden fixture test |
| Signing code shipped to customers | Build tag plus a symbol check in verification |
| No revocation | Accepted and documented. Bounded by term length; Self Hosted is annual-only |
@@ -0,0 +1,271 @@
# Spec 5 — Paddle Billing
Date: 2026-07-24
Status: Design approved, not implemented
Depends on: spec 3 (`admin-backend`)
Ships: after spec 3. Can be developed in parallel with spec 4.
## Context
Paddle is merchant of record: it owns checkout, tax, invoices, dunning and the
customer billing portal. This spec connects Paddle's subscription lifecycle to
the licence issuance functions spec 3 defines, and moves cloud signup off
sitesvc.
The central constraint, restated because every table below follows from it:
**licences are offline-verified, so nothing Paddle says can revoke one early.**
Cancellation takes effect when the licence expires. Self Hosted is annual-only to
bound that window; the alternative — a customer holding a valid key for eleven
months after cancelling a monthly plan — is not acceptable.
## Goals
1. A catalog in Paddle sandbox, promotable to production by configuration alone.
2. Webhooks that issue and renew licences reliably, including under retries and
out-of-order delivery.
3. Cloud signup owned by one service instead of two.
## Non-goals
- Building any part of billing Paddle already provides.
- Usage-based or metered pricing. Tiers are flat.
- Proration logic. Paddle handles money; we react to the resulting subscription
state.
## Design
### Catalog
Three products, created in **sandbox** first. Production is a configuration
change: the same `plans` rows carry different `paddle_product_id` and
`paddle_price_ids`, selected by `PADDLE_ENV`.
| Product | Prices | Notes |
|---|---|---|
| Vantage Free | monthly, £0 | Yes, a real £0 subscription. It gives every account a Paddle customer, a lifecycle, and an upgrade path with no special-case code. |
| Vantage Professional | monthly, annual | Cloud |
| Vantage Self Hosted | **annual only** | No monthly price exists, so the offline-revocation window is at most a year |
**No price ID is ever hard-coded.** They live in `plans.paddle_price_ids` and are
edited through the staff UI. A price change in Paddle is a data edit, not a
deploy.
`custom_data` on every checkout carries `{ account_id, instance_id, tier }`. This
is what lets a webhook route without a lookup table, and it is why the
self-hosted flow creates the instance record *before* checkout completes.
### Checkout
Paddle Checkout, overlay mode, in the admin site.
**Cloud upgrade** — instance exists, `instance_id` in `custom_data`, existing
Paddle customer reused.
**Self-hosted purchase** — the instance does not exist yet. Order:
```
Customer creates an admin-site account (verified email)
Account row created, then an admin_instances row with status awaiting_link
and a generated placeholder instance record
Checkout opened with account_id and that instance row's id in custom_data
subscription.created fires → subscription recorded, status awaiting_link,
NO licence issued
Customer pastes their install's UUID → instance_id set, status active
→ licence issued and delivered
```
The instance row exists before payment so the webhook has something to attach to.
The licence is not issued until the UUID is known, because a licence with no
instance to bind to cannot be signed — spec 1 has no unbound licence.
A customer who pays and never links has a subscription and no licence. Spec 4's
staff dashboard flags `awaiting_link` older than 48 hours, and a reminder email
goes out at 24 hours and 72 hours. This is the most likely place for a paying
customer to get stuck, so it gets active chasing rather than a support queue.
### Webhooks
`POST /api/paddle/webhook`, signature-verified with `PADDLE_WEBHOOK_SECRET`.
An unsigned or badly signed request is rejected `401` and logged — never
processed.
**Idempotency is mandatory.** Paddle retries. Every event ID is recorded in
`paddle_events` with a unique index before processing; a duplicate returns `200`
without acting. `200` on duplicates matters — returning an error would make
Paddle retry a message we have already handled, forever.
| Event | Action |
|---|---|
| `subscription.created` | Record the subscription. Cloud: issue and inject. Self-hosted: leave `awaiting_link`, issue nothing. |
| `subscription.updated` | Tier or term changed: issue a replacement licence at the new tier, supersede the old. Cloud injects; self-hosted emails a new blob and flags the site. Reflects Paddle's resulting state; no proration maths here. |
| `subscription.canceled` | Mark `cancelled`. **No licence action.** The current licence runs to expiry, then the instance degrades per spec 2. |
| `subscription.past_due` | Mark `past_due`, notify the customer, flag for staff. Licence untouched. Dunning is Paddle's job; ours is not to punish a retryable card failure. |
| `transaction.completed` where the transaction is a subscription renewal | Issue the next term's licence, supersede, inject or email. Reset `RelinkCount`. |
| `transaction.payment_failed` | Record for staff visibility. No licence action. |
| `customer.updated` | Sync `billing_email` onto the account. |
Out-of-order delivery is handled by making every handler a function of the
subscription's *current* state as reported in the event payload, rather than of
the transition. An `updated` arriving before its `created` creates the
subscription row and proceeds.
Renewal licences are issued with a **3-day grace** past the period end (spec 3),
so a webhook delayed by hours never produces a gap in coverage.
**Webhook failures must be visible.** Every failed handler writes to
`admin_audit` and appears on the staff dashboard. A licence that silently failed
to issue is a customer who paid and got nothing.
### Cancellation, stated plainly
When a customer cancels:
- Paddle stops billing at period end.
- We issue no further licences.
- Their current licence keeps working until it expires — up to a month for
Professional monthly, up to a year for Self Hosted.
- On expiry the instance degrades per spec 2: monitors keep running, changes stop.
This is documented in the terms and shown on the cancellation confirmation
screen, because a customer who cancels and sees their instance keep working
should understand why rather than assume the cancellation failed.
### Signup migration off sitesvc
Cloud signup currently lives in sitesvc: `site_pending_signups`, a verification
email, and provisioning on link click. It now needs to also create an Account, a
Paddle customer, a Free subscription and a licence.
**Signup moves to the admin backend.** The form stays on the marketing site where
customers find it, but it posts to admin instead of sitesvc. sitesvc keeps the
contact form only.
The reason is the one `CLAUDE.md` already names: provisioning logic duplicated
across services drifts. Spec 0a removed the second copy; adding signup to admin
while leaving it in sitesvc would create a third.
New flow, preserving every property of the current one:
```
Marketing site form → POST /api/signup on admin
→ pending record, password bcrypt cost 12, token 32 random bytes,
only the SHA-256 hash stored, 24h expiry, TTL index
→ verification email
Link opened → FindOneAndDelete the pending record (atomic, before provisioning)
→ shared.CreateInstance + shared.CreateUser in the control plane
→ Account created
→ Paddle customer created, Free subscription created
→ Free licence issued and injected
→ redirect to APP_LOGIN_URL with {slug} filled in
```
Properties that must survive, verified by test:
- Nothing written to `instances` or `users` until the link is opened.
- `FindOneAndDelete` before provisioning, so a double-clicked link cannot create
two instances.
- Instance rollback if the owner insert fails, refusing to delete an instance
that has users.
- Re-submitting for the same address replaces the pending record.
- Rate limited to 3 signups per IP per hour, plus the honeypot field.
Two failure modes are new, because provisioning now spans two systems:
- **Paddle customer creation fails** — the instance and user are already created.
Complete the signup, record the account with an empty `PaddleCustomerID`, issue
the Free licence anyway, and flag for staff. A new customer must never be
blocked from signing in by a billing-system hiccup.
- **Licence issuance fails** — the instance exists with no licence and is
read-only. Flagged for staff, and the 15-minute reconciliation job (spec 3)
retries. The customer can log in and sees the licence banner.
Both resolve toward "the customer gets in", because a signup that half-fails
silently is worse than either outcome.
sitesvc changes: signup, verify, `site_pending_signups` and the provisioning
calls are deleted. `SITE_API_URL` gains a sibling for the admin endpoint, or the
marketing site posts signup to `ADMIN_API_URL` directly — the latter, so the two
form targets are explicit rather than implied.
### Configuration
| Variable | Required | Notes |
|---|---|---|
| `PADDLE_ENV` | yes | `sandbox` or `production`; selects which price IDs the plans table serves |
| `PADDLE_API_KEY` | yes | server-side API |
| `PADDLE_CLIENT_TOKEN` | yes | browser checkout; baked into the admin site build |
| `PADDLE_WEBHOOK_SECRET` | yes | signature verification. Boot fails without it — an unverified webhook endpoint is an endpoint anyone can issue licences through |
| `APP_LOGIN_URL` | yes | moved from sitesvc; `{slug}` template |
### Cutover
Signup migration is the only user-visible switch:
1. Deploy admin with signup enabled; sitesvc still serving its own.
2. Point the marketing site's form at admin. Deploy.
3. Let sitesvc's outstanding pending signups expire naturally — 24 hours — while
its verify endpoint stays live. **Do not delete the collection until it is
empty**, or someone's verification link breaks.
4. Deploy sitesvc with signup removed.
## Testing
**Webhooks:**
1. Each event type produces its documented action against a mock Paddle payload.
2. Replaying an event ID is a no-op returning `200`.
3. A bad signature is rejected `401` and processes nothing.
4. `subscription.updated` before `subscription.created` creates the subscription
and applies the update.
5. `subscription.canceled` issues nothing and leaves the current licence intact.
6. `past_due` leaves the licence intact and flags the account.
7. Renewal issues the next term, supersedes, resets `RelinkCount`, and the new
`ExpiresAt` is period end plus 3 days.
8. A handler failure writes to `admin_audit` and surfaces on the dashboard.
**Checkout:**
9. `custom_data` round-trips account, instance and tier through to the webhook.
10. Self-hosted checkout leaves the instance `awaiting_link` with no licence.
11. Linking after checkout issues the licence.
**Signup:**
12. Nothing is written to `instances` or `users` before the link is opened.
13. A double-clicked verification link creates exactly one instance.
14. Owner-insert failure rolls the instance back; rollback refuses an instance
with users.
15. Re-submitting replaces the pending record and invalidates the earlier link.
16. Rate limit and honeypot both reject.
17. Paddle customer creation failure still completes signup and issues the Free
licence.
18. Licence issuance failure still lets the user log in, showing the banner.
19. Expired pending records are dropped by the TTL index.
## Verification before merge
1. Full suite green.
2. Against Paddle **sandbox**, end to end for each tier: checkout with a test
card, confirm the licence is issued, confirm the instance reports `valid`.
3. Trigger a sandbox renewal and confirm the next term's licence arrives and is
injected.
4. Cancel in sandbox and confirm the licence keeps working to expiry, then the
instance degrades correctly — monitors still running.
5. Replay every webhook from Paddle's dashboard and confirm no duplicate licences
are created.
6. Full signup end to end through admin, then confirm the new user can log into
their control-plane instance and sees a valid Free licence.
7. Confirm sitesvc's pending-signup collection is empty before its signup code is
removed.
## Risks
| Risk | Mitigation |
|---|---|
| Duplicate licences from webhook retries | Unique index on event ID, checked before processing |
| Webhook missed entirely | 15-minute reconciliation job (spec 3) compares subscription state against issued licences |
| Cancellation not enforceable until expiry | Accepted, bounded by term; Self Hosted annual-only; stated in terms and on the cancellation screen |
| Signup cutover breaks in-flight verification links | Staged cutover; sitesvc's verify stays live until its collection is empty |
| Sandbox price IDs reaching production | `PADDLE_ENV` selects them from the plans table; environment badge in the admin site |
| Webhook endpoint unauthenticated | Signature verification mandatory; boot fails without the secret |
| Customer pays and never links | Reminder emails at 24h and 72h, staff dashboard alert at 48h |
@@ -0,0 +1,286 @@
# Spec 0a — Shared Module Extraction
Date: 2026-07-24
Status: Design approved, not implemented
Ships: independently. No dependency on any other licensing spec.
## Context
Vantage is three independent Go modules: `server`, `sitesvc`, `agent`. There is no
root `go.mod` and no `go.work`.
`sitesvc` writes into the same MongoDB collections the control plane reads, but
cannot import the control plane, so it carries hand-copied duplicates:
- `sitesvc/internal/models/models.go``Org` and `User` mirrored field for field
- `sitesvc/internal/provision/provision.go``Slugify`, `ReservedSlugs`,
`MinSlugLength`, `MaxSlugLength`, `BcryptCost`, slug-collision rules
Both files carry comments saying they must be changed in lockstep with the
control plane, and `CLAUDE.md` names the hazard explicitly: nothing enforces the
match. **The duplication has already drifted.** The control plane's `CreateOrg`
resolves slug collisions with an inline `fmt.Sprintf("%s-%d", base, i)` loop,
while sitesvc exposes the same rule as a separate `NextSlug(base, attempt)`
helper. They currently agree by luck, not by construction.
The licensing programme adds a fourth service (`admin`) that writes the license
blob onto the same tenant document. Adding a third copy of these rules is not
acceptable. This spec removes the duplication before any licensing code is
written.
This spec is a **pure refactor**. No database document changes. No behaviour
changes. Names stay as they are today (`Org`, `org_id`) — renaming happens in
spec 0b, deliberately kept separate so that a failed deploy has one suspect
rather than two.
## Goals
1. One authoritative definition of every document shape written by more than one
service.
2. One authoritative definition of provisioning rules (slug, bcrypt cost,
creation, rollback).
3. `sitesvc` keeps its independence from `server` — it depends on `shared`, not
on the control plane. The original design intent survives; only the copying
dies.
4. The agent is untouched.
## Non-goals
- Renaming anything. That is spec 0b.
- Moving control-plane-only models. `workflow.go`, `monitor.go`, `key.go`,
`server.go`, `secret.go`, `assignment.go`, `channel.go`, `console_session.go`,
`audit.go`, `org_oidc.go` stay in `server/internal/models`. Only the control
plane touches them, and hoisting them would make `shared` a dumping ground.
- Merging the repo into a single module.
## Design
### Module layout
```
vantage/
├── go.work # NEW: server, sitesvc, shared (NOT agent)
├── shared/ # NEW module: github.com/mrhid6/vantage/shared
│ ├── go.mod
│ ├── models/
│ │ ├── org.go # Org
│ │ ├── user.go # User, RoleOwner/RoleAdmin/RoleMember, ValidRole
│ │ └── settings.go # Settings, AlertSettings, EmailSettings, SecretsSettings
│ ├── provision/
│ │ ├── slug.go # Slugify, BaseSlug, NextSlug, ReservedSlugs, limits
│ │ ├── org.go # CreateOrg
│ │ ├── user.go # CreateUser, BcryptCost
│ │ └── rollback.go # RollbackOrg
│ └── indexes/
│ └── indexes.go # EnsureCoreIndexes
├── server/ # replace => ../shared
├── sitesvc/ # replace => ../shared
└── agent/ # untouched
```
`go.work`:
```
go 1.26
use (
./shared
./server
./sitesvc
)
```
Each consumer's `go.mod` also carries an explicit replace:
```
require github.com/mrhid6/vantage/shared v0.0.0
replace github.com/mrhid6/vantage/shared => ../shared
```
Both are needed. `go.work` makes editors, `go test ./...` and local tooling work
across modules. The `replace` directives make Docker builds work whether or not
`go.work` is present, and stop `go build` outside the workspace from silently
trying to resolve `shared` from the network.
`shared` depends only on `go.mongodb.org/mongo-driver/v2`,
`golang.org/x/crypto/bcrypt` and `github.com/google/uuid`. It must not import
gin, redis, guac or anything else from the control plane's tree — that is what
keeps sitesvc small.
### What moves
**`shared/models`** — the three documents written by more than one service:
| Type | From | Written by |
|---|---|---|
| `Org` | `server/internal/models/org.go` | server, sitesvc, later admin |
| `User` + role constants + `ValidRole` | `server/internal/models/user.go` | server, sitesvc |
| `Settings` and its sub-structs | `server/internal/models/settings.go` | server today; admin reads it later |
`Settings` moves now rather than later because spec 3's admin service reads it,
and moving it later would mean a second round of import churn across both
services.
`PendingSignup` does **not** move. Only sitesvc writes `site_pending_signups`,
and the control plane does not know the collection exists.
**`shared/provision`** — the rules, promoted from private helpers to a real API:
```go
const (
MinSlugLength = 3
MaxSlugLength = 40
BcryptCost = 12
)
var ReservedSlugs = map[string]bool{ /* www, api, app, admin, auth, install, static, _next, default */ }
func Slugify(name string) string
func BaseSlug(name string) (string, error) // validates length + reserved
func NextSlug(base string, attempt int) string
// CreateOrg resolves a free slug and inserts. The caller supplies the
// collection handle so shared does not own a Mongo connection.
func CreateOrg(ctx context.Context, db *mongo.Database, name string) (*models.Org, error)
func CreateUser(ctx context.Context, db *mongo.Database, orgID, email, password, role string) (*models.User, error)
// RollbackOrg deletes an org only if it has no users. Refuses otherwise.
func RollbackOrg(ctx context.Context, db *mongo.Database, orgID string) error
```
`shared.CreateOrg` becomes the single implementation. The control plane's
`services.CreateOrg` shrinks to a wrapper that calls it and then runs
`SeedDefaultSteps` — seeding stays in the server, because `shared` must not know
about workflow steps. sitesvc calls `shared.CreateOrg` directly and does not
seed, which is the behaviour it has today.
Note on the slug loop: it is count-then-insert and therefore racy. It is safe
only because of the unique index on `orgs.slug`. `CreateOrg` must keep handling
`mongo.IsDuplicateKeyError` and returning a clean error — moving the code must
not lose that. Document the reliance in a comment at the loop.
**`shared/indexes`** — `EnsureCoreIndexes(ctx, db)` declares the unique indexes
on `users.email` and `orgs.slug`. Both services call it at boot; creating an
existing index is a no-op. These indexes are a security property, not an
optimisation (see `CLAUDE.md`: `GetUserByEmail` does an unscoped `FindOne`, so
duplicates would break the OIDC cross-org guard), so the shared version is
**fatal on failure** for both callers.
Server-only index builders (`EnsureSettingsIndexes`, `EnsureSecretIndexes`,
`EnsureWorkflowIndexes`) stay in the server and keep their current
fatal/warn behaviour.
### What is deleted
- `sitesvc/internal/models/models.go` — reduced to `PendingSignup` only
- `sitesvc/internal/provision/` — deleted entirely
- `sitesvc/internal/store/store.go` — org/user creation replaced by calls into
`shared/provision`; pending-signup storage stays
### Docker and CI
Both Go Dockerfiles currently build with the module directory as context:
```dockerfile
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN go build ... ./cmd
```
A `replace => ../shared` cannot resolve from that context. Build contexts move
to the repo root:
```dockerfile
WORKDIR /src
COPY shared/go.mod shared/go.sum ./shared/
COPY server/go.mod server/go.sum ./server/
RUN cd server && go mod download
COPY shared/ ./shared/
COPY server/ ./server/
ARG VERSION=dev
RUN cd server && CGO_ENABLED=0 GOOS=linux go build \
-ldflags="-s -w -X main.Version=${VERSION}" -o /vantage-server ./cmd
```
The two-stage copy keeps the dependency-download layer cached, which is the
reason the current Dockerfiles are written the way they are.
`.gitea/workflows/server-deploy.yml` must set `context: .` and
`file: server/Dockerfile` (and likewise for sitesvc) for the two Go images. The
`web` and `site` image builds are unaffected.
`agent-release.yml` is untouched. The agent is not in the workspace, has no
`replace`, and cross-compiles exactly as it does today.
### Error handling
No new error paths. `shared/provision` returns the same error strings the two
callers produce today so that API responses do not change. The one place to be
careful is wording: sitesvc says "organisation" and the control plane says
"organization". `shared` standardises on **"organisation"**; the control plane's
two error strings change spelling. This is user-visible in API error text and is
called out here so it is a decision rather than an accident.
## Testing
**No automated tests.** Decision taken 2026-07-24: the repo has no Go test suite
and one is not being started here. Verification is by compiler, `grep`, and
running both services end to end.
That places the whole weight on three manual checks, which the implementation
plan makes mandatory steps rather than suggestions:
1. **bson tag diff**`diff` the `bson:"…"` tags of each moved struct against
the originals. A changed tag orphans production data silently, and this is
the only thing that catches it.
2. **Slug behaviour walkthrough** — a throwaway `main` printing `Slugify`,
`BaseSlug` and `NextSlug` output for a fixed input table, compared against
expected output recorded in the plan.
3. **End-to-end agreement** — sign up through sitesvc against a scratch
database, open the verification link, then log into the control plane with
those credentials. This is the check that proves the two services still agree
about the documents they share. If it passes, the refactor worked.
Plus `grep` assertions that exactly one definition of `Slugify` and
`ReservedSlugs` survives repo-wide, and that no struct under `sitesvc/` carries
a `bson:"org_id"` tag.
## Verification before merge
Evidence required, not assertions:
1. `go build ./...` succeeds in `shared`, `server` and `sitesvc`.
2. `go vet ./...` clean in all three.
3. `docker build -f server/Dockerfile .` and `docker build -f sitesvc/Dockerfile .`
both succeed from the repo root.
4. `grep -r "org_id" sitesvc/` returns hits only in `PendingSignup` context and
`shared` imports — no local struct redefinitions.
5. End-to-end against a scratch database: sitesvc signup form → verification link
→ org and owner created → that owner logs into the control plane
successfully. This is the test that proves the two services still agree.
6. The agent still builds for `linux/amd64`, `linux/arm64` and `windows/amd64`.
## Rollout
Single release. `server` and `sitesvc` images must be deployed together — a skew
is harmless here (documents are unchanged) but there is no reason to split it.
No database migration. No downtime.
## Risks
| Risk | Mitigation |
|---|---|
| Docker context change breaks CI | Verified locally by building both images from root before pushing |
| Behaviour drift while moving `CreateOrg` | Unit tests written against current behaviour first, then the move |
| `shared` accumulating control-plane concerns | Explicit non-goals above; keep its `go.mod` dependency list to three entries and review any addition |
| Error-string spelling change | Called out as a decision; grep the web UI for hard-coded matches on the old strings |
## Follow-on
Spec 0b (`instance-rename`) becomes a rename inside one module plus its
consumers, rather than a rename across three independent copies. That is the
whole reason this spec goes first.
+70
View File
@@ -0,0 +1,70 @@
# Vantage Licensing Programme — Spec Index
Seven specs, designed 2026-07-24. Build in this order.
| # | Spec | Plan | Status |
|---|---|---|---|
| 0a | [shared-module](2026-07-24-shared-module-design.md) | [plan](../plans/2026-07-24-shared-module.md) | **shipped** |
| 0b | [instance-rename](2026-07-24-instance-rename-design.md) | [plan](../plans/2026-07-24-instance-rename.md) | **shipped**, migration verified on live |
| 1 | [licensing-core](2026-07-24-licensing-core-design.md) | [plan](../plans/2026-07-24-licensing-core.md) | **shipped** |
| 2 | [instance-licensing](2026-07-24-instance-licensing-design.md) | [plan](../plans/2026-07-24-instance-licensing.md) | **shipped**, no grandfathering — existing cloud instances are read-only until admin backfills |
| 3 | [admin-backend](2026-07-24-admin-backend-design.md) | [plan](../plans/2026-07-24-admin-backend.md) | planned |
| 4 | [admin-site](2026-07-24-admin-site-design.md) | — | needs 3 |
| 5 | [paddle-billing](2026-07-24-paddle-billing-design.md) | — | needs 3 |
Specs 1 and 2 together give working licensing with licences cut by hand with
`lkctl` — no admin service needed. 4 and 5 can run in parallel once 3 lands.
4 and 5 can run in parallel once 3 lands.
## The shape
```
Account (admin only)
├── Instance 1 cloud vantage.hostxtra.co.uk/<slug> licence auto-injected
├── Instance 2 cloud licence auto-injected
└── Instance 3 self-hosted customer's own deployment licence pasted by hand
```
The control plane knows only **Instance**. Accounts exist solely in the admin
service, because a self-hosted instance has no row in the cloud database at all.
## Decisions that everything else follows from
**Licences are offline-verified signed blobs.** ECDSA P-384 with SHA-256 via
`github.com/hyperboloide/lk`, public key compiled into the server, no phone-home
anywhere. This buys air-gapped self-hosting and means no Vantage instance ever
depends on the licensing service being up. It costs revocation: a licence is
valid until it expires whatever Paddle later says. Self Hosted is annual-only to
bound that window.
**Every licence is bound to one instance UUID.** Self-hosted customers link their
UUID before the licence is signed, so there is no unbound licence and no claim
protocol.
**Expiry degrades, it does not break.** Monitors keep executing, alerts keep
firing, agents keep their keys, in-flight workflow runs finish. Mutations stop.
Deletes and OS-update application stay open so a customer is never trapped
over-limit or unpatched.
**Tiers are data, not code.** The server reads `Limits` and `Features` and never
branches on tier name. Tier contents live in the admin `plans` table and are
snapshotted into each issued licence, so editing a plan never rewrites history —
the same rule as `workflow_runs.steps_snapshot`.
| | Free | Professional | Self Hosted |
|---|---|---|---|
| deployment | cloud only | cloud | self-hosted |
| max servers | 3 | unlimited | unlimited |
| max secret groups | 1 | unlimited | unlimited |
| max channels | 1 | unlimited | unlimited |
| console | no | yes | yes |
| OIDC | no | yes | yes |
| term | monthly, £0 | monthly or annual | annual only |
Free is cloud-only by construction: it is only ever signed with
`deployment: "cloud"`, and verification rejects a deployment mismatch. There is
no server-side flag to edit. One Free instance per account.
**Existing cloud tenants** are grandfathered to Professional, one year out, by
migration `0005`.
+7
View File
@@ -0,0 +1,7 @@
go 1.26.4
use (
./server
./shared
./sitesvc
)
+18
View File
@@ -0,0 +1,18 @@
cloud.google.com/go/compute v1.25.1/go.mod h1:oopOIR53ly6viBYxaDhBfJwzUAxf1zE//uf3IB011ls=
cloud.google.com/go/compute/metadata v0.3.0/go.mod h1:zFmK7XCadkQkj6TtorcaGlCW1hT1fIilQDwofLpJ20k=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137/go.mod h1:OMCwj8VM1Kc9e19TLln2VL61YJF0x1XFtfdL4JdbSyE=
github.com/census-instrumentation/opencensus-proto v0.4.1/go.mod h1:4T9NM4+4Vw91VeyqjLS6ao50K5bOcLKN6Q42XnYaRYw=
github.com/cncf/xds/go v0.0.0-20240318125728-8a4994d93e50/go.mod h1:5e1+Vvlzido69INQaVO6d87Qn543Xr6nooe9Kz7oBFM=
github.com/envoyproxy/go-control-plane v0.12.0/go.mod h1:ZBTaoJ23lqITozF0M6G4/IragXCQKCnYbmlmtHvwRG0=
github.com/envoyproxy/protoc-gen-validate v1.0.4/go.mod h1:qys6tmnRsYrQqIhm2bvKZH4Blx/1gTIZ2UKVY1M+Yew=
github.com/golang/glog v1.2.0/go.mod h1:6AhwSGph0fcJtXVM/PEHPqZlFeoLxhs7/t5UDAwmO+w=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w=
golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/appengine v1.6.8/go.mod h1:1jJ3jBArFh5pcgW8gCtRJnepW8FzD1V44FJffLiz/Ds=
google.golang.org/genproto/googleapis/api v0.0.0-20240318140521-94a12d6c2237/go.mod h1:Z5Iiy3jtmioajWHDGFk7CeugTyHtPvMHA4UTmUkyalE=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
-2
View File
@@ -27,8 +27,6 @@ function Invoke-Native {
}
}
# Like Invoke-Native but never throws — for teardown, where a missing/stopped
# service must not abort the uninstall.
function Invoke-NativeSoft {
param([string]$File, [string[]]$Arguments)
Write-Log ("RUN(soft): {0} {1}" -f $File, ($Arguments -join " "))
+85
View File
@@ -9,6 +9,9 @@ service Vantage {
rpc SyncKeys(SyncRequest) returns (SyncResponse);
rpc UploadGeneratedKey(UploadKeyRequest) returns (UploadKeyResponse);
rpc ReportUpdates(ReportUpdatesRequest) returns (ReportUpdatesResponse);
rpc ReportInventory(InventoryReport) returns (InventoryReportResponse);
rpc SyncMonitors(SyncMonitorsRequest) returns (SyncMonitorsResponse);
rpc ReportChecks(ReportChecksRequest) returns (ReportChecksResponse);
// Bidirectional stream: agent sends auth once, server pushes commands.
rpc CommandStream(stream AgentMessage) returns (stream ServerCommand);
}
@@ -82,6 +85,80 @@ message ReportUpdatesRequest {
message ReportUpdatesResponse {}
message CPUReport {
string model = 1;
int32 cores = 2;
double usage_pct = 3;
double load1 = 4;
}
message MemReport {
uint64 total_bytes = 1;
uint64 used_bytes = 2;
}
message PartitionReport {
string device = 1;
string mountpoint = 2;
string fstype = 3;
uint64 total_bytes = 4;
uint64 used_bytes = 5;
}
message InventoryReport {
string server_id = 1;
string agent_token = 2;
bool include_static = 3;
CPUReport cpu = 4;
MemReport memory = 5;
uint64 swap_total = 6;
uint64 swap_used = 7;
repeated PartitionReport partitions = 8;
string kernel = 9;
}
message InventoryReportResponse {}
message MonitorSpec {
string monitor_id = 1;
string type = 2;
string url = 3;
string host = 4;
int32 port = 5;
string method = 6;
int32 expected_status = 7;
string keyword = 8;
int32 tls_warn_days = 9;
int32 interval_sec = 10;
int32 retries = 11;
bool insecure = 12;
}
message SyncMonitorsRequest {
string server_id = 1;
string agent_token = 2;
}
message SyncMonitorsResponse {
repeated MonitorSpec monitors = 1;
}
message CheckResult {
string monitor_id = 1;
bool up = 2;
int32 latency_ms = 3;
string message = 4;
int64 cert_expiry_unix = 5;
}
message ReportChecksRequest {
string server_id = 1;
string agent_token = 2;
repeated CheckResult results = 3;
}
message ReportChecksResponse {}
message ApplyUpdatesCmd {}
message ServerCommand {
@@ -92,9 +169,16 @@ message ServerCommand {
UpdateAgentCmd update_agent = 4;
ApplyUpdatesCmd apply_updates = 5;
RunStepCmd run_step = 6;
CleanupWorkspaceCmd cleanup_workspace = 7;
}
}
// CleanupWorkspaceCmd tells the agent to recursively remove the run's working
// directory once all steps on that server have finished.
message CleanupWorkspaceCmd {
string workspace_id = 1;
}
message DeleteKeyCmd {
string label = 1;
}
@@ -117,6 +201,7 @@ message RunStepCmd {
string script = 2;
map<string, string> env = 3;
int32 timeout_seconds = 4;
string workspace_id = 5; // per-run working dir the agent creates & uses as cwd
}
message StepResult {
+12 -7
View File
@@ -1,17 +1,22 @@
# Build stage
#
# Context is the repository root, not server/, because server depends on the
# shared module through a replace directive.
FROM golang:1.26 AS builder
WORKDIR /app
WORKDIR /src
# Download dependencies first (layer cache)
COPY go.mod go.sum ./
RUN go mod download
# Manifests first so the dependency layer caches independently of source edits.
COPY shared/go.mod shared/go.sum ./shared/
COPY server/go.mod server/go.sum ./server/
RUN cd server && go mod download
# Copy source and build
COPY . .
COPY shared/ ./shared/
COPY server/ ./server/
ARG VERSION=dev
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-s -w -X main.Version=${VERSION}" -o /vantage-server ./cmd
RUN cd server && CGO_ENABLED=0 GOOS=linux go build \
-ldflags="-s -w -X main.Version=${VERSION}" -o /vantage-server ./cmd
# Runtime stage
FROM scratch
+60 -7
View File
@@ -11,6 +11,7 @@ import (
"github.com/mrhid6/vantage/server/internal/auth"
"github.com/mrhid6/vantage/server/internal/db"
grpcserver "github.com/mrhid6/vantage/server/internal/grpc"
"github.com/mrhid6/vantage/server/internal/monitorsched"
"github.com/mrhid6/vantage/server/internal/services"
)
@@ -18,30 +19,82 @@ func main() {
mongoURI := getEnv("MONGO_URI", "mongodb://localhost:27017")
dbName := getEnv("MONGO_DB", "vantage")
if os.Getenv("GRPC_HOST") == "" {
log.Fatal("GRPC_HOST is required (host:port agents dial for gRPC)")
}
if err := db.Connect(mongoURI, dbName); err != nil {
log.Fatalf("failed to connect to MongoDB: %v", err)
}
log.Println("connected to MongoDB")
// Migrations 0001 to 0003 still speak the pre-rename shape (orgs, org_id),
// so they must run before 0004 renames everything underneath them.
if err := services.RunMigrations(); err != nil {
log.Fatalf("migration failed: %v", err)
}
// 0002 must precede 0003: 0003 can create a "default" org, which pushes
// 0002 into its ambiguous multi-org branch.
if err := services.MigrateSettingsOrg(); err != nil {
log.Fatalf("settings org migration failed: %v", err)
}
if err := services.MigrateMissedOrgScopes(); err != nil {
log.Fatalf("missed org scope migration failed: %v", err)
}
// 0004 renames orgs to instances. It must run BEFORE the index builders:
// EnsureAuthIndexes creates instances.slug, which would create an empty
// instances collection and make 0004 refuse to rename onto it.
migCtx, migCancel := context.WithTimeout(context.Background(), 10*time.Minute)
migErr := services.MigrateOrgToInstance(migCtx, db.Database)
migCancel()
if migErr != nil {
log.Fatalf("instance rename migration failed: %v", migErr)
}
assertCtx, assertCancel := context.WithTimeout(context.Background(), 30*time.Second)
assertErr := services.AssertNoScopedCollectionMissed(assertCtx, db.Database)
assertCancel()
if assertErr != nil {
log.Fatalf("scoped collection check failed: %v", assertErr)
}
if err := services.EnsureAuthIndexes(); err != nil {
log.Fatalf("failed to ensure auth indexes: %v", err)
}
if err := services.EnsureSecretIndexes(); err != nil {
log.Printf("warning: failed to ensure secret indexes: %v", err)
}
if err := services.EnsureSettingsIndexes(); err != nil {
log.Fatalf("failed to ensure settings indexes: %v", err)
}
if err := services.EnsureWorkflowIndexes(); err != nil {
log.Printf("warning: failed to ensure workflow indexes: %v", err)
}
if instanceIDs, err := services.ListInstanceIDs(); err != nil {
log.Printf("warning: failed to list instances for default step seeding: %v", err)
} else {
for _, instanceID := range instanceIDs {
if created, updated, err := services.SeedDefaultSteps(instanceID); err != nil {
log.Printf("warning: failed to seed default steps for instance %s: %v", instanceID, err)
} else {
log.Printf("default steps seeded for instance %s: %d created, %d updated", instanceID, created, updated)
}
}
}
services.StartLogSweeper()
redisAddr := getEnv("REDIS_ADDR", "localhost:6379")
if err := auth.InitRedis(redisAddr); err != nil {
log.Fatalf("failed to connect to Redis: %v", err)
}
log.Println("connected to Redis")
if err := auth.InitOIDC(context.Background()); err != nil {
log.Fatalf("failed to initialise OIDC: %v", err)
}
// Background goroutine to mark offline servers
go func() {
ticker := time.NewTicker(2 * time.Minute)
defer ticker.Stop()
@@ -52,14 +105,14 @@ func main() {
}
}()
// Start gRPC server
go func() {
if err := grpcserver.StartGRPC(9090); err != nil {
log.Fatalf("gRPC server error: %v", err)
}
}()
// Start REST server
monitorsched.Start(context.Background())
r := gin.New()
r.Use(gin.Recovery())
r.Use(gin.LoggerWithConfig(gin.LoggerConfig{SkipPaths: []string{"/api/console/tunnel"}}))
+14 -10
View File
@@ -1,6 +1,6 @@
module github.com/mrhid6/vantage/server
go 1.26
go 1.26.4
require (
github.com/coreos/go-oidc/v3 v3.18.0
@@ -8,11 +8,14 @@ require (
github.com/google/uuid v1.6.0
github.com/redis/go-redis/v9 v9.20.1
github.com/wwt/guac v1.3.2
go.mongodb.org/mongo-driver/v2 v2.2.2
go.mongodb.org/mongo-driver/v2 v2.8.0
golang.org/x/crypto v0.54.0
golang.org/x/oauth2 v0.36.0
google.golang.org/grpc v1.64.0
)
require github.com/hyperboloide/lk v0.0.0-20251220053519-b291812e3216 // indirect
require (
github.com/bytedance/sonic v1.11.6 // indirect
github.com/bytedance/sonic/loader v0.1.1 // indirect
@@ -26,32 +29,33 @@ require (
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.20.0 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/golang/snappy v1.0.0 // indirect
github.com/gorilla/websocket v1.4.1 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.16.7 // indirect
github.com/klauspost/compress v1.17.6 // indirect
github.com/klauspost/cpuid/v2 v2.2.10 // indirect
github.com/konsorten/go-windows-terminal-sequences v1.0.1 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/mrhid6/vantage/shared v0.0.0
github.com/pelletier/go-toml/v2 v2.2.2 // indirect
github.com/sirupsen/logrus v1.4.2 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.12 // indirect
github.com/xdg-go/pbkdf2 v1.0.0 // indirect
github.com/xdg-go/scram v1.1.2 // indirect
github.com/xdg-go/scram v1.2.0 // indirect
github.com/xdg-go/stringprep v1.0.4 // indirect
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
go.uber.org/atomic v1.11.0 // indirect
golang.org/x/arch v0.8.0 // indirect
golang.org/x/crypto v0.33.0 // indirect
golang.org/x/net v0.25.0 // indirect
golang.org/x/sync v0.11.0 // indirect
golang.org/x/sys v0.30.0 // indirect
golang.org/x/text v0.22.0 // indirect
golang.org/x/net v0.56.0 // indirect
golang.org/x/sync v0.22.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/text v0.40.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240521202816-d264139d666e // indirect
google.golang.org/protobuf v1.34.2 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
replace github.com/mrhid6/vantage/shared => ../shared
+20 -19
View File
@@ -35,8 +35,6 @@ github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBEx
github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/golang/snappy v1.0.0 h1:Oy607GVXHs7RtbggtPBnr2RmDArIsAefDwvrdWvRhGs=
github.com/golang/snappy v1.0.0/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
@@ -45,10 +43,12 @@ github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/websocket v1.4.1 h1:q7AeDBpnBk8AogcD4DSag/Ukw/KV+YhzLj2bP5HvKCM=
github.com/gorilla/websocket v1.4.1/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/hyperboloide/lk v0.0.0-20251220053519-b291812e3216 h1:Luh+sE/W2M+V0Y+jlZN7nJefLNHc4/y93xxl+rFD7k0=
github.com/hyperboloide/lk v0.0.0-20251220053519-b291812e3216/go.mod h1:/OLW9HZj6qtQ7gWTGwuO3JrUZ+MC7I7TLRuNl14TYuo=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/klauspost/compress v1.16.7 h1:2mk3MPGNzKyxErAw8YaohYh69+pa4sIQSC0fPGCFR9I=
github.com/klauspost/compress v1.16.7/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE=
github.com/klauspost/compress v1.17.6 h1:60eq2E/jlfwQXtvZEeBUYADs+BwKBWURIY+Gj2eRGjI=
github.com/klauspost/compress v1.17.6/go.mod h1:/dCuZOvVtNoHsyb+cuJD3itjs3NbnF6KH9zAO4BDxPM=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2skhE=
github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
@@ -84,8 +84,9 @@ github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE=
@@ -94,8 +95,8 @@ github.com/wwt/guac v1.3.2 h1:sH6OFGa/1tBs7ieWBVlZe7t6F5JAOWBry/tqQL/Vup4=
github.com/wwt/guac v1.3.2/go.mod h1:eKm+NrnK7A88l4UBEcYNpZQGMpZRryYKoz4D/0/n1C0=
github.com/xdg-go/pbkdf2 v1.0.0 h1:Su7DPu48wXMwC3bs7MCNG+z4FhcyEuz5dlvchbq0B0c=
github.com/xdg-go/pbkdf2 v1.0.0/go.mod h1:jrpuAogTd400dnrH08LKmI/xc1MbPOebTwRqcT5RDeI=
github.com/xdg-go/scram v1.1.2 h1:FHX5I5B4i4hKRVRBCFRxq1iQRej7WO3hhBuJf+UUySY=
github.com/xdg-go/scram v1.1.2/go.mod h1:RT/sEzTbU5y00aCK8UOx6R7YryM0iF1N2MOmC3kKLN4=
github.com/xdg-go/scram v1.2.0 h1:bYKF2AEwG5rqd1BumT4gAnvwU/M9nBp2pTSxeZw7Wvs=
github.com/xdg-go/scram v1.2.0/go.mod h1:3dlrS0iBaWKYVt2ZfA4cj48umJZ+cAEbR6/SjLA88I8=
github.com/xdg-go/stringprep v1.0.4 h1:XLI/Ng3O1Atzq0oBs3TWm+5ZVgkq2aqdlvP9JtoZ6c8=
github.com/xdg-go/stringprep v1.0.4/go.mod h1:mPGuuIYwz7CmR2bT9j4GbQqutWS1zV24gijq1dTyGkM=
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 h1:ilQV1hzziu+LLM3zUTJ0trRztfwgjqKnBWNtSRkbmwM=
@@ -103,8 +104,8 @@ github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78/go.mod h1:aL8wCCfTfS
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
go.mongodb.org/mongo-driver/v2 v2.2.2 h1:9cYuS3fl1Xhqwpfazso10V7BHQD58kCgtzhfAmJYz9c=
go.mongodb.org/mongo-driver/v2 v2.2.2/go.mod h1:qQkDMhCGWl3FN509DfdPd4GRBLU/41zqF/k8eTRceps=
go.mongodb.org/mongo-driver/v2 v2.8.0 h1:CxWDGQYY8QQwNjAl/aq2sfWakdnWZynnqJ9F4DhHbP8=
go.mongodb.org/mongo-driver/v2 v2.8.0/go.mod h1:yOI9kBsufol30iFsl1slpdq1I0eHPzybRWdyYUs8K/0=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
@@ -112,20 +113,20 @@ golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc=
golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.33.0 h1:IOBPskki6Lysi0lo9qQvbxiQ+FvsCC/YWOecCHAixus=
golang.org/x/crypto v0.33.0/go.mod h1:bVdXmD7IV/4GdElGPozy6U7lWdRXA4qyRVGJV57uQ5M=
golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.25.0 h1:d/OCCoBEUq33pjydKrGQhw7IlUPI2Oylr+8qLx49kac=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.11.0 h1:GGz8+XQP4FvTTrjZPzNKTMFtSXH80RAzG+5ghFPgK9w=
golang.org/x/sync v0.11.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -133,16 +134,16 @@ golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc=
golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
golang.org/x/text v0.22.0 h1:bofq7m3/HAFvbF51jz3Q9wLg3jkvSPuiZu/pD1XwgtM=
golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY=
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
+100
View File
@@ -0,0 +1,100 @@
package api
import (
"net/http"
"github.com/gin-gonic/gin"
"github.com/mrhid6/vantage/server/internal/auth"
"github.com/mrhid6/vantage/server/internal/models"
"github.com/mrhid6/vantage/server/internal/services"
"go.mongodb.org/mongo-driver/v2/bson"
)
func registerChannelRoutes(g *gin.RouterGroup) {
g.GET("/channels", listChannels)
g.POST("/channels", createChannel)
g.PUT("/channels/:id", updateChannel)
g.DELETE("/channels/:id", deleteChannel)
g.POST("/channels/:id/test", testChannel)
}
func listChannels(c *gin.Context) {
channels, err := services.ListChannels(auth.InstanceID(c))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, channels)
}
func createChannel(c *gin.Context) {
var ch models.NotificationChannel
if err := c.ShouldBindJSON(&ch); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
if ch.Name == "" || ch.Type == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "name and type are required"})
return
}
created, err := services.CreateChannel(auth.InstanceID(c), &ch)
if err != nil {
if limitStatus(c, err) {
return
}
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusCreated, created)
}
func updateChannel(c *gin.Context) {
var body struct {
Name *string `json:"name"`
Type *string `json:"type"`
Config *map[string]string `json:"config"`
Enabled *bool `json:"enabled"`
}
if err := c.ShouldBindJSON(&body); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
upd := bson.M{}
if body.Name != nil {
upd["name"] = *body.Name
}
if body.Type != nil {
upd["type"] = *body.Type
}
if body.Config != nil {
upd["config"] = *body.Config
}
if body.Enabled != nil {
upd["enabled"] = *body.Enabled
}
if len(upd) == 0 {
c.JSON(http.StatusBadRequest, gin.H{"error": "no fields to update"})
return
}
if err := services.UpdateChannel(auth.InstanceID(c), c.Param("id"), upd); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.Status(http.StatusNoContent)
}
func deleteChannel(c *gin.Context) {
if err := services.DeleteChannel(auth.InstanceID(c), c.Param("id")); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.Status(http.StatusNoContent)
}
func testChannel(c *gin.Context) {
if err := services.TestChannel(auth.InstanceID(c), c.Param("id")); err != nil {
c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"status": "sent"})
}
+13 -22
View File
@@ -8,13 +8,11 @@ import (
"time"
"github.com/gin-gonic/gin"
"github.com/mrhid6/vantage/server/internal/auth"
"github.com/mrhid6/vantage/server/internal/services"
"github.com/wwt/guac"
)
// POST /api/console/connect
// Body: { server_id, protocol, key_id?, rdp_username?, rdp_password? }
// Returns: { session_id, token, ws_path }
func consoleConnect(c *gin.Context) {
var body struct {
ServerID string `json:"server_id" binding:"required"`
@@ -29,13 +27,13 @@ func consoleConnect(c *gin.Context) {
return
}
srv, err := services.GetServer(body.ServerID)
srv, err := services.GetServer(auth.InstanceID(c), body.ServerID)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "server not found"})
return
}
sess, err := services.CreateConsoleSession(body.ServerID, body.Protocol, body.KeyID, actorFromCtx(c), c.ClientIP())
sess, err := services.CreateConsoleSession(auth.InstanceID(c), body.ServerID, body.Protocol, body.KeyID, actorFromCtx(c), c.ClientIP())
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -47,20 +45,20 @@ func consoleConnect(c *gin.Context) {
}
if (body.Protocol == "rdp" || body.Protocol == "vnc") && (body.RDPUsername != "" || body.RDPPassword != "") {
if err := services.StashConsoleRDPCreds(sess.SessionID, body.RDPUsername, body.RDPPassword); err != nil {
if err := services.StashConsoleRDPCreds(auth.InstanceID(c), sess.SessionID, body.RDPUsername, body.RDPPassword); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
}
if body.Protocol == "ssh" {
if err := services.SetConsoleSSHUser(sess.SessionID, body.SSHUsername); err != nil {
if err := services.SetConsoleSSHUser(auth.InstanceID(c), sess.SessionID, body.SSHUsername); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
}
services.LogEvent("console.opened", actorFromCtx(c), srv.ServerID, "",
services.LogEvent(auth.InstanceID(c), "console.opened", actorFromCtx(c), srv.ServerID, "",
"console session opened ("+body.Protocol+")")
c.JSON(http.StatusOK, gin.H{
@@ -70,8 +68,6 @@ func consoleConnect(c *gin.Context) {
})
}
// queryIntDefault reads a positive integer query param, falling back to def
// when absent, unparseable, or non-positive.
func queryIntDefault(r *http.Request, key string, def int) int {
v, err := strconv.Atoi(r.URL.Query().Get(key))
if err != nil || v <= 0 {
@@ -80,7 +76,6 @@ func queryIntDefault(r *http.Request, key string, def int) int {
return v
}
// GET /api/console/tunnel?token=... (WebSocket upgrade)
func consoleTunnel(c *gin.Context) {
token := c.Query("token")
sessionID, err := services.VerifySessionToken(token)
@@ -88,35 +83,32 @@ func consoleTunnel(c *gin.Context) {
c.JSON(http.StatusUnauthorized, gin.H{"error": "invalid token"})
return
}
sess, err := services.GetConsoleSession(sessionID)
instanceID := auth.InstanceID(c)
sess, err := services.GetConsoleSession(instanceID, sessionID)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "session not found"})
return
}
// User-bound: the caller (authenticated via session cookie) must be the same
// user who opened the session. Blocks a leaked token being used by someone else.
if actor := actorFromCtx(c); actor != sess.User {
c.JSON(http.StatusForbidden, gin.H{"error": "session belongs to another user"})
return
}
// Single-use: atomically spend the token so a replay within its TTL is rejected.
if err := services.ConsumeSessionToken(sessionID); err != nil {
if err := services.ConsumeSessionToken(instanceID, sessionID); err != nil {
c.JSON(http.StatusUnauthorized, gin.H{"error": "token already used"})
return
}
srv, err := services.GetServer(sess.ServerID)
srv, err := services.GetServer(auth.InstanceID(c), sess.ServerID)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "server not found"})
return
}
// Decrypt private key + passphrase in-memory only (ssh).
var privKey, passphrase string
if sess.Protocol == "ssh" && sess.KeyID != "" {
privKey, err = services.GetPrivateKey(sess.KeyID)
privKey, err = services.GetPrivateKey(auth.InstanceID(c), sess.KeyID)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "selected key has no private material"})
return
@@ -126,7 +118,7 @@ func consoleTunnel(c *gin.Context) {
var rdpUser, rdpPass string
if sess.Protocol == "rdp" || sess.Protocol == "vnc" {
rdpUser, rdpPass, err = services.ConsumeConsoleRDPCreds(sessionID)
rdpUser, rdpPass, err = services.ConsumeConsoleRDPCreds(instanceID, sessionID)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "could not load credentials"})
return
@@ -143,7 +135,6 @@ func consoleTunnel(c *gin.Context) {
guacdAddr = "guacd:4822"
}
// Build a guac tunnel config from our params.
connect := func(r *http.Request) (guac.Tunnel, error) {
config := guac.NewGuacamoleConfiguration()
config.Protocol = gp.Protocol
@@ -171,7 +162,7 @@ func consoleTunnel(c *gin.Context) {
wsServer := guac.NewWebsocketServer(connect)
wsServer.OnDisconnect = func(id string, r *http.Request, t guac.Tunnel) {
_ = services.EndConsoleSession(sessionID)
_ = services.EndConsoleSession(instanceID, sessionID)
}
wsServer.ServeHTTP(c.Writer, c.Request)
}
+81 -67
View File
@@ -25,23 +25,26 @@ func RegisterRoutes(r *gin.Engine) {
r.GET("/update", handleUpdateScript)
r.GET("/update.ps1", handleUpdateScriptWindows)
// ESO read endpoint — bearer-token auth, not session auth, so Kubernetes
// External Secrets Operator can call it. Lives under /api (so the reverse
// proxy routes it to the backend) but on a distinct subpath to avoid
// colliding with the session-authed GET /api/secrets/:group. Returns a
// group as flat JSON.
r.GET("/api/secrets/:group/values", secretsReadAuth(), esoGetGroup)
// Auth endpoints (no session required)
r.GET("/auth/login", auth.HandleLogin)
r.GET("/auth/callback", auth.HandleCallback)
r.GET("/auth/logout", auth.HandleLogout)
r.GET("/auth/bootstrap-status", auth.HandleBootstrapStatus)
r.POST("/auth/bootstrap", auth.HandleBootstrap)
r.POST("/auth/login", auth.HandleLocalLogin)
r.POST("/auth/logout", auth.HandleLogout)
r.GET("/auth/me", auth.HandleMe)
r.GET("/auth/oidc/start", auth.HandleOIDCStart)
r.GET("/auth/oidc/callback", auth.HandleOIDCCallback)
// API endpoints protected by session middleware
apiGroup := r.Group("/api")
apiGroup.Use(auth.Middleware())
// Deny by default: every non-GET route under /api is gated unless it is on
// the exemption list in licence.go. A route added later is covered because
// of where it is mounted, not because someone remembered.
apiGroup.Use(RequireActiveLicense())
{
apiGroup.GET("/license", getLicence)
apiGroup.POST("/license", auth.RequireRole("owner"), postLicence)
apiGroup.GET("/servers", listServers)
apiGroup.POST("/servers", createServer)
apiGroup.GET("/servers/new", newServer)
@@ -56,9 +59,13 @@ func RegisterRoutes(r *gin.Engine) {
apiGroup.GET("/audit", listAuditEvents)
apiGroup.GET("/settings", getSettings)
apiGroup.PUT("/settings", saveSettings)
apiGroup.POST("/settings/secrets-token", rotateSecretsToken)
settings := apiGroup.Group("/settings")
settings.Use(auth.RequireRole("owner", "admin"))
{
settings.GET("", getSettings)
settings.PUT("", saveSettings)
settings.POST("/secrets-token", rotateSecretsToken)
}
apiGroup.GET("/secrets", listSecretGroups)
apiGroup.POST("/secrets", createSecretGroup)
@@ -76,15 +83,28 @@ func RegisterRoutes(r *gin.Engine) {
apiGroup.POST("/keys/:id/assign", assignKey)
apiGroup.DELETE("/keys/:id/assign/:serverId", revokeAssignment)
apiGroup.POST("/console/connect", consoleConnect)
apiGroup.GET("/console/tunnel", consoleTunnel)
apiGroup.POST("/console/connect", RequireFeature("console"), consoleConnect)
apiGroup.GET("/console/tunnel", RequireFeature("console"), consoleTunnel)
registerWorkflowRoutes(apiGroup)
registerMonitorRoutes(apiGroup)
registerChannelRoutes(apiGroup)
instance := apiGroup.Group("/instance")
instance.Use(auth.RequireRole("owner", "admin"))
{
instance.GET("/users", listInstanceUsers)
instance.POST("/users", createInstanceUser)
instance.PUT("/users/:id/role", updateInstanceUserRole)
instance.DELETE("/users/:id", deleteInstanceUser)
instance.GET("/oidc", RequireFeature("oidc"), getInstanceOIDC)
instance.PUT("/oidc", RequireFeature("oidc"), putInstanceOIDC)
}
}
}
func listServers(c *gin.Context) {
servers, err := services.ListServers()
servers, err := services.ListServers(auth.InstanceID(c))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -93,8 +113,11 @@ func listServers(c *gin.Context) {
}
func createServer(c *gin.Context) {
s, token, err := services.CreateServer()
s, token, err := services.CreateServer(auth.InstanceID(c))
if err != nil {
if limitStatus(c, err) {
return
}
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
@@ -106,21 +129,22 @@ func createServer(c *gin.Context) {
}
func newServer(c *gin.Context) {
s, token, err := services.CreateServer()
s, token, err := services.CreateServer(auth.InstanceID(c))
if err != nil {
if limitStatus(c, err) {
return
}
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("server.created", actorFromCtx(c), s.ServerID, "", "pre-registration token issued")
services.LogEvent(auth.InstanceID(c), "server.created", actorFromCtx(c), s.ServerID, "", "pre-registration token issued")
giteaHost := os.Getenv("GITEA_HOST")
if giteaHost == "" {
giteaHost = "gitea.example.com"
}
host := os.Getenv("PUBLIC_HOST")
if host == "" {
host = "https://vantage.example.com"
}
host := publicHostFromRequest(c)
installCmd := fmt.Sprintf(
`curl -fsSL "%s/install?server_id=%s&token=%s" | bash`,
@@ -142,15 +166,14 @@ func newServer(c *gin.Context) {
func getServer(c *gin.Context) {
id := c.Param("id")
s, err := services.GetServer(id)
s, err := services.GetServer(auth.InstanceID(c), id)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "server not found"})
return
}
assignments, _ := services.GetAssignmentsWithKeysForServer(id)
assignments, _ := services.GetAssignmentsWithKeysForServer(auth.InstanceID(c), id)
// Build response matching ServerWithKeys shape expected by frontend
type serverResponse struct {
*models.Server
Keys interface{} `json:"keys"`
@@ -163,8 +186,8 @@ func getServer(c *gin.Context) {
func deleteServer(c *gin.Context) {
id := c.Param("id")
s, _ := services.GetServer(id)
if err := services.DeleteServer(id); err != nil {
s, _ := services.GetServer(auth.InstanceID(c), id)
if err := services.DeleteServer(auth.InstanceID(c), id); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
@@ -172,7 +195,7 @@ func deleteServer(c *gin.Context) {
if s != nil {
hostname = s.Hostname
}
services.LogEvent("server.deleted", actorFromCtx(c), id, "", fmt.Sprintf("server %s deleted", hostname))
services.LogEvent(auth.InstanceID(c), "server.deleted", actorFromCtx(c), id, "", fmt.Sprintf("server %s deleted", hostname))
c.JSON(http.StatusOK, gin.H{"deleted": true})
}
@@ -191,7 +214,7 @@ func generateKey(c *gin.Context) {
body.Label = "generated"
}
s, err := services.GetServer(id)
s, err := services.GetServer(auth.InstanceID(c), id)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "server not found"})
return
@@ -209,7 +232,7 @@ func generateKey(c *gin.Context) {
return
}
services.LogEvent("key.generation_dispatched", actorFromCtx(c), s.ServerID, "", fmt.Sprintf("key generation dispatched (label=%s type=%s)", body.Label, body.KeyType))
services.LogEvent(auth.InstanceID(c), "key.generation_dispatched", actorFromCtx(c), s.ServerID, "", fmt.Sprintf("key generation dispatched (label=%s type=%s)", body.Label, body.KeyType))
c.JSON(http.StatusAccepted, gin.H{
"message": "key generation command sent to agent",
"command_id": cmdID,
@@ -218,7 +241,7 @@ func generateKey(c *gin.Context) {
}
func listKeys(c *gin.Context) {
keys, err := services.ListKeys()
keys, err := services.ListKeys(auth.InstanceID(c))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -238,18 +261,18 @@ func createKey(c *gin.Context) {
return
}
key, err := services.CreateKey(body.Label, body.PublicKey, "uploaded", "", body.PrivateKey, body.Passphrase)
key, err := services.CreateKey(auth.InstanceID(c), body.Label, body.PublicKey, "uploaded", "", body.PrivateKey, body.Passphrase)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("key.uploaded", actorFromCtx(c), "", key.KeyID, fmt.Sprintf("key '%s' uploaded", key.Label))
services.LogEvent(auth.InstanceID(c), "key.uploaded", actorFromCtx(c), "", key.KeyID, fmt.Sprintf("key '%s' uploaded", key.Label))
c.JSON(http.StatusCreated, key)
}
func getPrivateKey(c *gin.Context) {
id := c.Param("id")
plaintext, err := services.GetPrivateKey(id)
plaintext, err := services.GetPrivateKey(auth.InstanceID(c), id)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": err.Error()})
return
@@ -259,13 +282,13 @@ func getPrivateKey(c *gin.Context) {
func getKey(c *gin.Context) {
id := c.Param("id")
key, err := services.GetKey(id)
key, err := services.GetKey(auth.InstanceID(c), id)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "key not found"})
return
}
assignments, _ := services.GetAssignmentsWithServers(id)
assignments, _ := services.GetAssignmentsWithServers(auth.InstanceID(c), id)
type keyResponse struct {
*models.Key
@@ -279,8 +302,8 @@ func getKey(c *gin.Context) {
func deleteKey(c *gin.Context) {
id := c.Param("id")
k, _ := services.GetKey(id)
if err := services.DeleteKey(id); err != nil {
k, _ := services.GetKey(auth.InstanceID(c), id)
if err := services.DeleteKey(auth.InstanceID(c), id); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
@@ -288,7 +311,7 @@ func deleteKey(c *gin.Context) {
if k != nil {
label = k.Label
}
services.LogEvent("key.deleted", actorFromCtx(c), "", id, fmt.Sprintf("key '%s' deleted", label))
services.LogEvent(auth.InstanceID(c), "key.deleted", actorFromCtx(c), "", id, fmt.Sprintf("key '%s' deleted", label))
c.JSON(http.StatusOK, gin.H{"deleted": true})
}
@@ -302,12 +325,12 @@ func assignKey(c *gin.Context) {
return
}
a, err := services.AssignKey(keyID, body.ServerID)
a, err := services.AssignKey(auth.InstanceID(c), keyID, body.ServerID)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("key.assigned", actorFromCtx(c), body.ServerID, keyID, fmt.Sprintf("key %s assigned to server %s", keyID, body.ServerID))
services.LogEvent(auth.InstanceID(c), "key.assigned", actorFromCtx(c), body.ServerID, keyID, fmt.Sprintf("key %s assigned to server %s", keyID, body.ServerID))
c.JSON(http.StatusCreated, a)
}
@@ -315,11 +338,11 @@ func revokeAssignment(c *gin.Context) {
keyID := c.Param("id")
serverID := c.Param("serverId")
if err := services.RevokeAssignment(keyID, serverID); err != nil {
if err := services.RevokeAssignment(auth.InstanceID(c), keyID, serverID); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("key.revoked", actorFromCtx(c), serverID, keyID, fmt.Sprintf("key %s revoked from server %s", keyID, serverID))
services.LogEvent(auth.InstanceID(c), "key.revoked", actorFromCtx(c), serverID, keyID, fmt.Sprintf("key %s revoked from server %s", keyID, serverID))
c.JSON(http.StatusOK, gin.H{"revoked": true})
}
@@ -334,7 +357,7 @@ func getLatestAgentVersion(c *gin.Context) {
func updateAgent(c *gin.Context) {
id := c.Param("id")
s, err := services.GetServer(id)
s, err := services.GetServer(auth.InstanceID(c), id)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "server not found"})
return
@@ -345,7 +368,7 @@ func updateAgent(c *gin.Context) {
c.JSON(http.StatusServiceUnavailable, gin.H{"error": err.Error()})
return
}
services.LogEvent("agent.update_dispatched", actorFromCtx(c), s.ServerID, "", fmt.Sprintf("agent update dispatched to %s (version %s)", s.Hostname, version))
services.LogEvent(auth.InstanceID(c), "agent.update_dispatched", actorFromCtx(c), s.ServerID, "", fmt.Sprintf("agent update dispatched to %s (version %s)", s.Hostname, version))
c.JSON(http.StatusAccepted, gin.H{
"message": "update command sent to agent",
"version": version,
@@ -354,7 +377,7 @@ func updateAgent(c *gin.Context) {
func applyUpdates(c *gin.Context) {
id := c.Param("id")
s, err := services.GetServer(id)
s, err := services.GetServer(auth.InstanceID(c), id)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "server not found"})
return
@@ -364,7 +387,7 @@ func applyUpdates(c *gin.Context) {
c.JSON(http.StatusServiceUnavailable, gin.H{"error": err.Error()})
return
}
services.LogEvent("updates.applied", actorFromCtx(c), s.ServerID, "", fmt.Sprintf("package update command dispatched to %s", s.Hostname))
services.LogEvent(auth.InstanceID(c), "updates.applied", actorFromCtx(c), s.ServerID, "", fmt.Sprintf("package update command dispatched to %s", s.Hostname))
c.JSON(http.StatusAccepted, gin.H{"message": "apply updates command sent to agent"})
}
@@ -396,7 +419,7 @@ if [ -z "$LATEST" ]; then
fi
VERSION="${LATEST#agent/}"
LATEST_ENCODED="${LATEST/\//%%2F}"
LATEST_ENCODED="${LATEST/\
BINARY_URL="https://${GITEA_HOST}/mrhid6/vantage/releases/download/${LATEST_ENCODED}/vantage-agent-linux-${ARCH}"
CHECKSUM_URL="https://${GITEA_HOST}/mrhid6/vantage/releases/download/${LATEST_ENCODED}/checksums.txt"
@@ -431,7 +454,7 @@ func listAuditEvents(c *gin.Context) {
limit = n
}
}
events, err := services.ListAuditEvents(limit)
events, err := services.ListAuditEvents(auth.InstanceID(c), limit)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -440,7 +463,7 @@ func listAuditEvents(c *gin.Context) {
}
func getSettings(c *gin.Context) {
s, err := services.GetSettings()
s, err := services.GetSettings(auth.InstanceID(c))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -450,18 +473,19 @@ func getSettings(c *gin.Context) {
func saveSettings(c *gin.Context) {
var body struct {
Alerts models.AlertSettings `json:"alerts"`
Email models.EmailSettings `json:"email"`
Alerts models.AlertSettings `json:"alerts"`
Email models.EmailSettings `json:"email"`
WorkflowLogRetentionDays *int `json:"workflow_log_retention_days"`
}
if err := c.ShouldBindJSON(&body); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
if err := services.SaveSettings(body.Alerts, body.Email); err != nil {
if err := services.SaveSettings(auth.InstanceID(c), body.Alerts, body.Email, body.WorkflowLogRetentionDays); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("settings.updated", actorFromCtx(c), "", "", "alert settings updated")
services.LogEvent(auth.InstanceID(c), "settings.updated", actorFromCtx(c), "", "", "alert settings updated")
c.JSON(http.StatusOK, gin.H{"saved": true})
}
@@ -473,14 +497,7 @@ func handleInstallScript(c *gin.Context) {
if giteaHost == "" {
giteaHost = "gitea.example.com"
}
publicHost := os.Getenv("PUBLIC_HOST")
if publicHost == "" {
publicHost = "vantage.example.com"
}
grpcHost := os.Getenv("GRPC_HOST")
if grpcHost == "" {
grpcHost = publicHost
}
script := fmt.Sprintf(`#!/usr/bin/env bash
set -euo pipefail
@@ -488,9 +505,6 @@ set -euo pipefail
SERVER_ID="%s"
TOKEN="%s"
GITEA_HOST="%s"
KM_HOST="%s"
KM_HOST="${KM_HOST#https://}"
KM_HOST="${KM_HOST#http://}"
GRPC_HOST="%s"
GRPC_HOST="${GRPC_HOST#https://}"
GRPC_HOST="${GRPC_HOST#http://}"
@@ -512,7 +526,7 @@ if [ -z "$LATEST" ]; then
fi
VERSION="${LATEST#agent/}"
LATEST_ENCODED="${LATEST/\//%%2F}"
LATEST_ENCODED="${LATEST/\
BINARY_URL="https://${GITEA_HOST}/mrhid6/vantage/releases/download/${LATEST_ENCODED}/vantage-agent-linux-${ARCH}"
CHECKSUM_URL="https://${GITEA_HOST}/mrhid6/vantage/releases/download/${LATEST_ENCODED}/checksums.txt"
@@ -563,7 +577,7 @@ systemctl daemon-reload
systemctl enable --now vantage-agent
echo "vantage-agent installed and started."
`, serverID, token, giteaHost, publicHost, grpcHost)
`, serverID, token, giteaHost, grpcHost)
c.Header("Content-Type", "text/x-shellscript")
c.String(http.StatusOK, script)
+1 -9
View File
@@ -16,13 +16,8 @@ func handleInstallScriptWindows(c *gin.Context) {
if giteaHost == "" {
giteaHost = "gitea.example.com"
}
grpcHost := os.Getenv("GRPC_HOST")
if grpcHost == "" {
grpcHost = os.Getenv("PUBLIC_HOST")
}
if grpcHost == "" {
grpcHost = "vantage.example.com"
}
script := fmt.Sprintf(
"#Requires -RunAsAdministrator\n"+
@@ -55,9 +50,6 @@ func handleInstallScriptWindows(c *gin.Context) {
c.String(http.StatusOK, script)
}
// handleUpdateScriptWindows serves a PowerShell one-liner that upgrades an
// already-installed Windows agent. No server_id/token needed: the MSI is a
// MajorUpgrade and setup.ps1 preserves the existing config on upgrade.
func handleUpdateScriptWindows(c *gin.Context) {
giteaHost := os.Getenv("GITEA_HOST")
if giteaHost == "" {
+155
View File
@@ -0,0 +1,155 @@
package api
import (
"errors"
"net/http"
"github.com/gin-gonic/gin"
"github.com/mrhid6/vantage/server/internal/auth"
"github.com/mrhid6/vantage/server/internal/models"
"github.com/mrhid6/vantage/server/internal/services"
)
func listInstanceUsers(c *gin.Context) {
users, err := services.ListUsers(auth.InstanceID(c))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, users)
}
func actorMayGrantOwner(c *gin.Context) bool {
return auth.Role(c) == models.RoleOwner
}
func createInstanceUser(c *gin.Context) {
var body struct {
Email string `json:"email"`
Password string `json:"password"`
Role string `json:"role"`
}
if err := c.ShouldBindJSON(&body); err != nil || body.Email == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "email required"})
return
}
if body.Role == "" {
body.Role = models.RoleMember
}
if !models.ValidRole(body.Role) {
c.JSON(http.StatusBadRequest, gin.H{"error": "role must be one of owner, admin, or member"})
return
}
if body.Role == models.RoleOwner && !actorMayGrantOwner(c) {
c.JSON(http.StatusForbidden, gin.H{"error": "only an owner can create another owner"})
return
}
u, err := services.CreateUser(auth.InstanceID(c), body.Email, body.Password, body.Role, "local")
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusCreated, u)
}
func updateInstanceUserRole(c *gin.Context) {
var body struct {
Role string `json:"role"`
}
if err := c.ShouldBindJSON(&body); err != nil || body.Role == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "role required"})
return
}
if !models.ValidRole(body.Role) {
c.JSON(http.StatusBadRequest, gin.H{"error": "role must be one of owner, admin, or member"})
return
}
instanceID, targetID := auth.InstanceID(c), c.Param("id")
if targetID == auth.UserID(c) {
c.JSON(http.StatusForbidden, gin.H{"error": "you cannot change your own role"})
return
}
target, err := services.GetUserInInstance(instanceID, targetID)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "user not found"})
return
}
if (body.Role == models.RoleOwner || target.Role == models.RoleOwner) && !actorMayGrantOwner(c) {
c.JSON(http.StatusForbidden, gin.H{"error": "only an owner can change owner roles"})
return
}
if err := services.UpdateUserRole(instanceID, targetID, body.Role); err != nil {
c.JSON(orgUserErrStatus(err), gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"ok": true})
}
func deleteInstanceUser(c *gin.Context) {
instanceID, targetID := auth.InstanceID(c), c.Param("id")
if targetID == auth.UserID(c) {
c.JSON(http.StatusForbidden, gin.H{"error": "you cannot remove your own account"})
return
}
target, err := services.GetUserInInstance(instanceID, targetID)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "user not found"})
return
}
if target.Role == models.RoleOwner && !actorMayGrantOwner(c) {
c.JSON(http.StatusForbidden, gin.H{"error": "only an owner can remove another owner"})
return
}
if err := services.DeleteUser(instanceID, targetID); err != nil {
c.JSON(orgUserErrStatus(err), gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"deleted": true})
}
func orgUserErrStatus(err error) int {
if errors.Is(err, services.ErrLastOwner) {
return http.StatusConflict
}
return http.StatusInternalServerError
}
func getInstanceOIDC(c *gin.Context) {
cfg, err := services.GetInstanceOIDC(auth.InstanceID(c))
if err != nil {
c.JSON(http.StatusOK, gin.H{"enabled": false, "client_secret_set": false})
return
}
c.JSON(http.StatusOK, gin.H{
"instance_id": cfg.InstanceID,
"issuer": cfg.Issuer,
"client_id": cfg.ClientID,
"enabled": cfg.Enabled,
"updated_at": cfg.UpdatedAt,
"client_secret_set": cfg.ClientSecretEnc != "",
})
}
func putInstanceOIDC(c *gin.Context) {
var body struct {
Issuer string `json:"issuer"`
ClientID string `json:"client_id"`
ClientSecret string `json:"client_secret"`
Enabled bool `json:"enabled"`
}
if err := c.ShouldBindJSON(&body); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
if err := services.SaveInstanceOIDC(auth.InstanceID(c), body.Issuer, body.ClientID, body.ClientSecret, body.Enabled); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
auth.EvictOIDCProvider(auth.InstanceID(c))
c.JSON(http.StatusOK, gin.H{"saved": true})
}
+227
View File
@@ -0,0 +1,227 @@
package api
import (
"errors"
"net/http"
"sync"
"time"
"github.com/gin-gonic/gin"
"github.com/mrhid6/vantage/server/internal/auth"
"github.com/mrhid6/vantage/server/internal/services"
"github.com/mrhid6/vantage/shared/license"
)
// licenceExemptPaths are routes that must work while a licence is expired or
// missing, because they are how a customer recovers or stays safe.
//
// /api/license pasting a valid licence is the way out of degraded mode
// apply-updates security patching is never paywalled
//
// All DELETE requests are exempt separately (see RequireActiveLicense): a
// customer downgraded below their current usage must be able to delete their
// way back under the cap.
var licenceExemptPaths = map[string]bool{
"/api/license": true,
}
// mutatingGETs are routes that change state despite their method. GET is
// otherwise always allowed through, so these have to be named explicitly:
// GET /api/servers/new mints a pre-registration token, which is a creation.
var mutatingGETs = map[string]bool{
"/api/servers/new": true,
}
func licenceExempt(c *gin.Context) bool {
if c.Request.Method == http.MethodDelete {
return true
}
if licenceExemptPaths[c.FullPath()] {
return true
}
if c.FullPath() == "/api/servers/:id/apply-updates" {
return true
}
return false
}
// RequireActiveLicense blocks mutating requests when the licence is not valid.
//
// Mounted on the /api group, so a route added tomorrow is gated because of where
// it lives rather than because someone remembered. GET and HEAD always pass —
// reading is never blocked.
func RequireActiveLicense() gin.HandlerFunc {
return func(c *gin.Context) {
if (c.Request.Method == http.MethodGet || c.Request.Method == http.MethodHead) &&
!mutatingGETs[c.FullPath()] {
c.Next()
return
}
if licenceExempt(c) {
c.Next()
return
}
st := services.GetLicenseState(auth.InstanceID(c))
if st.Active() {
c.Next()
return
}
c.AbortWithStatusJSON(http.StatusForbidden, gin.H{
"error": "license_required",
"state": st.Status,
"reason": st.Reason,
})
}
}
// RequireFeature blocks a route when the licence does not grant a feature.
func RequireFeature(name string) gin.HandlerFunc {
return func(c *gin.Context) {
st := services.GetLicenseState(auth.InstanceID(c))
if st.Feature(name) {
c.Next()
return
}
c.AbortWithStatusJSON(http.StatusForbidden, gin.H{
"error": "feature_unavailable",
"feature": name,
})
}
}
type licenceResponse struct {
InstanceID string `json:"instance_id"`
State license.State `json:"state"`
Reason string `json:"reason,omitempty"`
Tier string `json:"tier,omitempty"`
ExpiresAt *time.Time `json:"expires_at,omitempty"`
DaysRemaining *int `json:"days_remaining,omitempty"`
Limits license.Limits `json:"limits"`
Features map[string]bool `json:"features"`
Usage licenceUsageResponse `json:"usage"`
Source string `json:"source"`
}
type licenceUsageResponse struct {
Servers int `json:"servers"`
SecretGroups int `json:"secret_groups"`
Channels int `json:"channels"`
}
func getLicence(c *gin.Context) {
instanceID := auth.InstanceID(c)
st := services.GetLicenseState(instanceID)
servers, groups, channels := services.LicenseUsage(instanceID)
resp := licenceResponse{
InstanceID: instanceID,
State: st.Status,
Reason: st.Reason,
Tier: st.Tier,
ExpiresAt: st.ExpiresAt,
Limits: st.Limits,
Features: st.Features,
Usage: licenceUsageResponse{Servers: servers, SecretGroups: groups, Channels: channels},
Source: st.Source,
}
if st.ExpiresAt != nil {
d := int(time.Until(*st.ExpiresAt).Hours() / 24)
resp.DaysRemaining = &d
}
c.JSON(http.StatusOK, resp)
}
var (
licencePostMu sync.Mutex
licencePostCounts = map[string][]time.Time{}
)
const licencePostLimit = 10
// licencePostAllowed permits 10 attempts per instance per hour.
func licencePostAllowed(instanceID string) bool {
cutoff := time.Now().Add(-time.Hour)
licencePostMu.Lock()
defer licencePostMu.Unlock()
kept := licencePostCounts[instanceID][:0]
for _, t := range licencePostCounts[instanceID] {
if t.After(cutoff) {
kept = append(kept, t)
}
}
if len(kept) >= licencePostLimit {
licencePostCounts[instanceID] = kept
return false
}
licencePostCounts[instanceID] = append(kept, time.Now())
return true
}
func postLicence(c *gin.Context) {
instanceID := auth.InstanceID(c)
if !licencePostAllowed(instanceID) {
c.JSON(http.StatusTooManyRequests, gin.H{
"error": "Too many licence attempts. Try again later.",
})
return
}
var body struct {
Blob string `json:"blob"`
}
if err := c.ShouldBindJSON(&body); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "a licence key is required"})
return
}
st, err := services.StoreLicense(instanceID, body.Blob)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{
"error": licenceRejectionMessage(err.Error(), instanceID),
"reason": err.Error(),
})
return
}
services.LogEvent(instanceID, "license.updated", actorFromCtx(c), "", "",
"licence accepted (tier "+st.Tier+")")
c.JSON(http.StatusOK, gin.H{"state": st.Status, "tier": st.Tier, "expires_at": st.ExpiresAt})
}
// licenceRejectionMessage turns a machine reason into something a person can act
// on. The instance ID is included in the mismatch case because that is the one
// piece of information the customer needs and cannot guess.
func licenceRejectionMessage(reason, instanceID string) string {
switch reason {
case license.ReasonBadSignature:
return "This licence key is not valid. Check it was copied in full."
case license.ReasonDeploymentMismatch:
return "This licence is for Vantage Cloud and cannot be used on a self-hosted install."
case license.ReasonInstanceMismatch:
return "This licence was issued for a different instance. Your instance ID is " + instanceID + "."
case license.ReasonNoLicense:
return "No licence key was provided."
default:
return "This licence could not be accepted."
}
}
// limitStatus maps a LimitError to a 403 body. Handlers that create countable
// resources call this so the UI gets a machine-readable limit name.
func limitStatus(c *gin.Context, err error) bool {
var le *services.LimitError
if !errors.As(err, &le) {
return false
}
c.JSON(http.StatusForbidden, gin.H{
"error": "limit_exceeded",
"limit": le.Limit,
"current": le.Current,
"max": le.Max,
})
return true
}
+158
View File
@@ -0,0 +1,158 @@
package api
import (
"net/http"
"time"
"github.com/gin-gonic/gin"
"github.com/mrhid6/vantage/server/internal/auth"
"github.com/mrhid6/vantage/server/internal/models"
"github.com/mrhid6/vantage/server/internal/services"
"go.mongodb.org/mongo-driver/v2/bson"
)
func registerMonitorRoutes(g *gin.RouterGroup) {
g.GET("/monitors", listMonitors)
g.POST("/monitors", createMonitor)
g.GET("/monitors/:id", getMonitor)
g.PUT("/monitors/:id", updateMonitor)
g.DELETE("/monitors/:id", deleteMonitor)
g.GET("/monitors/:id/incidents", getMonitorIncidents)
g.GET("/monitors/:id/uptime", getMonitorUptime)
}
func listMonitors(c *gin.Context) {
monitors, err := services.ListMonitors(auth.InstanceID(c))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, monitors)
}
func createMonitor(c *gin.Context) {
var m models.Monitor
if err := c.ShouldBindJSON(&m); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
if m.Name == "" || m.Type == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "name and type are required"})
return
}
created, err := services.CreateMonitor(auth.InstanceID(c), &m)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusCreated, created)
}
func getMonitor(c *gin.Context) {
m, err := services.GetMonitor(auth.InstanceID(c), c.Param("id"))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
if m == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "monitor not found"})
return
}
c.JSON(http.StatusOK, m)
}
func updateMonitor(c *gin.Context) {
var body struct {
Name *string `json:"name"`
Type *string `json:"type"`
Target *models.MonitorTarget `json:"target"`
IntervalSec *int `json:"interval_sec"`
Runner *string `json:"runner"`
Retries *int `json:"retries"`
Enabled *bool `json:"enabled"`
ChannelIDs *[]string `json:"channel_ids"`
}
if err := c.ShouldBindJSON(&body); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
upd := bson.M{}
if body.Name != nil {
upd["name"] = *body.Name
}
if body.Type != nil {
upd["type"] = *body.Type
}
if body.Target != nil {
upd["target"] = *body.Target
}
if body.IntervalSec != nil {
upd["interval_sec"] = *body.IntervalSec
}
if body.Runner != nil {
upd["runner"] = *body.Runner
}
if body.Retries != nil {
upd["retries"] = *body.Retries
}
if body.Enabled != nil {
upd["enabled"] = *body.Enabled
}
if body.ChannelIDs != nil {
upd["channel_ids"] = *body.ChannelIDs
}
if len(upd) == 0 {
c.JSON(http.StatusBadRequest, gin.H{"error": "no fields to update"})
return
}
if err := services.UpdateMonitor(auth.InstanceID(c), c.Param("id"), upd); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.Status(http.StatusNoContent)
}
func deleteMonitor(c *gin.Context) {
if err := services.DeleteMonitor(auth.InstanceID(c), c.Param("id")); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.Status(http.StatusNoContent)
}
func getMonitorIncidents(c *gin.Context) {
m, err := services.GetMonitor(auth.InstanceID(c), c.Param("id"))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
if m == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "monitor not found"})
return
}
incidents, err := services.ListIncidents(auth.InstanceID(c), c.Param("id"), 50)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, incidents)
}
func getMonitorUptime(c *gin.Context) {
m, err := services.GetMonitor(auth.InstanceID(c), c.Param("id"))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
if m == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "monitor not found"})
return
}
since := time.Now().Add(-30 * 24 * time.Hour)
rollups, err := services.UptimeRollups(auth.InstanceID(c), c.Param("id"), since)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, rollups)
}
+51
View File
@@ -0,0 +1,51 @@
package api
import (
"net"
"strings"
"github.com/gin-gonic/gin"
)
func publicHostFromRequest(c *gin.Context) string {
host := c.Request.Host
if h := firstForwarded(c.GetHeader("X-Forwarded-Host")); h != "" {
host = h
}
if host == "" {
return "https://vantage.example.com"
}
return schemeFor(c, host) + "://" + host
}
func schemeFor(c *gin.Context, host string) string {
if p := firstForwarded(c.GetHeader("X-Forwarded-Proto")); p != "" {
return p
}
if c.Request.TLS != nil {
return "https"
}
if isLoopback(host) {
return "http"
}
return "https"
}
func firstForwarded(v string) string {
if v == "" {
return ""
}
return strings.TrimSpace(strings.Split(v, ",")[0])
}
func isLoopback(host string) bool {
h, _, err := net.SplitHostPort(host)
if err != nil {
h = host
}
if h == "localhost" || strings.HasSuffix(h, ".localhost") {
return true
}
ip := net.ParseIP(h)
return ip != nil && ip.IsLoopback()
}
+36 -27
View File
@@ -7,40 +7,46 @@ import (
"strings"
"github.com/gin-gonic/gin"
"github.com/mrhid6/vantage/server/internal/auth"
"github.com/mrhid6/vantage/server/internal/services"
)
// groupNamePattern restricts group and key names to characters that are safe
// in URLs and Kubernetes/env contexts.
var groupNamePattern = regexp.MustCompile(`^[A-Za-z0-9._-]+$`)
func validName(s string) bool {
return s != "" && len(s) <= 128 && groupNamePattern.MatchString(s)
}
// secretsReadAuth validates the ESO bearer token on the public read endpoint.
const ctxSecretsInstanceKey = "km_secrets_instance"
func secretsReadAuth() gin.HandlerFunc {
return func(c *gin.Context) {
const prefix = "Bearer "
auth := c.GetHeader("Authorization")
if len(auth) <= len(prefix) || !strings.EqualFold(auth[:len(prefix)], prefix) {
authHeader := c.GetHeader("Authorization")
if len(authHeader) <= len(prefix) || !strings.EqualFold(authHeader[:len(prefix)], prefix) {
c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "missing bearer token"})
return
}
if !services.VerifySecretsReadToken(auth[len(prefix):]) {
instanceID, ok := services.ResolveSecretsReadToken(authHeader[len(prefix):])
if !ok {
c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "invalid token"})
return
}
c.Set(ctxSecretsInstanceKey, instanceID)
c.Next()
}
}
// esoGetGroup handles GET /secrets/:group for the External Secrets Operator.
// Returns a flat JSON object { "KEY": "value", ... }; 404 if the group is empty
// (ESO treats 404 as "deleted").
func esoGetGroup(c *gin.Context) {
group := c.Param("group")
values, err := services.GetSecretGroupDecrypted(group)
instanceID := c.GetString(ctxSecretsInstanceKey)
if instanceID == "" {
c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "invalid token"})
return
}
values, err := services.GetSecretGroupDecrypted(instanceID, group)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "store error"})
return
@@ -53,7 +59,7 @@ func esoGetGroup(c *gin.Context) {
}
func listSecretGroups(c *gin.Context) {
groups, err := services.ListSecretGroups()
groups, err := services.ListSecretGroups(auth.InstanceID(c))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -61,8 +67,6 @@ func listSecretGroups(c *gin.Context) {
c.JSON(http.StatusOK, groups)
}
// createSecretGroup handles POST /api/secrets. A group is implicit, so it must
// be created with at least one key/value pair.
func createSecretGroup(c *gin.Context) {
var body struct {
Group string `json:"group" binding:"required"`
@@ -86,17 +90,20 @@ func createSecretGroup(c *gin.Context) {
return
}
}
if err := services.UpsertSecrets(body.Group, body.Values); err != nil {
if err := services.UpsertSecrets(auth.InstanceID(c), body.Group, body.Values); err != nil {
if limitStatus(c, err) {
return
}
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("secret.updated", actorFromCtx(c), "", "", fmt.Sprintf("group '%s' created with keys: %s", body.Group, strings.Join(services.SortedKeys(body.Values), ", ")))
services.LogEvent(auth.InstanceID(c), "secret.updated", actorFromCtx(c), "", "", fmt.Sprintf("group '%s' created with keys: %s", body.Group, strings.Join(services.SortedKeys(body.Values), ", ")))
c.JSON(http.StatusCreated, gin.H{"group": body.Group})
}
func getSecretGroup(c *gin.Context) {
group := c.Param("group")
secrets, err := services.GetSecretGroup(group)
secrets, err := services.GetSecretGroup(auth.InstanceID(c), group)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -108,7 +115,6 @@ func getSecretGroup(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{"group": group, "secrets": secrets})
}
// putSecretGroup upserts one or more keys into an existing (or new) group.
func putSecretGroup(c *gin.Context) {
group := c.Param("group")
if !validName(group) {
@@ -130,11 +136,14 @@ func putSecretGroup(c *gin.Context) {
return
}
}
if err := services.UpsertSecrets(group, values); err != nil {
if err := services.UpsertSecrets(auth.InstanceID(c), group, values); err != nil {
if limitStatus(c, err) {
return
}
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("secret.updated", actorFromCtx(c), "", "", fmt.Sprintf("group '%s' keys updated: %s", group, strings.Join(services.SortedKeys(values), ", ")))
services.LogEvent(auth.InstanceID(c), "secret.updated", actorFromCtx(c), "", "", fmt.Sprintf("group '%s' keys updated: %s", group, strings.Join(services.SortedKeys(values), ", ")))
c.JSON(http.StatusOK, gin.H{"saved": true})
}
@@ -147,42 +156,42 @@ func revealSecret(c *gin.Context) {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
value, err := services.RevealSecret(group, body.Key)
value, err := services.RevealSecret(auth.InstanceID(c), group, body.Key)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": err.Error()})
return
}
services.LogEvent("secret.revealed", actorFromCtx(c), "", "", fmt.Sprintf("value of '%s/%s' revealed", group, body.Key))
services.LogEvent(auth.InstanceID(c), "secret.revealed", actorFromCtx(c), "", "", fmt.Sprintf("value of '%s/%s' revealed", group, body.Key))
c.JSON(http.StatusOK, gin.H{"value": value})
}
func deleteSecretKey(c *gin.Context) {
group := c.Param("group")
key := c.Param("key")
if err := services.DeleteSecret(group, key); err != nil {
if err := services.DeleteSecret(auth.InstanceID(c), group, key); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("secret.deleted", actorFromCtx(c), "", "", fmt.Sprintf("key '%s' deleted from group '%s'", key, group))
services.LogEvent(auth.InstanceID(c), "secret.deleted", actorFromCtx(c), "", "", fmt.Sprintf("key '%s' deleted from group '%s'", key, group))
c.JSON(http.StatusOK, gin.H{"deleted": true})
}
func deleteSecretGroup(c *gin.Context) {
group := c.Param("group")
if err := services.DeleteSecretGroup(group); err != nil {
if err := services.DeleteSecretGroup(auth.InstanceID(c), group); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("secretgroup.deleted", actorFromCtx(c), "", "", fmt.Sprintf("group '%s' deleted", group))
services.LogEvent(auth.InstanceID(c), "secretgroup.deleted", actorFromCtx(c), "", "", fmt.Sprintf("group '%s' deleted", group))
c.JSON(http.StatusOK, gin.H{"deleted": true})
}
func rotateSecretsToken(c *gin.Context) {
token, err := services.RotateSecretsReadToken()
token, err := services.RotateSecretsReadToken(auth.InstanceID(c))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("secrets.token_rotated", actorFromCtx(c), "", "", "ESO read token rotated")
services.LogEvent(auth.InstanceID(c), "secrets.token_rotated", actorFromCtx(c), "", "", "ESO read token rotated")
c.JSON(http.StatusOK, gin.H{"token": token})
}
+206 -22
View File
@@ -2,10 +2,16 @@ package api
import (
"fmt"
"io"
"net/http"
"os"
"regexp"
"strconv"
"strings"
"time"
"github.com/gin-gonic/gin"
"github.com/mrhid6/vantage/server/internal/auth"
"github.com/mrhid6/vantage/server/internal/models"
"github.com/mrhid6/vantage/server/internal/services"
)
@@ -15,6 +21,11 @@ func registerWorkflowRoutes(g *gin.RouterGroup) {
g.POST("/steps", createStep)
g.PUT("/steps/:id", updateStep)
g.DELETE("/steps/:id", deleteStep)
g.GET("/steps/:id/export", exportStep)
g.POST("/steps/import", importStep)
g.POST("/steps/seed-defaults", seedDefaults)
g.GET("/steps/usage", stepUsage)
g.POST("/steps/parse", parseStep)
g.GET("/workflows", listWorkflows)
g.POST("/workflows", createWorkflow)
@@ -26,10 +37,116 @@ func registerWorkflowRoutes(g *gin.RouterGroup) {
g.GET("/runs/:runId", getRun)
g.POST("/runs/:runId/cancel", cancelRun)
g.GET("/runs/:runId/servers/:serverId/logs", getServerRunLog)
g.GET("/runs/:runId/servers/:serverId/logs/stream", streamServerRunLog)
}
var uuidLike = regexp.MustCompile(`^[a-zA-Z0-9-]{1,64}$`)
func getServerRunLog(c *gin.Context) {
runID, serverID := c.Param("runId"), c.Param("serverId")
if !uuidLike.MatchString(runID) || !uuidLike.MatchString(serverID) {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid id"})
return
}
path := services.ServerRunLogPath(runID, serverID)
b, err := os.ReadFile(path)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "no logs"})
return
}
c.Data(http.StatusOK, "text/plain; charset=utf-8", b)
}
func streamServerRunLog(c *gin.Context) {
runID, serverID := c.Param("runId"), c.Param("serverId")
if !uuidLike.MatchString(runID) || !uuidLike.MatchString(serverID) {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid id"})
return
}
path := services.ServerRunLogPath(runID, serverID)
c.Writer.Header().Set("Content-Type", "text/event-stream")
c.Writer.Header().Set("Cache-Control", "no-cache")
c.Writer.Header().Set("Connection", "keep-alive")
c.Writer.Header().Set("X-Accel-Buffering", "no")
flusher, ok := c.Writer.(http.Flusher)
if !ok {
c.JSON(http.StatusInternalServerError, gin.H{"error": "stream unsupported"})
return
}
var offset int64
sendNew := func() {
f, err := os.Open(path)
if err != nil {
return
}
defer f.Close()
if _, err := f.Seek(offset, 0); err != nil {
return
}
buf := make([]byte, 32*1024)
for {
n, _ := f.Read(buf)
if n <= 0 {
break
}
offset += int64(n)
for _, line := range splitSSE(buf[:n]) {
_, _ = c.Writer.WriteString("data: " + line + "\n")
}
_, _ = c.Writer.WriteString("\n")
flusher.Flush()
}
}
ctx := c.Request.Context()
ticker := time.NewTicker(500 * time.Millisecond)
defer ticker.Stop()
instanceID := auth.InstanceID(c)
for {
sendNew()
if serverRunTerminal(instanceID, runID, serverID) {
sendNew()
_, _ = c.Writer.WriteString("event: done\ndata: end\n\n")
flusher.Flush()
return
}
select {
case <-ctx.Done():
return
case <-ticker.C:
}
}
}
func serverRunTerminal(instanceID, runID, serverID string) bool {
r, err := services.GetRun(instanceID, runID)
if err != nil {
return true
}
for _, sr := range r.ServerRuns {
if sr.ServerID == serverID {
switch sr.Status {
case "success", "failed", "skipped", "cancelled":
return true
}
return false
}
}
return true
}
func splitSSE(b []byte) []string {
s := strings.ReplaceAll(string(b), "\r", "")
return strings.Split(s, "\n")
}
func listSteps(c *gin.Context) {
steps, err := services.ListSteps()
steps, err := services.ListSteps(auth.InstanceID(c))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -37,18 +154,27 @@ func listSteps(c *gin.Context) {
c.JSON(http.StatusOK, steps)
}
func stepUsage(c *gin.Context) {
counts, err := services.StepUsageCounts(auth.InstanceID(c))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, counts)
}
func createStep(c *gin.Context) {
var s models.WorkflowStep
if err := c.ShouldBindJSON(&s); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
out, err := services.CreateStep(s)
out, err := services.CreateStep(auth.InstanceID(c), s)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("workflow.step_created", actorFromCtx(c), "", out.StepID, fmt.Sprintf("step '%s' created", out.Name))
services.LogEvent(auth.InstanceID(c), "workflow.step_created", actorFromCtx(c), "", out.StepID, fmt.Sprintf("step '%s' created", out.Name))
c.JSON(http.StatusCreated, out)
}
@@ -58,25 +184,78 @@ func updateStep(c *gin.Context) {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
if err := services.UpdateStep(c.Param("id"), s); err != nil {
if err := services.UpdateStep(auth.InstanceID(c), c.Param("id"), s); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("workflow.step_updated", actorFromCtx(c), "", c.Param("id"), "step updated")
services.LogEvent(auth.InstanceID(c), "workflow.step_updated", actorFromCtx(c), "", c.Param("id"), "step updated")
c.JSON(http.StatusOK, gin.H{"updated": true})
}
func deleteStep(c *gin.Context) {
if err := services.DeleteStep(c.Param("id")); err != nil {
if err := services.DeleteStep(auth.InstanceID(c), c.Param("id")); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("workflow.step_deleted", actorFromCtx(c), "", c.Param("id"), "step deleted")
services.LogEvent(auth.InstanceID(c), "workflow.step_deleted", actorFromCtx(c), "", c.Param("id"), "step deleted")
c.JSON(http.StatusOK, gin.H{"deleted": true})
}
func exportStep(c *gin.Context) {
b, err := services.ExportStep(auth.InstanceID(c), c.Param("id"))
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": err.Error()})
return
}
c.Header("Content-Disposition", fmt.Sprintf("attachment; filename=step-%s.json", c.Param("id")))
c.Data(http.StatusOK, "application/json", b)
}
func seedDefaults(c *gin.Context) {
created, updated, err := services.SeedDefaultSteps(auth.InstanceID(c))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent(auth.InstanceID(c), "workflow.defaults_synced", actorFromCtx(c), "", "", fmt.Sprintf("default steps synced: %d created, %d updated", created, updated))
c.JSON(http.StatusOK, gin.H{"created": created, "updated": updated})
}
const maxStepBodyBytes = 1 << 20
func importStep(c *gin.Context) {
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, maxStepBodyBytes)
body, err := io.ReadAll(c.Request.Body)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
out, err := services.ImportStepToLibrary(auth.InstanceID(c), body)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
services.LogEvent(auth.InstanceID(c), "workflow.step_imported", actorFromCtx(c), "", out.StepID, fmt.Sprintf("step '%s' imported", out.Name))
c.JSON(http.StatusCreated, out)
}
func parseStep(c *gin.Context) {
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, maxStepBodyBytes)
body, err := io.ReadAll(c.Request.Body)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
s, err := services.ParseStepDoc(body)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, s)
}
func listWorkflows(c *gin.Context) {
wfs, err := services.ListWorkflows()
wfs, err := services.ListWorkflows(auth.InstanceID(c))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -90,17 +269,17 @@ func createWorkflow(c *gin.Context) {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
out, err := services.CreateWorkflow(w)
out, err := services.CreateWorkflow(auth.InstanceID(c), w)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("workflow.created", actorFromCtx(c), "", out.WorkflowID, fmt.Sprintf("workflow '%s' created", out.Name))
services.LogEvent(auth.InstanceID(c), "workflow.created", actorFromCtx(c), "", out.WorkflowID, fmt.Sprintf("workflow '%s' created", out.Name))
c.JSON(http.StatusCreated, out)
}
func getWorkflow(c *gin.Context) {
w, err := services.GetWorkflow(c.Param("id"))
w, err := services.GetWorkflow(auth.InstanceID(c), c.Param("id"))
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": err.Error()})
return
@@ -114,30 +293,35 @@ func updateWorkflow(c *gin.Context) {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
if err := services.UpdateWorkflow(c.Param("id"), w); err != nil {
if err := services.UpdateWorkflow(auth.InstanceID(c), c.Param("id"), w); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("workflow.updated", actorFromCtx(c), "", c.Param("id"), "workflow updated")
c.JSON(http.StatusOK, gin.H{"updated": true})
services.LogEvent(auth.InstanceID(c), "workflow.updated", actorFromCtx(c), "", c.Param("id"), "workflow updated")
updated, err := services.GetWorkflow(auth.InstanceID(c), c.Param("id"))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, updated)
}
func deleteWorkflow(c *gin.Context) {
if err := services.DeleteWorkflow(c.Param("id")); err != nil {
if err := services.DeleteWorkflow(auth.InstanceID(c), c.Param("id")); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("workflow.deleted", actorFromCtx(c), "", c.Param("id"), "workflow deleted")
services.LogEvent(auth.InstanceID(c), "workflow.deleted", actorFromCtx(c), "", c.Param("id"), "workflow deleted")
c.JSON(http.StatusOK, gin.H{"deleted": true})
}
func runWorkflow(c *gin.Context) {
runID, err := services.TriggerWorkflow(c.Param("id"), actorFromCtx(c))
runID, err := services.TriggerWorkflow(auth.InstanceID(c), c.Param("id"), actorFromCtx(c))
if err != nil {
c.JSON(http.StatusServiceUnavailable, gin.H{"error": err.Error()})
return
}
services.LogEvent("workflow.run_triggered", actorFromCtx(c), "", c.Param("id"), fmt.Sprintf("run %s triggered", runID))
services.LogEvent(auth.InstanceID(c), "workflow.run_triggered", actorFromCtx(c), "", c.Param("id"), fmt.Sprintf("run %s triggered", runID))
c.JSON(http.StatusAccepted, gin.H{"run_id": runID})
}
@@ -148,7 +332,7 @@ func listWorkflowRuns(c *gin.Context) {
limit = n
}
}
runs, err := services.ListRuns(c.Param("id"), limit)
runs, err := services.ListRuns(auth.InstanceID(c), c.Param("id"), limit)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
@@ -157,7 +341,7 @@ func listWorkflowRuns(c *gin.Context) {
}
func getRun(c *gin.Context) {
r, err := services.GetRun(c.Param("runId"))
r, err := services.GetRun(auth.InstanceID(c), c.Param("runId"))
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": err.Error()})
return
@@ -166,10 +350,10 @@ func getRun(c *gin.Context) {
}
func cancelRun(c *gin.Context) {
if err := services.CancelRun(c.Param("runId")); err != nil {
if err := services.CancelRun(auth.InstanceID(c), c.Param("runId")); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
services.LogEvent("workflow.run_cancelled", actorFromCtx(c), "", c.Param("runId"), "run cancelled")
services.LogEvent(auth.InstanceID(c), "workflow.run_cancelled", actorFromCtx(c), "", c.Param("runId"), "run cancelled")
c.JSON(http.StatusOK, gin.H{"cancelled": true})
}
+74
View File
@@ -0,0 +1,74 @@
package auth
import (
"os"
"strings"
"sync"
"time"
"github.com/gin-gonic/gin"
"github.com/mrhid6/vantage/server/internal/models"
"github.com/mrhid6/vantage/server/internal/services"
)
type cachedInstance struct {
instance *models.Instance
at time.Time
}
var (
instanceCacheMu sync.Mutex
instanceCache = map[string]cachedInstance{}
)
const instanceCacheTTL = 60 * time.Second
func appRootLabel() string {
if v := os.Getenv("APP_ROOT_LABEL"); v != "" {
return strings.ToLower(v)
}
return "vantage"
}
func hostSlug(host string) string {
host = strings.ToLower(host)
if i := strings.IndexByte(host, ':'); i >= 0 {
host = host[:i]
}
root := appRootLabel()
parts := strings.Split(host, ".")
if len(parts) < 3 {
return ""
}
if parts[1] != root {
return ""
}
if parts[0] == root || parts[0] == "www" {
return ""
}
return parts[0]
}
func InstanceFromHost(c *gin.Context) (*models.Instance, bool) {
slug := hostSlug(c.Request.Host)
if slug == "" {
return nil, false
}
instanceCacheMu.Lock()
if e, ok := instanceCache[slug]; ok && time.Since(e.at) < instanceCacheTTL {
instanceCacheMu.Unlock()
return e.instance, e.instance != nil
}
instanceCacheMu.Unlock()
inst, err := services.GetInstanceBySlug(slug)
if err != nil || inst == nil {
return nil, false
}
instanceCacheMu.Lock()
instanceCache[slug] = cachedInstance{instance: inst, at: time.Now()}
instanceCacheMu.Unlock()
return inst, true
}
+155
View File
@@ -0,0 +1,155 @@
package auth
import (
"fmt"
"net/http"
"github.com/gin-gonic/gin"
"github.com/mrhid6/vantage/server/internal/models"
"github.com/mrhid6/vantage/server/internal/services"
)
func SetSessionCookie(c *gin.Context, sessionID string) {
secure := c.Request.TLS != nil || c.GetHeader("X-Forwarded-Proto") == "https"
http.SetCookie(c.Writer, &http.Cookie{
Name: sessionCookieName,
Value: sessionID,
Path: "/",
HttpOnly: true,
Secure: secure,
SameSite: http.SameSiteLaxMode,
MaxAge: int(sessionTTL.Seconds()),
})
}
func HandleLocalLogin(c *gin.Context) {
var body struct {
Email string `json:"email"`
Password string `json:"password"`
}
if err := c.ShouldBindJSON(&body); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "email and password required"})
return
}
u, err := services.GetUserByEmail(body.Email)
if err != nil || !services.VerifyPassword(u, body.Password) {
c.JSON(http.StatusUnauthorized, gin.H{"error": "invalid credentials"})
return
}
sessionID, err := SaveSession(c.Request.Context(), &Session{
UserID: u.UserID, InstanceID: u.InstanceID, Role: u.Role, Email: u.Email,
})
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "session save failed"})
return
}
_ = services.TouchLastLogin(u.UserID)
SetSessionCookie(c, sessionID)
c.JSON(http.StatusOK, gin.H{"ok": true})
}
func HandleBootstrapStatus(c *gin.Context) {
var (
n int64
err error
instName string
)
if inst, ok := InstanceFromHost(c); ok {
n, err = services.CountInstanceUsers(inst.InstanceID)
instName = inst.Name
} else {
n, err = services.CountUsers()
}
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"needs_setup": n == 0, "instance_name": instName})
}
func HandleBootstrap(c *gin.Context) {
n, err := services.CountUsers()
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
if n > 0 {
c.JSON(http.StatusConflict, gin.H{"error": "setup already complete"})
return
}
var body struct {
InstanceName string `json:"instance_name"`
Email string `json:"email"`
Password string `json:"password"`
}
if err := c.ShouldBindJSON(&body); err != nil || body.InstanceName == "" || body.Email == "" || len(body.Password) < 8 {
c.JSON(http.StatusBadRequest, gin.H{"error": "instance_name, email, and password (>=8 chars) required"})
return
}
instanceCount, err := services.CountInstances()
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
var inst *models.Instance
switch instanceCount {
case 0:
inst, err = services.CreateInstance(body.InstanceName)
case 1:
var existing *models.Instance
existing, err = services.FirstInstance()
if err == nil {
inst, err = services.AdoptInstance(existing.InstanceID, body.InstanceName)
}
default:
c.JSON(http.StatusConflict, gin.H{"error": fmt.Sprintf(
"cannot bootstrap: %d organizations already exist but no users do; "+
"create the owner against the intended inst rather than through setup, "+
"or remove the unintended orgs and retry", instanceCount)})
return
}
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
u, err := services.CreateUser(inst.InstanceID, body.Email, body.Password, "owner", "local")
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
sessionID, err := SaveSession(c.Request.Context(), &Session{
UserID: u.UserID, InstanceID: u.InstanceID, Role: u.Role, Email: u.Email,
})
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "session save failed"})
return
}
SetSessionCookie(c, sessionID)
c.JSON(http.StatusCreated, gin.H{
"instance": inst,
"slug": inst.Slug,
"instance_id": inst.InstanceID,
})
}
func HandleMe(c *gin.Context) {
cookie, err := c.Request.Cookie(sessionCookieName)
if err != nil {
c.JSON(http.StatusUnauthorized, gin.H{"error": "not authenticated"})
return
}
sess, err := GetSession(c.Request.Context(), cookie.Value)
if err != nil {
c.JSON(http.StatusUnauthorized, gin.H{"error": "session expired"})
return
}
if hostInstance, ok := InstanceFromHost(c); ok && hostInstance.InstanceID != sess.InstanceID {
c.JSON(http.StatusForbidden, gin.H{"error": "instance host mismatch"})
return
}
inst, _ := services.GetInstance(sess.InstanceID)
c.JSON(http.StatusOK, gin.H{"user": sess, "instance": inst})
}
+45 -5
View File
@@ -14,13 +14,42 @@ func GetSessionFromContext(c *gin.Context) *Session {
return sess
}
func InstanceID(c *gin.Context) string {
if s := GetSessionFromContext(c); s != nil {
return s.InstanceID
}
return ""
}
func Role(c *gin.Context) string {
if s := GetSessionFromContext(c); s != nil {
return s.Role
}
return ""
}
func UserID(c *gin.Context) string {
if s := GetSessionFromContext(c); s != nil {
return s.UserID
}
return ""
}
func RequireRole(roles ...string) gin.HandlerFunc {
return func(c *gin.Context) {
r := Role(c)
for _, want := range roles {
if r == want {
c.Next()
return
}
}
c.AbortWithStatusJSON(http.StatusForbidden, gin.H{"error": "insufficient role"})
}
}
func Middleware() gin.HandlerFunc {
return func(c *gin.Context) {
if !authEnabled {
c.Next()
return
}
cookie, err := c.Request.Cookie(sessionCookieName)
if err != nil {
c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "not authenticated"})
@@ -33,7 +62,18 @@ func Middleware() gin.HandlerFunc {
return
}
if sess.InstanceID == "" {
c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "session has no organization"})
return
}
c.Set(ctxSessionKey, sess)
if hostInstance, ok := InstanceFromHost(c); ok && hostInstance.InstanceID != sess.InstanceID {
c.AbortWithStatusJSON(http.StatusForbidden, gin.H{"error": "instance host mismatch"})
return
}
c.Next()
}
}
+96 -83
View File
@@ -2,123 +2,154 @@ package auth
import (
"context"
"log"
"fmt"
"net/http"
"os"
"strings"
"sync"
"github.com/coreos/go-oidc/v3/oidc"
"github.com/gin-gonic/gin"
"github.com/mrhid6/vantage/server/internal/services"
"golang.org/x/oauth2"
)
var (
oidcProvider *oidc.Provider
oauth2Cfg *oauth2.Config
authEnabled bool
provMu sync.Mutex
provCache = map[string]*oidc.Provider{}
)
func InitOIDC(ctx context.Context) error {
issuer := os.Getenv("OIDC_ISSUER")
if issuer == "" {
log.Println("OIDC_ISSUER not set; authentication disabled")
return nil
}
p, err := oidc.NewProvider(ctx, issuer)
if err != nil {
return err
}
oidcProvider = p
oauth2Cfg = &oauth2.Config{
ClientID: os.Getenv("OIDC_CLIENT_ID"),
ClientSecret: os.Getenv("OIDC_CLIENT_SECRET"),
RedirectURL: os.Getenv("OIDC_REDIRECT_URL"),
Endpoint: p.Endpoint(),
Scopes: []string{oidc.ScopeOpenID, "profile", "email"},
}
authEnabled = true
log.Println("OIDC authentication enabled")
return nil
func EvictOIDCProvider(instanceID string) {
provMu.Lock()
delete(provCache, instanceID)
provMu.Unlock()
}
func Enabled() bool { return authEnabled }
func redirectURL(c *gin.Context) string {
scheme := "https"
if c.Request.TLS == nil && c.GetHeader("X-Forwarded-Proto") != "https" {
scheme = "http"
}
return fmt.Sprintf("%s://%s/auth/oidc/callback", scheme, c.Request.Host)
}
func HandleLogin(c *gin.Context) {
state, err := randomHex(16)
func providerForInstance(ctx context.Context, c *gin.Context, instanceID string) (*oidc.Provider, *oauth2.Config, error) {
cfg, err := services.GetInstanceOIDC(instanceID)
if err != nil || !cfg.Enabled {
return nil, nil, fmt.Errorf("inst SSO not configured")
}
secret, err := services.GetInstanceOIDCSecret(instanceID)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "state generation failed"})
return nil, nil, err
}
provMu.Lock()
p := provCache[instanceID]
provMu.Unlock()
if p == nil {
p, err = oidc.NewProvider(ctx, cfg.Issuer)
if err != nil {
return nil, nil, err
}
provMu.Lock()
provCache[instanceID] = p
provMu.Unlock()
}
return p, &oauth2.Config{
ClientID: cfg.ClientID, ClientSecret: secret,
RedirectURL: redirectURL(c), Endpoint: p.Endpoint(),
Scopes: []string{oidc.ScopeOpenID, "profile", "email"},
}, nil
}
func HandleOIDCStart(c *gin.Context) {
inst, ok := InstanceFromHost(c)
if !ok {
c.JSON(http.StatusBadRequest, gin.H{"error": "unknown instance host"})
return
}
if err := SaveState(c.Request.Context(), state); err != nil {
// Losing the feature stops new SSO logins. It deliberately does not touch
// session validation, so nobody is evicted mid-session.
if !services.GetLicenseState(inst.InstanceID).Feature("oidc") {
c.Redirect(http.StatusFound, "/login?error=oidc_unavailable")
return
}
ctx := c.Request.Context()
_, oauthCfg, err := providerForInstance(ctx, c, inst.InstanceID)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
state, err := randomHex(16)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "state gen failed"})
return
}
if err := SaveStateInstance(ctx, state, inst.InstanceID); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "state save failed"})
return
}
c.Redirect(http.StatusFound, oauth2Cfg.AuthCodeURL(state))
c.Redirect(http.StatusFound, oauthCfg.AuthCodeURL(state))
}
func HandleCallback(c *gin.Context) {
func HandleOIDCCallback(c *gin.Context) {
ctx := c.Request.Context()
if !ConsumeState(ctx, c.Query("state")) {
instanceID, ok := ConsumeStateInstance(ctx, c.Query("state"))
if !ok {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid state"})
return
}
token, err := oauth2Cfg.Exchange(ctx, c.Query("code"))
provider, oauthCfg, err := providerForInstance(ctx, c, instanceID)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
token, err := oauthCfg.Exchange(ctx, c.Query("code"))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "token exchange failed"})
return
}
rawIDToken, ok := token.Extra("id_token").(string)
if !ok {
c.JSON(http.StatusInternalServerError, gin.H{"error": "missing id_token"})
return
}
verifier := oidcProvider.Verifier(&oidc.Config{ClientID: oauth2Cfg.ClientID})
idToken, err := verifier.Verify(ctx, rawIDToken)
idToken, err := provider.Verifier(&oidc.Config{ClientID: oauthCfg.ClientID}).Verify(ctx, rawIDToken)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "token verification failed"})
return
}
var claims struct {
Sub string `json:"sub"`
Email string `json:"email"`
Name string `json:"name"`
}
if err := idToken.Claims(&claims); err != nil {
if err := idToken.Claims(&claims); err != nil || claims.Email == "" {
c.JSON(http.StatusInternalServerError, gin.H{"error": "claims extraction failed"})
return
}
email := strings.ToLower(claims.Email)
u, err := services.GetUserByEmail(email)
if err != nil {
u, err = services.CreateUser(instanceID, email, "", "member", "oidc")
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "provisioning failed"})
return
}
} else if u.InstanceID != instanceID {
c.JSON(http.StatusForbidden, gin.H{"error": "email belongs to a different organization"})
return
}
sessionID, err := SaveSession(ctx, &Session{
UserID: claims.Sub,
Email: claims.Email,
Name: claims.Name,
UserID: u.UserID, InstanceID: u.InstanceID, Role: u.Role, Email: u.Email, Name: claims.Name,
})
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "session save failed"})
return
}
secure := c.Request.TLS != nil || c.GetHeader("X-Forwarded-Proto") == "https"
http.SetCookie(c.Writer, &http.Cookie{
Name: sessionCookieName,
Value: sessionID,
Path: "/",
HttpOnly: true,
Secure: secure,
SameSite: http.SameSiteLaxMode,
MaxAge: int(sessionTTL.Seconds()),
})
frontendURL := os.Getenv("PUBLIC_HOST")
if frontendURL == "" {
frontendURL = "/"
}
c.Redirect(http.StatusFound, frontendURL)
_ = services.TouchLastLogin(u.UserID)
SetSessionCookie(c, sessionID)
c.Redirect(http.StatusFound, "/")
}
func HandleLogout(c *gin.Context) {
@@ -134,21 +165,3 @@ func HandleLogout(c *gin.Context) {
})
c.Redirect(http.StatusFound, "/")
}
func HandleMe(c *gin.Context) {
if !authEnabled {
c.JSON(http.StatusOK, gin.H{"auth_enabled": false})
return
}
cookie, err := c.Request.Cookie(sessionCookieName)
if err != nil {
c.JSON(http.StatusUnauthorized, gin.H{"error": "not authenticated"})
return
}
sess, err := GetSession(c.Request.Context(), cookie.Value)
if err != nil {
c.JSON(http.StatusUnauthorized, gin.H{"error": "session expired"})
return
}
c.JSON(http.StatusOK, sess)
}
+13 -8
View File
@@ -16,9 +16,11 @@ const sessionPrefix = "km:session:"
const statePrefix = "km:state:"
type Session struct {
UserID string `json:"user_id"`
Email string `json:"email"`
Name string `json:"name"`
UserID string `json:"user_id"`
InstanceID string `json:"instance_id"`
Role string `json:"role"`
Email string `json:"email"`
Name string `json:"name"`
}
var rdb *redis.Client
@@ -69,11 +71,14 @@ func DeleteSession(ctx context.Context, id string) error {
return rdb.Del(ctx, sessionPrefix+id).Err()
}
func SaveState(ctx context.Context, state string) error {
return rdb.Set(ctx, statePrefix+state, "1", 10*time.Minute).Err()
func SaveStateInstance(ctx context.Context, state, instanceID string) error {
return rdb.Set(ctx, statePrefix+state, instanceID, 10*time.Minute).Err()
}
func ConsumeState(ctx context.Context, state string) bool {
n, err := rdb.Del(ctx, statePrefix+state).Result()
return err == nil && n > 0
func ConsumeStateInstance(ctx context.Context, state string) (string, bool) {
instanceID, err := rdb.GetDel(ctx, statePrefix+state).Result()
if err != nil || instanceID == "" {
return "", false
}
return instanceID, true
}
+215
View File
@@ -0,0 +1,215 @@
package checker
import (
"context"
"crypto/tls"
"fmt"
"io"
"net"
"net/http"
"os"
"strings"
"time"
)
const (
TypeHTTP = "http"
TypeTCP = "tcp"
TypeICMP = "icmp"
TypeTLS = "tls"
)
type Spec struct {
Type string
URL string
Host string
Port int
Method string
ExpectedStatus int
Keyword string
TLSWarnDays int
Insecure bool
TimeoutSec int
}
type Result struct {
Up bool
LatencyMs int
Message string
CertExpiry *time.Time
}
func (s Spec) timeout() time.Duration {
t := s.TimeoutSec
if t <= 0 || t > 10 {
t = 10
}
return time.Duration(t) * time.Second
}
func Run(ctx context.Context, s Spec) Result {
switch s.Type {
case TypeHTTP:
return runHTTP(ctx, s)
case TypeTCP:
return runTCP(ctx, s)
case TypeICMP:
return runICMP(ctx, s)
case TypeTLS:
return runTLS(ctx, s)
default:
return Result{Message: "unknown check type: " + s.Type}
}
}
func runHTTP(ctx context.Context, s Spec) Result {
method := s.Method
if method == "" {
method = http.MethodGet
}
expect := s.ExpectedStatus
if expect == 0 {
expect = 200
}
client := &http.Client{Timeout: s.timeout()}
if s.Insecure {
client.Transport = &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}}
}
start := time.Now()
req, err := http.NewRequestWithContext(ctx, method, s.URL, nil)
if err != nil {
return Result{Message: err.Error()}
}
resp, err := client.Do(req)
if err != nil {
return Result{LatencyMs: msSince(start), Message: err.Error()}
}
defer resp.Body.Close()
res := Result{LatencyMs: msSince(start), Up: true}
if resp.TLS != nil && len(resp.TLS.PeerCertificates) > 0 {
exp := resp.TLS.PeerCertificates[0].NotAfter
res.CertExpiry = &exp
}
if resp.StatusCode != expect {
return Result{LatencyMs: res.LatencyMs, CertExpiry: res.CertExpiry, Message: fmt.Sprintf("status %d (want %d)", resp.StatusCode, expect)}
}
if s.Keyword != "" {
body, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if !strings.Contains(string(body), s.Keyword) {
return Result{LatencyMs: res.LatencyMs, CertExpiry: res.CertExpiry, Message: "keyword not found"}
}
}
return res
}
func runTCP(ctx context.Context, s Spec) Result {
addr := net.JoinHostPort(s.Host, fmt.Sprint(s.Port))
start := time.Now()
d := net.Dialer{Timeout: s.timeout()}
conn, err := d.DialContext(ctx, "tcp", addr)
if err != nil {
return Result{LatencyMs: msSince(start), Message: err.Error()}
}
conn.Close()
return Result{Up: true, LatencyMs: msSince(start)}
}
func runTLS(ctx context.Context, s Spec) Result {
port := s.Port
if port == 0 {
port = 443
}
addr := net.JoinHostPort(s.Host, fmt.Sprint(port))
start := time.Now()
d := net.Dialer{Timeout: s.timeout()}
conn, err := tls.DialWithDialer(&d, "tcp", addr, &tls.Config{ServerName: s.Host})
if err != nil {
return Result{LatencyMs: msSince(start), Message: err.Error()}
}
defer conn.Close()
certs := conn.ConnectionState().PeerCertificates
if len(certs) == 0 {
return Result{LatencyMs: msSince(start), Message: "no peer certificate"}
}
exp := certs[0].NotAfter
res := Result{LatencyMs: msSince(start), CertExpiry: &exp}
warn := s.TLSWarnDays
if warn <= 0 {
warn = 14
}
remaining := time.Until(exp)
if remaining <= 0 {
res.Message = "certificate expired"
return res
}
if remaining <= time.Duration(warn)*24*time.Hour {
res.Message = fmt.Sprintf("certificate expires in %d days", int(remaining.Hours()/24))
return res
}
res.Up = true
return res
}
func msSince(t time.Time) int { return int(time.Since(t).Milliseconds()) }
func runICMP(ctx context.Context, s Spec) Result {
dst, err := net.ResolveIPAddr("ip4", s.Host)
if err != nil {
return Result{Message: err.Error()}
}
conn, err := net.ListenPacket("ip4:icmp", "0.0.0.0")
if err != nil {
return Result{Message: "icmp socket: " + err.Error()}
}
defer conn.Close()
id := os.Getpid() & 0xffff
pkt := icmpEcho(id, 1)
deadline := time.Now().Add(s.timeout())
if d, ok := ctx.Deadline(); ok && d.Before(deadline) {
deadline = d
}
_ = conn.SetDeadline(deadline)
start := time.Now()
if _, err := conn.WriteTo(pkt, dst); err != nil {
return Result{Message: err.Error()}
}
reply := make([]byte, 1500)
for {
n, peer, err := conn.ReadFrom(reply)
if err != nil {
return Result{LatencyMs: msSince(start), Message: "no reply"}
}
if n < 28 || peer.String() != dst.String() {
continue
}
if reply[20] == 0 {
return Result{Up: true, LatencyMs: msSince(start)}
}
}
}
func icmpEcho(id, seq int) []byte {
b := []byte{8, 0, 0, 0, byte(id >> 8), byte(id), byte(seq >> 8), byte(seq)}
cs := icmpChecksum(b)
b[2] = byte(cs >> 8)
b[3] = byte(cs)
return b
}
func icmpChecksum(b []byte) uint16 {
var sum uint32
for i := 0; i < len(b)-1; i += 2 {
sum += uint32(b[i])<<8 | uint32(b[i+1])
}
if len(b)%2 == 1 {
sum += uint32(b[len(b)-1]) << 8
}
for sum>>16 != 0 {
sum = (sum & 0xffff) + (sum >> 16)
}
return ^uint16(sum)
}
+1 -2
View File
@@ -4,7 +4,6 @@ import (
"encoding/json"
)
// JSONCodec is a gRPC codec that uses JSON encoding.
type JSONCodec struct{}
func (JSONCodec) Marshal(v interface{}) ([]byte, error) {
@@ -16,5 +15,5 @@ func (JSONCodec) Unmarshal(data []byte, v interface{}) error {
}
func (JSONCodec) Name() string {
return "proto" // override default proto codec name so gRPC uses it
return "proto"
}
+174 -29
View File
@@ -1,6 +1,3 @@
// Hand-written gRPC bindings for vantage.proto using JSON codec.
// To use: register the JSON codec before creating gRPC servers/clients.
package pb
import (
@@ -11,8 +8,6 @@ import (
"google.golang.org/grpc/status"
)
// Message types
type RegisterRequest struct {
ServerId string `json:"server_id"`
PreRegToken string `json:"pre_reg_token"`
@@ -47,8 +42,6 @@ type UploadKeyResponse struct {
KeyId string `json:"key_id"`
}
// CommandStream message types
type PackageUpdate struct {
Name string `json:"name"`
CurrentVersion string `json:"current_version,omitempty"`
@@ -63,15 +56,85 @@ type ReportUpdatesRequest struct {
type ReportUpdatesResponse struct{}
type CPUReport struct {
Model string `json:"model,omitempty"`
Cores int `json:"cores,omitempty"`
UsagePct float64 `json:"usage_pct"`
Load1 float64 `json:"load1,omitempty"`
}
type MemReport struct {
TotalBytes uint64 `json:"total_bytes"`
UsedBytes uint64 `json:"used_bytes"`
}
type PartitionReport struct {
Device string `json:"device"`
Mountpoint string `json:"mountpoint"`
Fstype string `json:"fstype,omitempty"`
TotalBytes uint64 `json:"total_bytes"`
UsedBytes uint64 `json:"used_bytes"`
}
type InventoryReport struct {
ServerId string `json:"server_id"`
AgentToken string `json:"agent_token"`
IncludeStatic bool `json:"include_static"`
CPU *CPUReport `json:"cpu,omitempty"`
Memory *MemReport `json:"memory,omitempty"`
SwapTotal uint64 `json:"swap_total"`
SwapUsed uint64 `json:"swap_used"`
Partitions []PartitionReport `json:"partitions,omitempty"`
Kernel string `json:"kernel,omitempty"`
}
type InventoryReportResponse struct{}
type MonitorSpec struct {
MonitorId string `json:"monitor_id"`
Type string `json:"type"`
URL string `json:"url,omitempty"`
Host string `json:"host,omitempty"`
Port int `json:"port,omitempty"`
Method string `json:"method,omitempty"`
ExpectedStatus int `json:"expected_status,omitempty"`
Keyword string `json:"keyword,omitempty"`
TLSWarnDays int `json:"tls_warn_days,omitempty"`
Insecure bool `json:"insecure,omitempty"`
IntervalSec int `json:"interval_sec"`
Retries int `json:"retries"`
}
type SyncMonitorsRequest struct {
ServerId string `json:"server_id"`
AgentToken string `json:"agent_token"`
}
type SyncMonitorsResponse struct {
Monitors []MonitorSpec `json:"monitors,omitempty"`
}
type CheckResult struct {
MonitorId string `json:"monitor_id"`
Up bool `json:"up"`
LatencyMs int `json:"latency_ms"`
Message string `json:"message,omitempty"`
CertExpiryUnix int64 `json:"cert_expiry_unix,omitempty"`
}
type ReportChecksRequest struct {
ServerId string `json:"server_id"`
AgentToken string `json:"agent_token"`
Results []CheckResult `json:"results,omitempty"`
}
type ReportChecksResponse struct{}
type ApplyUpdatesCmd struct{}
type ServerCommand struct {
CommandId string `json:"command_id"`
GenerateKey *GenerateKeyCmd `json:"generate_key,omitempty"`
DeleteKey *DeleteKeyCmd `json:"delete_key,omitempty"`
UpdateAgent *UpdateAgentCmd `json:"update_agent,omitempty"`
ApplyUpdates *ApplyUpdatesCmd `json:"apply_updates,omitempty"`
RunStep *RunStepCmd `json:"run_step,omitempty"`
CommandId string `json:"command_id"`
GenerateKey *GenerateKeyCmd `json:"generate_key,omitempty"`
DeleteKey *DeleteKeyCmd `json:"delete_key,omitempty"`
UpdateAgent *UpdateAgentCmd `json:"update_agent,omitempty"`
ApplyUpdates *ApplyUpdatesCmd `json:"apply_updates,omitempty"`
RunStep *RunStepCmd `json:"run_step,omitempty"`
CleanupWorkspace *CleanupWorkspaceCmd `json:"cleanup_workspace,omitempty"`
}
type CleanupWorkspaceCmd struct {
WorkspaceId string `json:"workspace_id"`
}
type DeleteKeyCmd struct {
@@ -92,12 +155,12 @@ type GenerateKeyCmd struct {
}
type AgentMessage struct {
ServerId string `json:"server_id"`
AgentToken string `json:"agent_token"`
Ready *AgentReady `json:"ready,omitempty"`
Result *CommandResult `json:"result,omitempty"`
StepResult *StepResult `json:"step_result,omitempty"`
StepOutput *StepOutputChunk `json:"step_output,omitempty"`
ServerId string `json:"server_id"`
AgentToken string `json:"agent_token"`
Ready *AgentReady `json:"ready,omitempty"`
Result *CommandResult `json:"result,omitempty"`
StepResult *StepResult `json:"step_result,omitempty"`
StepOutput *StepOutputChunk `json:"step_output,omitempty"`
}
type AgentReady struct{}
@@ -113,6 +176,8 @@ type RunStepCmd struct {
Script string `json:"script"`
Env map[string]string `json:"env,omitempty"`
TimeoutSeconds int `json:"timeout_seconds,omitempty"`
WorkspaceId string `json:"workspace_id,omitempty"`
}
type StepResult struct {
@@ -130,8 +195,6 @@ type StepOutputChunk struct {
Eof bool `json:"eof,omitempty"`
}
// CommandStream server-side interface
type Vantage_CommandStreamServer interface {
Send(*ServerCommand) error
Recv() (*AgentMessage, error)
@@ -154,8 +217,6 @@ func (s *keyManagerCommandStreamServer) Recv() (*AgentMessage, error) {
return m, nil
}
// CommandStream client-side interface
type Vantage_CommandStreamClient interface {
Send(*AgentMessage) error
Recv() (*ServerCommand, error)
@@ -178,13 +239,14 @@ func (c *vantageCommandStreamClient) Recv() (*ServerCommand, error) {
return m, nil
}
// Server interface
type VantageServer interface {
Register(context.Context, *RegisterRequest) (*RegisterResponse, error)
SyncKeys(context.Context, *SyncRequest) (*SyncResponse, error)
UploadGeneratedKey(context.Context, *UploadKeyRequest) (*UploadKeyResponse, error)
ReportUpdates(context.Context, *ReportUpdatesRequest) (*ReportUpdatesResponse, error)
ReportInventory(context.Context, *InventoryReport) (*InventoryReportResponse, error)
SyncMonitors(context.Context, *SyncMonitorsRequest) (*SyncMonitorsResponse, error)
ReportChecks(context.Context, *ReportChecksRequest) (*ReportChecksResponse, error)
CommandStream(Vantage_CommandStreamServer) error
}
@@ -206,17 +268,30 @@ func (UnimplementedVantageServer) ReportUpdates(context.Context, *ReportUpdatesR
return nil, status.Errorf(codes.Unimplemented, "method ReportUpdates not implemented")
}
func (UnimplementedVantageServer) ReportInventory(context.Context, *InventoryReport) (*InventoryReportResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method ReportInventory not implemented")
}
func (UnimplementedVantageServer) SyncMonitors(context.Context, *SyncMonitorsRequest) (*SyncMonitorsResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method SyncMonitors not implemented")
}
func (UnimplementedVantageServer) ReportChecks(context.Context, *ReportChecksRequest) (*ReportChecksResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method ReportChecks not implemented")
}
func (UnimplementedVantageServer) CommandStream(Vantage_CommandStreamServer) error {
return status.Errorf(codes.Unimplemented, "method CommandStream not implemented")
}
// Client interface
type VantageClient interface {
Register(ctx context.Context, in *RegisterRequest, opts ...grpc.CallOption) (*RegisterResponse, error)
SyncKeys(ctx context.Context, in *SyncRequest, opts ...grpc.CallOption) (*SyncResponse, error)
UploadGeneratedKey(ctx context.Context, in *UploadKeyRequest, opts ...grpc.CallOption) (*UploadKeyResponse, error)
ReportUpdates(ctx context.Context, in *ReportUpdatesRequest, opts ...grpc.CallOption) (*ReportUpdatesResponse, error)
ReportInventory(ctx context.Context, in *InventoryReport, opts ...grpc.CallOption) (*InventoryReportResponse, error)
SyncMonitors(ctx context.Context, in *SyncMonitorsRequest, opts ...grpc.CallOption) (*SyncMonitorsResponse, error)
ReportChecks(ctx context.Context, in *ReportChecksRequest, opts ...grpc.CallOption) (*ReportChecksResponse, error)
CommandStream(ctx context.Context, opts ...grpc.CallOption) (Vantage_CommandStreamClient, error)
}
@@ -260,6 +335,30 @@ func (c *keyManagerClient) ReportUpdates(ctx context.Context, in *ReportUpdatesR
return out, nil
}
func (c *keyManagerClient) ReportInventory(ctx context.Context, in *InventoryReport, opts ...grpc.CallOption) (*InventoryReportResponse, error) {
out := new(InventoryReportResponse)
if err := c.cc.Invoke(ctx, "/vantage.v1.Vantage/ReportInventory", in, out, opts...); err != nil {
return nil, err
}
return out, nil
}
func (c *keyManagerClient) SyncMonitors(ctx context.Context, in *SyncMonitorsRequest, opts ...grpc.CallOption) (*SyncMonitorsResponse, error) {
out := new(SyncMonitorsResponse)
if err := c.cc.Invoke(ctx, "/vantage.v1.Vantage/SyncMonitors", in, out, opts...); err != nil {
return nil, err
}
return out, nil
}
func (c *keyManagerClient) ReportChecks(ctx context.Context, in *ReportChecksRequest, opts ...grpc.CallOption) (*ReportChecksResponse, error) {
out := new(ReportChecksResponse)
if err := c.cc.Invoke(ctx, "/vantage.v1.Vantage/ReportChecks", in, out, opts...); err != nil {
return nil, err
}
return out, nil
}
func (c *keyManagerClient) CommandStream(ctx context.Context, opts ...grpc.CallOption) (Vantage_CommandStreamClient, error) {
stream, err := c.cc.NewStream(ctx, &Vantage_ServiceDesc.Streams[0], "/vantage.v1.Vantage/CommandStream", opts...)
if err != nil {
@@ -268,8 +367,6 @@ func (c *keyManagerClient) CommandStream(ctx context.Context, opts ...grpc.CallO
return &vantageCommandStreamClient{stream}, nil
}
// Server registration
func RegisterVantageServer(s grpc.ServiceRegistrar, srv VantageServer) {
s.RegisterService(&Vantage_ServiceDesc, srv)
}
@@ -282,6 +379,9 @@ var Vantage_ServiceDesc = grpc.ServiceDesc{
{MethodName: "SyncKeys", Handler: _Vantage_SyncKeys_Handler},
{MethodName: "UploadGeneratedKey", Handler: _Vantage_UploadGeneratedKey_Handler},
{MethodName: "ReportUpdates", Handler: _Vantage_ReportUpdates_Handler},
{MethodName: "ReportInventory", Handler: _Vantage_ReportInventory_Handler},
{MethodName: "SyncMonitors", Handler: _Vantage_SyncMonitors_Handler},
{MethodName: "ReportChecks", Handler: _Vantage_ReportChecks_Handler},
},
Streams: []grpc.StreamDesc{
{
@@ -354,6 +454,51 @@ func _Vantage_ReportUpdates_Handler(srv interface{}, ctx context.Context, dec fu
return interceptor(ctx, in, info, handler)
}
func _Vantage_ReportInventory_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(InventoryReport)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(VantageServer).ReportInventory(ctx, in)
}
info := &grpc.UnaryServerInfo{Server: srv, FullMethod: "/vantage.v1.Vantage/ReportInventory"}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(VantageServer).ReportInventory(ctx, req.(*InventoryReport))
}
return interceptor(ctx, in, info, handler)
}
func _Vantage_SyncMonitors_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(SyncMonitorsRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(VantageServer).SyncMonitors(ctx, in)
}
info := &grpc.UnaryServerInfo{Server: srv, FullMethod: "/vantage.v1.Vantage/SyncMonitors"}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(VantageServer).SyncMonitors(ctx, req.(*SyncMonitorsRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Vantage_ReportChecks_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ReportChecksRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(VantageServer).ReportChecks(ctx, in)
}
info := &grpc.UnaryServerInfo{Server: srv, FullMethod: "/vantage.v1.Vantage/ReportChecks"}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(VantageServer).ReportChecks(ctx, req.(*ReportChecksRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Vantage_CommandStream_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(VantageServer).CommandStream(&keyManagerCommandStreamServer{stream})
}
+81 -11
View File
@@ -7,6 +7,7 @@ import (
"net"
"time"
"github.com/mrhid6/vantage/server/internal/checker"
"github.com/mrhid6/vantage/server/internal/grpc/pb"
"github.com/mrhid6/vantage/server/internal/models"
"github.com/mrhid6/vantage/server/internal/services"
@@ -25,6 +26,16 @@ type vantageServer struct {
pb.UnimplementedVantageServer
}
// Register carries no licence check, deliberately.
//
// A server row only ever comes from CreateServer, which checks the cap before
// issuing a pre-registration token. By the time an agent calls Register its row
// already exists, so counting here would count the caller itself: an instance
// sitting exactly at its cap would reject the very agent it just authorised, and
// every re-registration after a reinstall would fail too.
//
// The cap is enforced where rows are created, which is the only place it can be
// enforced correctly.
func (s *vantageServer) Register(ctx context.Context, req *pb.RegisterRequest) (*pb.RegisterResponse, error) {
agentToken, err := services.RegisterServer(req.ServerId, req.PreRegToken, req.Hostname, req.IpAddress, req.OsInfo)
if err != nil {
@@ -61,14 +72,12 @@ func (s *vantageServer) UploadGeneratedKey(ctx context.Context, req *pb.UploadKe
return nil, status.Errorf(codes.Unauthenticated, "invalid agent token")
}
// Agent-generated keys carry no passphrase over the wire (proto has no field).
key, err := services.CreateKey(req.Label, req.PublicKey, "generated", srv.ServerID, req.PrivateKey, "")
key, err := services.CreateKey(srv.InstanceID, req.Label, req.PublicKey, "generated", srv.ServerID, req.PrivateKey, "")
if err != nil {
return nil, status.Errorf(codes.Internal, "failed to store key: %v", err)
}
// Auto-assign to the generating server
if _, err := services.AssignKey(key.KeyID, srv.ServerID); err != nil {
if _, err := services.AssignKey(srv.InstanceID, key.KeyID, srv.ServerID); err != nil {
log.Printf("failed to auto-assign generated key: %v", err)
}
@@ -95,8 +104,66 @@ func (s *vantageServer) ReportUpdates(ctx context.Context, req *pb.ReportUpdates
return &pb.ReportUpdatesResponse{}, nil
}
func (s *vantageServer) ReportInventory(ctx context.Context, req *pb.InventoryReport) (*pb.InventoryReportResponse, error) {
srv, err := services.ValidateAgentToken(req.ServerId, req.AgentToken)
if err != nil {
return nil, status.Errorf(codes.Unauthenticated, "invalid agent token")
}
if err := services.StoreInventory(srv.ServerID, req); err != nil {
log.Printf("store inventory for %s: %v", srv.ServerID, err)
}
return &pb.InventoryReportResponse{}, nil
}
func (s *vantageServer) SyncMonitors(ctx context.Context, req *pb.SyncMonitorsRequest) (*pb.SyncMonitorsResponse, error) {
srv, err := services.ValidateAgentToken(req.ServerId, req.AgentToken)
if err != nil {
return nil, status.Errorf(codes.Unauthenticated, "invalid agent token")
}
monitors, err := services.ListMonitorsForRunner(srv.InstanceID, srv.ServerID)
if err != nil {
return nil, status.Errorf(codes.Internal, "list monitors")
}
specs := make([]pb.MonitorSpec, 0, len(monitors))
for _, m := range monitors {
specs = append(specs, pb.MonitorSpec{
MonitorId: m.MonitorID,
Type: m.Type,
URL: m.Target.URL,
Host: m.Target.Host,
Port: m.Target.Port,
Method: m.Target.Method,
ExpectedStatus: m.Target.ExpectedStatus,
Keyword: m.Target.Keyword,
TLSWarnDays: m.Target.TLSWarnDays,
Insecure: m.Target.Insecure,
IntervalSec: m.IntervalSec,
Retries: m.Retries,
})
}
return &pb.SyncMonitorsResponse{Monitors: specs}, nil
}
func (s *vantageServer) ReportChecks(ctx context.Context, req *pb.ReportChecksRequest) (*pb.ReportChecksResponse, error) {
srv, err := services.ValidateAgentToken(req.ServerId, req.AgentToken)
if err != nil {
return nil, status.Errorf(codes.Unauthenticated, "invalid agent token")
}
for _, r := range req.Results {
res := checker.Result{Up: r.Up, LatencyMs: r.LatencyMs, Message: r.Message}
if r.CertExpiryUnix > 0 {
t := time.Unix(r.CertExpiryUnix, 0)
res.CertExpiry = &t
}
if err := services.IngestResult(srv.InstanceID, srv.ServerID, r.MonitorId, res); err != nil {
log.Printf("ingest check %s: %v", r.MonitorId, err)
}
}
return &pb.ReportChecksResponse{}, nil
}
func (s *vantageServer) CommandStream(stream pb.Vantage_CommandStreamServer) error {
// First message authenticates the agent and signals readiness.
msg, err := stream.Recv()
if err != nil {
return status.Errorf(codes.InvalidArgument, "expected initial auth message: %v", err)
@@ -117,8 +184,6 @@ func (s *vantageServer) CommandStream(stream pb.Vantage_CommandStreamServer) err
log.Printf("agent %s connected command stream", srv.ServerID)
defer log.Printf("agent %s disconnected command stream", srv.ServerID)
// Drain inbound results in the background so client Send calls never block.
// UploadGeneratedKey handles the real storage; these are just confirmation logs.
go func() {
for {
m, err := stream.Recv()
@@ -132,6 +197,13 @@ func (s *vantageServer) CommandStream(stream pb.Vantage_CommandStreamServer) err
if m.StepResult != nil {
services.StepResults.Deliver(m.StepResult)
}
if m.StepOutput != nil {
if m.StepOutput.Eof {
services.StepLogs.Close(m.StepOutput.CommandId)
} else {
services.StepLogs.Append(m.StepOutput.CommandId, m.StepOutput.Data)
}
}
}
}()
@@ -158,15 +230,13 @@ func StartGRPC(port int) error {
}
s := grpc.NewServer(
// Accept client keepalive pings as fast as every 20s so the 30s agent
// ping interval is always within the allowed window.
grpc.KeepaliveEnforcementPolicy(keepalive.EnforcementPolicy{
MinTime: 20 * time.Second,
PermitWithoutStream: false,
}),
grpc.KeepaliveParams(keepalive.ServerParameters{
// Server also pings the client after 45s of inactivity so both
// sides can detect a dead connection without waiting for a timeout.
Time: 45 * time.Second,
Timeout: 10 * time.Second,
}),
+5 -4
View File
@@ -8,8 +8,9 @@ import (
type Assignment struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
KeyID string `bson:"key_id" json:"key_id"`
ServerID string `bson:"server_id" json:"server_id"`
AssignedAt time.Time `bson:"assigned_at" json:"assigned_at"`
RevokedAt *time.Time `bson:"revoked_at,omitempty" json:"revoked_at,omitempty"`
InstanceID string `bson:"instance_id" json:"instance_id"`
KeyID string `bson:"key_id" json:"key_id"`
ServerID string `bson:"server_id" json:"server_id"`
AssignedAt time.Time `bson:"assigned_at" json:"assigned_at"`
RevokedAt *time.Time `bson:"revoked_at,omitempty" json:"revoked_at,omitempty"`
}
+8 -7
View File
@@ -7,11 +7,12 @@ import (
)
type AuditEvent struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"id"`
EventType string `bson:"event_type" json:"event_type"`
Actor string `bson:"actor" json:"actor"`
ServerID string `bson:"server_id,omitempty" json:"server_id,omitempty"`
KeyID string `bson:"key_id,omitempty" json:"key_id,omitempty"`
Details string `bson:"details" json:"details"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
ID bson.ObjectID `bson:"_id,omitempty" json:"id"`
InstanceID string `bson:"instance_id" json:"instance_id"`
EventType string `bson:"event_type" json:"event_type"`
Actor string `bson:"actor" json:"actor"`
ServerID string `bson:"server_id,omitempty" json:"server_id,omitempty"`
KeyID string `bson:"key_id,omitempty" json:"key_id,omitempty"`
Details string `bson:"details" json:"details"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
}
+26
View File
@@ -0,0 +1,26 @@
package models
import (
"time"
"go.mongodb.org/mongo-driver/v2/bson"
)
const (
ChannelWebhook = "webhook"
ChannelSMTP = "smtp"
ChannelDiscord = "discord"
ChannelSlack = "slack"
ChannelTelegram = "telegram"
)
type NotificationChannel struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
InstanceID string `bson:"instance_id" json:"instance_id"`
ChannelID string `bson:"channel_id" json:"channel_id"`
Name string `bson:"name" json:"name"`
Type string `bson:"type" json:"type"`
Config map[string]string `bson:"config" json:"config"`
Enabled bool `bson:"enabled" json:"enabled"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
}
+10 -11
View File
@@ -7,18 +7,17 @@ import (
)
type ConsoleSession struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
SessionID string `bson:"session_id" json:"session_id"`
ServerID string `bson:"server_id" json:"server_id"`
Protocol string `bson:"protocol" json:"protocol"` // ssh | rdp | vnc
KeyID string `bson:"key_id,omitempty" json:"key_id,omitempty"`
User string `bson:"user" json:"user"`
StartedAt time.Time `bson:"started_at" json:"started_at"`
EndedAt *time.Time `bson:"ended_at,omitempty" json:"ended_at,omitempty"`
ClientIP string `bson:"client_ip,omitempty" json:"client_ip,omitempty"`
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
InstanceID string `bson:"instance_id" json:"instance_id"`
SessionID string `bson:"session_id" json:"session_id"`
ServerID string `bson:"server_id" json:"server_id"`
Protocol string `bson:"protocol" json:"protocol"`
KeyID string `bson:"key_id,omitempty" json:"key_id,omitempty"`
User string `bson:"user" json:"user"`
StartedAt time.Time `bson:"started_at" json:"started_at"`
EndedAt *time.Time `bson:"ended_at,omitempty" json:"ended_at,omitempty"`
ClientIP string `bson:"client_ip,omitempty" json:"client_ip,omitempty"`
// TokenConsumedAt marks the one-time session token as spent. Set atomically
// when the tunnel opens; a second open with the same token is rejected.
TokenConsumedAt *time.Time `bson:"token_consumed_at,omitempty" json:"-"`
SSHUsername string `bson:"ssh_username,omitempty" json:"ssh_username,omitempty"`
+7
View File
@@ -0,0 +1,7 @@
package models
import shared "github.com/mrhid6/vantage/shared/models"
// Instance is defined in the shared module because sitesvc and the admin
// control plane write the same documents.
type Instance = shared.Instance
+17
View File
@@ -0,0 +1,17 @@
package models
import (
"time"
"go.mongodb.org/mongo-driver/v2/bson"
)
type InstanceOIDC struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
InstanceID string `bson:"instance_id" json:"instance_id"`
Issuer string `bson:"issuer" json:"issuer"`
ClientID string `bson:"client_id" json:"client_id"`
ClientSecretEnc string `bson:"client_secret_enc,omitempty" json:"-"`
Enabled bool `bson:"enabled" json:"enabled"`
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
}
+2 -1
View File
@@ -8,11 +8,12 @@ import (
type Key struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
InstanceID string `bson:"instance_id" json:"instance_id"`
KeyID string `bson:"key_id" json:"key_id"`
Label string `bson:"label" json:"label"`
PublicKey string `bson:"public_key" json:"public_key"`
Fingerprint string `bson:"fingerprint" json:"fingerprint"`
Source string `bson:"source" json:"source"` // uploaded | generated
Source string `bson:"source" json:"source"`
GeneratedByServerID string `bson:"generated_by_server_id,omitempty" json:"generated_by_server_id,omitempty"`
PrivateKeyEncrypted string `bson:"private_key_enc,omitempty" json:"-"`
HasPrivateKey bool `bson:"-" json:"has_private_key"`
+77
View File
@@ -0,0 +1,77 @@
package models
import (
"time"
"go.mongodb.org/mongo-driver/v2/bson"
)
const (
MonitorHTTP = "http"
MonitorTCP = "tcp"
MonitorICMP = "icmp"
MonitorTLS = "tls"
)
const (
StatusUp = "up"
StatusDown = "down"
StatusPending = "pending"
)
const RunnerServer = "server"
type MonitorTarget struct {
URL string `bson:"url,omitempty" json:"url,omitempty"`
Host string `bson:"host,omitempty" json:"host,omitempty"`
Port int `bson:"port,omitempty" json:"port,omitempty"`
Method string `bson:"method,omitempty" json:"method,omitempty"`
ExpectedStatus int `bson:"expected_status,omitempty" json:"expected_status,omitempty"`
Keyword string `bson:"keyword,omitempty" json:"keyword,omitempty"`
TLSWarnDays int `bson:"tls_warn_days,omitempty" json:"tls_warn_days,omitempty"`
Insecure bool `bson:"insecure,omitempty" json:"insecure,omitempty"`
}
type MonitorState struct {
Status string `bson:"status" json:"status"`
LastCheckAt *time.Time `bson:"last_check_at,omitempty" json:"last_check_at,omitempty"`
LatencyMs int `bson:"latency_ms" json:"latency_ms"`
Message string `bson:"message,omitempty" json:"message,omitempty"`
CertExpiryAt *time.Time `bson:"cert_expiry_at,omitempty" json:"cert_expiry_at,omitempty"`
Fails int `bson:"fails" json:"fails"`
LastNotifiedAt *time.Time `bson:"last_notified_at,omitempty" json:"last_notified_at,omitempty"`
}
type Monitor struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
InstanceID string `bson:"instance_id" json:"instance_id"`
MonitorID string `bson:"monitor_id" json:"monitor_id"`
Name string `bson:"name" json:"name"`
Type string `bson:"type" json:"type"`
Target MonitorTarget `bson:"target" json:"target"`
IntervalSec int `bson:"interval_sec" json:"interval_sec"`
Runner string `bson:"runner" json:"runner"`
Retries int `bson:"retries" json:"retries"`
Enabled bool `bson:"enabled" json:"enabled"`
ChannelIDs []string `bson:"channel_ids,omitempty" json:"channel_ids,omitempty"`
State MonitorState `bson:"state" json:"state"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
}
type Incident struct {
InstanceID string `bson:"instance_id" json:"instance_id"`
IncidentID string `bson:"incident_id" json:"incident_id"`
MonitorID string `bson:"monitor_id" json:"monitor_id"`
StartedAt time.Time `bson:"started_at" json:"started_at"`
ResolvedAt *time.Time `bson:"resolved_at,omitempty" json:"resolved_at,omitempty"`
Cause string `bson:"cause,omitempty" json:"cause,omitempty"`
}
type Rollup struct {
InstanceID string `bson:"instance_id" json:"instance_id"`
MonitorID string `bson:"monitor_id" json:"monitor_id"`
PeriodStart time.Time `bson:"period_start" json:"period_start"`
Checks int `bson:"checks" json:"checks"`
UpCount int `bson:"up_count" json:"up_count"`
SumLatency int64 `bson:"sum_latency" json:"sum_latency"`
}
+1 -3
View File
@@ -6,17 +6,15 @@ import (
"go.mongodb.org/mongo-driver/v2/bson"
)
// Secret is a single key/value pair within a group. The value is stored
// encrypted (AES-256-GCM) and is never serialized to JSON.
type Secret struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
InstanceID string `bson:"instance_id" json:"instance_id"`
Group string `bson:"group" json:"group"`
Key string `bson:"key" json:"key"`
EncryptedValue string `bson:"encrypted_value" json:"-"`
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
}
// GroupSummary describes a group in the list view.
type GroupSummary struct {
Group string `json:"group"`
KeyCount int `json:"key_count"`
+52 -19
View File
@@ -12,23 +12,56 @@ type PackageUpdate struct {
NewVersion string `bson:"new_version" json:"new_version"`
}
type Server struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
ServerID string `bson:"server_id" json:"server_id"`
Hostname string `bson:"hostname" json:"hostname"`
IPAddress string `bson:"ip_address" json:"ip_address"`
OSInfo string `bson:"os_info" json:"os_info"`
OSType string `bson:"os_type,omitempty" json:"os_type,omitempty"`
ConsoleProtocols []string `bson:"console_protocols,omitempty" json:"console_protocols,omitempty"`
SSHPort int `bson:"ssh_port,omitempty" json:"ssh_port,omitempty"`
RDPPort int `bson:"rdp_port,omitempty" json:"rdp_port,omitempty"`
PreRegToken string `bson:"pre_reg_token,omitempty" json:"pre_reg_token,omitempty"`
PreRegExpires *time.Time `bson:"pre_reg_expires,omitempty" json:"pre_reg_expires,omitempty"`
AgentTokenHash string `bson:"agent_token_hash,omitempty" json:"-"`
Status string `bson:"status" json:"status"`
AgentVersion string `bson:"agent_version,omitempty" json:"agent_version,omitempty"`
LastSeen *time.Time `bson:"last_seen,omitempty" json:"last_seen,omitempty"`
AvailableUpdates []PackageUpdate `bson:"available_updates,omitempty" json:"available_updates,omitempty"`
UpdatesCheckedAt *time.Time `bson:"updates_checked_at,omitempty" json:"updates_checked_at,omitempty"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
type CPUInfo struct {
Model string `bson:"model,omitempty" json:"model,omitempty"`
Cores int `bson:"cores,omitempty" json:"cores,omitempty"`
UsagePct float64 `bson:"usage_pct" json:"usage_pct"`
Load1 float64 `bson:"load1,omitempty" json:"load1,omitempty"`
}
type MemInfo struct {
TotalBytes uint64 `bson:"total_bytes" json:"total_bytes"`
UsedBytes uint64 `bson:"used_bytes" json:"used_bytes"`
}
type Partition struct {
Device string `bson:"device" json:"device"`
Mountpoint string `bson:"mountpoint" json:"mountpoint"`
Fstype string `bson:"fstype,omitempty" json:"fstype,omitempty"`
TotalBytes uint64 `bson:"total_bytes" json:"total_bytes"`
UsedBytes uint64 `bson:"used_bytes" json:"used_bytes"`
}
type Inventory struct {
CPU CPUInfo `bson:"cpu" json:"cpu"`
Memory MemInfo `bson:"memory" json:"memory"`
SwapTotalBytes uint64 `bson:"swap_total_bytes" json:"swap_total_bytes"`
SwapUsedBytes uint64 `bson:"swap_used_bytes" json:"swap_used_bytes"`
Partitions []Partition `bson:"partitions,omitempty" json:"partitions,omitempty"`
Kernel string `bson:"kernel,omitempty" json:"kernel,omitempty"`
MetricsAt *time.Time `bson:"metrics_at,omitempty" json:"metrics_at,omitempty"`
StaticAt *time.Time `bson:"static_at,omitempty" json:"static_at,omitempty"`
}
type Server struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
InstanceID string `bson:"instance_id" json:"instance_id"`
ServerID string `bson:"server_id" json:"server_id"`
Hostname string `bson:"hostname" json:"hostname"`
IPAddress string `bson:"ip_address" json:"ip_address"`
OSInfo string `bson:"os_info" json:"os_info"`
OSType string `bson:"os_type,omitempty" json:"os_type,omitempty"`
ConsoleProtocols []string `bson:"console_protocols,omitempty" json:"console_protocols,omitempty"`
SSHPort int `bson:"ssh_port,omitempty" json:"ssh_port,omitempty"`
RDPPort int `bson:"rdp_port,omitempty" json:"rdp_port,omitempty"`
PreRegToken string `bson:"pre_reg_token,omitempty" json:"pre_reg_token,omitempty"`
PreRegExpires *time.Time `bson:"pre_reg_expires,omitempty" json:"pre_reg_expires,omitempty"`
AgentTokenHash string `bson:"agent_token_hash,omitempty" json:"-"`
Status string `bson:"status" json:"status"`
AgentVersion string `bson:"agent_version,omitempty" json:"agent_version,omitempty"`
LastSeen *time.Time `bson:"last_seen,omitempty" json:"last_seen,omitempty"`
AvailableUpdates []PackageUpdate `bson:"available_updates,omitempty" json:"available_updates,omitempty"`
UpdatesCheckedAt *time.Time `bson:"updates_checked_at,omitempty" json:"updates_checked_at,omitempty"`
Inventory *Inventory `bson:"inventory,omitempty" json:"inventory,omitempty"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
}
+6 -35
View File
@@ -1,39 +1,10 @@
package models
import (
"time"
import shared "github.com/mrhid6/vantage/shared/models"
"go.mongodb.org/mongo-driver/v2/bson"
type (
Settings = shared.Settings
AlertSettings = shared.AlertSettings
EmailSettings = shared.EmailSettings
SecretsSettings = shared.SecretsSettings
)
type AlertSettings struct {
Enabled bool `bson:"enabled" json:"enabled"`
WebhookURL string `bson:"webhook_url" json:"webhook_url"`
OfflineThresholdMinutes int `bson:"offline_threshold_minutes" json:"offline_threshold_minutes"`
}
type EmailSettings struct {
Enabled bool `bson:"enabled" json:"enabled"`
SMTPHost string `bson:"smtp_host" json:"smtp_host"`
SMTPPort int `bson:"smtp_port" json:"smtp_port"`
Username string `bson:"username" json:"username"`
Password string `bson:"password" json:"password"`
FromAddr string `bson:"from_addr" json:"from_addr"`
ToAddrs []string `bson:"to_addrs" json:"to_addrs"`
UseTLS bool `bson:"use_tls" json:"use_tls"`
}
// SecretsSettings holds configuration for the secrets vault / ESO integration.
// The read token is stored as a SHA-256 hash and never returned to clients.
type SecretsSettings struct {
ReadTokenHash string `bson:"read_token_hash,omitempty" json:"-"`
ReadTokenSet bool `bson:"-" json:"read_token_set"`
RotatedAt time.Time `bson:"rotated_at,omitempty" json:"rotated_at,omitempty"`
}
type Settings struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
Alerts AlertSettings `bson:"alerts" json:"alerts"`
Email EmailSettings `bson:"email" json:"email"`
Secrets SecretsSettings `bson:"secrets" json:"secrets"`
}
+13
View File
@@ -0,0 +1,13 @@
package models
import shared "github.com/mrhid6/vantage/shared/models"
type User = shared.User
const (
RoleOwner = shared.RoleOwner
RoleAdmin = shared.RoleAdmin
RoleMember = shared.RoleMember
)
func ValidRole(role string) bool { return shared.ValidRole(role) }
+48 -31
View File
@@ -1,26 +1,42 @@
package models
import "time"
import (
"time"
"go.mongodb.org/mongo-driver/v2/bson"
)
type InputParam struct {
Name string `bson:"name" json:"name"`
Default string `bson:"default" json:"default"`
Description string `bson:"description" json:"description"`
}
type WorkflowStep struct {
ID string `bson:"_id,omitempty" json:"-"`
StepID string `bson:"step_id" json:"step_id"`
Name string `bson:"name" json:"name"`
Description string `bson:"description" json:"description"`
Interpreter string `bson:"interpreter" json:"interpreter"` // "bash" | "powershell"
Script string `bson:"script" json:"script"`
DeclaredOutputs []string `bson:"declared_outputs" json:"declared_outputs"`
SecretRefs []string `bson:"secret_refs" json:"secret_refs"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
InstanceID string `bson:"instance_id" json:"instance_id"`
StepID string `bson:"step_id" json:"step_id"`
Name string `bson:"name" json:"name"`
Description string `bson:"description" json:"description"`
Interpreter string `bson:"interpreter" json:"interpreter"`
Script string `bson:"script" json:"script"`
DeclaredOutputs []string `bson:"declared_outputs" json:"declared_outputs"`
DeclaredInputs []InputParam `bson:"declared_inputs" json:"declared_inputs"`
SecretRefs []string `bson:"secret_refs" json:"secret_refs"`
Source string `bson:"source" json:"source"`
Slug string `bson:"slug,omitempty" json:"slug,omitempty"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
}
type WorkflowStepRef struct {
StepID string `bson:"step_id" json:"step_id"`
Order int `bson:"order" json:"order"`
OnFailure string `bson:"on_failure" json:"on_failure"` // "stop" | "continue" | "retry"
MaxRetries int `bson:"max_retries" json:"max_retries"`
Overrides *StepOverride `bson:"overrides,omitempty" json:"overrides,omitempty"`
StepID string `bson:"step_id,omitempty" json:"step_id,omitempty"`
Inline *WorkflowStep `bson:"inline,omitempty" json:"inline,omitempty"`
Order int `bson:"order" json:"order"`
OnFailure string `bson:"on_failure" json:"on_failure"`
MaxRetries int `bson:"max_retries" json:"max_retries"`
Overrides *StepOverride `bson:"overrides,omitempty" json:"overrides,omitempty"`
Inputs map[string]string `bson:"inputs,omitempty" json:"inputs,omitempty"`
}
type StepOverride struct {
@@ -29,7 +45,8 @@ type StepOverride struct {
}
type Workflow struct {
ID string `bson:"_id,omitempty" json:"-"`
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
InstanceID string `bson:"instance_id" json:"instance_id"`
WorkflowID string `bson:"workflow_id" json:"workflow_id"`
Name string `bson:"name" json:"name"`
TargetServerIDs []string `bson:"target_server_ids" json:"target_server_ids"`
@@ -38,25 +55,24 @@ type Workflow struct {
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
}
// ResolvedStep is a step frozen into a run snapshot (library step + overrides applied).
type ResolvedStep struct {
Order int `bson:"order" json:"order"`
Name string `bson:"name" json:"name"`
Interpreter string `bson:"interpreter" json:"interpreter"`
Script string `bson:"script" json:"script"`
SecretRefs []string `bson:"secret_refs" json:"secret_refs"`
OnFailure string `bson:"on_failure" json:"on_failure"`
MaxRetries int `bson:"max_retries" json:"max_retries"`
Order int `bson:"order" json:"order"`
Name string `bson:"name" json:"name"`
Interpreter string `bson:"interpreter" json:"interpreter"`
Script string `bson:"script" json:"script"`
SecretRefs []string `bson:"secret_refs" json:"secret_refs"`
OnFailure string `bson:"on_failure" json:"on_failure"`
MaxRetries int `bson:"max_retries" json:"max_retries"`
Inputs map[string]string `bson:"inputs" json:"inputs"`
}
type StepRun struct {
Order int `bson:"order" json:"order"`
Name string `bson:"name" json:"name"`
Status string `bson:"status" json:"status"` // queued|running|success|failed|skipped
Status string `bson:"status" json:"status"`
Attempts int `bson:"attempts" json:"attempts"`
ExitCode int `bson:"exit_code" json:"exit_code"`
Stdout string `bson:"stdout" json:"stdout"`
Stderr string `bson:"stderr" json:"stderr"`
LogOffset int64 `bson:"log_offset" json:"log_offset"`
OutputEnv map[string]string `bson:"output_env" json:"output_env"`
StartedAt *time.Time `bson:"started_at,omitempty" json:"started_at,omitempty"`
FinishedAt *time.Time `bson:"finished_at,omitempty" json:"finished_at,omitempty"`
@@ -65,7 +81,7 @@ type StepRun struct {
type ServerRun struct {
ServerID string `bson:"server_id" json:"server_id"`
Hostname string `bson:"hostname" json:"hostname"`
Status string `bson:"status" json:"status"` // queued|running|success|failed|skipped
Status string `bson:"status" json:"status"`
StartedAt *time.Time `bson:"started_at,omitempty" json:"started_at,omitempty"`
FinishedAt *time.Time `bson:"finished_at,omitempty" json:"finished_at,omitempty"`
RunEnv map[string]string `bson:"run_env" json:"run_env"`
@@ -73,12 +89,13 @@ type ServerRun struct {
}
type WorkflowRun struct {
ID string `bson:"_id,omitempty" json:"-"`
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
InstanceID string `bson:"instance_id" json:"instance_id"`
RunID string `bson:"run_id" json:"run_id"`
WorkflowID string `bson:"workflow_id" json:"workflow_id"`
Name string `bson:"name" json:"name"`
Steps []ResolvedStep `bson:"steps_snapshot" json:"steps_snapshot"`
Status string `bson:"status" json:"status"` // running|success|failed|cancelled
Status string `bson:"status" json:"status"`
TriggeredBy string `bson:"triggered_by" json:"triggered_by"`
StartedAt time.Time `bson:"started_at" json:"started_at"`
FinishedAt *time.Time `bson:"finished_at,omitempty" json:"finished_at,omitempty"`
+105
View File
@@ -0,0 +1,105 @@
package monitorsched
import (
"context"
"log"
"sync"
"time"
"github.com/mrhid6/vantage/server/internal/checker"
"github.com/mrhid6/vantage/server/internal/models"
"github.com/mrhid6/vantage/server/internal/services"
)
const reloadInterval = 30 * time.Second
type runner struct {
monitorID string
intervalSec int
cancel context.CancelFunc
}
func Start(ctx context.Context) {
go loop(ctx)
}
func loop(ctx context.Context) {
active := map[string]*runner{}
var mu sync.Mutex
// Monitors run regardless of licence state, deliberately.
//
// A customer whose card failed must not lose the ability to know their
// infrastructure is on fire. Creating and editing monitors is blocked by the
// API gate; executing the ones that already exist is not.
sync := func() {
monitors, err := services.ListServerScheduledMonitors()
if err != nil {
log.Printf("monitorsched: list monitors: %v", err)
return
}
want := map[string]models.Monitor{}
for _, m := range monitors {
want[m.MonitorID] = m
}
mu.Lock()
defer mu.Unlock()
for id, r := range active {
m, ok := want[id]
if !ok || m.IntervalSec != r.intervalSec {
r.cancel()
delete(active, id)
}
}
for id, m := range want {
if _, ok := active[id]; ok {
continue
}
rctx, cancel := context.WithCancel(ctx)
active[id] = &runner{monitorID: id, intervalSec: m.IntervalSec, cancel: cancel}
go runMonitor(rctx, m)
}
}
sync()
t := time.NewTicker(reloadInterval)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
sync()
}
}
}
func runMonitor(ctx context.Context, m models.Monitor) {
interval := time.Duration(m.IntervalSec) * time.Second
if interval <= 0 {
interval = 60 * time.Second
}
spec := services.SpecFor(&m)
run := func() {
res := checker.Run(ctx, spec)
if err := services.IngestServerScheduledResult(m.MonitorID, res); err != nil {
log.Printf("monitorsched: ingest %s: %v", m.MonitorID, err)
}
}
run()
t := time.NewTicker(interval)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
run()
}
}
}
+57
View File
@@ -0,0 +1,57 @@
package notify
import (
"fmt"
"time"
"github.com/mrhid6/vantage/server/internal/models"
)
type Event struct {
MonitorName string
Type string
OldStatus string
NewStatus string
Message string
Time time.Time
}
func (e Event) title() string {
verb := "recovered"
if e.NewStatus == models.StatusDown {
verb = "is DOWN"
}
s := fmt.Sprintf("[Vantage] %s (%s) %s", e.MonitorName, e.Type, verb)
if e.Message != "" {
s += ": " + e.Message
}
return s
}
func Dispatch(ch models.NotificationChannel, ev Event) error {
switch ch.Type {
case models.ChannelWebhook:
return dispatchWebhook(ch, ev)
case models.ChannelDiscord:
return dispatchDiscord(ch, ev)
case models.ChannelSlack:
return dispatchSlack(ch, ev)
case models.ChannelTelegram:
return dispatchTelegram(ch, ev)
case models.ChannelSMTP:
return dispatchSMTP(ch, ev)
default:
return fmt.Errorf("unknown channel type: %s", ch.Type)
}
}
func Test(ch models.NotificationChannel) error {
return Dispatch(ch, Event{
MonitorName: "Test monitor",
Type: "http",
OldStatus: models.StatusUp,
NewStatus: models.StatusDown,
Message: "this is a test alert from Vantage",
Time: time.Now(),
})
}
+70
View File
@@ -0,0 +1,70 @@
package notify
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
"time"
"github.com/mrhid6/vantage/server/internal/models"
)
var httpClient = &http.Client{Timeout: 10 * time.Second}
func postJSON(target string, payload any) error {
body, err := json.Marshal(payload)
if err != nil {
return err
}
resp, err := httpClient.Post(target, "application/json", bytes.NewReader(body))
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode >= 300 {
return fmt.Errorf("HTTP %d from %s", resp.StatusCode, target)
}
return nil
}
func dispatchWebhook(ch models.NotificationChannel, ev Event) error {
target := ch.Config["url"]
if target == "" {
return fmt.Errorf("webhook: missing url")
}
return postJSON(target, map[string]any{
"monitor": ev.MonitorName,
"type": ev.Type,
"old_status": ev.OldStatus,
"new_status": ev.NewStatus,
"message": ev.Message,
"time": ev.Time.Format(time.RFC3339),
})
}
func dispatchDiscord(ch models.NotificationChannel, ev Event) error {
target := ch.Config["url"]
if target == "" {
return fmt.Errorf("discord: missing url")
}
return postJSON(target, map[string]string{"content": ev.title()})
}
func dispatchSlack(ch models.NotificationChannel, ev Event) error {
target := ch.Config["url"]
if target == "" {
return fmt.Errorf("slack: missing url")
}
return postJSON(target, map[string]string{"text": ev.title()})
}
func dispatchTelegram(ch models.NotificationChannel, ev Event) error {
token := ch.Config["token"]
chatID := ch.Config["chat_id"]
if token == "" || chatID == "" {
return fmt.Errorf("telegram: missing token or chat_id")
}
api := fmt.Sprintf("https://api.telegram.org/bot%s/sendMessage", token)
return postJSON(api, map[string]string{"chat_id": chatID, "text": ev.title()})
}
+90
View File
@@ -0,0 +1,90 @@
package notify
import (
"crypto/tls"
"fmt"
"net"
"net/smtp"
"strings"
"time"
"github.com/mrhid6/vantage/server/internal/models"
)
const smtpTimeout = 15 * time.Second
func dispatchSMTP(ch models.NotificationChannel, ev Event) error {
host := ch.Config["host"]
port := ch.Config["port"]
from := ch.Config["from"]
to := ch.Config["to"]
if host == "" || port == "" || from == "" || to == "" {
return fmt.Errorf("smtp: missing host/port/from/to")
}
addr := net.JoinHostPort(host, port)
conn, err := net.DialTimeout("tcp", addr, smtpTimeout)
if err != nil {
return fmt.Errorf("smtp: dial %s: %w", addr, err)
}
_ = conn.SetDeadline(time.Now().Add(smtpTimeout))
if port == "465" {
conn = tls.Client(conn, &tls.Config{ServerName: host})
}
c, err := smtp.NewClient(conn, host)
if err != nil {
conn.Close()
return fmt.Errorf("smtp: client: %w", err)
}
defer c.Close()
if port != "465" {
if ok, _ := c.Extension("STARTTLS"); ok {
if err := c.StartTLS(&tls.Config{ServerName: host}); err != nil {
return fmt.Errorf("smtp: starttls: %w", err)
}
}
}
if user := ch.Config["username"]; user != "" {
if err := c.Auth(smtp.PlainAuth("", user, ch.Config["password"], host)); err != nil {
return fmt.Errorf("smtp: auth: %w", err)
}
}
recipients := strings.Split(to, ",")
for i := range recipients {
recipients[i] = strings.TrimSpace(recipients[i])
}
if err := c.Mail(from); err != nil {
return fmt.Errorf("smtp: mail from: %w", err)
}
for _, rcpt := range recipients {
if rcpt == "" {
continue
}
if err := c.Rcpt(rcpt); err != nil {
return fmt.Errorf("smtp: rcpt %s: %w", rcpt, err)
}
}
msg, err := buildMIME(from, to, ev.title(), textEmail(ev), htmlEmail(ev))
if err != nil {
return fmt.Errorf("smtp: build message: %w", err)
}
w, err := c.Data()
if err != nil {
return fmt.Errorf("smtp: data: %w", err)
}
if _, err := w.Write(msg); err != nil {
return fmt.Errorf("smtp: write: %w", err)
}
if err := w.Close(); err != nil {
return fmt.Errorf("smtp: close data: %w", err)
}
return c.Quit()
}
+161
View File
@@ -0,0 +1,161 @@
package notify
import (
"fmt"
"html"
"mime/multipart"
"net/textproto"
"strings"
"github.com/mrhid6/vantage/server/internal/models"
)
const (
colBg = "#0f1117"
colSurface = "#1a1d27"
colSurface2 = "#232635"
colBorder = "#2e3147"
colText = "#e8eaf0"
colTextMuted = "#9095a8"
colAccent = "#6366f1"
colSuccess = "#22c55e"
colDanger = "#ef4444"
)
func statusColor(status string) string {
switch status {
case models.StatusUp:
return colSuccess
case models.StatusDown:
return colDanger
default:
return colTextMuted
}
}
func buildMIME(from, to, subject, text, htmlBody string) ([]byte, error) {
var buf strings.Builder
w := multipart.NewWriter(&buf)
var head strings.Builder
head.WriteString("From: " + from + "\r\n")
head.WriteString("To: " + to + "\r\n")
head.WriteString("Subject: " + subject + "\r\n")
head.WriteString("MIME-Version: 1.0\r\n")
head.WriteString(fmt.Sprintf("Content-Type: multipart/alternative; boundary=%s\r\n\r\n", w.Boundary()))
textPart, err := w.CreatePart(textproto.MIMEHeader{"Content-Type": {"text/plain; charset=UTF-8"}})
if err != nil {
return nil, err
}
textPart.Write([]byte(text))
htmlPart, err := w.CreatePart(textproto.MIMEHeader{"Content-Type": {"text/html; charset=UTF-8"}})
if err != nil {
return nil, err
}
htmlPart.Write([]byte(htmlBody))
if err := w.Close(); err != nil {
return nil, err
}
return []byte(head.String() + buf.String()), nil
}
func htmlEmail(ev Event) string {
accent := statusColor(ev.NewStatus)
label := "Recovered"
if ev.NewStatus == models.StatusDown {
label = "Down"
}
esc := html.EscapeString
row := func(k, v string) string {
if v == "" {
v = ""
}
return fmt.Sprintf(
`<tr>`+
`<td style="padding:8px 0;color:%s;font-size:13px;width:120px;">%s</td>`+
`<td style="padding:8px 0;color:%s;font-size:13px;font-weight:500;">%s</td>`+
`</tr>`,
colTextMuted, k, colText, esc(v))
}
message := ""
if ev.Message != "" {
message = fmt.Sprintf(
`<p style="margin:0 0 20px;padding:12px 14px;background:%s;border:1px solid %s;border-radius:8px;color:%s;font-size:13px;">%s</p>`,
colSurface2, colBorder, colText, esc(ev.Message))
}
transition := esc(ev.OldStatus) + " → " + esc(ev.NewStatus)
return fmt.Sprintf(`<!DOCTYPE html>
<html>
<body style="margin:0;padding:0;background:%s;">
<table role="presentation" width="100%%" cellpadding="0" cellspacing="0" style="background:%s;padding:32px 0;">
<tr>
<td align="center">
<table role="presentation" width="480" cellpadding="0" cellspacing="0" style="max-width:480px;background:%s;border:1px solid %s;border-radius:12px;overflow:hidden;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,Helvetica,Arial,sans-serif;">
<tr><td style="height:4px;background:%s;"></td></tr>
<tr>
<td style="padding:28px 28px 20px;">
<table role="presentation" cellpadding="0" cellspacing="0" style="margin-bottom:18px;">
<tr>
<td style="font-size:18px;font-weight:700;color:%s;">Vantage</td>
</tr>
</table>
<span style="display:inline-block;padding:4px 12px;border-radius:9999px;background:%s22;color:%s;font-size:12px;font-weight:600;letter-spacing:.02em;">%s</span>
<h1 style="margin:14px 0 6px;font-size:20px;font-weight:700;color:%s;">%s</h1>
<p style="margin:0 0 20px;color:%s;font-size:13px;">%s check</p>
%s
<table role="presentation" width="100%%" cellpadding="0" cellspacing="0" style="border-top:1px solid %s;">
%s
%s
%s
</table>
</td>
</tr>
<tr>
<td style="padding:16px 28px;border-top:1px solid %s;">
<p style="margin:0;color:%s;font-size:12px;">Sent by Vantage · self-hosted service monitoring</p>
</td>
</tr>
</table>
</td>
</tr>
</table>
</body>
</html>`,
colBg,
colBg,
colSurface, colBorder,
accent,
colText,
accent, accent, label,
colText, esc(ev.MonitorName),
colTextMuted, esc(ev.Type),
message,
colBorder,
row("Status", transition),
row("Type", ev.Type),
row("Time", ev.Time.Format("2006-01-02 15:04:05 MST")),
colBorder,
colTextMuted,
)
}
func textEmail(ev Event) string {
return strings.Join([]string{
ev.title(),
"",
"Monitor: " + ev.MonitorName,
"Type: " + ev.Type,
"Status: " + ev.OldStatus + " -> " + ev.NewStatus,
"Message: " + ev.Message,
"Time: " + ev.Time.Format("2006-01-02 15:04:05 MST"),
"",
"Sent by Vantage · self-hosted service monitoring",
}, "\r\n")
}
+10 -9
View File
@@ -11,24 +11,25 @@ import (
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
func LogEvent(eventType, actor, serverID, keyID, details string) {
func LogEvent(instanceID, eventType, actor, serverID, keyID, details string) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
event := models.AuditEvent{
EventType: eventType,
Actor: actor,
ServerID: serverID,
KeyID: keyID,
Details: details,
CreatedAt: time.Now(),
InstanceID: instanceID,
EventType: eventType,
Actor: actor,
ServerID: serverID,
KeyID: keyID,
Details: details,
CreatedAt: time.Now(),
}
if _, err := db.Col("audit_logs").InsertOne(ctx, event); err != nil {
log.Printf("audit log error: %v", err)
}
}
func ListAuditEvents(limit int64) ([]models.AuditEvent, error) {
func ListAuditEvents(instanceID string, limit int64) ([]models.AuditEvent, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
@@ -36,7 +37,7 @@ func ListAuditEvents(limit int64) ([]models.AuditEvent, error) {
SetSort(bson.D{{Key: "created_at", Value: -1}}).
SetLimit(limit)
cursor, err := db.Col("audit_logs").Find(ctx, bson.M{}, opts)
cursor, err := db.Col("audit_logs").Find(ctx, bson.M{"instance_id": instanceID}, opts)
if err != nil {
return nil, err
}
+115
View File
@@ -0,0 +1,115 @@
package services
import (
"errors"
"time"
"github.com/google/uuid"
"github.com/mrhid6/vantage/server/internal/db"
"github.com/mrhid6/vantage/server/internal/models"
"github.com/mrhid6/vantage/server/internal/notify"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
func ListChannels(instanceID string) ([]models.NotificationChannel, error) {
ctx, cancel := monCtx()
defer cancel()
cur, err := db.Col("notification_channels").Find(ctx, bson.M{"instance_id": instanceID}, options.Find().SetSort(bson.M{"created_at": 1}))
if err != nil {
return nil, err
}
var out []models.NotificationChannel
if err := cur.All(ctx, &out); err != nil {
return nil, err
}
return out, nil
}
func GetChannel(instanceID, channelID string) (*models.NotificationChannel, error) {
ctx, cancel := monCtx()
defer cancel()
var ch models.NotificationChannel
err := db.Col("notification_channels").FindOne(ctx, bson.M{"channel_id": channelID, "instance_id": instanceID}).Decode(&ch)
if errors.Is(err, mongo.ErrNoDocuments) {
return nil, nil
}
if err != nil {
return nil, err
}
return &ch, nil
}
func GetChannels(instanceID string, channelIDs []string) ([]models.NotificationChannel, error) {
if len(channelIDs) == 0 {
return nil, nil
}
ctx, cancel := monCtx()
defer cancel()
cur, err := db.Col("notification_channels").Find(ctx, bson.M{"instance_id": instanceID, "channel_id": bson.M{"$in": channelIDs}})
if err != nil {
return nil, err
}
var out []models.NotificationChannel
if err := cur.All(ctx, &out); err != nil {
return nil, err
}
return out, nil
}
func validateChannelIDs(instanceID string, channelIDs []string) error {
for _, id := range channelIDs {
ch, err := GetChannel(instanceID, id)
if err != nil {
return err
}
if ch == nil {
return errors.New("channel " + id + " not found")
}
}
return nil
}
func CreateChannel(instanceID string, ch *models.NotificationChannel) (*models.NotificationChannel, error) {
if err := CheckChannelLimit(instanceID); err != nil {
return nil, err
}
ctx, cancel := monCtx()
defer cancel()
ch.InstanceID = instanceID
ch.ChannelID = uuid.NewString()
ch.CreatedAt = time.Now()
if ch.Config == nil {
ch.Config = map[string]string{}
}
if _, err := db.Col("notification_channels").InsertOne(ctx, ch); err != nil {
return nil, err
}
return ch, nil
}
func UpdateChannel(instanceID, channelID string, upd bson.M) error {
ctx, cancel := monCtx()
defer cancel()
_, err := db.Col("notification_channels").UpdateOne(ctx, bson.M{"channel_id": channelID, "instance_id": instanceID}, bson.M{"$set": upd})
return err
}
func DeleteChannel(instanceID, channelID string) error {
ctx, cancel := monCtx()
defer cancel()
_, err := db.Col("notification_channels").DeleteOne(ctx, bson.M{"channel_id": channelID, "instance_id": instanceID})
return err
}
func TestChannel(instanceID, channelID string) error {
ch, err := GetChannel(instanceID, channelID)
if err != nil {
return err
}
if ch == nil {
return errors.New("channel not found")
}
return notify.Test(*ch)
}
+23 -37
View File
@@ -17,7 +17,7 @@ import (
)
func sessionHMACKey() ([]byte, error) {
// Reuse the AES key material as the HMAC secret. Distinct domain via prefix.
k, err := encryptionKey()
if err != nil {
return nil, err
@@ -29,7 +29,6 @@ func sessionHMACKey() ([]byte, error) {
func b64(b []byte) string { return base64.RawURLEncoding.EncodeToString(b) }
// SignSessionToken returns a signed, expiring token binding a session id.
func SignSessionToken(sessionID string, ttl time.Duration) (string, error) {
key, err := sessionHMACKey()
if err != nil {
@@ -42,7 +41,6 @@ func SignSessionToken(sessionID string, ttl time.Duration) (string, error) {
return payload + "." + b64(mac.Sum(nil)), nil
}
// VerifySessionToken checks signature + expiry and returns the session id.
func VerifySessionToken(token string) (string, error) {
parts := strings.Split(token, ".")
if len(parts) != 3 {
@@ -86,10 +84,6 @@ func portOr(v, def int) string {
return strconv.Itoa(v)
}
// BuildGuacParams assembles the guacd connection parameter map for a protocol.
// privateKey/passphrase are the decrypted SSH private key and its optional
// passphrase (ssh only); rdpUser/rdpPass are used for rdp, and rdpPass carries
// the password for vnc. None of these values are persisted or logged by the caller.
func BuildGuacParams(srv *models.Server, protocol, sshUser, privateKey, passphrase, rdpUser, rdpPass string) (*GuacParams, error) {
host := srv.IPAddress
switch protocol {
@@ -129,18 +123,19 @@ func BuildGuacParams(srv *models.Server, protocol, sshUser, privateKey, passphra
}
}
func CreateConsoleSession(serverID, protocol, keyID, user, clientIP string) (*models.ConsoleSession, error) {
func CreateConsoleSession(instanceID, serverID, protocol, keyID, user, clientIP string) (*models.ConsoleSession, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
s := &models.ConsoleSession{
SessionID: uuid.NewString(),
ServerID: serverID,
Protocol: protocol,
KeyID: keyID,
User: user,
ClientIP: clientIP,
StartedAt: time.Now(),
InstanceID: instanceID,
SessionID: uuid.NewString(),
ServerID: serverID,
Protocol: protocol,
KeyID: keyID,
User: user,
ClientIP: clientIP,
StartedAt: time.Now(),
}
if _, err := db.Col("console_sessions").InsertOne(ctx, s); err != nil {
return nil, err
@@ -148,19 +143,17 @@ func CreateConsoleSession(serverID, protocol, keyID, user, clientIP string) (*mo
return s, nil
}
func GetConsoleSession(sessionID string) (*models.ConsoleSession, error) {
func GetConsoleSession(instanceID, sessionID string) (*models.ConsoleSession, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
var s models.ConsoleSession
if err := db.Col("console_sessions").FindOne(ctx, bson.M{"session_id": sessionID}).Decode(&s); err != nil {
if err := db.Col("console_sessions").FindOne(ctx, bson.M{"session_id": sessionID, "instance_id": instanceID}).Decode(&s); err != nil {
return nil, err
}
return &s, nil
}
// StashConsoleRDPCreds encrypts and stores single-use RDP credentials on the
// session document. They are consumed (and cleared) when the tunnel opens.
func StashConsoleRDPCreds(sessionID, username, password string) error {
func StashConsoleRDPCreds(instanceID, sessionID, username, password string) error {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
u, err := encryptString(username)
@@ -172,17 +165,14 @@ func StashConsoleRDPCreds(sessionID, username, password string) error {
return err
}
_, err = db.Col("console_sessions").UpdateOne(ctx,
bson.M{"session_id": sessionID},
bson.M{"session_id": sessionID, "instance_id": instanceID},
bson.M{"$set": bson.M{"rdp_user_enc": u, "rdp_pass_enc": p}},
)
return err
}
// ConsumeConsoleRDPCreds decrypts and returns the stored RDP credentials, then
// clears them from the session document (single-use). Returns empty strings if
// none were stored.
func ConsumeConsoleRDPCreds(sessionID string) (username, password string, err error) {
s, err := GetConsoleSession(sessionID)
func ConsumeConsoleRDPCreds(instanceID, sessionID string) (username, password string, err error) {
s, err := GetConsoleSession(instanceID, sessionID)
if err != nil {
return "", "", err
}
@@ -202,31 +192,27 @@ func ConsumeConsoleRDPCreds(sessionID string) (username, password string, err er
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
_, _ = db.Col("console_sessions").UpdateOne(ctx,
bson.M{"session_id": sessionID},
bson.M{"session_id": sessionID, "instance_id": instanceID},
bson.M{"$unset": bson.M{"rdp_user_enc": "", "rdp_pass_enc": ""}},
)
return username, password, nil
}
// SetConsoleSSHUser persists the SSH username to use on the session doc.
func SetConsoleSSHUser(sessionID, username string) error {
func SetConsoleSSHUser(instanceID, sessionID, username string) error {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
_, err := db.Col("console_sessions").UpdateOne(ctx,
bson.M{"session_id": sessionID},
bson.M{"session_id": sessionID, "instance_id": instanceID},
bson.M{"$set": bson.M{"ssh_username": username}})
return err
}
// ConsumeSessionToken atomically marks a session's one-time token as spent.
// It returns an error if the token was already consumed (replay) or the session
// does not exist, so the tunnel can be opened at most once per issued token.
func ConsumeSessionToken(sessionID string) error {
func ConsumeSessionToken(instanceID, sessionID string) error {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
now := time.Now()
res, err := db.Col("console_sessions").UpdateOne(ctx,
bson.M{"session_id": sessionID, "token_consumed_at": nil},
bson.M{"session_id": sessionID, "instance_id": instanceID, "token_consumed_at": nil},
bson.M{"$set": bson.M{"token_consumed_at": now}},
)
if err != nil {
@@ -238,12 +224,12 @@ func ConsumeSessionToken(sessionID string) error {
return nil
}
func EndConsoleSession(sessionID string) error {
func EndConsoleSession(instanceID, sessionID string) error {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
now := time.Now()
_, err := db.Col("console_sessions").UpdateOne(ctx,
bson.M{"session_id": sessionID, "ended_at": nil},
bson.M{"session_id": sessionID, "instance_id": instanceID, "ended_at": nil},
bson.M{"$set": bson.M{"ended_at": now}},
)
return err
-111
View File
@@ -1,111 +0,0 @@
package services
import (
"testing"
"time"
"github.com/mrhid6/vantage/server/internal/models"
)
func TestSessionTokenRoundTrip(t *testing.T) {
t.Setenv("KEY_ENCRYPTION_KEY", "00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff")
tok, err := SignSessionToken("sess-123", time.Minute)
if err != nil {
t.Fatalf("sign: %v", err)
}
got, err := VerifySessionToken(tok)
if err != nil {
t.Fatalf("verify: %v", err)
}
if got != "sess-123" {
t.Fatalf("got %q want sess-123", got)
}
}
func TestSessionTokenExpired(t *testing.T) {
t.Setenv("KEY_ENCRYPTION_KEY", "00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff")
tok, err := SignSessionToken("sess-123", -time.Second)
if err != nil {
t.Fatalf("sign: %v", err)
}
if _, err := VerifySessionToken(tok); err == nil {
t.Fatalf("expected expiry error, got nil")
}
}
func TestSessionTokenTampered(t *testing.T) {
t.Setenv("KEY_ENCRYPTION_KEY", "00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff")
tok, _ := SignSessionToken("sess-123", time.Minute)
if _, err := VerifySessionToken(tok + "x"); err == nil {
t.Fatalf("expected signature error, got nil")
}
}
func TestBuildGuacParamsSSH(t *testing.T) {
srv := &models.Server{IPAddress: "10.0.0.5", SSHPort: 22}
p, err := BuildGuacParams(srv, "ssh", "", "PRIVATE-KEY-DATA", "", "", "")
if err != nil {
t.Fatalf("err: %v", err)
}
if p.Protocol != "ssh" {
t.Fatalf("protocol %q", p.Protocol)
}
if p.Params["hostname"] != "10.0.0.5" || p.Params["port"] != "22" {
t.Fatalf("bad host/port: %+v", p.Params)
}
if p.Params["private-key"] != "PRIVATE-KEY-DATA" {
t.Fatalf("missing private-key")
}
if p.Params["username"] != "root" {
t.Fatalf("expected default username root, got %q", p.Params["username"])
}
}
func TestBuildGuacParamsRDP(t *testing.T) {
srv := &models.Server{IPAddress: "10.0.0.9", RDPPort: 3389}
p, err := BuildGuacParams(srv, "rdp", "", "", "", "administrator", "s3cret")
if err != nil {
t.Fatalf("err: %v", err)
}
if p.Params["port"] != "3389" || p.Params["username"] != "administrator" || p.Params["password"] != "s3cret" {
t.Fatalf("bad rdp params: %+v", p.Params)
}
if p.Params["ignore-cert"] != "true" {
t.Fatalf("expected ignore-cert=true")
}
}
func TestBuildGuacParamsUnknownProtocol(t *testing.T) {
srv := &models.Server{IPAddress: "10.0.0.9"}
if _, err := BuildGuacParams(srv, "telnet", "", "", "", "", ""); err == nil {
t.Fatalf("expected error for unknown protocol")
}
}
func TestBuildGuacParamsSSHPassphrase(t *testing.T) {
srv := &models.Server{IPAddress: "10.0.0.5", SSHPort: 22}
p, err := BuildGuacParams(srv, "ssh", "deploy", "PK", "s3cret-phrase", "", "")
if err != nil {
t.Fatalf("err: %v", err)
}
if p.Params["username"] != "deploy" {
t.Fatalf("username %q", p.Params["username"])
}
if p.Params["passphrase"] != "s3cret-phrase" {
t.Fatalf("missing passphrase: %+v", p.Params)
}
}
func TestBuildGuacParamsVNC(t *testing.T) {
srv := &models.Server{IPAddress: "10.0.0.7"}
p, err := BuildGuacParams(srv, "vnc", "", "", "", "", "vncpass")
if err != nil {
t.Fatalf("err: %v", err)
}
if p.Protocol != "vnc" || p.Params["hostname"] != "10.0.0.7" || p.Params["port"] != "5900" || p.Params["password"] != "vncpass" {
t.Fatalf("bad vnc params: %+v", p.Params)
}
}
+41
View File
@@ -0,0 +1,41 @@
package services
import (
"context"
"time"
"github.com/mrhid6/vantage/server/internal/db"
"github.com/mrhid6/vantage/shared/indexes"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
// EnsureAuthIndexes declares the indexes tenant isolation depends on.
//
// It lives here rather than in migrate.go so that the org-to-instance rename
// could touch it without touching migrations 0001 to 0003, which deliberately
// still speak the pre-rename shape.
//
// It MUST run after MigrateOrgToInstance. Creating the instances.slug index
// first would create an empty instances collection, and migration 0004 refuses
// to rename orgs when instances already exists.
func EnsureAuthIndexes() error {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// users.email and instances.slug are declared in the shared module so the
// control plane and sitesvc cannot disagree about them.
if err := indexes.EnsureCoreIndexes(ctx, db.Database); err != nil {
return err
}
// instance_oidc is control-plane only, so its index stays here.
if _, err := db.Col("instance_oidc").Indexes().CreateOne(ctx, mongo.IndexModel{
Keys: bson.D{{Key: "instance_id", Value: 1}},
Options: options.Index().SetUnique(true),
}); err != nil {
return err
}
return nil
}
-3
View File
@@ -22,8 +22,6 @@ func encryptionKey() ([]byte, error) {
return key, nil
}
// encryptString encrypts a plaintext value with AES-256-GCM using the
// shared KEY_ENCRYPTION_KEY, returning hex(nonce + ciphertext).
func encryptString(plaintext string) (string, error) {
key, err := encryptionKey()
if err != nil {
@@ -45,7 +43,6 @@ func encryptString(plaintext string) (string, error) {
return hex.EncodeToString(sealed), nil
}
// decryptString reverses encryptString.
func decryptString(ciphertextHex string) (string, error) {
key, err := encryptionKey()
if err != nil {
+90
View File
@@ -0,0 +1,90 @@
package services
import (
"os"
"path/filepath"
"time"
"github.com/google/uuid"
"github.com/mrhid6/vantage/server/internal/db"
"github.com/mrhid6/vantage/server/internal/models"
"github.com/mrhid6/vantage/shared/provision"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
func DefaultStepsDir() string {
dir := os.Getenv("VANTAGE_DEFAULT_STEPS_DIR")
if dir == "" {
dir = filepath.Join("data", "default-steps")
}
_ = os.MkdirAll(dir, 0700)
return dir
}
func readDefaultStepFiles() ([]models.WorkflowStep, error) {
matches, err := filepath.Glob(filepath.Join(DefaultStepsDir(), "*.json"))
if err != nil {
return nil, err
}
out := []models.WorkflowStep{}
for _, path := range matches {
b, err := os.ReadFile(path)
if err != nil {
continue
}
s, err := ParseStepDoc(b)
if err != nil {
continue
}
s.Source = "default"
s.Slug = provision.Slugify(s.Name)
if s.Slug == "" {
continue
}
out = append(out, s)
}
return out, nil
}
func SeedDefaultSteps(instanceID string) (created, updated int, err error) {
steps, err := readDefaultStepFiles()
if err != nil {
return 0, 0, err
}
ctx, cancel := wfCtx()
defer cancel()
col := db.Col("workflow_steps")
for _, s := range steps {
filter := bson.M{"instance_id": instanceID, "slug": s.Slug, "source": "default"}
set := bson.M{
"name": s.Name,
"description": s.Description,
"interpreter": s.Interpreter,
"script": s.Script,
"declared_outputs": s.DeclaredOutputs,
"declared_inputs": s.DeclaredInputs,
"secret_refs": s.SecretRefs,
"updated_at": time.Now(),
}
res, uerr := col.UpdateOne(ctx, filter, bson.M{
"$set": set,
"$setOnInsert": bson.M{
"instance_id": instanceID,
"step_id": uuid.New().String(),
"slug": s.Slug,
"source": "default",
"created_at": time.Now(),
},
}, options.UpdateOne().SetUpsert(true))
if uerr != nil {
return created, updated, uerr
}
if res.UpsertedCount > 0 {
created++
} else if res.ModifiedCount > 0 {
updated++
}
}
return created, updated, nil
}
+12 -19
View File
@@ -17,13 +17,10 @@ type commandDispatcher struct {
channels map[string]chan *pb.ServerCommand
}
// Dispatcher is the singleton command dispatcher used by both the gRPC server
// and the REST API to push commands to connected agents.
var Dispatcher = &commandDispatcher{
channels: make(map[string]chan *pb.ServerCommand),
}
// Connect registers an agent's command channel. Returns the channel to drain.
func (d *commandDispatcher) Connect(serverID string) chan *pb.ServerCommand {
ch := make(chan *pb.ServerCommand, 16)
d.mu.Lock()
@@ -32,14 +29,12 @@ func (d *commandDispatcher) Connect(serverID string) chan *pb.ServerCommand {
return ch
}
// Disconnect removes the agent's channel on stream close.
func (d *commandDispatcher) Disconnect(serverID string) {
d.mu.Lock()
delete(d.channels, serverID)
d.mu.Unlock()
}
// IsConnected reports whether an agent is currently holding a CommandStream.
func (d *commandDispatcher) IsConnected(serverID string) bool {
d.mu.RLock()
_, ok := d.channels[serverID]
@@ -62,13 +57,20 @@ func (d *commandDispatcher) dispatch(serverID string, cmd *pb.ServerCommand) err
}
}
// DispatchRunStep pushes a RunStepCmd to a server's agent. Caller must have
// registered StepResults.Await(commandID) first.
func DispatchRunStep(serverID, commandID string, cmd *pb.RunStepCmd) error {
return Dispatcher.dispatch(serverID, &pb.ServerCommand{CommandId: commandID, RunStep: cmd})
}
// KeyGenParams carries all options for a generate-key command.
func DispatchCleanupWorkspace(serverID, workspaceID string) {
if !Dispatcher.IsConnected(serverID) {
return
}
_ = Dispatcher.dispatch(serverID, &pb.ServerCommand{
CommandId: uuid.New().String(),
CleanupWorkspace: &pb.CleanupWorkspaceCmd{WorkspaceId: workspaceID},
})
}
type KeyGenParams struct {
Label string
KeyType string
@@ -77,15 +79,13 @@ type KeyGenParams struct {
Comment string
}
// GetLatestAgentVersion queries the Gitea API for the latest agent/v* release tag
// and returns just the version number (e.g. "1.2.3").
func GetLatestAgentVersion() (string, error) {
giteaHost := os.Getenv("GITEA_HOST")
if giteaHost == "" {
giteaHost = "gitea.example.com"
}
url := fmt.Sprintf("https://%s/api/v1/repos/mrhid6/vantage/releases?limit=20", giteaHost)
resp, err := http.Get(url) //nolint:gosec
resp, err := http.Get(url)
if err != nil {
return "", fmt.Errorf("fetch releases: %w", err)
}
@@ -109,8 +109,6 @@ func GetLatestAgentVersion() (string, error) {
return "", fmt.Errorf("no agent release found")
}
// DispatchUpdateAgent sends an update command to the named server's agent.
// It fetches the latest version from Gitea and includes the download base URL.
func DispatchUpdateAgent(serverID string) (string, error) {
if !Dispatcher.IsConnected(serverID) {
return "", fmt.Errorf("agent is not connected to the command stream")
@@ -140,7 +138,6 @@ func DispatchUpdateAgent(serverID string) (string, error) {
return version, nil
}
// DispatchApplyUpdates sends an apply-updates command to the named server's agent.
func DispatchApplyUpdates(serverID string) error {
if !Dispatcher.IsConnected(serverID) {
return fmt.Errorf("agent is not connected to the command stream")
@@ -152,8 +149,6 @@ func DispatchApplyUpdates(serverID string) error {
return Dispatcher.dispatch(serverID, cmd)
}
// DispatchDeleteKey sends a delete-key command to the named server's agent.
// It is best-effort: if the agent is offline the local files will remain until next connection.
func DispatchDeleteKey(serverID, label string) {
if !Dispatcher.IsConnected(serverID) {
return
@@ -163,13 +158,11 @@ func DispatchDeleteKey(serverID, label string) {
DeleteKey: &pb.DeleteKeyCmd{Label: label},
}
if err := Dispatcher.dispatch(serverID, cmd); err != nil {
// Non-fatal: agent will clean up files on next manual intervention or reinstall.
_ = err
}
}
// DispatchGenerateKey sends a generate-key command to the named server's agent.
// Returns the command ID that can be used to correlate the agent's result.
func DispatchGenerateKey(serverID string, p KeyGenParams) (string, error) {
if !Dispatcher.IsConnected(serverID) {
return "", fmt.Errorf("agent is not connected to the command stream")
+50
View File
@@ -0,0 +1,50 @@
package services
import (
"context"
"time"
"github.com/mrhid6/vantage/server/internal/db"
"github.com/mrhid6/vantage/server/internal/models"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
func GetInstanceOIDC(instanceID string) (*models.InstanceOIDC, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
var o models.InstanceOIDC
err := db.Col("instance_oidc").FindOne(ctx, bson.M{"instance_id": instanceID}).Decode(&o)
if err != nil {
return nil, err
}
return &o, nil
}
func GetInstanceOIDCSecret(instanceID string) (string, error) {
o, err := GetInstanceOIDC(instanceID)
if err != nil {
return "", err
}
return decryptString(o.ClientSecretEnc)
}
func SaveInstanceOIDC(instanceID, issuer, clientID, clientSecret string, enabled bool) error {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
set := bson.M{
"instance_id": instanceID, "issuer": issuer, "client_id": clientID,
"enabled": enabled, "updated_at": time.Now(),
}
if clientSecret != "" {
enc, err := encryptString(clientSecret)
if err != nil {
return err
}
set["client_secret_enc"] = enc
}
_, err := db.Col("instance_oidc").UpdateOne(ctx,
bson.M{"instance_id": instanceID}, bson.M{"$set": set},
options.UpdateOne().SetUpsert(true))
return err
}
+122
View File
@@ -0,0 +1,122 @@
package services
import (
"context"
"fmt"
"log"
"time"
"github.com/mrhid6/vantage/server/internal/db"
"github.com/mrhid6/vantage/server/internal/models"
"github.com/mrhid6/vantage/shared/provision"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
)
func GetInstance(instanceID string) (*models.Instance, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
var o models.Instance
err := db.Col("instances").FindOne(ctx, bson.M{"instance_id": instanceID}).Decode(&o)
if err != nil {
return nil, err
}
return &o, nil
}
func GetInstanceBySlug(slug string) (*models.Instance, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
var o models.Instance
err := db.Col("instances").FindOne(ctx, bson.M{"slug": slug}).Decode(&o)
if err != nil {
return nil, err
}
return &o, nil
}
func ListInstanceIDs() ([]string, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
cursor, err := db.Col("instances").Find(ctx, bson.M{})
if err != nil {
return nil, err
}
defer cursor.Close(ctx)
var orgs []models.Instance
if err := cursor.All(ctx, &orgs); err != nil {
return nil, err
}
ids := make([]string, 0, len(orgs))
for _, o := range orgs {
ids = append(ids, o.InstanceID)
}
return ids, nil
}
func CountInstances() (int64, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
return db.Col("instances").CountDocuments(ctx, bson.M{})
}
func FirstInstance() (*models.Instance, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
var o models.Instance
if err := db.Col("instances").FindOne(ctx, bson.M{}).Decode(&o); err != nil {
return nil, err
}
return &o, nil
}
func AdoptInstance(instanceID, name string) (*models.Instance, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
set := bson.M{"name": name}
slug := provision.Slugify(name)
if len(slug) > provision.MaxSlugLength {
slug = slug[:provision.MaxSlugLength]
}
if len(slug) >= provision.MinSlugLength && !provision.ReservedSlugs[slug] {
n, err := db.Col("instances").CountDocuments(ctx, bson.M{"slug": slug, "instance_id": bson.M{"$ne": instanceID}})
if err != nil {
return nil, err
}
if n == 0 {
set["slug"] = slug
}
}
if _, err := db.Col("instances").UpdateOne(ctx, bson.M{"instance_id": instanceID}, bson.M{"$set": set}); err != nil {
if mongo.IsDuplicateKeyError(err) {
return nil, fmt.Errorf("instance slug already taken")
}
return nil, err
}
return GetInstance(instanceID)
}
// CreateInstance creates an organisation and seeds its default workflow steps.
//
// The creation rules live in shared/provision because sitesvc creates
// organisations too. Seeding stays here: shared must not know about workflow
// steps.
func CreateInstance(name string) (*models.Instance, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
o, err := provision.CreateInstance(ctx, db.Database, name)
if err != nil {
return nil, err
}
if created, updated, err := SeedDefaultSteps(o.InstanceID); err != nil {
log.Printf("warning: failed to seed default steps for new org %s: %v", o.InstanceID, err)
} else {
log.Printf("default steps seeded for new org %s: %d created, %d updated", o.InstanceID, created, updated)
}
return o, nil
}
+50
View File
@@ -0,0 +1,50 @@
package services
import (
"context"
"time"
"github.com/mrhid6/vantage/server/internal/db"
"github.com/mrhid6/vantage/server/internal/grpc/pb"
"go.mongodb.org/mongo-driver/v2/bson"
)
func StoreInventory(serverID string, r *pb.InventoryReport) error {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
now := time.Now()
set := bson.M{"inventory.metrics_at": now}
if r.CPU != nil {
set["inventory.cpu.usage_pct"] = r.CPU.UsagePct
set["inventory.cpu.load1"] = r.CPU.Load1
}
if r.Memory != nil {
set["inventory.memory.used_bytes"] = r.Memory.UsedBytes
}
set["inventory.swap_used_bytes"] = r.SwapUsed
if r.IncludeStatic {
set["inventory.static_at"] = now
set["inventory.swap_total_bytes"] = r.SwapTotal
set["inventory.kernel"] = r.Kernel
if r.CPU != nil {
set["inventory.cpu.model"] = r.CPU.Model
set["inventory.cpu.cores"] = r.CPU.Cores
}
if r.Memory != nil {
set["inventory.memory.total_bytes"] = r.Memory.TotalBytes
}
parts := make([]bson.M, 0, len(r.Partitions))
for _, p := range r.Partitions {
parts = append(parts, bson.M{
"device": p.Device, "mountpoint": p.Mountpoint, "fstype": p.Fstype,
"total_bytes": p.TotalBytes, "used_bytes": p.UsedBytes,
})
}
set["inventory.partitions"] = parts
}
_, err := db.Col("servers").UpdateOne(ctx, bson.M{"server_id": serverID}, bson.M{"$set": set})
return err
}
+38 -30
View File
@@ -36,8 +36,9 @@ func setKeyMeta(k *models.Key) {
k.HasPassphrase = k.PassphraseEncrypted != ""
}
func CreateKey(label, publicKey, source, generatedByServerID, privateKey, passphrase string) (*models.Key, error) {
func CreateKey(instanceID, label, publicKey, source, generatedByServerID, privateKey, passphrase string) (*models.Key, error) {
key := &models.Key{
InstanceID: instanceID,
KeyID: uuid.NewString(),
Label: label,
PublicKey: publicKey,
@@ -71,12 +72,12 @@ func CreateKey(label, publicKey, source, generatedByServerID, privateKey, passph
return key, nil
}
func GetKey(keyID string) (*models.Key, error) {
func GetKey(instanceID, keyID string) (*models.Key, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
var key models.Key
err := db.Col("keys").FindOne(ctx, bson.M{"key_id": keyID}).Decode(&key)
err := db.Col("keys").FindOne(ctx, bson.M{"key_id": keyID, "instance_id": instanceID}).Decode(&key)
if err != nil {
return nil, err
}
@@ -84,12 +85,12 @@ func GetKey(keyID string) (*models.Key, error) {
return &key, nil
}
func GetPrivateKey(keyID string) (string, error) {
func GetPrivateKey(instanceID, keyID string) (string, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
var key models.Key
if err := db.Col("keys").FindOne(ctx, bson.M{"key_id": keyID}).Decode(&key); err != nil {
if err := db.Col("keys").FindOne(ctx, bson.M{"key_id": keyID, "instance_id": instanceID}).Decode(&key); err != nil {
return "", err
}
if key.PrivateKeyEncrypted == "" {
@@ -98,8 +99,6 @@ func GetPrivateKey(keyID string) (string, error) {
return decryptPrivateKey(key.PrivateKeyEncrypted)
}
// GetPassphrase returns the decrypted passphrase for a key, or an empty string
// if the key has none stored.
func GetPassphrase(keyID string) (string, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
@@ -119,11 +118,11 @@ type KeyWithCount struct {
AssignedCount int `bson:"-" json:"assigned_count"`
}
func ListKeys() ([]KeyWithCount, error) {
func ListKeys(instanceID string) ([]KeyWithCount, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
cursor, err := db.Col("keys").Find(ctx, bson.M{})
cursor, err := db.Col("keys").Find(ctx, bson.M{"instance_id": instanceID})
if err != nil {
return nil, err
}
@@ -138,27 +137,28 @@ func ListKeys() ([]KeyWithCount, error) {
for _, k := range keys {
setKeyMeta(&k)
count, _ := db.Col("assignments").CountDocuments(ctx, bson.M{
"key_id": k.KeyID,
"revoked_at": nil,
"instance_id": instanceID,
"key_id": k.KeyID,
"revoked_at": nil,
})
result = append(result, KeyWithCount{Key: k, AssignedCount: int(count)})
}
return result, nil
}
func DeleteKey(keyID string) error {
func DeleteKey(instanceID, keyID string) error {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
var key models.Key
if err := db.Col("keys").FindOne(ctx, bson.M{"key_id": keyID}).Decode(&key); err != nil {
if err := db.Col("keys").FindOne(ctx, bson.M{"key_id": keyID, "instance_id": instanceID}).Decode(&key); err != nil {
return err
}
if _, err := db.Col("keys").DeleteOne(ctx, bson.M{"key_id": keyID}); err != nil {
if _, err := db.Col("keys").DeleteOne(ctx, bson.M{"key_id": keyID, "instance_id": instanceID}); err != nil {
return err
}
if _, err := db.Col("assignments").DeleteMany(ctx, bson.M{"key_id": keyID}); err != nil {
if _, err := db.Col("assignments").DeleteMany(ctx, bson.M{"key_id": keyID, "instance_id": instanceID}); err != nil {
return err
}
@@ -168,22 +168,30 @@ func DeleteKey(keyID string) error {
return nil
}
func AssignKey(keyID, serverID string) (*models.Assignment, error) {
func AssignKey(instanceID, keyID, serverID string) (*models.Assignment, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
// Check if already assigned and active
if _, err := GetKey(instanceID, keyID); err != nil {
return nil, fmt.Errorf("key not found")
}
if _, err := GetServer(instanceID, serverID); err != nil {
return nil, fmt.Errorf("server not found")
}
var existing models.Assignment
err := db.Col("assignments").FindOne(ctx, bson.M{
"key_id": keyID,
"server_id": serverID,
"revoked_at": nil,
"instance_id": instanceID,
"key_id": keyID,
"server_id": serverID,
"revoked_at": nil,
}).Decode(&existing)
if err == nil {
return &existing, nil
}
a := &models.Assignment{
InstanceID: instanceID,
KeyID: keyID,
ServerID: serverID,
AssignedAt: time.Now(),
@@ -195,23 +203,23 @@ func AssignKey(keyID, serverID string) (*models.Assignment, error) {
return a, nil
}
func RevokeAssignment(keyID, serverID string) error {
func RevokeAssignment(instanceID, keyID, serverID string) error {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
now := time.Now()
_, err := db.Col("assignments").UpdateOne(ctx,
bson.M{"key_id": keyID, "server_id": serverID, "revoked_at": nil},
bson.M{"instance_id": instanceID, "key_id": keyID, "server_id": serverID, "revoked_at": nil},
bson.M{"$set": bson.M{"revoked_at": now}},
)
return err
}
func GetAssignmentsForKey(keyID string) ([]models.Assignment, error) {
func GetAssignmentsForKey(instanceID, keyID string) ([]models.Assignment, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
cursor, err := db.Col("assignments").Find(ctx, bson.M{"key_id": keyID, "revoked_at": nil})
cursor, err := db.Col("assignments").Find(ctx, bson.M{"instance_id": instanceID, "key_id": keyID, "revoked_at": nil})
if err != nil {
return nil, err
}
@@ -229,11 +237,11 @@ type AssignmentWithServer struct {
Server *models.Server `json:"server,omitempty"`
}
func GetAssignmentsWithServers(keyID string) ([]AssignmentWithServer, error) {
func GetAssignmentsWithServers(instanceID, keyID string) ([]AssignmentWithServer, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
cursor, err := db.Col("assignments").Find(ctx, bson.M{"key_id": keyID})
cursor, err := db.Col("assignments").Find(ctx, bson.M{"instance_id": instanceID, "key_id": keyID})
if err != nil {
return nil, err
}
@@ -248,7 +256,7 @@ func GetAssignmentsWithServers(keyID string) ([]AssignmentWithServer, error) {
for _, a := range assignments {
item := AssignmentWithServer{Assignment: a}
var srv models.Server
if err := db.Col("servers").FindOne(ctx, bson.M{"server_id": a.ServerID}).Decode(&srv); err == nil {
if err := db.Col("servers").FindOne(ctx, bson.M{"server_id": a.ServerID, "instance_id": instanceID}).Decode(&srv); err == nil {
item.Server = &srv
}
result = append(result, item)
@@ -261,11 +269,11 @@ type AssignmentWithKey struct {
Key *models.Key `json:"key,omitempty"`
}
func GetAssignmentsWithKeysForServer(serverID string) ([]AssignmentWithKey, error) {
func GetAssignmentsWithKeysForServer(instanceID, serverID string) ([]AssignmentWithKey, error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
cursor, err := db.Col("assignments").Find(ctx, bson.M{"server_id": serverID})
cursor, err := db.Col("assignments").Find(ctx, bson.M{"instance_id": instanceID, "server_id": serverID})
if err != nil {
return nil, err
}
@@ -279,7 +287,7 @@ func GetAssignmentsWithKeysForServer(serverID string) ([]AssignmentWithKey, erro
result := make([]AssignmentWithKey, 0, len(assignments))
for _, a := range assignments {
var key models.Key
if err := db.Col("keys").FindOne(ctx, bson.M{"key_id": a.KeyID}).Decode(&key); err != nil {
if err := db.Col("keys").FindOne(ctx, bson.M{"key_id": a.KeyID, "instance_id": instanceID}).Decode(&key); err != nil {
continue
}
setKeyMeta(&key)
+175
View File
@@ -0,0 +1,175 @@
package services
import (
"context"
"fmt"
"os"
"strings"
"sync"
"time"
"github.com/mrhid6/vantage/server/internal/db"
"github.com/mrhid6/vantage/shared/license"
"go.mongodb.org/mongo-driver/v2/bson"
)
// LicenseState is the resolved licence for one instance.
type LicenseState struct {
Status license.State `json:"state"`
Reason string `json:"reason,omitempty"`
Tier string `json:"tier,omitempty"`
ExpiresAt *time.Time `json:"expires_at,omitempty"`
Limits license.Limits `json:"limits"`
Features map[string]bool `json:"features"`
// Source is "stored", "env" or "none" — useful when a self-hosted operator
// asks why the licence they pasted is not the one in effect.
Source string `json:"source"`
}
// Active reports whether mutations are allowed.
func (s LicenseState) Active() bool { return s.Status == license.StateValid }
// Feature reports whether a named feature is granted.
func (s LicenseState) Feature(name string) bool { return s.Features[name] }
// DeploymentMode is how this install describes itself to the verifier.
//
// It defaults to self_hosted, the stricter mode. An operator who removes the
// variable gets the tighter behaviour, not the looser one.
func DeploymentMode() string {
if strings.ToLower(os.Getenv("VANTAGE_DEPLOYMENT")) == license.DeploymentCloud {
return license.DeploymentCloud
}
return license.DeploymentSelfHosted
}
type cachedLicense struct {
state LicenseState
at time.Time
}
var (
licenseCacheMu sync.Mutex
licenseCache = map[string]cachedLicense{}
)
const licenseCacheTTL = 60 * time.Second
// InvalidateLicenseCache drops the cached state for one instance, so a pasted
// licence takes effect immediately rather than within the TTL.
func InvalidateLicenseCache(instanceID string) {
licenseCacheMu.Lock()
delete(licenseCache, instanceID)
licenseCacheMu.Unlock()
}
// GetLicenseState resolves the licence for an instance, cached for 60 seconds.
//
// Resolution order:
//
// 1. the blob stored on the instance document
// 2. VANTAGE_LICENSE, used ONLY when the instance has no stored blob, so an
// automated self-hosted deployment can ship a licence without a human
// pasting one
// 3. neither -> invalid / no_license
//
// A blob stored through the UI always wins afterwards, so an operator is never
// locked out by a stale environment value.
func GetLicenseState(instanceID string) LicenseState {
licenseCacheMu.Lock()
if e, ok := licenseCache[instanceID]; ok && time.Since(e.at) < licenseCacheTTL {
licenseCacheMu.Unlock()
return e.state
}
licenseCacheMu.Unlock()
state := resolveLicenseState(instanceID)
licenseCacheMu.Lock()
licenseCache[instanceID] = cachedLicense{state: state, at: time.Now()}
licenseCacheMu.Unlock()
return state
}
func resolveLicenseState(instanceID string) LicenseState {
inst, err := GetInstance(instanceID)
if err != nil {
return LicenseState{
Status: license.StateInvalid,
Reason: license.ReasonNoLicense,
Features: map[string]bool{},
Source: "none",
}
}
blob, source := inst.LicenseBlob, "stored"
if blob == "" {
blob, source = os.Getenv("VANTAGE_LICENSE"), "env"
}
if blob == "" {
return LicenseState{
Status: license.StateInvalid,
Reason: license.ReasonNoLicense,
Features: map[string]bool{},
Source: "none",
}
}
res := license.Verify(blob, license.VerifyOpts{
InstanceID: instanceID,
Deployment: DeploymentMode(),
})
return stateFromResult(res, source)
}
func stateFromResult(res license.Result, source string) LicenseState {
feats := map[string]bool{}
for _, f := range res.License.Features {
feats[f] = true
}
s := LicenseState{
Status: res.State,
Reason: res.Reason,
Tier: res.License.Tier,
Limits: res.License.Limits,
Features: feats,
Source: source,
}
if !res.License.ExpiresAt.IsZero() {
exp := res.License.ExpiresAt
s.ExpiresAt = &exp
}
return s
}
// StoreLicense verifies a blob against this instance and stores it.
//
// An expired-but-otherwise-valid blob IS stored, so the UI can show what expired
// and when. An invalid blob is rejected and the previous one kept.
func StoreLicense(instanceID, blob string) (LicenseState, error) {
blob = strings.TrimSpace(blob)
res := license.Verify(blob, license.VerifyOpts{
InstanceID: instanceID,
Deployment: DeploymentMode(),
})
if res.State == license.StateInvalid {
return LicenseState{}, fmt.Errorf("%s", res.Reason)
}
set := bson.M{
"license_blob": blob,
"license_tier": res.License.Tier,
"license_expiry": res.License.ExpiresAt,
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if _, err := db.Col("instances").UpdateOne(ctx,
bson.M{"instance_id": instanceID}, bson.M{"$set": set}); err != nil {
return LicenseState{}, err
}
InvalidateLicenseCache(instanceID)
return stateFromResult(res, "stored"), nil
}
+111
View File
@@ -0,0 +1,111 @@
package services
import (
"context"
"fmt"
"time"
"github.com/mrhid6/vantage/server/internal/db"
"github.com/mrhid6/vantage/shared/license"
"go.mongodb.org/mongo-driver/v2/bson"
)
// LimitError is returned when a licence cap would be exceeded. The API maps it
// to 403 with a machine-readable body.
type LimitError struct {
Limit string
Current int
Max int
}
func (e *LimitError) Error() string {
return fmt.Sprintf("licence limit reached: %s (%d of %d)", e.Limit, e.Current, e.Max)
}
func limitCtx() (context.Context, context.CancelFunc) {
return context.WithTimeout(context.Background(), 5*time.Second)
}
// CheckServerLimit refuses a new server when the instance is at its cap.
//
// Counts live rows only. An instance already over its cap keeps every server it
// has — nothing is truncated — it simply cannot add another.
func CheckServerLimit(instanceID string) error {
st := GetLicenseState(instanceID)
ctx, cancel := limitCtx()
defer cancel()
n, err := db.Col("servers").CountDocuments(ctx, bson.M{"instance_id": instanceID})
if err != nil {
return err
}
if !license.WithinLimit(int(n), st.Limits.MaxServers) {
return &LimitError{Limit: "max_servers", Current: int(n), Max: st.Limits.MaxServers}
}
return nil
}
// CheckSecretGroupLimit refuses a NEW group at the cap. Writing to a group that
// already exists is always allowed, so a capped customer can still rotate the
// secrets they have.
func CheckSecretGroupLimit(instanceID, group string) error {
st := GetLicenseState(instanceID)
ctx, cancel := limitCtx()
defer cancel()
existing, err := db.Col("secrets").CountDocuments(ctx,
bson.M{"instance_id": instanceID, "group": group})
if err != nil {
return err
}
if existing > 0 {
return nil
}
var groups []string
if err := db.Col("secrets").Distinct(ctx, "group",
bson.M{"instance_id": instanceID}).Decode(&groups); err != nil {
return err
}
if !license.WithinLimit(len(groups), st.Limits.MaxSecretGroups) {
return &LimitError{Limit: "max_secret_groups", Current: len(groups), Max: st.Limits.MaxSecretGroups}
}
return nil
}
// CheckChannelLimit refuses a new notification channel at the cap.
func CheckChannelLimit(instanceID string) error {
st := GetLicenseState(instanceID)
ctx, cancel := limitCtx()
defer cancel()
n, err := db.Col("notification_channels").CountDocuments(ctx, bson.M{"instance_id": instanceID})
if err != nil {
return err
}
if !license.WithinLimit(int(n), st.Limits.MaxChannels) {
return &LimitError{Limit: "max_channels", Current: int(n), Max: st.Limits.MaxChannels}
}
return nil
}
// LicenseUsage reports current counts, so the UI can say "12 of 3 servers"
// honestly when an instance is over its cap rather than pretending.
func LicenseUsage(instanceID string) (servers, secretGroups, channels int) {
ctx, cancel := limitCtx()
defer cancel()
if n, err := db.Col("servers").CountDocuments(ctx, bson.M{"instance_id": instanceID}); err == nil {
servers = int(n)
}
var groups []string
if err := db.Col("secrets").Distinct(ctx, "group",
bson.M{"instance_id": instanceID}).Decode(&groups); err == nil {
secretGroups = len(groups)
}
if n, err := db.Col("notification_channels").CountDocuments(ctx,
bson.M{"instance_id": instanceID}); err == nil {
channels = int(n)
}
return
}
+210
View File
@@ -0,0 +1,210 @@
package services
import (
"context"
"errors"
"fmt"
"time"
"github.com/google/uuid"
"github.com/mrhid6/vantage/server/internal/db"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
)
// legacyOrg is the pre-0004 shape of the orgs collection.
//
// Migrations 0001 to 0003 run BEFORE the org-to-instance rename and must keep
// reading and writing org_id in the orgs collection. They deliberately do not
// use shared/models, which has moved on to Instance and instance_id.
type legacyOrg struct {
OrgID string `bson:"org_id"`
Name string `bson:"name"`
Slug string `bson:"slug"`
CreatedAt time.Time `bson:"created_at"`
}
var backfillCollections = []string{
"servers", "keys", "assignments", "secrets",
"workflows", "workflow_steps", "workflow_runs",
"audit_logs", "monitors", "notification_channels",
"console_sessions", "incidents", "monitor_rollups",
}
func defaultBackfillOrg(ctx context.Context) (*legacyOrg, error) {
var org legacyOrg
err := db.Col("orgs").FindOne(ctx, bson.M{"slug": "default"}).Decode(&org)
switch {
case err == nil:
case errors.Is(err, mongo.ErrNoDocuments):
org = legacyOrg{OrgID: uuid.NewString(), Name: "Default", Slug: "default", CreatedAt: time.Now()}
if _, err := db.Col("orgs").InsertOne(ctx, org); err != nil {
return nil, err
}
default:
return nil, err
}
return &org, nil
}
func RunMigrations() error {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
const marker = "0001_default_org_backfill"
if n, _ := db.Col("migrations").CountDocuments(ctx, bson.M{"_id": marker}); n > 0 {
return nil
}
needs := false
for _, col := range backfillCollections {
n, _ := db.Col(col).CountDocuments(ctx, bson.M{"org_id": bson.M{"$exists": false}})
if n > 0 {
needs = true
break
}
}
if needs {
org, err := defaultBackfillOrg(ctx)
if err != nil {
return err
}
for _, col := range backfillCollections {
if _, err := db.Col(col).UpdateMany(ctx,
bson.M{"org_id": bson.M{"$exists": false}},
bson.M{"$set": bson.M{"org_id": org.OrgID}},
); err != nil {
return err
}
}
}
_, err := db.Col("migrations").InsertOne(ctx, bson.M{"_id": marker, "applied_at": time.Now()})
return err
}
func MigrateMissedOrgScopes() error {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
const marker = "0003_missed_org_scopes"
if n, _ := db.Col("migrations").CountDocuments(ctx, bson.M{"_id": marker}); n > 0 {
return nil
}
missed := []string{"audit_logs", "notification_channels"}
needs := false
for _, col := range missed {
n, _ := db.Col(col).CountDocuments(ctx, bson.M{"org_id": bson.M{"$exists": false}})
if n > 0 {
needs = true
break
}
}
if needs {
org, err := defaultBackfillOrg(ctx)
if err != nil {
return err
}
for _, col := range missed {
if _, err := db.Col(col).UpdateMany(ctx,
bson.M{"org_id": bson.M{"$exists": false}},
bson.M{"$set": bson.M{"org_id": org.OrgID}},
); err != nil {
return err
}
}
}
if err := backfillOrgFromOwner(ctx, "console_sessions", "server_id", "servers", "server_id"); err != nil {
return err
}
if err := backfillOrgFromOwner(ctx, "incidents", "monitor_id", "monitors", "monitor_id"); err != nil {
return err
}
if err := backfillOrgFromOwner(ctx, "monitor_rollups", "monitor_id", "monitors", "monitor_id"); err != nil {
return err
}
_, err := db.Col("migrations").InsertOne(ctx, bson.M{"_id": marker, "applied_at": time.Now()})
return err
}
func backfillOrgFromOwner(ctx context.Context, col, localField, ownerCol, ownerField string) error {
var raw []bson.RawValue
if err := db.Col(col).Distinct(ctx, localField,
bson.M{"org_id": bson.M{"$exists": false}}).Decode(&raw); err != nil {
return err
}
for _, rv := range raw {
id, ok := rv.StringValueOK()
if !ok || id == "" {
continue
}
var owner struct {
OrgID string `bson:"org_id"`
}
if err := db.Col(ownerCol).FindOne(ctx, bson.M{ownerField: id}).Decode(&owner); err != nil {
continue
}
if owner.OrgID == "" {
continue
}
if _, err := db.Col(col).UpdateMany(ctx,
bson.M{localField: id, "org_id": bson.M{"$exists": false}},
bson.M{"$set": bson.M{"org_id": owner.OrgID}},
); err != nil {
return err
}
}
return nil
}
func MigrateSettingsOrg() error {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
const marker = "0002_settings_org_backfill"
if n, _ := db.Col("migrations").CountDocuments(ctx, bson.M{"_id": marker}); n > 0 {
return nil
}
n, _ := db.Col("settings").CountDocuments(ctx, bson.M{"org_id": bson.M{"$exists": false}})
if n > 0 {
var org legacyOrg
orgCount, err := db.Col("orgs").CountDocuments(ctx, bson.M{})
if err != nil {
return err
}
switch orgCount {
case 1:
if err := db.Col("orgs").FindOne(ctx, bson.M{}).Decode(&org); err != nil {
return err
}
case 0:
org = legacyOrg{OrgID: uuid.NewString(), Name: "Default", Slug: "default", CreatedAt: time.Now()}
if _, err := db.Col("orgs").InsertOne(ctx, org); err != nil {
return err
}
default:
return fmt.Errorf(
"settings org migration: %d settings document(s) have no org_id but %d orgs exist; "+
"cannot infer the owner. Set org_id manually on each settings document "+
"(db.settings.updateOne({_id:<id>},{$set:{org_id:\"<org uuid>\"}})), deleting any "+
"duplicates, then restart", n, orgCount)
}
if org.OrgID != "" {
if _, err := db.Col("settings").UpdateMany(ctx,
bson.M{"org_id": bson.M{"$exists": false}},
bson.M{"$set": bson.M{"org_id": org.OrgID}},
); err != nil {
return err
}
}
}
_, err := db.Col("migrations").InsertOne(ctx, bson.M{"_id": marker, "applied_at": time.Now()})
return err
}

Some files were not shown because too many files have changed in this diff Show More