Compare commits

..
Author SHA1 Message Date
mrhid6 7b13944c24 fix: Build the vantagectl image on its release tag, not on every push to main
Chart Release / chart (push) Successful in 29s
vantagectl Release / build (push) Successful in 1m28s
vantagectl Release / image (push) Successful in 1m52s
Server Deploy / deploy (push) Successful in 3m46s
vantagectl is a released tool, not a running service. An operator restoring a
database should be able to name the version they ran, and ":latest, rebuilt
whenever main moved" cannot be named after the fact.

The image now builds in vantagectl-release.yml on a vantagectl/v* tag, tagged
with that version as well as latest, with VERSION passed through so the binary
inside reports the tag rather than "dev". server-deploy.yml no longer builds
it and is back to seven images.

The cost is that a shared/ fix reaches the image only at the next release
rather than the next push to main. That is the intended trade and is written
down in CLAUDE.md next to the trigger table.
2026-09-07 15:11:39 +00:00
mrhid6 e7384d334a feat: Add vantagectl, the control plane backup and restore CLI
Chart Release / chart (push) Successful in 33s
vantagectl Release / build (push) Successful in 5m56s
Server Deploy / deploy (push) Successful in 9m40s
Vantage encrypts SSH private keys, vault secrets, SSO client secrets and
console credentials with KEY_ENCRYPTION_KEY. That key is in no backup and is
not recoverable, so restoring a database without it produces a control plane
whose every secret is permanently unreadable.

vantagectl dumps and restores a whole database and stamps a SHA-256
fingerprint of the key into every archive, so a restore refuses rather than
producing that database. The key itself is never written.

- shared/cryptobox: AES-GCM extracted so the server and the CLI share one
  implementation rather than a copy that can drift
- shared/backup: manifest, tar+gzip archive with per-member checksums
  verified before the first write, dump, restore, verify
- vantagectl: its own module, so cobra stays out of the server, admin and
  sitesvc module graphs
- container image, tagged release workflow, CI rebuild trigger
- optional Helm CronJob, off by default
- operator documentation, reconciled with the existing backups page
2026-09-07 14:52:13 +00:00
mrhid6 be299845ca fix: document inspect and --confirm-db's actual behaviour 2026-09-07 14:45:37 +00:00
mrhid6 d83061786c fix: validate manifest collection names and route archive accessors through safeJoin 2026-09-07 14:45:37 +00:00
mrhid6 b28a2263bb fix: read KEY_ENCRYPTION_KEY in archive-only verify; never leave a partial archive 2026-09-07 14:45:36 +00:00
mrhid6 b6f45390c4 fix: correct verify's ciphertext field map against the models 2026-09-07 14:45:36 +00:00
mrhid6 2e4bc687d4 fix: replay index specs verbatim instead of reconstructing them 2026-09-07 14:45:36 +00:00
mrhid6 d328d3aaca docs: Reconcile backups and backup-and-restore pages
backups.md kept its store-level table and danger note but now points to
vantagectl as the supported path, with mongodump/mongorestore demoted to
an explicit fallback and a warning that a plain dump records no key
fingerprint. backup-and-restore.md links back for the store-level
overview.
2026-09-07 14:32:31 +00:00
mrhid6 89cc524f3f docs: Document backup and restore
The page leads with KEY_ENCRYPTION_KEY rather than mentioning it in a
note, because holding a good database dump and no key is the way this goes
wrong.
2026-09-07 14:29:03 +00:00
mrhid6 f0f2600bed feat: Add an optional scheduled backup CronJob to the chart
Off by default: a backup with nowhere durable to land is a false sense of
safety and the chart cannot know where that is. NOTES.txt says so when it
is off.

No restore manifest ships: a restore must never be something a helm
upgrade can trigger.
2026-09-07 14:17:42 +00:00
mrhid6 884fd189fb fix: Report the real reason verify falls back to archive-only checks
resolveGlobals can fail for two distinct reasons — no MongoDB URI, or no
resolvable database name — and verify.go was printing a hardcoded
no-URI note regardless of which one occurred, misleading an operator
whose URI was fine but whose database name could not be resolved.
2026-09-07 14:11:58 +00:00
mrhid6 e08cc2f928 feat: Build and publish vantagectl
The scratch runtime stage copies an explicit /tmp: restore extracts an
archive there before verifying it, and a scratch image has none.

shared/ now fans out to four Go images rather than three.
2026-09-07 14:09:00 +00:00
mrhid6 3ce963b0cd feat: Add the vantagectl restore and verify subcommands
--force requires a typed database name on a terminal and --confirm-db
without one, so a copy-pasted restore command carries its intended target
and cannot destroy a different database.

Also silences cobra's own error print (root.go) so a failure is reported
once by main.go instead of twice, and pins the Changed()-based env
fallback in resolveGlobals with a test for an explicitly empty --db.
2026-09-07 14:04:25 +00:00
mrhid6 461a79277d feat: Add the vantagectl backup and inspect subcommands
Progress output goes to stderr so --out - stays a clean pipe into restic,
age or aws s3 cp. Archive names carry no colon, because these get copied
onto Windows shares.
2026-09-07 13:57:04 +00:00
mrhid6 d9f7fa6993 feat: Add the vantagectl module and its cobra root
Its own module rather than a package under shared, so cobra and pflag stay
out of the module graphs of server, admin and sitesvc, which never use
them.
2026-09-07 13:45:20 +00:00
mrhid6 060fa64339 feat: Add backup verify with a live decrypt probe
A fingerprint comparison proves two archives agree about a key. Only
opening real ciphertext from the target proves the key in hand reads the
data, which is the question an operator actually has.
2026-09-07 11:26:34 +00:00
mrhid6 30d83c4c32 feat: Add the backup restore
Every refusal happens before the first write: format, checksums, key
policy, then target inspection. A unique index that will not build aborts,
because the unique indexes here are tenant-isolation properties rather
than optimisations.
2026-09-07 11:23:27 +00:00
mrhid6 a918b1bdc1 feat: Add the backup dump
Collections are enumerated live rather than from a list, so a collection
added later is backed up with no code change. Documents are written as the
raw BSON the driver returned, so Decimal128, ObjectId, DateTime and binary
subtypes survive byte for byte.
2026-09-07 11:16:38 +00:00
mrhid6 66b1a041ba feat: Add the backup archive writer and reader
Open extracts and verifies every member against the manifest before the
reader is usable, so a corrupt archive is refused before a restore writes
its first document rather than halfway through.
2026-09-07 11:10:17 +00:00
mrhid6 d55ed2b19a feat: Add the backup archive manifest
KeyFingerprint is a pointer so an archive that recorded no key is a state
restore can report, not a default it silently treats as a match.
2026-09-07 11:08:02 +00:00
mrhid6 8135c8d781 feat: Add key fingerprinting for backup archives
Fingerprint hashes the raw key bytes rather than the hex string, so the
same key written in different cases fingerprints identically.
2026-09-07 11:03:48 +00:00
mrhid6 577b060b8a feat: Extract AES-GCM into shared/cryptobox
services/crypto.go keeps its function names and its KEY_ENCRYPTION_KEY
lookup and delegates the cipher, so vantagectl's verify probe can decrypt
with the same implementation rather than a second copy.
2026-09-07 10:55:54 +00:00
mrhid6 1028a2e43a docs: Add the backup-restore implementation plan
Thirteen tasks, each ending in a testable deliverable and a commit.

Also drops --log-level from the spec: the tool's whole output is what it is
telling the operator, and a level that could hide a key warning is worth not
having.
2026-09-07 10:30:56 +00:00
mrhid6 eba93a812e docs: Correct Dockerfile location and specify the verify probe
vantagectl/Dockerfile follows the repo's per-module convention rather than
living under deploy/docker.

verify's live check needs AES-GCM open, so the cipher primitives move to
shared/cryptobox and services/crypto.go delegates, rather than a second
copy of the cipher existing in another module.
2026-09-07 10:10:04 +00:00
mrhid6 83b7256b60 docs: Design for control plane backup and restore
Standalone vantagectl CLI (cobra, own module) that dumps and restores a
whole Vantage MongoDB database, stamping a sha256 fingerprint of
KEY_ENCRYPTION_KEY into the manifest so a restore cannot silently produce
a database whose secrets are unreadable.

The key itself never enters the archive.
2026-09-07 10:07:50 +00:00
mrhid6 9d17f539b5 fix: Fixed card header margin
Chart Release / chart (push) Successful in 19s
Server Deploy / deploy (push) Successful in 1m0s
2026-09-07 08:36:18 +00:00
mrhid6 28f746c7e2 feat: Compact vitals
Chart Release / chart (push) Successful in 27s
Server Deploy / deploy (push) Successful in 1m4s
2026-09-07 08:26:07 +00:00
mrhid6 9d218cb19f feat: Reduce server vitials panel height
Chart Release / chart (push) Successful in 18s
Server Deploy / deploy (push) Successful in 1m10s
2026-09-07 08:14:47 +00:00
mrhid6 f9ef9c4929 fix: Fixed mobile scroll bar
Chart Release / chart (push) Successful in 25s
Server Deploy / deploy (push) Successful in 2m17s
2026-09-07 08:02:09 +00:00
mrhid6 049e005873 feat: Updated monitor chart
Chart Release / chart (push) Successful in 15s
Server Deploy / deploy (push) Successful in 1m9s
2026-08-25 15:02:04 +00:00
mrhid6 c440b59b93 feat: Updated affected components on status page incidents
Chart Release / chart (push) Successful in 13s
Server Deploy / deploy (push) Canceled after 6m59s
2026-08-25 14:56:50 +00:00
mrhid6 3e4865884c feat: Updated plans and catalogue pages
Chart Release / chart (push) Successful in 13s
Server Deploy / deploy (push) Successful in 1m58s
2026-08-25 14:35:47 +00:00
mrhid6 270d55e6a6 feat: Updated status page title 2026-08-25 14:11:21 +00:00
mrhid6 7a0a1953f6 fix: Fixed api url on web
Chart Release / chart (push) Successful in 10s
Server Deploy / deploy (push) Successful in 7m37s
2026-08-25 13:53:59 +00:00
mrhid6 3e4ccc9720 feat: Added more debug logging
Chart Release / chart (push) Successful in 16s
Server Deploy / deploy (push) Successful in 7m1s
2026-08-25 13:26:23 +00:00
mrhid6 e5947489e4 fix: Fixed status page published switch 2026-08-25 13:26:10 +00:00
mrhid6 0a7a10aeed feat: Added status pages to license page 2026-08-25 13:10:28 +00:00
mrhid6 28b813ba64 feat: sell status pages as a per-instance licence feature
Chart Release / chart (push) Successful in 13s
Server Deploy / deploy (push) Successful in 9m30s
2026-08-25 09:25:05 +00:00
mrhid6 68160dc681 docs: correct the status page URL for self-hosted, trim to what ships
- The status page URL was given only as `<instance>.vantage.<tld>`, which a
  self-hosted install does not serve. Both deployments are now described.
- The banner is documented as one notice: the editor exposes no level picker
  and the view renders every level identically.
- `pending` added to the component states, which a monitor with no result yet
  renders.
- Delete page documented alongside un-publish.
- `TRUSTED_PROXIES` names the LAN case: with the RFC1918 default, a client on a
  private range reaching the server directly is itself trusted and can spoof
  `X-Forwarded-For` — and now `X-Forwarded-Host`. Narrow it to the proxy.
- CLAUDE.md: scopes are nine resources, not eight; `status-pages` added to the
  REST route table; the host-resolution rules recorded under Status pages.
2026-08-25 09:05:06 +00:00
mrhid6 32e7420d89 fix: wire status page and incident delete, stop promising a name we do not publish
- The display-name placeholder showed the monitor's own name, reading as "leave
  this blank and we will use it". The server deliberately does the opposite: a
  blank `display_name` publishes the raw monitor id, because publishing an
  internal name has to be a decision. The placeholder now says "Public name
  (required)" and Save is refused until every component has one, so nobody adds
  five monitors and discovers five UUIDs on their public page. The server
  fallback is unchanged.
- `deleteStatusPage` and `deleteStatusIncident` existed in the api client and
  were wired to nothing, and the page address is immutable — delete was the
  only correction for a typo and there was no way to reach it. The editor
  header gains a typed-confirmation Delete page, and each incident row a
  confirmed delete, both on the existing ConfirmDialog.
- The create modal's address hint had lost its em dash and read as a broken
  sentence.
2026-08-25 09:05:06 +00:00
mrhid6 7e1d67dba4 fix: file live outages as active, refuse "operational" over zero components
- A derived monitor outage with no `resolved_at` went to `History`, so an
  ongoing disruption was listed under "Past incidents" while the component pill
  beside it read Down. Unresolved now goes to `ActiveIncidents`.
- `overallState` returned `up` when nothing was counted: "all systems
  operational" claimed from no evidence at all. A page with no components now
  reports `no_data`, which the view already renders as "Status unknown".
- `EnsureStatusPageIndexes` returned on the first failure, so a transient
  failure on the `status_pages` index left `status_incidents` with no unique
  `(instance_id, incident_id)` index — a correctness property, not a scan
  optimisation. All three are attempted and the failures joined.
2026-08-25 09:04:54 +00:00
mrhid6 da1dc90ac5 fix: resolve the public status page's tenant from a trusted X-Forwarded-Host
The SSR fetch set `Host` to the visitor's hostname. `Host` is a forbidden
header name and undici discards it silently, so the Go server saw
`server:8080`, `hostSlug` returned "", `InstanceFromHost` returned false and
every public status page 404'd on every deployment. The feature did not work.

- `web/` now forwards the visitor's host as `X-Forwarded-Host`, and their
  address on `X-Forwarded-For` — without the latter gin sees a request from the
  Next pod with no XFF and every visitor of every page shares one 120/min
  bucket, tripped by exactly the traffic an outage produces.
- `publicStatusInstance` honours `X-Forwarded-Host` only when `c.RemoteIP()` is
  in `TRUSTED_PROXIES`. It is a tenant selector, so an untrusted peer must not
  be able to name one; `RemoteIP()` rather than `ClientIP()` because the latter
  is reconstructed from the very headers being judged. `TrustedProxies()` moves
  from main.go into the api package so the variable keeps one parser.
- A host naming no slug on a non-cloud deployment resolves the sole instance,
  the way bootstrap does. A self-hosted install at vantage.acme.com or an IP
  has no slug and could never serve a status page; more than one instance is a
  404 rather than a guess, and an unknown-but-well-formed slug stays a 404.
- `InstanceFromHost` gains an explicit-host variant rather than a second copy
  of the slug rules, and now caches negative lookups: an unknown host cost a
  Mongo query per anonymous request, which is also a timing oracle separating
  "no such instance" from "instance exists, page does not".
- The handler's `@Router` annotation is dropped. openapi.json declares one
  server of `/api`, so it published `/api/public/status/{pageId}` — a path that
  does not exist. The real address is described in prose instead.
2026-08-25 09:04:47 +00:00
mrhid6 72c9492223 docs: status pages 2026-08-25 08:44:38 +00:00
mrhid6 fa67d839cd fix: status page editor error handling, maintenance validation and UTC display 2026-08-25 08:40:05 +00:00
mrhid6 1452928b75 feat: status page authoring UI 2026-08-25 08:32:43 +00:00
mrhid6 bf10023f35 fix: empty slices rather than null on the unavailable status snapshot 2026-08-24 19:42:16 +00:00
mrhid6 f4f41e400b feat: public status page 2026-08-24 14:53:42 +00:00
mrhid6 3abbdc41d6 feat: status page authoring API
Adds owner|admin routes under /api/status-pages for authoring status pages
and their incidents/maintenance windows, gated by the status_pages licence
feature. Adds the "status" token scope resource and the ten route-scope
entries, and regenerates the committed OpenAPI document.

Also types ErrPageInvalid as a sentinel for status page/incident validation
failures (previously bare errors), so statusPageError maps them to 400
instead of 500, and createStatusIncident/updateStatusIncident route through
the shared error mapper rather than hand-rolling a 400 for any service error.
2026-08-24 14:43:17 +00:00
mrhid6 6ba54f690c fix: default TRUSTED_PROXIES in shipped deployments and route /public/ through ingress 2026-08-24 14:35:29 +00:00
mrhid6 a3c6b2a305 feat: public status page endpoint with per-address rate limit 2026-08-24 14:29:24 +00:00
mrhid6 21a2d077d8 feat: cached public status snapshot with licence gate 2026-08-24 14:24:07 +00:00
mrhid6 6263c7e16f fix: clear resolved_at when reopening a status incident via appended update 2026-08-24 14:21:26 +00:00
mrhid6 161835802d feat: authored status incidents and maintenance windows 2026-08-24 14:18:06 +00:00
mrhid6 6f998ff506 feat: status page CRUD and cache invalidation 2026-08-24 14:15:25 +00:00
mrhid6 d192589790 fix: maintenance repaint no longer zeroes no_data uptime; strengthen redaction test 2026-08-24 14:12:38 +00:00
mrhid6 21c2bb2646 feat: public status snapshot assembly and redaction boundary 2026-08-24 14:07:10 +00:00
mrhid6 9c0bbd13dd feat: status page id validation and cache key 2026-08-24 14:02:46 +00:00
mrhid6 1c15961309 feat: status page schema, licence feature and indexes 2026-08-24 14:00:00 +00:00
mrhid6 383b763a66 docs: attach approved status page mockup to the implementation plan 2026-08-24 13:52:07 +00:00
mrhid6 3e99a9df33 docs: public status pages implementation plan 2026-08-24 13:42:44 +00:00
mrhid6 f1b6f90345 docs: public status pages design spec 2026-08-24 13:32:24 +00:00
mrhid6 2a660697c5 docs: Updated troubleshotting doc
Chart Release / chart (push) Successful in 12s
Server Deploy / deploy (push) Successful in 1m4s
2026-08-24 12:50:44 +00:00
mrhid6 0c08dda635 feat: Useragent
Chart Release / chart (push) Successful in 14s
Server Deploy / deploy (push) Successful in 6m5s
2026-08-24 12:17:46 +00:00
mrhid6 22b99ff895 feat: Monitor grath zoom
Chart Release / chart (push) Successful in 13s
Server Deploy / deploy (push) Successful in 6m31s
2026-08-24 10:58:50 +00:00
mrhid6 2fab784ba7 feat: Monitor groups and chart information
Chart Release / chart (push) Successful in 15s
Server Deploy / deploy (push) Successful in 6m35s
2026-08-24 10:30:23 +00:00
mrhid6 83cdf92575 feat: Updated edit monitor page
Chart Release / chart (push) Successful in 14s
Server Deploy / deploy (push) Successful in 43s
2026-08-24 09:25:43 +00:00
mrhid6 aa1c8e4aa1 feat: Hide secrets on api and channels
Chart Release / chart (push) Successful in 15s
Server Deploy / deploy (push) Successful in 8m5s
2026-08-14 12:23:36 +00:00
mrhid6 ac61015cc0 fix: Fixed incorrect openapi doc
Chart Release / chart (push) Canceled after 0s
Server Deploy / deploy (push) Successful in 9m51s
2026-08-13 13:04:44 +00:00
mrhid6 a0fbf5b9ba fix: Word and colour Windows workloads correctly across the UI
Server Deploy / deploy (push) Canceled after 0s
Chart Release / chart (push) Canceled after 0s
Agent Release / build (push) Successful in 14m1s
Agent Release / msi (push) Successful in 1m22s
2026-08-13 12:57:37 +00:00
mrhid6 ddf0814803 docs: Fix Windows package-inventory and poll-loop claims in CLAUDE.md 2026-08-13 12:28:51 +00:00
138 changed files with 20943 additions and 1185 deletions
+18
View File
@@ -76,6 +76,13 @@ jobs:
fi
echo "ok: reaper configured in cloud mode only"
- name: Render with backups enabled
run: |
helm template test "$CHART_DIR" \
--set backup.enabled=true \
--set backup.image=gitea.hostxtra.co.uk/mrhid6/vantage/vantagectl:latest \
--set backup.pvcName=vantage-backups > /dev/null
- name: Render against external Redis and MongoDB
run: |
helm template test "$CHART_DIR" \
@@ -140,10 +147,21 @@ jobs:
--set ingress.enabled=true \
--set ingress.web.host=vantage.example.com \
--set server.env.grpcHost=agents.example.com:443
refuses "an ingress that leaves /api unrouted" \
--set ingress.enabled=true \
--set ingress.web.host=vantage.example.com \
--set ingress.grpc.enabled=false \
--set ingress.api.enabled=false
refuses "gRPC ingress while grpcHost is still in-cluster" \
--set ingress.enabled=true \
--set ingress.web.host=vantage.example.com \
--set ingress.grpc.host=agents.example.com
refuses "backup enabled with no pvcName" \
--set backup.enabled=true \
--set backup.image=gitea.hostxtra.co.uk/mrhid6/vantage/vantagectl:latest
refuses "backup enabled with no image" \
--set backup.enabled=true \
--set backup.pvcName=vantage-backups
- name: Read the chart version
id: chart
+10 -3
View File
@@ -71,9 +71,16 @@ jobs:
fi
}
# The three Go images build from the repo root and COPY
# shared/ plus their own directory, so shared/ rebuilds all
# three. proto/ is in server's list as insurance: the
# The three Go images here build from the repo root and
# COPY shared/ plus their own directory, so shared/ rebuilds
# all three. vantagectl also depends on shared/ but is NOT
# built here: it is a released tool, so its image is built and
# version-tagged by vantagectl-release.yml on a vantagectl/v*
# tag. A shared/ change therefore reaches it at the next
# release rather than on the next push to main, which is the
# point — an operator restoring a database should be running a
# version they can name, not whatever main built last night.
# proto/ is in server's list as insurance: the
# generated pb is committed under server/, but a proto change
# that someone regenerates in the same push should not depend
# on that ordering.
+111
View File
@@ -0,0 +1,111 @@
name: vantagectl Release
on:
push:
tags:
- "vantagectl/v*"
jobs:
build:
runs-on: ubuntu-docker
container: node:26
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: "1.26"
cache: true
cache-dependency-path: vantagectl/go.sum
- name: Extract version
id: version
run: echo "VERSION=${GITHUB_REF_NAME#vantagectl/}" >> $GITHUB_OUTPUT
- name: Test
working-directory: vantagectl
run: go test ./...
- name: Build
working-directory: vantagectl
env:
VERSION: ${{ steps.version.outputs.VERSION }}
run: |
mkdir -p dist
for target in linux/amd64 linux/arm64 darwin/arm64 windows/amd64; do
goos="${target%/*}"
goarch="${target#*/}"
out="dist/vantagectl-${goos}-${goarch}"
if [ "$goos" = "windows" ]; then out="${out}.exe"; fi
CGO_ENABLED=0 GOOS="$goos" GOARCH="$goarch" go build \
-ldflags="-s -w -X main.Version=${VERSION}" \
-o "$out" .
done
- name: Checksums
working-directory: vantagectl/dist
run: sha256sum vantagectl-* > checksums.txt
- name: Create release
uses: https://gitea.com/actions/gitea-release-action@v1
with:
token: ${{ secrets.RELEASE_TOKEN }}
files: |
vantagectl/dist/vantagectl-linux-amd64
vantagectl/dist/vantagectl-linux-arm64
vantagectl/dist/vantagectl-darwin-arm64
vantagectl/dist/vantagectl-windows-amd64.exe
vantagectl/dist/checksums.txt
# The image is built here rather than in server-deploy.yml on every push to
# main, because vantagectl is a released tool rather than a running service.
# An operator restoring a database should be able to name the version they
# ran; ":latest, rebuilt whenever main moved" cannot be named after the
# fact. It is a separate job from the binaries because it needs a
# docker-capable runner rather than a Go one, and it does not need the
# binaries — the image builds from source in its own stage.
image:
runs-on: ubuntu-docker
container: docker:dind
steps:
- name: Setup
run: apk add --update nodejs npm git
- name: Checkout
uses: actions/checkout@v4
- name: Extract version
id: version
run: |
# v0.1.0 for the binary stamp, 0.1.0 for the image tag: a
# leading v is conventional on a git tag and unconventional on
# a container tag.
VERSION="${GITHUB_REF_NAME#vantagectl/}"
echo "VERSION=${VERSION}" >> $GITHUB_OUTPUT
echo "IMAGE_TAG=${VERSION#v}" >> $GITHUB_OUTPUT
- name: Log in to registry
run: |
echo "${{ secrets.RELEASE_TOKEN }}" | \
docker login ${{ vars.DOCKER_HOST }} \
-u "${{ secrets.REGISTRY_USER }}" --password-stdin
- name: Build and push image
env:
VERSION: ${{ steps.version.outputs.VERSION }}
IMAGE_TAG: ${{ steps.version.outputs.IMAGE_TAG }}
run: |
REPO="${{ vars.DOCKER_HOST }}/${{ github.repository_owner }}/vantage/vantagectl"
# Root context: vantagectl depends on the shared module through
# a replace directive, so the build needs shared/ alongside it.
# VERSION is passed through so `vantagectl --version` inside the
# image reports the tag it was built from rather than "dev".
docker build \
--build-arg VERSION="${VERSION}" \
-t "${REPO}:${IMAGE_TAG}" \
-t "${REPO}:latest" \
-f vantagectl/Dockerfile .
docker push "${REPO}:${IMAGE_TAG}"
docker push "${REPO}:latest"
+263 -16
View File
@@ -91,7 +91,7 @@ vantage/
│ └── models/ # accounts, instances, licences, plans
├── adminsite/ # staff + customer console (vantage-hq)
│ ├── app/(customer)/ # overview, instance, link, billing
│ ├── app/(staff)/staff/ # operations, accounts, licences, plans, audit
│ ├── app/(staff)/staff/ # operations, accounts, licences, pricing, audit
│ ├── components/ # AppBar, PageHeader, PageFrame, InstanceRecord
│ └── lib/ # api client, session guards, formatters
├── docsite/ # user documentation (Docusaurus, static)
@@ -383,9 +383,11 @@ one wire shape, worded per platform in the UI, which is the only layer that
knows the host's OS. The platform split lives entirely in the agent, as build
tags (`systemd_linux.go` / `services_windows.go` and the matching `control_`
and `logs_` pairs); the control plane is OS-blind and needed no changes.
Windows collection runs PowerShell through `agent/internal/winexec`, and every
script emits JSON that a build-tag-free parser reads, so the parsers are tested
on Linux — the agent module has no Windows CI.
Windows collection runs PowerShell through `agent/internal/winexec`. Every
script that reports data emits JSON that a build-tag-free parser reads, so
those parsers are tested on Linux — the agent module has no Windows CI. The
control verbs and `serviceDisplayName` emit no JSON and have no parser; they
are exercised only by running the agent on Windows.
**Not gated by licence**: this reads as core fleet management, so v1 ships
everywhere with no `HasFeature` check. If that changes the check belongs at
@@ -454,10 +456,213 @@ there are two copies — `agent/internal/grpc/pb` and `server/internal/grpc/pb`.
A message added to one must be added to the other and to the `.proto`, in the
same commit.
### Status pages
Two collections: `status_pages` is the page itself — title, banner, published
flag, and an ordered list of sections each holding entries that pair a
`monitor_id` with a per-page display name. `status_incidents` holds both
operator-authored incidents and maintenance windows, sharing one document
shape because they share a timeline, an impact and a set of affected
components; each carries an explicit `page_ids` rather than deriving it from
`affected_monitors`, because adding a monitor to a page later must not
retroactively republish that monitor's old incidents to a new audience.
**`services.assembleSnapshot` is the redaction boundary, and it is the only
one.** It takes a `snapshotInput` built from already-fetched
`models.Monitor`/`models.Rollup`/`models.Incident` documents and returns a
`StatusSnapshot` built entirely from a parallel, deliberately smaller
vocabulary (`PublicComponent`, `PublicIncident`, …) that has no field for a
target URL, host, port, expected status, keyword, failure message,
certificate expiry, latency, runner or notification channel — `models.Monitor`
itself never reaches an anonymous caller, only the handful of fields
`assembleSnapshot` chooses to copy out of it. Being a pure function of already-
fetched data (no DB calls inside it) is what makes the boundary testable
without a database, which is the only thing standing between an editor adding
a field to `PublicComponent` and that field being a hostname.
**An incident may only name components the page already carries.**
`services.checkAffectedOnPages` refuses an `affected_monitors` entry that no
page in the incident's `page_ids` lists, and the editor offers only the saved
page's components — labelled by their per-page display name, since that is the
name the reader sees. Naming an arbitrary monitor would publish a machine the
page deliberately does not, which is the same leak `assembleSnapshot`'s
redaction boundary exists to prevent, reached from the authoring side instead
of the read side. It is a separate pass rather than part of `validateIncident`
because it reads the database and `validateIncident` is a pure function of the
document. A component dropped from the page **after** an incident named it
makes the next edit of that incident fail, deliberately: the editor renders the
stale entry flagged and checked so it is one click from being dropped, and the
alternative is a page quietly publishing a component it no longer has.
Monitor-detected outages are **derived at read time, never copied**: each
snapshot assembly reads recent `incidents` for the page's monitors and folds
them into the timeline alongside the authored ones. There is no second
incidents table for automatic ones and no reconciliation between two records
of the same outage. A maintenance window in progress **repaints how a day is
drawn, never the uptime number** — `buildDays` computes each day's up/down
state and the 90-day percentage from rollups first, and
`applyMaintenanceRepaint` only overwrites today's display state afterward, so
a component that stayed up throughout a maintenance window still shows as up
in its history.
The public route, `GET /public/status/:pageId`, is mounted on the gin **root**,
outside `/api`, on purpose: `/api` carries `auth.Middleware`, `RequireScopes`,
`RateLimitTokens` and `RequireActiveLicense` by virtue of where it is mounted,
and a public route living there would need four exemptions — each one a hole a
later change to any of those four could widen back open. A missing page, an
unpublished page, and a page on the wrong host all answer the same 404;
inventing a distinct code for "exists but unpublished" would itself leak that
the page exists. A lapsed licence or a tier lacking `status_pages` answers 200
with `available:false` and a `reason`, never a 403 or a blank page — the
reader is a member of the public who can do nothing about either condition and
deserves an explanation, not a browser error.
**The instance is resolved from `X-Forwarded-Host`, not `Host`.** The public
page is server-rendered by `web/`, and the SSR fetch cannot set `Host` at all:
it is a forbidden header name and undici drops it silently, so the Go server
saw `server:8080` and every status page 404'd on every deployment. `web/`
forwards the visitor's host in `X-Forwarded-Host` (and their address in
`X-Forwarded-For`, or the whole deployment shares one rate-limit bucket), and
`publicStatusInstance` honours that header **only when `c.RemoteIP()` is in
`TRUSTED_PROXIES`** — it selects a tenant, so an untrusted peer must not be
able to name one. It uses `RemoteIP()` and not `ClientIP()` deliberately: the
latter is reconstructed from the very headers being judged.
**A host naming no slug falls back to the sole instance on a non-cloud
deployment.** `hostSlug` requires `<slug>.vantage.<tld>`; a self-hosted install
at `vantage.acme.com` or an IP has no slug and would otherwise 404 forever. It
has exactly one instance, resolved with the same count-then-read bootstrap
uses, cached alongside the slug lookups. More than one instance is a 404, not a
guess. A host that *does* name a slug which does not exist stays a 404 —
falling back there would serve one tenant's page on another's address.
Assembled snapshots are cached in Redis for **30 seconds**, keyed per
instance and page, and every authoring write (`UpdateStatusPage`,
`DeleteStatusPage`, and every incident mutation) invalidates its page's entry
immediately rather than waiting out the TTL — an operator posting an update
mid-incident should not wonder for half a minute whether it saved. A cache
miss, on Redis being down or on any read error, degrades to reassembly rather
than an error: the status page has to survive the outage it exists to report.
The public endpoint itself is rate limited to **120 requests per minute per
client address**, answering 429 with `Retry-After`, on the same fixed-window
pattern as `RateLimitTokens`.
**`TRUSTED_PROXIES` is load-bearing for that limiter, not cosmetic.** `main.go`
always calls `gin.SetTrustedProxies` with it; left unset, gin trusts no proxy
and `c.ClientIP()` falls back to the direct peer address — which, sat behind a
real reverse proxy, is the proxy's own address for every visitor. The rate
limiter then keys on one address for the whole fleet of readers, and the first
burst of legitimate traffic during an incident is what trips it. Set it to the
proxy's real address or CIDR, not merely a private range guess; the shipped
compose file and Helm chart default it to the RFC1918 ranges, which is right
for their own bundled reverse proxy but wrong the moment another one is
inserted in front. The same setting also decides the address recorded in
`audit_logs` and `console_sessions`.
### Agent self-update
`UpdateAgentCmd` carries a target version and Gitea base URL; the agent downloads and replaces itself.
### Backup and restore
`vantagectl` is a standalone Go module (`vantagectl/`), not a subcommand of
`server`. It needs its own module rather than living inside `server`'s for the
same reason `admin` and `sitesvc` already do: `server` imports the rest of
`server`'s dependency graph, and `spf13/cobra` has no business in a process
that also terminates gRPC streams and serves the REST API. More to the point,
`vantagectl` has to run when the control plane **does not** — a backup or
restore against a database with no server container alive at all — so it
cannot be a mode of the binary whose crash is the reason you need it.
The actual logic lives in `shared/backup` (dump, restore, verify, manifest,
fingerprint), not in `vantagectl/internal/cmd`, which holds only argument
parsing and operator-facing output. That split is what lets `server` import
`shared/backup` later — a scheduled in-process backup, say — without a second
implementation to keep in sync. `shared/cryptobox` is the same move one layer
down: it is now the **single** AES-256-GCM implementation, and
`server/internal/services/crypto.go` delegates to it rather than keeping its
own copy that `shared/backup` would otherwise have had to duplicate to decrypt
a probe value during `verify`.
**The archive stores a SHA-256 fingerprint of `KEY_ENCRYPTION_KEY`, never the
key.** `backup` refuses to run without the key set in the environment unless
`--allow-no-key` is passed, because an archive with no fingerprint at all
cannot later tell a restore that the wrong key is in hand — it can only find
that out when the data comes back as noise. The fingerprint is what turns that
failure into a refusal at `restore` time instead.
**Collections are enumerated live**`shared/backup` lists what the database
actually holds rather than reading `services.ScopedCollections`, the opposite
choice from the one instance-deletion purge makes. Purge must never miss a
tenant-scoped collection, so it keeps one hand-maintained registry; a backup
must never miss **any** collection, tenant-scoped or not (`migrations`,
`vulndb_meta`), so a static list is the wrong shape twice over — once for the
collections it would still owe `instance_id` deletion but not a backup, and
once for the two singleton collections that carry neither `instance_id` nor a
release note.
**Restore refuses a non-empty target database and has no merge semantics.**
There is no code path that upserts an archive's documents over existing ones:
merging two control planes' data reconciles nothing about which SSH keys are
still valid or which users still exist, and an upsert would resurrect a
revoked key or a deleted member from the older side. `--force` drops each
collection in the archive first, and is gated behind a second assurance:
`--confirm-db NAME` matching the target exactly, which works everywhere, or —
on a terminal only, and only when `--confirm-db` was not given — the target
database's name typed back at a prompt. `--confirm-db` is accepted on a
terminal too: it is the stronger of the two, because naming the target in the
command itself means a copied command carries its intended target with it and
cannot destroy a different one by accident. Without a terminal and without
`--confirm-db`, `--force` is refused.
**`--force` drops only what the archive names.** Collections already in the
target that the archive does not carry are left untouched and **named in a
warning** — an archive taken with `--exclude workflow_log_lines` restored over
a live database leaves the old lines joined to restored runs, which the
operator must be told. Dropping them instead would delete data nobody asked to
delete, and there is no way back from that.
**Index specifications are replayed verbatim, never reconstructed.**
`dumpIndexes` stores each spec as extended JSON over the raw BSON the server
reported, and `replayIndexes` hands it back to `createIndexes` through
`RunCommand` with only `v` and `ns` stripped and `_id_` skipped. Rebuilding a
`mongo.IndexModel` from a hand-picked set of options dropped
`partialFilterExpression` — which this codebase relies on in
`services/workflows.go` and `services/settings.go` — so a partial unique index
came back as a full one, failed on duplicate keys, and aborted the restore
mid-write. Reconstructing the key document from JSON also lost compound key
order, which is significant.
**`backup.ciphertextFields` mirrors `server/internal/models` by hand.**
`shared/` is a separate module and `models` is under `server/internal`, so
`shared/backup` cannot import it; the map naming each collection's `*_enc`
fields (`keys`, `secrets`, `auth_providers`, `console_sessions`) must change in
the same commit as any of those bson tags, the same hazard as
`web/lib/targets.ts` and `services.MaxWorkloadLogLines`. Wrong field names are
silent: `verify`'s live probe simply finds no ciphertext and reports "this
database stores no ciphertext yet", so the one gate that catches what a
fingerprint cannot no-ops. `settings` is deliberately in neither that map nor
`CiphertextCollections()` — its ESO read token is a SHA-256 hash, not
ciphertext.
**A file-backed `backup` writes to `<name>.tar.gz.partial` and renames on
success**, the same discipline the agent uses for `authorized_keys`. A failed
dump must not leave a partial file named exactly like a good archive; `--out -`
is untouched, since a broken pipe has no file to mislead anyone.
**`vantagectl/Dockerfile`'s runtime stage is `scratch`, and needs the same
explicit `/tmp` as `server/Dockerfile`.** `restore` extracts an archive to a
temporary directory before verifying its checksums, and a scratch image has no
`/tmp` for `os.MkdirTemp` to find — the same failure mode `vulnsched` hits on
`server`, but here it would break every restore rather than only vulnerability
scanning.
**`shared/` reaches four Go images, but only three of them from
`server-deploy.yml`** (`server`, `sitesvc`, `admin`). The `vantagectl` image is
built by `vantagectl-release.yml` on a `vantagectl/v*` tag instead, so a
`shared/` change reaches it at the next release rather than the next push to
main — see the CI section below.
### API tokens and OpenAPI
A token is `vt_` plus 32 random bytes hex, shown once at creation and stored
@@ -465,9 +670,9 @@ only as sha256 — the same shape as `servers.agent_token_hash` and the ESO read
token, and for the same reason: nothing downstream ever needs the plaintext
back. It belongs to the user who created it, and its role can never exceed
theirs; see the `api_tokens` note under MongoDB Collections for how that stays
true across a demotion rather than only at issuance. Scopes are eight
true across a demotion rather than only at issuance. Scopes are nine
resources — `servers`, `keys`, `secrets`, `workflows`, `monitors`, `vulns`,
`workloads`, `settings` — each split into `:read` and `:write`, with `:write`
`workloads`, `status`, `settings` — each split into `:read` and `:write`, with `:write`
satisfying a `:read` requirement on the same resource so a caller does not have
to hold both. Any signed-in member may mint and revoke their **own** tokens —
there is no `RequireRole` on `POST /tokens` or `DELETE /tokens/:id` — because
@@ -746,6 +951,10 @@ workloads GET /workloads · GET /servers/:id/workloads
POST /servers/:id/workloads/refresh
POST /servers/:id/workloads/:wid/action (owner|admin)
GET /servers/:id/workloads/:wid/logs (owner|admin)
status-pages GET,POST /status-pages · GET,PUT,DELETE /status-pages/:pageId (owner|admin)
GET,POST /status-pages/:pageId/incidents
PUT,DELETE /status-pages/:pageId/incidents/:incidentId
POST /status-pages/:pageId/incidents/:incidentId/updates
audit GET /audit
agent GET /agent/latest-version
settings GET,PUT /settings · POST /settings/secrets-token (owner|admin)
@@ -829,7 +1038,7 @@ Paddle is merchant of record; `admin/internal/paddle` is a thin REST client (no
## MongoDB Collections
`servers` · `keys` · `assignments` · `orgs` · `users` · `auth_providers` · `settings` · `secrets` · `workflows` · `workflow_steps` · `workflow_runs` · `workflow_log_lines` · `workflow_log_seq` · `monitors` · `incidents` · `monitor_rollups` · `notification_channels` · `console_sessions` · `audit_logs` · `server_packages` · `vuln_findings` · `vuln_alert_rules` · `vulndb_meta` · `server_workloads` · `api_tokens` · `migrations`
`servers` · `keys` · `assignments` · `orgs` · `users` · `auth_providers` · `settings` · `secrets` · `workflows` · `workflow_steps` · `workflow_runs` · `workflow_log_lines` · `workflow_log_seq` · `monitors` · `incidents` · `monitor_rollups` · `notification_channels` · `console_sessions` · `audit_logs` · `server_packages` · `vuln_findings` · `vuln_alert_rules` · `vulndb_meta` · `server_workloads` · `api_tokens` · `status_pages` · `status_incidents` · `migrations`
Every document except `migrations` carries `org_id`. Struct definitions are the source of truth — see `server/internal/models/`.
@@ -852,7 +1061,9 @@ Notes that are not obvious from the structs:
Admin's own database is separate and holds `accounts` · `admin_instances` · `licenses` · `subscriptions` · `plans` · `catalogue` · `entitlements` · `paddle_events` · `staff_users` · `customer_users` · `instance_members` · `admin_audit`. `paddle_events` is the webhook idempotency log, unique on `event_id`: an event is claimed there before processing, and a duplicate of a handled event is a 200 no-op. `instance_members` is unique on `(instance_id, customer_user_id)` — one person holds at most one user in one instance, which makes a grant idempotent-by-refusal rather than silently doubling a projection. It is an _index_ of the control-plane rows, not the authority (see "Grants project, they do not federate"). Admin has no migrations collection; `models.Backfill` runs on every boot and is idempotent by filtering on the absence of what it writes.
`plans` is keyed on `(deployment, tier)` — six rows, two deployments times three tiers — and holds base allowances only. **Every Paddle price ID lives in `catalogue`**, one row per priceable component (`base`, `limit`, `feature`), because a metered plan is priced by several prices and one map on a plan row cannot express that. `entitlements` holds one row per instance with `desired` beside `granted`: the checkout is built from `desired`, a licence is only ever signed from `granted`, and an abandoned checkout therefore leaves a `desired` that reached nothing. The two Free plans have **no catalogue rows at all**, which is what keeps Free outside Paddle.
`plans` is keyed on `(deployment, tier)` — six rows, two deployments times three tiers — and holds base allowances only. **Every Paddle price ID lives in `catalogue`**, one row per priceable component (`base`, `limit`, `feature`), because a metered plan is priced by several prices and one map on a plan row cannot express that. A row carries a `scope`: `plan` rows name a `deployment` and `tier` and belong to that plan alone, `shared` rows leave both empty and are sold by every paid plan. **How many rows a component needs follows from how many Paddle products it is** — the base fee is a different product per plan, every add-on is one product at one price, so the catalogue is four base rows plus five shared rows, nine instead of twenty-four, and an add-on's price ID is typed once rather than four times. `models.CatalogueFor` is the seam: it returns a plan's base row plus every shared row, and **nothing may filter the catalogue by `deployment` and `tier` itself** or it sees a plan priced by its base fee alone. `adminsite/lib/catalogue.ts`'s `rowsForPlan` is the TypeScript half of that and must change in the same commit, the same shape of hazard as `web/lib/targets.ts`. `models.MigrateSharedCatalogue` runs at boot after `SeedCatalogue`, merges the old per-plan copies onto the shared row and deletes them; it **refuses rather than guesses** when the four copies disagree, because four rows meant to be one price and are not is a pricing decision somebody made and picking one silently moves a customer's bill. `entitlements` holds one row per instance with `desired` beside `granted`: the checkout is built from `desired`, a licence is only ever signed from `granted`, and an abandoned checkout therefore leaves a `desired` that reached nothing. The two Free plans have **no catalogue rows at all**, which is what keeps Free outside Paddle.
**No tier bundles a feature.** `console`, `oidc`, `vuln_scanning` and `status_pages` are each a per-customer priceable add-on: every plan row carries an empty `base_features`, and the grant comes from a `catalogue` row the customer buys. Adding a fifth feature therefore means one more shared `KindFeature` row in `SeedCatalogue`'s `seedRows` and one entry in `adminsite/lib/features.ts` — that map is what the customer's grant list, the staff configurator and the purchase form all enumerate, so a feature missing from it exists in the licence and is invisible in the portal. `SeedCatalogue` upserts on the row's natural key `(kind, deployment, tier, limit_key, feature_key)` — a shared row's empty deployment and tier are part of that key, not a wildcard — so a new row reaches an existing database on the next admin boot with no migration; `SeedPlans` is `$setOnInsert` on the whole document and would not, which is the other reason bundling into a tier is the harder path.
### Migrations
@@ -895,7 +1106,7 @@ tls: true
```
1. SyncKeys(server_id, agent_token, agent_version)
2. Non-Linux hosts stop here — Windows agents register and heartbeat only
2. Non-Linux hosts stop here — the key-management steps below are Linux-only; a Windows agent's other work (workflow steps, inventory, OS updates, workloads) runs from the goroutines started above, not from this loop
3. Diff desired keys against /root/.ssh/authorized_keys; unchanged → no write
4. Changed → write .tmp, os.Rename() over the real file, chmod 0600
```
@@ -938,6 +1149,7 @@ Windows: MSI built by CI (WiX), or `installer/setup.ps1` registering the agent a
| `PROXY_ADVERTISE_HOST` | no | default `server`; the hostname guacd resolves the control plane by, handed to guacd as the relay's address. Wrong here and every console session fails at connect |
| `PROXY_LISTEN_HOST` | no | default `0.0.0.0`; the interface the ephemeral relay listener binds |
| `APP_ROOT_LABEL` | no | default `vantage`; wrong value disables the host/session org guard |
| `TRUSTED_PROXIES` | no | comma-separated CIDRs or addresses gin trusts for `X-Forwarded-For`. Empty means trust none: `c.ClientIP()` falls back to the direct peer address, which behind a real reverse proxy is that proxy's own address for every visitor — the public status page's per-address rate limit then keys on one address for the whole fleet of readers. Also the address recorded in `audit_logs` and `console_sessions`. Compose and the Helm chart default it to the RFC1918 ranges, right for their own bundled proxy and wrong the moment another one is inserted in front |
| `POD_IP` | no | this pod's own address, set by the Helm chart from the downward API. **Takes precedence over `PROXY_ADVERTISE_HOST`** — a console relay listener belongs to one replica, and a Service address names all of them |
| `VANTAGE_MIGRATE_ONLY` | no | run schema setup (migrations, index builders, default-step seeding) and exit without serving. `GRPC_HOST` is not required in this mode. Set by the Helm chart's pre-upgrade Job |
| `VANTAGE_SKIP_MIGRATIONS` | no | serve without running schema setup, on the assumption a Job already did. Set by the chart's Deployment whenever `server.migrationJob.enabled`. Unset under Compose, where one process still migrates and then serves |
@@ -969,7 +1181,7 @@ Windows: MSI built by CI (WiX), or `installer/setup.ps1` registering the agent a
**`ingress.web.host` is normally a wildcard.** `*.vantage.example.com` is the per-tenant instance namespace — `APP_ROOT_LABEL` resolves the instance from the label. A Kubernetes wildcard host matches **exactly one** label, so it does not match the apex, and here that is correct rather than a gap: `vantage.hostxtra.co.uk` is the marketing site (`site/`, in `docker-compose.site.yml`), which this chart does not deploy. `extraHosts` is for a genuine second name; adding the apex to it would put the control plane on the marketing host. Every host in the list gets identical paths.
**`ingress.api.enabled` routes `/api` and `/auth` straight to the server.** Both arrangements work — without it `web` proxies those prefixes onward itself (`web/next.config.ts`) — but edge routing is one hop shorter and matches what the Nginx Proxy Manager in front of the Docker deployment already does, so leaving it off makes the request path a different shape on Kubernetes than in production. It stays **off by default** because it only helps where the server is reachable on the same host and certificate as `web`; turning it on blindly moves the whole API onto a route that may not be provisioned. Traefik derives router priority from rule length, so `PathPrefix(/api)` outranks the catch-all `/` with no priority annotation needed.
**`ingress.api.enabled` routes `/api`, `/auth`, `/public`, `/install*` and `/update*` straight to the server, and it is not optional.** It defaults to **true** and the chart refuses to render with it off, because `web` proxies nothing: with those prefixes unrouted the UI loads and every request it makes 404s against Next. The value survives only for an installation whose own terminator sits in front of this ingress and routes them there instead. Traefik derives router priority from rule length, so `PathPrefix(/api)` outranks the catch-all `/` with no priority annotation needed.
**The gRPC route needs its own Service.** The server terminates no TLS; it speaks plain h2c and always has, with TLS terminated by whatever sits in front. Traefik will not use h2c to a backend unless the *Service* says so, and that annotation applies to every port on the Service — so annotating the shared two-port `<release>-server` would force h2c on its HTTP port too.
@@ -979,6 +1191,8 @@ TLS is `ingress.tls.secretName` / `grpcSecretName` (pre-existing certificates) *
---
**Neither compose file ships a reverse proxy, and both now need one.** `web:3000` serves the UI only; a request to `/api` there is a Next 404. Route `/api`, `/auth`, `/public`, `/install`, `/install.ps1`, `/update`, `/update.ps1` to `server:8080` and everything else to `web:3000` — on vantage.hostxtra.co.uk that is the Nginx Proxy Manager already in front, and it is what a self-hosted install has to configure before the UI works at all.
`deploy/docker-compose.yml` runs four services: `redis`, `guacd`, `server` (8080 + 9090), `web` (3000). MongoDB is external. `deploy/docker-compose.site.yml` adds five more — `site` (3003), `sitesvc` (8082), `admin` (8083), `adminsite` (3004) and `docsite` (3005) — and is only used on vantage.hostxtra.co.uk.
`docsite` is the odd one: a **static** build served by `nginx:alpine-slim`, not a Node runtime, and it listens on `80` rather than `3000`. It is reached at **`vantage.hostxtra.co.uk/docs`** — a path on the marketing host, routed by its own Nginx Proxy Manager location, which must sort **above** the catch-all forwarding to `site:3003` or Next answers the 404. A path and not a subdomain because `*.vantage.hostxtra.co.uk` is the per-tenant instance namespace and `APP_ROOT_LABEL` would read a `docs.` label as a tenant slug. NPM forwards the **full** path upstream — it does not strip `/docs` — so `DOCS_BASE_URL`, the proxy location and the directory the image copies the build into (`/usr/share/nginx/html/docs`) must all agree. When they do not, the HTML loads and every asset 404s.
@@ -1028,6 +1242,27 @@ Tailwind in all three maps `var(--…)` references only, so **no component in an
`web/` collapses Tailwind's radius scale — `md`, `lg` and `xl` all resolve to site/'s 4px — rather than rewriting the ~140 `rounded-lg` classes across its pages. Every one of them meant "a panel corner", and `tailwind.config.ts` is now where that decision lives. `rounded-full` is untouched: status dots and pills still need it.
**Plans and the catalogue are one page, `/staff/pricing`.** They were two nav
entries and the split asked staff to hold one half in their head while looking
at the other: a tier's allowance is what the metered component charges above,
and a base fee means nothing without the allowance it includes. The page is
`PlansSection` then `CatalogueSection`, in the order the decision is made —
what a tier grants, then what it costs. `next.config.ts` keeps permanent
redirects from `/staff/plans` and `/staff/catalogue`, which are bookmarked in
staff browsers. **The tier list is cards, not forms**: six plans with five
number fields, a select, a checkbox and four feature toggles each was forty-odd
controls on one screen, and the page could not be read for the thing it exists
to answer. A card states what the tier grants and `Modal` — a native
`<dialog>`, for the focus trap and Escape handling a hand-rolled overlay gets
wrong — is where it is changed. Every feature key renders on every card, lit or
unlit: no tier bundles one today, so the unlit row is the information.
**The catalogue's coverage ledger is not decoration.** A missing production
price is invisible in a grid of text inputs — every cell looks like every other
until twenty-six characters of each are read — and it is the one thing staff
come to the page to check before a launch, so each component draws one filled
or empty square per environment and term.
**The `adminsite/` shell.** `AppBar` is the single masthead — identity, nav, environment, account menu — and it belongs to the two authenticated layouts, never to `app/layout.tsx`, so `/login` and `/accept-invite` do not render navigation they cannot use. Nav active state is derived from `usePathname`; do not hardcode it. `PageHeader` gives every screen the same back link, title, actions and **record line** (the reference number in mono, click-to-copy) — the reference is what people paste into support tickets, so it has a fixed slot rather than a per-page treatment. `PageFrame` is the main-plus-320px-rail split; the rail carries only what is true account-wide, which is why there is no plan card in it — **tier, limits and expiry belong to a licence, and a licence belongs to one instance**, so an account holding a Free cloud instance and a Professional self-hosted one has no single plan.
Customer nav is three destinations — Overview, People, Billing. Settings is in the account menu because it is your password, not a place, and appearance lives there too: `AccountMenu` is the only thing that sets `data-theme`, which the token blocks have always supported in both directions.
@@ -1086,7 +1321,7 @@ GOOS=linux GOARCH=amd64 go build \
### `server-deploy.yml` — triggered on every push to `main`
Builds and pushes seven images to the Gitea container registry: `server`, `web`, `site`, `sitesvc`, `admin`, `adminsite` and `docsite`.
Builds and pushes seven images to the Gitea container registry: `server`, `web`, `site`, `sitesvc`, `admin`, `adminsite` and `docsite`. **`vantagectl` is deliberately not among them** — it is a released tool rather than a running service, and its image is version-tagged by `vantagectl-release.yml`.
Note that despite the name, **this workflow does not deploy** — it only builds and pushes. There is no SSH step. Rolling images out is a separate manual step on the host:
@@ -1104,7 +1339,17 @@ cd /opt/vantage && docker compose -f docker-compose.yml -f docker-compose.site.y
| `sitesvc` | `sitesvc/`, `shared/`, `go.work` |
| `web` · `site` · `adminsite` · `docsite` | their own directory only |
`shared/` fans out to all three Go images because each of their Dockerfiles copies `shared/` from a root context — **if a fourth service ever imports `shared/`, add it to that list or it will ship stale**. A change to the workflow file rebuilds everything, since a build arg is baked into the image. So does anything that leaves no trustworthy base commit: a manual `workflow_dispatch`, a new branch, or a force-push whose old head is gone.
`shared/` fans out to **three** images here (`server`, `sitesvc`, `admin`)
because each of their Dockerfiles copies `shared/` from a root context — **if a
fourth service ever imports `shared/`, add it to that list or it will ship
stale**. `vantagectl` also imports `shared/` and is the exception: it is built
by `vantagectl-release.yml`, so a `shared/` fix reaches it only when someone
cuts a `vantagectl/v*` tag. That is deliberate — an operator restoring a
database should be running a version they can name — but it does mean a
`shared/backup` fix is not live until it is released. A change to the workflow file rebuilds everything, since
a build arg is baked into the image. So does anything that leaves no
trustworthy base commit: a manual `workflow_dispatch`, a new branch, or a
force-push whose old head is gone.
The gap this leaves: **changing a repo variable pushes no commit, so nothing rebuilds.** After editing `ADMIN_API_URL`, `HQ_URL` or `ADMIN_ENV`, run the workflow manually — that is what `workflow_dispatch` is there for. Base images also stop being refreshed on a service nobody touches; a periodic manual run covers that.
@@ -1137,7 +1382,7 @@ git push origin main # server + web deploy
| `REGISTRY_USER` | Secret | Gitea username. Must own `RELEASE_TOKEN`, or basic auth is rejected |
| ~~`REGISTRY_PASSWORD`~~ | — | **Not used.** Named here historically; no workflow reads it. Referencing an unset secret yields an empty password and a `401 Failed to authenticate user` that looks like a token scope problem. Use `RELEASE_TOKEN` |
| `DOCKER_HOST` | Variable | registry host used for image tags |
| `API_URL` | **not** a CI variable | `web` reads it at **runtime**, from the container environment — `next.config.ts` is evaluated when `server.js` boots in standalone mode, and the rewrites it feeds are server-side, never browser-side. Default `http://localhost:8080`; compose sets `http://server:8080`. `NEXT_PUBLIC_API_URL` is still honoured as a fallback for existing deployments. |
| ~~`API_URL`~~ | — | **Gone.** `web` proxies nothing and holds no address for the control plane. `/api`, `/auth`, `/public`, `/install*` and `/update*` must be routed to `server:8080` by the reverse proxy in front of both; everything else goes to `web:3000`. One variable that could name the wrong host was one request path too many — pointed at the marketing site, `/public/status/…` answered a Next 404 indistinguishable from a status page that does not exist. |
| `SITE_API_URL` | Variable | **browser-reachable** sitesvc URL, baked into the `site` image. Required — if empty, both forms report "not connected" and submit nowhere. Must also be in sitesvc's `SITE_ORIGIN`. |
| `SITE_CONTACT_EMAIL` | Variable | optional; address shown when a form is misconfigured |
| `SITE_URL` | Variable | browser URL of the marketing site, baked into `adminsite` so `/login` can point at `/start`. **Signup has no page in `adminsite` at all** — one signup form, on `site/`. Empty renders no link rather than one that 404s. |
@@ -1171,9 +1416,11 @@ git push origin main # server + web deploy
- **Windows agents cover the fleet-management path** — register, heartbeat, run
steps, report inventory, OS updates through the Windows Update COM API, and
workloads (services plus containers, with control and logs). They still do no
`authorized_keys` management, and no package inventory or CVE matching: the
vulnerability feeds this project uses carry no Windows data, so a Windows host
correctly reports `unsupported` rather than a clean bill of health.
`authorized_keys` management, and no package inventory or CVE matching: a
Windows agent never calls `ReportPackages`, so no `server_packages` document
exists for it and it reports no package inventory at all — a different,
earlier state than the `unsupported` a Linux distribution reaches when its
family has no security feed.
- **Both `server` and `web` scale horizontally** — see "Running more than one server replica" below. `web` holds nothing; `server` holds per-agent state that is routed between replicas over Redis rather than duplicated.
- **Deletion lives in the control plane** — admin sends the warnings because it knows the billing address; the control plane performs the delete because it is the only service that knows which collections carry `instance_id`. Mirroring that list into admin would drift, and a drift there deletes the wrong rows.
+4
View File
@@ -86,6 +86,10 @@ func main() {
idxCancel()
log.Fatalf("seed catalogue: %v", err)
}
if err := models.MigrateSharedCatalogue(idxCtx); err != nil {
idxCancel()
log.Fatalf("migrate catalogue: %v", err)
}
if err := models.Backfill(idxCtx); err != nil {
idxCancel()
log.Fatalf("backfill: %v", err)
+21 -1
View File
@@ -485,6 +485,7 @@ func staffListCatalogue(c *gin.Context) {
func staffUpdateCatalogue(c *gin.Context) {
var body struct {
Kind string `json:"kind"`
Scope string `json:"scope"`
Deployment string `json:"deployment"`
Tier string `json:"tier"`
LimitKey string `json:"limit_key"`
@@ -500,11 +501,19 @@ func staffUpdateCatalogue(c *gin.Context) {
// mean a resolved self-hosted monthly price later, which the resolver treats
// as a configuration error — better to refuse it at the point somebody
// pastes it, while they are looking at the screen.
//
// A shared row is sold by both deployments, so both terms are legitimate on
// it: the cloud checkout takes the monthly price and the self-hosted one
// never asks for it. Only a plan row can name a term its own deployment
// does not sell.
for env, byTerm := range body.PriceIDs {
for term, id := range byTerm {
if id == "" {
continue
}
if body.Scope == models.ScopeShared {
continue
}
if !termSold(body.Deployment, term) {
c.JSON(http.StatusBadRequest, gin.H{
"error": fmt.Sprintf("%s does not sell %s (environment %s)",
@@ -514,6 +523,8 @@ func staffUpdateCatalogue(c *gin.Context) {
}
}
// Addressed by its natural key, so the staff UI never holds a Mongo id. A
// shared row's empty deployment and tier are part of that key.
filter := bson.M{
"kind": body.Kind,
"deployment": body.Deployment,
@@ -534,12 +545,21 @@ func staffUpdateCatalogue(c *gin.Context) {
audit.Write(c.Request.Context(), models.AuditEntry{
Actor: auth.Current(c).Email,
Action: "catalogue.updated",
Target: body.Deployment + "/" + body.Tier + "/" + body.Kind,
Target: catalogueTarget(body.Scope, body.Deployment, body.Tier, body.Kind),
Detail: body.LimitKey + body.FeatureKey,
})
c.JSON(http.StatusOK, gin.H{"updated": true})
}
// catalogueTarget names an edited component in the audit log. A shared row has
// no plan to name, so it says so rather than logging "//feature".
func catalogueTarget(scope, deployment, tier, kind string) string {
if scope == models.ScopeShared {
return "shared/" + kind
}
return deployment + "/" + tier + "/" + kind
}
func termSold(deployment, term string) bool {
for _, t := range license.TermsFor(deployment) {
if t == term {
+12 -2
View File
@@ -29,8 +29,18 @@ func LineItems(ctx context.Context, env, term string, plan *models.Plan, cfg mod
if err != nil {
return nil, err
}
if len(rows) == 0 {
return nil, fmt.Errorf("%w: %s/%s is priced by nothing",
// A plan is identified by its base row, and shared add-on rows exist whether
// or not any plan sells them — so "the catalogue returned something" is no
// longer proof this plan is priced. Check for the base row itself.
hasBase := false
for _, r := range rows {
if r.Kind == models.KindBase {
hasBase = true
break
}
}
if !hasBase {
return nil, fmt.Errorf("%w: %s/%s has no base row",
ErrUnpriced, plan.Deployment, plan.Tier)
}
+183 -43
View File
@@ -2,6 +2,8 @@ package models
import (
"context"
"fmt"
"log"
"gitea.hostxtra.co.uk/mrhid6/vantage/admin/internal/db"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/license"
@@ -19,6 +21,23 @@ const (
KindFeature = "feature"
)
// Component scopes.
//
// A component is priced by one Paddle product, and how many catalogue rows it
// needs follows from how many products it is. The base fee is a different
// product per plan, so it is a row per plan. Every add-on — the server limit and
// all four features — is ONE product sold to every paid plan at one price, so it
// is one row, and its price ID is typed once instead of four times.
//
// Scope is stored rather than inferred from Kind so the rule is data. Pricing a
// future add-on per tier is then a scope on a row, not a rewrite of every reader.
const (
// ScopePlan rows carry a deployment and a tier and belong to that plan alone.
ScopePlan = "plan"
// ScopeShared rows leave deployment and tier empty and belong to every paid plan.
ScopeShared = "shared"
)
// LimitKeyServers is the only metered limit today.
//
// A limit_key is a field name in license.Limits, which is what lets a second
@@ -27,19 +46,23 @@ const (
// would be 1 in every row that will ever exist.
const LimitKeyServers = "max_servers"
// CatalogueRow is one priceable component of one plan.
// CatalogueRow is one priceable component.
//
// This is the ONLY place a Paddle price ID appears anywhere in Vantage. An empty
// PriceIDs means the component is free — a feature with no price is a toggle a
// customer may take at no charge, and giving it a price later is a staff edit
// rather than a migration or a deploy.
type CatalogueRow struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
Kind string `bson:"kind" json:"kind"`
Deployment string `bson:"deployment" json:"deployment"`
Tier string `bson:"tier" json:"tier"`
LimitKey string `bson:"limit_key,omitempty" json:"limit_key,omitempty"`
FeatureKey string `bson:"feature_key,omitempty" json:"feature_key,omitempty"`
ID bson.ObjectID `bson:"_id,omitempty" json:"-"`
Kind string `bson:"kind" json:"kind"`
Scope string `bson:"scope" json:"scope"`
// Deployment and Tier are empty on a shared row, and are what a plan row is
// keyed by. Readers must go through CatalogueFor rather than filtering on
// them, or a shared row is invisible to the plan that sells it.
Deployment string `bson:"deployment" json:"deployment"`
Tier string `bson:"tier" json:"tier"`
LimitKey string `bson:"limit_key,omitempty" json:"limit_key,omitempty"`
FeatureKey string `bson:"feature_key,omitempty" json:"feature_key,omitempty"`
// PriceIDs is environment -> term -> Paddle price ID, e.g.
// {"sandbox": {"monthly": "pri_…"}, "production": {"annual": "pri_…"}}.
//
@@ -67,57 +90,174 @@ func (r CatalogueRow) Priced(env string) bool {
return false
}
// SeedCatalogue inserts the twenty rows the four PAID plans need: a base, a
// server limit, and one row per feature key.
// Shared reports whether this row is sold by every paid plan.
func (r CatalogueRow) Shared() bool { return r.Scope == ScopeShared }
// naturalKey is how a row is addressed everywhere: by what it is, never by its
// ObjectID. A shared row's deployment and tier are empty, and that emptiness is
// part of the key rather than a wildcard.
func (r CatalogueRow) naturalKey() bson.M {
return bson.M{
"kind": r.Kind,
"deployment": r.Deployment,
"tier": r.Tier,
"limit_key": r.LimitKey,
"feature_key": r.FeatureKey,
}
}
// seedRows is the catalogue as it should exist: four base rows, one per paid
// plan, plus five shared add-on rows every paid plan sells.
//
// Nine rows, down from twenty-four. The count moves whenever shared/license
// gains a feature, and this comment is how the next person knows the number was
// chosen rather than drifted.
//
// The two Free plans get no rows at all, and that absence is what keeps Free
// outside Paddle: with nothing to price, no checkout can be built for it. Do not
// "fix" this by adding zero-priced Free rows.
func seedRows() []CatalogueRow {
rows := []CatalogueRow{}
paid := []string{license.TierProfessional, license.TierEnterprise}
for _, deployment := range license.Deployments() {
for _, tier := range paid {
rows = append(rows, CatalogueRow{
Kind: KindBase, Scope: ScopePlan, Deployment: deployment, Tier: tier,
})
}
}
rows = append(rows, CatalogueRow{
Kind: KindLimit, Scope: ScopeShared, LimitKey: LimitKeyServers,
})
for _, f := range []string{
license.FeatureConsole,
license.FeatureOIDC,
license.FeatureVulnScanning,
license.FeatureStatusPages,
} {
rows = append(rows, CatalogueRow{
Kind: KindFeature, Scope: ScopeShared, FeatureKey: f,
})
}
return rows
}
// SeedCatalogue inserts the nine rows the four paid plans need.
//
// $setOnInsert only, for the same reason as SeedPlans: the price IDs are pasted
// in by staff and a redeploy must not blank them.
func SeedCatalogue(ctx context.Context) error {
paid := []string{license.TierProfessional, license.TierEnterprise}
for _, deployment := range license.Deployments() {
for _, tier := range paid {
rows := []CatalogueRow{
{Kind: KindBase, Deployment: deployment, Tier: tier},
{Kind: KindLimit, Deployment: deployment, Tier: tier, LimitKey: LimitKeyServers},
{Kind: KindFeature, Deployment: deployment, Tier: tier, FeatureKey: license.FeatureConsole},
{Kind: KindFeature, Deployment: deployment, Tier: tier, FeatureKey: license.FeatureOIDC},
{Kind: KindFeature, Deployment: deployment, Tier: tier, FeatureKey: license.FeatureVulnScanning},
}
for _, r := range rows {
filter := bson.M{
"kind": r.Kind,
"deployment": r.Deployment,
"tier": r.Tier,
"limit_key": r.LimitKey,
"feature_key": r.FeatureKey,
}
if _, err := db.Admin("catalogue").UpdateOne(ctx, filter,
bson.M{"$setOnInsert": bson.M{
"kind": r.Kind,
"deployment": r.Deployment,
"tier": r.Tier,
"limit_key": r.LimitKey,
"feature_key": r.FeatureKey,
"price_ids": map[string]map[string]string{},
}},
options.UpdateOne().SetUpsert(true)); err != nil {
return err
}
}
for _, r := range seedRows() {
set := r.naturalKey()
set["scope"] = r.Scope
set["price_ids"] = map[string]map[string]string{}
if _, err := db.Admin("catalogue").UpdateOne(ctx, r.naturalKey(),
bson.M{"$setOnInsert": set},
options.UpdateOne().SetUpsert(true)); err != nil {
return err
}
}
return nil
}
// CatalogueFor returns every component of one plan.
// MigrateSharedCatalogue collapses the four per-plan copies of each add-on onto
// the one shared row, and deletes the copies.
//
// It runs after SeedCatalogue, which has already created the shared rows empty,
// and is idempotent: once the per-plan copies are gone there is nothing to move.
//
// It REFUSES rather than guesses when the copies disagree. Four rows that were
// meant to be one price and are not is a real pricing decision somebody made,
// and picking one of them silently would move a customer's bill.
func MigrateSharedCatalogue(ctx context.Context) error {
// Rows seeded before scope existed are all per-plan rows. Naming them so
// keeps CatalogueFor's $or honest for the base rows that survive.
if _, err := db.Admin("catalogue").UpdateMany(ctx,
bson.M{"scope": bson.M{"$exists": false}},
bson.M{"$set": bson.M{"scope": ScopePlan}}); err != nil {
return err
}
for _, shared := range seedRows() {
if !shared.Shared() {
continue
}
cur, err := db.Admin("catalogue").Find(ctx, bson.M{
"kind": shared.Kind,
"limit_key": shared.LimitKey,
"feature_key": shared.FeatureKey,
"deployment": bson.M{"$ne": ""},
})
if err != nil {
return err
}
old := []CatalogueRow{}
if err := cur.All(ctx, &old); err != nil {
return err
}
if len(old) == 0 {
continue
}
var target CatalogueRow
if err := db.Admin("catalogue").FindOne(ctx, shared.naturalKey()).Decode(&target); err != nil {
return err
}
merged := target.PriceIDs
if merged == nil {
merged = map[string]map[string]string{}
}
for _, o := range old {
for env, byTerm := range o.PriceIDs {
for term, id := range byTerm {
if id == "" {
continue
}
if merged[env] == nil {
merged[env] = map[string]string{}
}
if have := merged[env][term]; have != "" && have != id {
return fmt.Errorf(
"catalogue: %s%s was priced differently per plan (%s %s: %q and %q); "+
"decide which price is the shared one and delete the others before upgrading",
shared.LimitKey, shared.FeatureKey, env, term, have, id)
}
merged[env][term] = id
}
}
}
if _, err := db.Admin("catalogue").UpdateOne(ctx, shared.naturalKey(),
bson.M{"$set": bson.M{"price_ids": merged}}); err != nil {
return err
}
ids := make([]bson.ObjectID, 0, len(old))
for _, o := range old {
ids = append(ids, o.ID)
}
if _, err := db.Admin("catalogue").DeleteMany(ctx,
bson.M{"_id": bson.M{"$in": ids}}); err != nil {
return err
}
log.Printf("catalogue: merged %d per-plan rows into shared %s%s",
len(old), shared.LimitKey, shared.FeatureKey)
}
return nil
}
// CatalogueFor returns every component one plan sells: its own base row plus
// every shared add-on.
//
// This is the seam the whole shared-row change rests on. Every reader that used
// to filter the catalogue by deployment and tier must come through here instead,
// or it sees a plan priced by nothing but its base fee.
func CatalogueFor(ctx context.Context, deployment, tier string) ([]CatalogueRow, error) {
deployment, tier = license.NormaliseTier(deployment, tier)
cur, err := db.Admin("catalogue").Find(ctx,
bson.M{"deployment": deployment, "tier": tier})
cur, err := db.Admin("catalogue").Find(ctx, bson.M{"$or": []bson.M{
{"scope": ScopeShared},
{"deployment": deployment, "tier": tier},
}})
if err != nil {
return nil, err
}
@@ -4,6 +4,7 @@ import { useEffect, useMemo, useState } from "react";
import { useRouter } from "next/navigation";
import Link from "next/link";
import { useMutation, useQuery } from "@tanstack/react-query";
import { rowsForPlan, sharedRows } from "@/lib/catalogue";
import { ApiError, api, lineItemsFor, type CatalogueRow, type CheckoutOptions, type Deployment, type Plan, type Term, type Tier } from "@/lib/api";
import { initPaddle, previewPrices, type PricePreview } from "@/lib/paddle";
import { featureDesc, featureLabel } from "@/lib/features";
@@ -60,24 +61,24 @@ export function PurchaseForm() {
const options = optionsQ.data;
const accountId = account.data?.account.account_id ?? "";
// Distinct feature keys offered on this deployment, in a stable order.
// Every feature a paid plan can be sold, in a stable order. Features are
// shared rows now, so they no longer differ by deployment — the list is the
// same on both, and reads from one place rather than four.
const featureKeys = useMemo(() => {
if (!options) return [] as string[];
const keys = new Set<string>();
for (const r of options.catalogue) {
if (r.deployment === dep && r.kind === "feature" && r.feature_key) {
keys.add(r.feature_key);
}
for (const r of sharedRows(options.catalogue)) {
if (r.kind === "feature" && r.feature_key) keys.add(r.feature_key);
}
return [...keys];
}, [options, dep]);
}, [options]);
const activePlans = useMemo(() => (options?.plans ?? []).filter((p) => p.deployment === dep && p.active).sort((a, b) => TIER_ORDER.indexOf(a.tier) - TIER_ORDER.indexOf(b.tier)), [options, dep]);
const plan = activePlans.find((p) => p.tier === choice.tier);
const baseServers = plan?.base_limits.max_servers ?? 0;
const unlimited = baseServers === -1;
const rows = useMemo(() => (options?.catalogue ?? []).filter((r) => r.deployment === dep && r.tier === choice.tier), [options, dep, choice.tier]);
const rows = useMemo(() => rowsForPlan(options?.catalogue ?? [], dep, choice.tier), [options, dep, choice.tier]);
// Real line items for the current configuration the same builder the
// checkout uses, so the summary can never disagree with the overlay.
@@ -239,7 +240,7 @@ export function PurchaseForm() {
headline={p.tier === "free" ? "£0" : basePrices[p.tier]}
cycleLabel={cycleShort(dep, choice.term)}
featureKeys={featureKeys}
catalogue={options.catalogue.filter((r) => r.deployment === dep && r.tier === p.tier)}
catalogue={rowsForPlan(options.catalogue, dep, p.tier)}
env={options.env}
term={choice.term}
onSelect={() =>
@@ -252,7 +253,7 @@ export function PurchaseForm() {
features: c.features.filter((k) => {
const st = featureStateFor(
p,
options.catalogue.filter((r) => r.deployment === dep && r.tier === p.tier),
rowsForPlan(options.catalogue, dep, p.tier),
options.env,
c.term,
k,
@@ -658,7 +659,7 @@ function Receipt({
// Label each real line item from the catalogue, and price it from Paddle.
const base = plan?.base_limits.max_servers ?? 0;
const extra = base === -1 ? 0 : Math.max(0, choice.servers - base);
const rows = options.catalogue.filter((r) => r.deployment === dep && r.tier === choice.tier);
const rows = rowsForPlan(options.catalogue, dep, choice.tier);
const idFor = (predicate: (r: CatalogueRow) => boolean) => {
const row = rows.find(predicate);
return row?.price_ids?.[options.env]?.[choice.term] ?? "";
@@ -1,169 +0,0 @@
"use client";
import { useState } from "react";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { PageHeader } from "@/components/PageHeader";
import { PageFrame } from "@/components/PageFrame";
import { Panel } from "@/components/Panel";
import { TBody, TD, TH, THead, TR, Table } from "@/components/Table";
import { api, type CatalogueRow, type Term } from "@/lib/api";
const ENVS = ["sandbox", "production"] as const;
/* Self-hosted sells annual only, so the monthly cell is not rendered for it
* rather than rendered and rejected. The backend refuses one either way; this is
* so nobody types into a field that cannot be saved. */
function termsFor(deployment: string): Term[] {
return deployment === "self_hosted" ? ["annual"] : ["monthly", "annual"];
}
function componentLabel(r: CatalogueRow): string {
if (r.kind === "base") return "Base fee";
if (r.kind === "limit") return `Per ${r.limit_key?.replace("max_", "")}`;
return `Feature: ${r.feature_key}`;
}
function rowKey(r: CatalogueRow): string {
return [r.deployment, r.tier, r.kind, r.limit_key ?? "", r.feature_key ?? ""].join("/");
}
export default function CataloguePage() {
const qc = useQueryClient();
const { data: rows = [], isLoading } = useQuery({
queryKey: ["staff", "catalogue"],
queryFn: api.staff.catalogue,
});
const [drafts, setDrafts] = useState<Record<string, CatalogueRow["price_ids"]>>({});
const save = useMutation({
mutationFn: (r: CatalogueRow) => api.staff.updateCatalogue(r),
onSuccess: () => qc.invalidateQueries({ queryKey: ["staff", "catalogue"] }),
});
const groups = Array.from(new Set(rows.map((r) => `${r.deployment}/${r.tier}`)));
return (
<div className="grid gap-6">
<PageHeader
title="Catalogue"
back={{ href: "/staff", label: "Operations" }}
subtitle="Every priceable component. This is the only place a Paddle price ID lives."
/>
<PageFrame
aside={
<aside className="space-y-3 text-[0.82rem] text-ink-2">
<p>
A component with no price ID is free. A feature with no price is a
toggle a customer may take at no charge; giving it a price here is
all it takes to start charging for it.
</p>
<p>
Free is priced by nothing and has no rows. That absence is what
keeps it outside Paddle.
</p>
<p>
Changing a price affects the next checkout only. It cannot touch an
issued licence.
</p>
</aside>
}
>
{isLoading ? (
<p className="text-[0.85rem] text-ink-3">Loading</p>
) : (
<div className="grid gap-4">
{groups.map((g) => {
const [deployment, tier] = g.split("/");
const terms = termsFor(deployment);
return (
<Panel key={g} title={`${deployment === "cloud" ? "Cloud" : "Self-Hosted"} ${tier}`} meta={terms.join(" · ")} bodyless>
<Table className="min-w-[42rem]">
<THead>
<TR className="hover:bg-transparent">
<TH>Component</TH>
{ENVS.map((env) =>
terms.map((t) => (
<TH key={`${env}-${t}`}>
{env} / {t}
</TH>
)),
)}
<TH />
</TR>
</THead>
<TBody>
{rows
.filter(
(r) =>
r.deployment === deployment &&
r.tier === tier,
)
.map((r) => {
const k = rowKey(r);
const ids = drafts[k] ?? r.price_ids ?? {};
const dirty =
JSON.stringify(ids) !==
JSON.stringify(r.price_ids ?? {});
return (
<TR key={k}>
<TD className="text-ink">{componentLabel(r)}</TD>
{ENVS.map((env) =>
terms.map((t) => (
<TD key={`${env}-${t}`}>
<input
value={
ids[env]?.[t] ?? ""
}
placeholder="pri_…"
onChange={(e) =>
setDrafts({
...drafts,
[k]: {
...ids,
[env]: {
...(ids[
env
] ?? {}),
[t]: e
.target
.value,
},
},
})
}
className="w-40 rounded border border-rule bg-panel-2 px-2 py-1 font-mono text-[0.78rem] text-ink focus:border-accent focus:outline-none"
/>
</TD>
)),
)}
<TD numeric>
<button
type="button"
disabled={
!dirty || save.isPending
}
onClick={() =>
save.mutate({
...r,
price_ids: ids,
})
}
className="rounded border border-accent px-2.5 py-1 font-mono text-[0.7rem] uppercase tracking-[0.1em] text-accent disabled:opacity-40"
>
Save
</button>
</TD>
</TR>
);
})}
</TBody>
</Table>
</Panel>
);
})}
</div>
)}
</PageFrame>
</div>
);
}
+1 -2
View File
@@ -7,8 +7,7 @@ const LINKS: NavLink[] = [
{ href: "/staff", label: "Operations" },
{ href: "/staff/accounts", label: "Accounts" },
{ href: "/staff/licenses", label: "Licences" },
{ href: "/staff/plans", label: "Plans" },
{ href: "/staff/catalogue", label: "Catalogue" },
{ href: "/staff/pricing", label: "Pricing" },
{ href: "/staff/audit", label: "Audit" },
];
-153
View File
@@ -1,153 +0,0 @@
"use client";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { useState } from "react";
import { api, type Deployment, type Plan, type Tier } from "@/lib/api";
import { ConfirmPlanChange } from "@/components/ConfirmPlanChange";
import { PageHeader } from "@/components/PageHeader";
import { Panel } from "@/components/Panel";
const SUPPORT_LEVELS = [
{ value: "community", label: "Community" },
{ value: "email_24_5", label: "Email, 24/5" },
{ value: "email_call_24_7", label: "Email + call, 24/7" },
] as const;
const LIMIT_FIELDS = [
{ key: "max_servers", label: "Servers" },
{ key: "max_monitors", label: "Monitors" },
{ key: "max_secret_groups", label: "Secret groups" },
{ key: "max_channels", label: "Channels" },
{ key: "audit_retention_days", label: "Audit history (days)" },
] as const;
/*
* -1 is Unlimited everywhere in the licence payload, so the form takes it
* literally rather than inventing a checkbox. A staff screen that hides the
* sentinel is a staff screen where nobody can tell whether a plan says
* unlimited or nothing at all.
*/
function AllowanceForm({ plan, onSave, saving }: { plan: Plan; onSave: (next: Plan) => void; saving: boolean }) {
const [draft, setDraft] = useState<Plan>(plan);
const dirty = JSON.stringify(draft) !== JSON.stringify(plan);
return (
<div className="grid gap-3">
<div className="grid gap-2 sm:grid-cols-2 lg:grid-cols-3">
{LIMIT_FIELDS.map((f) => (
<label key={f.key} className="block">
<span className="mb-1 block text-[0.78rem] text-ink-3">{f.label}</span>
<input
type="number"
value={draft.base_limits[f.key]}
onChange={(e) =>
setDraft({
...draft,
base_limits: {
...draft.base_limits,
[f.key]: Number(e.target.value),
},
})
}
className="w-full rounded border border-rule bg-panel-2 px-2 py-1.5 text-[0.85rem] text-ink focus:border-accent focus:outline-none"
/>
<span className="mt-0.5 block text-[0.72rem] text-ink-3">1 is unlimited</span>
</label>
))}
<label className="block">
<span className="mb-1 block text-[0.78rem] text-ink-3">Support level</span>
<select
value={draft.support_level}
onChange={(e) => setDraft({ ...draft, support_level: e.target.value })}
className="w-full rounded border border-rule bg-panel-2 px-2 py-1.5 text-[0.85rem] text-ink focus:border-accent focus:outline-none"
>
{SUPPORT_LEVELS.map((s) => (
<option key={s.value} value={s.value}>
{s.label}
</option>
))}
</select>
</label>
</div>
<label className="flex items-center gap-2 text-[0.85rem] text-ink-2">
<input type="checkbox" checked={draft.active} onChange={(e) => setDraft({ ...draft, active: e.target.checked })} />
Offered to customers
</label>
<p className="text-[0.78rem] text-ink-3">Changes apply to licences issued from now on. Existing licences snapshotted their plan and are unaffected.</p>
<button
type="button"
disabled={!dirty || saving}
onClick={() => onSave(draft)}
className="justify-self-start rounded border border-accent bg-accent px-3.5 py-2 text-[0.86rem] font-semibold text-accent-ink disabled:opacity-40"
>
{saving ? "Saving…" : "Save allowances"}
</button>
</div>
);
}
export default function PlansPage() {
const qc = useQueryClient();
const plans = useQuery({ queryKey: ["plans"], queryFn: api.staff.plans });
const licenses = useQuery({
queryKey: ["staff-licenses"],
queryFn: () => api.staff.licenses(),
});
const [draft, setDraft] = useState<Plan | null>(null);
const [saving, setSaving] = useState<string | null>(null);
const save = useMutation({
mutationFn: (p: Plan) => api.staff.updatePlan(p.deployment, p.tier, p),
onSuccess: () => {
qc.invalidateQueries({ queryKey: ["plans"] });
setDraft(null);
setSaving(null);
},
onError: () => setSaving(null),
});
const original = plans.data?.find((p) => p.deployment === draft?.deployment && p.tier === draft?.tier);
return (
<div className="grid gap-6">
<PageHeader
title="Plans"
subtitle="The authoritative tier table six plans, two deployments by three tiers, base allowances only. Every issued licence snapshots the plan it was cut from, so editing one never rewrites an existing licence."
/>
{draft && original && (
<ConfirmPlanChange
plan={original}
next={draft}
issuedCount={(licenses.data ?? []).filter((l) => l.tier === draft.tier && l.deployment === draft.deployment).length}
onConfirm={() => {
setSaving(`${draft.deployment}/${draft.tier}`);
save.mutate(draft);
}}
onCancel={() => setDraft(null)}
/>
)}
{(["cloud", "self_hosted"] as const).map((deployment: Deployment) => (
<section key={deployment} className="grid gap-3">
<h2 className="font-mono text-[0.68rem] uppercase tracking-[0.14em] text-ink-3">{deployment === "cloud" ? "Cloud" : "Self-Hosted"}</h2>
{(plans.data ?? [])
.filter((p) => p.deployment === deployment)
.map((p) => (
<Panel
key={`${p.deployment}/${p.tier}`}
title={p.name}
meta={`${p.deployment}/${p.tier}`}
actions={!p.active ? <span className="font-mono text-[0.64rem] uppercase tracking-[0.12em] text-warn">Not offered</span> : undefined}
>
<AllowanceForm plan={p} saving={saving === `${p.deployment}/${p.tier}`} onSave={(next: Plan) => setDraft(next)} />
</Panel>
))}
</section>
))}
</div>
);
}
@@ -0,0 +1,162 @@
"use client";
import { useState } from "react";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { Panel } from "@/components/Panel";
import { SectionHeading } from "./SectionHeading";
import { planRows, rowKey, sharedRows } from "@/lib/catalogue";
import { featureLabel } from "@/lib/features";
import { api, type CatalogueRow, type Term } from "@/lib/api";
const ENVS = ["sandbox", "production"] as const;
/* A shared row is sold by both deployments, so it holds both terms: the cloud
* checkout takes the monthly price and the self-hosted one never asks for it. A
* plan row offers only the terms its own deployment sells self-hosted is
* annual only, and the field is not rendered rather than rendered and refused. */
function termsFor(r: CatalogueRow): Term[] {
if (r.scope === "shared") return ["monthly", "annual"];
return r.deployment === "self_hosted" ? ["annual"] : ["monthly", "annual"];
}
function componentLabel(r: CatalogueRow): string {
if (r.kind === "base") return `${r.tier === "enterprise" ? "Enterprise" : "Professional"} (${r.deployment === "cloud" ? "Cloud" : "Self-hosted"})`;
if (r.kind === "limit") return "Additional server";
return featureLabel(r.feature_key ?? "");
}
function componentDetail(r: CatalogueRow): string {
if (r.kind === "base") return "The plan's own fee, always quantity 1";
if (r.kind === "limit") return `Raises ${r.limit_key} by one per unit`;
return `feature · ${r.feature_key}`;
}
/*
* The coverage ledger: one square per environment and term, filled when that
* cell holds a price ID.
*
* A missing production price is invisible in a grid of text inputs every cell
* looks like every other until you read twenty-six characters of each. This is
* the one thing staff come to this page to check before a launch, so it reads
* before the IDs do.
*/
function Coverage({ row, terms }: { row: CatalogueRow; terms: Term[] }) {
const cells = ENVS.flatMap((env) => terms.map((t) => ({ env, t, filled: Boolean(row.price_ids?.[env]?.[t]) })));
const filled = cells.filter((c) => c.filled).length;
return (
<span className="flex items-center gap-1">
{cells.map((c) => (
<span key={`${c.env}-${c.t}`} title={`${c.env} ${c.t}`} className={["block h-2.5 w-2.5 rounded-[1px] border", c.filled ? "border-valid bg-valid" : "border-rule bg-panel-2"].join(" ")} />
))}
<span className="ml-1.5 font-mono text-[0.62rem] tracking-[0.08em] text-ink-3">
{filled}/{cells.length} priced
</span>
</span>
);
}
function ComponentRow({ row, scopeLabel }: { row: CatalogueRow; scopeLabel: string }) {
const qc = useQueryClient();
const [draft, setDraft] = useState<CatalogueRow["price_ids"] | null>(null);
const ids = draft ?? row.price_ids ?? {};
const dirty = JSON.stringify(ids) !== JSON.stringify(row.price_ids ?? {});
const terms = termsFor(row);
const save = useMutation({
mutationFn: () => api.staff.updateCatalogue({ ...row, price_ids: ids }),
onSuccess: () => {
setDraft(null);
qc.invalidateQueries({ queryKey: ["staff", "catalogue"] });
},
});
const set = (env: string, term: Term, value: string) =>
setDraft({ ...ids, [env]: { ...(ids[env] ?? {}), [term]: value } });
return (
<div className="grid gap-3 border-t border-rule-soft pt-3 first:border-0 first:pt-0 md:grid-cols-[minmax(0,17rem)_1fr]">
<div className="grid content-start gap-1.5">
<span className="text-[0.9rem] font-semibold">{componentLabel(row)}</span>
<span className={["w-max rounded border px-1.5 py-px font-mono text-[0.6rem] uppercase tracking-[0.1em]", row.scope === "shared" ? "border-accent text-accent" : "border-rule text-ink-3"].join(" ")}>{scopeLabel}</span>
<span className="text-[0.78rem] text-ink-3">{componentDetail(row)}</span>
<Coverage row={{ ...row, price_ids: ids }} terms={terms} />
</div>
<div className="grid gap-2">
<div className="grid gap-1.5 sm:grid-cols-2">
{ENVS.map((env) => (
<div key={env} className="grid content-start gap-1.5">
<span className="flex items-center gap-2 font-mono text-[0.62rem] uppercase tracking-[0.12em] text-ink-3">
{env}
<span className="h-px flex-1 bg-rule-soft" />
</span>
{terms.map((t) => (
<label key={t} className="grid gap-1">
<span className="font-mono text-[0.62rem] uppercase tracking-[0.1em] text-ink-3">{t}</span>
<input
value={ids[env]?.[t] ?? ""}
placeholder="pri_…"
onChange={(e) => set(env, t, e.target.value)}
className={["w-full rounded border bg-panel-2 px-2 py-1.5 font-mono text-[0.76rem] text-ink focus:border-accent focus:outline-none", ids[env]?.[t] ? "border-rule" : "border-dashed border-rule"].join(" ")}
aria-label={`${componentLabel(row)} ${env} ${t} price ID`}
/>
</label>
))}
</div>
))}
</div>
<div className="flex flex-wrap items-center gap-2.5">
<button type="button" disabled={!dirty || save.isPending} onClick={() => save.mutate()} className="rounded border border-accent px-2.5 py-1 font-mono text-[0.7rem] uppercase tracking-[0.1em] text-accent disabled:opacity-40">
{save.isPending ? "Saving…" : "Save"}
</button>
{save.error && <span className="text-[0.78rem] text-expired">{(save.error as Error).message}</span>}
</div>
</div>
</div>
);
}
/*
* The catalogue half of /staff/pricing: every priceable component, grouped by
* what it is rather than by which plan sells it.
*/
export function CatalogueSection() {
const { data: rows = [], isLoading } = useQuery({
queryKey: ["staff", "catalogue"],
queryFn: api.staff.catalogue,
});
const shared = sharedRows(rows);
const bases = planRows(rows);
return (
<section className="grid gap-3">
<SectionHeading
title="Catalogue"
note="Every priceable component, grouped by what it is rather than by which plan sells it. This is the only place a Paddle price ID lives."
/>
<div className="grid gap-1.5 rounded border-l-2 border-accent bg-accent-wash px-3 py-2.5 text-[0.82rem] text-ink-2">
<p>An add-on is one Paddle product sold to every paid plan, so its price is typed once. Only the base fee differs by plan, because only the base fee is a different product per plan.</p>
<p>A component with no price ID is free a feature with no price is a toggle a customer may take at no charge. Free is priced by nothing and has no rows at all, which is what keeps it outside Paddle. Changing a price affects the next checkout only; it cannot touch an issued licence.</p>
</div>
{isLoading ? (
<p className="text-[0.85rem] text-ink-3">Loading</p>
) : (
<div className="grid gap-3">
<Panel title="Add-ons" meta={`${shared.length} rows · every paid plan`}>
{shared.map((r) => (
<ComponentRow key={rowKey(r)} row={r} scopeLabel="All paid plans" />
))}
</Panel>
<Panel title="Base fee" meta={`${bases.length} rows · one per plan`}>
{bases.map((r) => (
<ComponentRow key={rowKey(r)} row={r} scopeLabel="This plan only" />
))}
</Panel>
</div>
)}
</section>
);
}
@@ -0,0 +1,248 @@
"use client";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { useState } from "react";
import { api, type Deployment, type Plan } from "@/lib/api";
import { featureDesc, featureLabel, FEATURE_LABEL } from "@/lib/features";
import { limitLabel } from "@/lib/format";
import { Button, controlClass } from "@/components/Button";
import { ConfirmPlanChange } from "@/components/ConfirmPlanChange";
import { SectionHeading } from "./SectionHeading";
import { Modal } from "@/components/Modal";
const SUPPORT_LEVELS = [
{ value: "community", label: "Community" },
{ value: "email_24_5", label: "Email, 24/5" },
{ value: "email_call_24_7", label: "Email + call, 24/7" },
] as const;
const LIMIT_FIELDS = [
{ key: "max_servers", label: "Servers" },
{ key: "max_monitors", label: "Monitors" },
{ key: "max_secret_groups", label: "Secret groups" },
{ key: "max_channels", label: "Channels" },
{ key: "audit_retention_days", label: "Audit history (days)" },
] as const;
const FEATURE_KEYS = Object.keys(FEATURE_LABEL);
const planKey = (p: Plan) => `${p.deployment}/${p.tier}`;
/*
* The list is tiers, and a tier's settings are behind a button.
*
* Six plans with five number fields, a select, a checkbox and four toggles each
* is forty-odd controls on one screen, and the page it made could not be read
* for the thing it exists to answer: what does each tier give you. The card
* answers that; the modal is where it is changed.
*/
function TierCard({ plan, onOpen }: { plan: Plan; onOpen: () => void }) {
return (
<button
type="button"
onClick={onOpen}
className={[
"grid w-full gap-2.5 rounded border bg-panel p-3.5 text-left",
"transition-[border-color,transform] duration-150 hover:-translate-y-px hover:border-accent",
plan.active ? "border-rule" : "border-dashed border-rule opacity-75",
].join(" ")}
>
<span className="flex flex-wrap items-center gap-2">
<span className="text-[1rem] font-semibold">{plan.name}</span>
{!plan.active && <span className="rounded border border-warn px-1.5 py-px font-mono text-[0.6rem] uppercase tracking-[0.1em] text-warn">Not offered</span>}
<span className="ml-auto font-mono text-[0.68rem] text-ink-3">{planKey(plan)}</span>
</span>
<dl className="grid grid-cols-[1fr_auto] gap-x-3 gap-y-0.5 text-[0.82rem]">
<dt className="text-ink-3">Servers</dt>
<dd className="text-right tabular-nums">{limitLabel(plan.base_limits.max_servers)}</dd>
<dt className="text-ink-3">Monitors</dt>
<dd className="text-right tabular-nums">{limitLabel(plan.base_limits.max_monitors)}</dd>
<dt className="text-ink-3">Audit history</dt>
<dd className="text-right tabular-nums">{limitLabel(plan.base_limits.audit_retention_days)} days</dd>
</dl>
{/* Every feature key, lit or unlit an absent chip cannot be told
* from a feature nobody has heard of, and no tier bundles one today,
* so the unlit row IS the information. */}
<span className="flex flex-wrap gap-1">
{FEATURE_KEYS.map((k) => {
const on = plan.base_features.includes(k);
return (
<span key={k} className={["rounded border px-1.5 py-px font-mono text-[0.6rem] uppercase tracking-[0.06em]", on ? "border-valid text-valid" : "border-rule text-ink-3"].join(" ")}>
{featureLabel(k)}
</span>
);
})}
</span>
<span className="justify-self-start rounded border border-accent px-2.5 py-1 font-mono text-[0.68rem] uppercase tracking-[0.1em] text-accent">Open plan</span>
</button>
);
}
/* -1 is Unlimited everywhere in the licence payload, so the form takes it
* literally rather than inventing a checkbox. A staff screen that hides the
* sentinel is a staff screen where nobody can tell whether a plan says
* unlimited or nothing at all. */
function PlanModal({ plan, onClose, onSave }: { plan: Plan; onClose: () => void; onSave: (next: Plan) => void }) {
const [draft, setDraft] = useState<Plan>(plan);
const dirty = JSON.stringify(draft) !== JSON.stringify(plan);
const toggleFeature = (key: string, on: boolean) =>
setDraft({
...draft,
base_features: on ? [...draft.base_features, key] : draft.base_features.filter((f) => f !== key),
});
return (
<Modal
open
onClose={onClose}
title={plan.name}
meta={planKey(plan)}
footer={
<>
<p className="mr-auto max-w-md text-[0.78rem] text-ink-3">Applies to licences issued from now on. Issued licences snapshotted their plan and are unaffected.</p>
<Button type="button" variant="line" onClick={onClose}>
Cancel
</Button>
<Button type="button" disabled={!dirty} onClick={() => onSave(draft)}>
Save plan
</Button>
</>
}
>
<section className="grid gap-2">
<span className="font-mono text-[0.66rem] uppercase tracking-[0.14em] text-ink-3">Base limits</span>
<div className="grid gap-2 sm:grid-cols-2 lg:grid-cols-3">
{LIMIT_FIELDS.map((f) => (
<label key={f.key} className="grid gap-1">
<span className="text-[0.78rem] text-ink-3">{f.label}</span>
<input
type="number"
value={draft.base_limits[f.key]}
onChange={(e) =>
setDraft({
...draft,
base_limits: { ...draft.base_limits, [f.key]: Number(e.target.value) },
})
}
className={controlClass("h-9 text-[0.84rem] tabular-nums")}
/>
</label>
))}
</div>
<p className="text-[0.78rem] text-ink-3">1 is unlimited. A metered dimension starts here and the customer buys upward from it.</p>
</section>
<section className="grid gap-2">
<span className="font-mono text-[0.66rem] uppercase tracking-[0.14em] text-ink-3">Base features</span>
<div className="grid gap-1.5">
{FEATURE_KEYS.map((k) => {
const on = draft.base_features.includes(k);
return (
<label key={k} className="flex items-center gap-2.5 rounded border border-rule-soft bg-panel-2 px-2.5 py-2">
<input type="checkbox" checked={on} onChange={(e) => toggleFeature(k, e.target.checked)} />
<span>
<span className="block text-[0.86rem]">{featureLabel(k)}</span>
<span className="block text-[0.75rem] text-ink-3">{featureDesc(k)}</span>
</span>
<span className="ml-auto font-mono text-[0.66rem] uppercase tracking-[0.1em] text-ink-3">{on ? "Included" : "Sold as add-on"}</span>
</label>
);
})}
</div>
<p className="text-[0.78rem] text-ink-3">No tier bundles a feature today. Including one here grants it with the plan and removes it from the customer&apos;s purchase form.</p>
</section>
<section className="grid gap-2">
<span className="font-mono text-[0.66rem] uppercase tracking-[0.14em] text-ink-3">Availability</span>
<div className="grid gap-2 sm:grid-cols-2">
<label className="grid gap-1">
<span className="text-[0.78rem] text-ink-3">Support level</span>
<select value={draft.support_level} onChange={(e) => setDraft({ ...draft, support_level: e.target.value })} className={controlClass("h-9 text-[0.84rem]")}>
{SUPPORT_LEVELS.map((s) => (
<option key={s.value} value={s.value}>
{s.label}
</option>
))}
</select>
</label>
<label className="flex items-center gap-2 self-end pb-2 text-[0.86rem]">
<input type="checkbox" checked={draft.active} onChange={(e) => setDraft({ ...draft, active: e.target.checked })} />
Offered to customers
</label>
</div>
</section>
</Modal>
);
}
/*
* The plans half of /staff/pricing. It is a section rather than a page because
* a tier's allowances and a tier's price are one decision made in one sitting,
* and they were two screens with no view showing both.
*/
export function PlansSection() {
const qc = useQueryClient();
const plans = useQuery({ queryKey: ["plans"], queryFn: api.staff.plans });
const licenses = useQuery({ queryKey: ["staff-licenses"], queryFn: () => api.staff.licenses() });
/* Two pieces of state, not one: `editing` is the plan whose modal is open,
* `confirming` is the edit awaiting the change summary. Collapsing them put
* the confirmation behind the modal it was confirming. */
const [editing, setEditing] = useState<Plan | null>(null);
const [confirming, setConfirming] = useState<Plan | null>(null);
const save = useMutation({
mutationFn: (p: Plan) => api.staff.updatePlan(p.deployment, p.tier, p),
onSuccess: () => {
qc.invalidateQueries({ queryKey: ["plans"] });
setConfirming(null);
},
});
const original = plans.data?.find((p) => p.deployment === confirming?.deployment && p.tier === confirming?.tier);
return (
<div className="grid gap-6">
<SectionHeading title="Plans" note="What each tier grants. Open a tier to change its base limits and features. Every issued licence snapshots the plan it was cut from, so editing one never rewrites an existing licence." />
{confirming && original && (
<ConfirmPlanChange
plan={original}
next={confirming}
issuedCount={(licenses.data ?? []).filter((l) => l.tier === confirming.tier && l.deployment === confirming.deployment).length}
onConfirm={() => save.mutate(confirming)}
onCancel={() => setConfirming(null)}
/>
)}
{(["cloud", "self_hosted"] as const).map((deployment: Deployment) => (
<section key={deployment} className="grid gap-2.5">
<h2 className="font-mono text-[0.68rem] uppercase tracking-[0.14em] text-ink-3">{deployment === "cloud" ? "Cloud" : "Self-hosted"}</h2>
<div className="grid gap-2.5 sm:grid-cols-2 lg:grid-cols-3">
{(plans.data ?? [])
.filter((p) => p.deployment === deployment)
.map((p) => (
<TierCard key={planKey(p)} plan={p} onOpen={() => setEditing(p)} />
))}
</div>
</section>
))}
{editing && (
<PlanModal
key={planKey(editing)}
plan={editing}
onClose={() => setEditing(null)}
onSave={(next) => {
setEditing(null);
setConfirming(next);
}}
/>
)}
</div>
);
}
@@ -0,0 +1,15 @@
/*
* The heading that separates the two halves of /staff/pricing.
*
* It is not PageHeader: the page has one of those, and a second title-sized
* heading under it would read as a second page. This is the same mono eyebrow
* idiom the deployment groups use, one level up.
*/
export function SectionHeading({ title, note }: { title: string; note: string }) {
return (
<div className="grid gap-1 border-b border-rule pb-2">
<h2 className="text-[1.05rem] font-bold tracking-[-0.01em]">{title}</h2>
<p className="max-w-[68ch] text-[0.84rem] text-ink-3">{note}</p>
</div>
);
}
@@ -0,0 +1,24 @@
"use client";
import { PageHeader } from "@/components/PageHeader";
import { CatalogueSection } from "./CatalogueSection";
import { PlansSection } from "./PlansSection";
/*
* Plans and catalogue on one page.
*
* They were two nav entries, and the split asked staff to hold one half in
* their head while looking at the other: a tier's allowances decide what the
* metered component charges for, and the base fee is meaningless without the
* allowance it includes. One page, two sections, in the order the decision is
* made what a tier grants, then what it costs.
*/
export default function PricingPage() {
return (
<div className="grid gap-7">
<PageHeader title="Pricing" back={{ href: "/staff", label: "Operations" }} subtitle="What each tier grants, and what every priceable component costs." />
<PlansSection />
<CatalogueSection />
</div>
);
}
+63
View File
@@ -0,0 +1,63 @@
"use client";
import { useEffect, useRef } from "react";
/*
* A native <dialog>, not a div with a fixed overlay.
*
* showModal() gives focus trapping, inert background, Escape and the top layer
* for free all four are things a hand-rolled overlay gets wrong, and the third
* is the one staff will actually reach for. The only wiring needed is keeping
* React state and the element's open state in step, and routing every close
* Escape, backdrop, button through one onClose.
*/
export function Modal({
open,
onClose,
title,
meta,
footer,
children,
}: {
open: boolean;
onClose: () => void;
title: string;
meta?: React.ReactNode;
footer?: React.ReactNode;
children: React.ReactNode;
}) {
const ref = useRef<HTMLDialogElement>(null);
useEffect(() => {
const el = ref.current;
if (!el) return;
if (open && !el.open) el.showModal();
if (!open && el.open) el.close();
}, [open]);
return (
<dialog
ref={ref}
onCancel={(e) => {
e.preventDefault();
onClose();
}}
/* Clicking the backdrop hits the dialog element itself, never a
* child so this closes on backdrop and not on content. */
onClick={(e) => {
if (e.target === ref.current) onClose();
}}
className="w-[min(44rem,94vw)] rounded border border-rule bg-panel p-0 text-ink shadow-lg backdrop:bg-[rgba(4,12,24,0.55)]"
>
<header className="flex flex-wrap items-center gap-3 border-b border-rule-soft bg-panel-2 px-4 py-3">
<h2 className="text-[1.02rem] font-bold tracking-[-0.01em]">{title}</h2>
{meta && <span className="font-mono text-[0.68rem] uppercase tracking-[0.12em] text-ink-3">{meta}</span>}
<button type="button" onClick={onClose} className="ml-auto rounded border border-rule px-2 py-1 font-mono text-[0.68rem] uppercase tracking-[0.1em] text-ink-2 hover:border-ink-3" aria-label="Close">
Esc
</button>
</header>
<div className="grid max-h-[68vh] gap-4 overflow-y-auto p-4">{children}</div>
{footer && <footer className="flex flex-wrap items-center gap-3 border-t border-rule-soft bg-panel-2 px-4 py-3">{footer}</footer>}
</dialog>
);
}
+2 -1
View File
@@ -2,6 +2,7 @@
import { useMemo } from "react";
import type { CatalogueRow, Deployment, Plan, Term, Tier } from "@/lib/api";
import { rowsForPlan } from "@/lib/catalogue";
import { featureLabel } from "@/lib/features";
export interface PlanChoice {
@@ -50,7 +51,7 @@ export default function PlanConfigurator({
);
const plan = available.find((p) => p.tier === value.tier);
const rows = useMemo(
() => catalogue.filter((r) => r.deployment === deployment && r.tier === value.tier),
() => rowsForPlan(catalogue, deployment, value.tier),
[catalogue, deployment, value.tier],
);
const featureRows = rows.filter((r) => r.kind === "feature");
+9 -3
View File
@@ -8,6 +8,9 @@
* customer-facing and should be shown verbatim).
*/
/* catalogue.ts imports only types from here, so this is not a cycle. */
import { rowsForPlan } from "@/lib/catalogue";
export const API_BASE = (process.env.NEXT_PUBLIC_ADMIN_API_URL ?? "").replace(/\/$/, "");
export class NotConnected extends Error {
@@ -176,6 +179,11 @@ export interface Plan {
export interface CatalogueRow {
kind: "base" | "limit" | "feature";
/* "plan" rows carry a deployment and tier and belong to that plan alone.
* "shared" rows leave both empty and are sold by every paid plan, which is
* why a price ID is typed once rather than four times. Read them through
* rowsForPlan in lib/catalogue, never by filtering on deployment. */
scope: "plan" | "shared";
deployment: Deployment;
tier: Tier;
limit_key?: string;
@@ -213,9 +221,7 @@ export function lineItemsFor(
const env = opts.env;
const plan = opts.plans.find((p) => p.deployment === deployment && p.tier === choice.tier);
if (!plan) return [];
const rows = opts.catalogue.filter(
(r) => r.deployment === deployment && r.tier === choice.tier,
);
const rows = rowsForPlan(opts.catalogue, deployment, choice.tier);
const priceOf = (r: CatalogueRow) => r.price_ids?.[env]?.[choice.term] ?? "";
const base = plan.base_limits.max_servers;
const items: { priceId: string; quantity: number }[] = [];
+38
View File
@@ -0,0 +1,38 @@
import type { CatalogueRow, Deployment, Tier } from "@/lib/api";
/*
* rowsForPlan is the TypeScript half of Go's models.CatalogueFor, and the two
* must change together the same shape of hazard as web/lib/targets.ts.
*
* A plan sells its own base row plus every shared add-on row. Shared rows leave
* deployment and tier empty, so the filter this replaced `r.deployment === dep
* && r.tier === tier` — now returns a plan priced by its base fee and nothing
* else. There were five copies of that filter; this is why it is a module.
*/
export function rowsForPlan(
catalogue: CatalogueRow[],
deployment: Deployment,
tier: Tier,
): CatalogueRow[] {
return catalogue.filter(
(r) => r.scope === "shared" || (r.deployment === deployment && r.tier === tier),
);
}
/* Every add-on a paid plan can be sold, in one list. The staff catalogue editor
* shows these once; the purchase form reads them per plan through rowsForPlan. */
export function sharedRows(catalogue: CatalogueRow[]): CatalogueRow[] {
return catalogue.filter((r) => r.scope === "shared");
}
/* The base fee rows, which are genuinely one per plan because each is its own
* Paddle product at its own price. */
export function planRows(catalogue: CatalogueRow[]): CatalogueRow[] {
return catalogue.filter((r) => r.scope !== "shared");
}
/* A stable identity for a row, used as a React key and as the draft key in the
* staff editor. Mirrors the natural key the API addresses a row by. */
export function rowKey(r: CatalogueRow): string {
return [r.scope ?? "plan", r.deployment ?? "", r.tier ?? "", r.kind, r.limit_key ?? "", r.feature_key ?? ""].join("/");
}
+2
View File
@@ -10,12 +10,14 @@ export const FEATURE_LABEL: Record<string, string> = {
console: "Browser console",
oidc: "Single sign-on",
vuln_scanning: "Vulnerability scanning",
status_pages: "Status pages",
};
export const FEATURE_DESC: Record<string, string> = {
console: "In-browser SSH, RDP and VNC sessions",
oidc: "OIDC sign-in for your whole team",
vuln_scanning: "Package inventory matched against distribution security advisories",
status_pages: "Public status pages for your customers, built from your monitors",
};
export function featureLabel(key: string): string {
+8
View File
@@ -8,6 +8,14 @@ import type { NextConfig } from "next";
*/
const nextConfig: NextConfig = {
output: "standalone",
/* Plans and catalogue became one page. Both old paths are bookmarked in
* staff browsers, so they redirect rather than 404. */
async redirects() {
return [
{ source: "/staff/plans", destination: "/staff/pricing", permanent: true },
{ source: "/staff/catalogue", destination: "/staff/pricing", permanent: true },
];
},
};
export default nextConfig;
+5
View File
@@ -18,6 +18,10 @@ const (
TypeTCP = "tcp"
TypeICMP = "icmp"
TypeTLS = "tls"
// UserAgent identifies Vantage monitor traffic so a WAF rule can single it
// out. Match on a prefix, not equality: the version moves.
UserAgent = "Vantage-Monitor/1.0 (+https://vantage.hostxtra.co.uk)"
)
@@ -84,6 +88,7 @@ func runHTTP(ctx context.Context, s Spec) Result {
if err != nil {
return Result{Message: err.Error()}
}
req.Header.Set("User-Agent", UserAgent)
resp, err := client.Do(req)
if err != nil {
return Result{LatencyMs: msSince(start), Message: err.Error()}
+7 -3
View File
@@ -19,12 +19,16 @@ func Run(ctx context.Context, script string) (string, error) {
out, err := cmd.Output()
if err != nil {
if ee, ok := err.(*exec.ExitError); ok && len(ee.Stderr) > 0 {
return "", fmt.Errorf("powershell: %s", strings.TrimSpace(string(ee.Stderr)))
}
// Checked before the ExitError/stderr branch: CommandContext kills the
// process on timeout, and that kill can itself produce an ExitError
// carrying stderr text, so a genuine timeout would otherwise surface
// as that stderr instead of the "timed out" message callers match on.
if ctx.Err() == context.DeadlineExceeded {
return "", fmt.Errorf("powershell: timed out")
}
if ee, ok := err.(*exec.ExitError); ok && len(ee.Stderr) > 0 {
return "", fmt.Errorf("powershell: %s", strings.TrimSpace(string(ee.Stderr)))
}
return "", fmt.Errorf("powershell: %w", err)
}
return string(out), nil
+10 -2
View File
@@ -12,8 +12,16 @@ const servicesTimeout = 60 * time.Second
const servicesScript = `
$ErrorActionPreference = 'Stop'
$svcs = Get-CimInstance Win32_Service |
Select-Object Name,DisplayName,State,StartMode,PathName,ExitCode
$svcs = Get-CimInstance Win32_Service | ForEach-Object {
[pscustomobject]@{
Name = $_.Name
DisplayName = $_.DisplayName
State = $_.State
StartMode = $_.StartMode
PathName = $_.PathName
ExitCode = $_.ExitCode
}
}
ConvertTo-Json -InputObject @($svcs) -Depth 3 -Compress
`
+11 -3
View File
@@ -113,10 +113,15 @@ func parseServices(jsonText, systemRoot string) ([]Workload, error) {
continue
}
state := "stopped"
// The wire shape is shared with the systemd collector — both report
// under kind "unit" — so the state word has to be too, or the UI
// (which colours and filters on it, and does so before it knows
// which platform sent the row) needs two vocabularies for one kind.
// running/stopped/failed become active/inactive/failed to match.
state := "inactive"
switch {
case running:
state = "running"
state = "active"
case failed:
state = "failed"
}
@@ -189,7 +194,10 @@ func parseEvents(jsonText, serviceName, displayName string, tail int) (string, e
continue
}
}
msg := strings.TrimSpace(strings.ReplaceAll(e.M, "\r\n", " "))
// Collapse every newline form, not just "\r\n": a message containing a
// bare "\n" would otherwise still break the one-line-per-event shape
// this renders for the log dialog, and undercount the tail trim above.
msg := strings.TrimSpace(strings.NewReplacer("\r\n", " ", "\r", " ", "\n", " ").Replace(e.M))
lines = append(lines, e.T+" "+e.L+" "+msg)
}
+25 -6
View File
@@ -59,12 +59,12 @@ func TestParseServicesFilters(t *testing.T) {
t.Fatalf("got %d workloads, want 3: %+v", len(got), got)
}
if w := byID["Contoso"]; w.Kind != "unit" || w.Name != "Contoso Broker" || w.State != "running" {
if w := byID["Contoso"]; w.Kind != "unit" || w.Name != "Contoso Broker" || w.State != "active" {
t.Errorf("Contoso = %+v", w)
}
// Enabled but not running is exactly the row worth seeing.
if byID["Fabrikam"].State != "stopped" {
t.Errorf("Fabrikam state = %q, want stopped", byID["Fabrikam"].State)
if byID["Fabrikam"].State != "inactive" {
t.Errorf("Fabrikam state = %q, want inactive", byID["Fabrikam"].State)
}
// A non-zero exit code on a stopped service is a crash, not a clean stop.
if byID["Crashed"].State != "failed" {
@@ -80,15 +80,15 @@ func TestParseServicesExitCode1077(t *testing.T) {
if err != nil {
t.Fatalf("parseServices: %v", err)
}
if len(got) != 1 || got[0].State != "stopped" {
t.Fatalf("got %+v, want one stopped workload", got)
if len(got) != 1 || got[0].State != "inactive" {
t.Fatalf("got %+v, want one inactive workload", got)
}
}
func TestParseServicesSingleObjectAndEmpty(t *testing.T) {
one := `{"Name":"Solo","DisplayName":"Solo","State":"Running","StartMode":"Auto","PathName":"C:\\Solo\\s.exe","ExitCode":0}`
got, err := parseServices(one, `C:\WINDOWS`)
if err != nil || len(got) != 1 {
if err != nil || len(got) != 1 || got[0].State != "active" {
t.Fatalf("single object: got %+v, err %v", got, err)
}
@@ -130,6 +130,25 @@ func TestParseEventsFormatsAndOrders(t *testing.T) {
}
}
// A message containing a bare "\n" (no carriage return) must still collapse to
// one line, or it silently multiplies into several output lines and throws
// off the tail trim's count.
func TestParseEventsCollapsesBareLF(t *testing.T) {
in := `[{"t":"2026-08-13T10:00:00Z","l":"Error","p":"Contoso","m":"broker died\nstack trace here"}]`
got, err := parseEvents(in, "Contoso", "Contoso Broker", 500)
if err != nil {
t.Fatalf("parseEvents: %v", err)
}
if strings.Count(got, "\n") != 0 {
t.Fatalf("parseEvents did not collapse bare LF into one line: %q", got)
}
want := "2026-08-13T10:00:00Z Error broker died stack trace here"
if got != want {
t.Fatalf("parseEvents =\n%q\nwant\n%q", got, want)
}
}
// Service Control Manager logs every service on the host under one provider, so
// its rows must be filtered down to the target or the log is somebody else's.
func TestParseEventsFiltersOtherServicesSCM(t *testing.T) {
+1 -1
View File
@@ -2,5 +2,5 @@ apiVersion: v2
name: vantage
description: Helm chart for the Vantage stack (Redis, MongoDB, guacd, server, web)
type: application
version: 1.0.8
version: 1.1.0
appVersion: "1.0.8"
+13 -6
View File
@@ -41,12 +41,9 @@ Ingress (Traefik):
{{- range .Values.ingress.web.extraHosts }}
https://{{ . }}
{{- end }}
{{- if .Values.ingress.api.enabled }}
{{ join ", " .Values.ingress.api.paths }} go straight to the server; everything else to web.
{{- else }}
Everything goes to web, which proxies /api and /auth onward. Set
ingress.api.enabled=true to route them at the edge instead.
{{- end }}
{{ join ", " .Values.ingress.api.paths }} go to the server; everything else to web.
web proxies nothing, so those paths must be routed here or by a terminator
in front of this ingress.
{{- if .Values.ingress.grpc.enabled }}
- Agents: {{ .Values.ingress.grpc.host }} (gRPC, h2c behind TLS)
Agents dial server.env.grpcHost, currently {{ tpl .Values.server.env.grpcHost . }}.
@@ -70,3 +67,13 @@ or add an Ingress on top of the -web and -server services.
Quick access via port-forward, e.g.:
kubectl port-forward svc/{{ .Release.Name }}-web {{ .Values.web.service.port }}:{{ .Values.web.service.port }}
kubectl port-forward svc/{{ .Release.Name }}-server {{ .Values.server.service.httpPort }}:{{ .Values.server.service.httpPort }}
{{- if not .Values.backup.enabled }}
No backups are scheduled. Vantage encrypts SSH private keys, vault secrets and
SSO client secrets with KEY_ENCRYPTION_KEY, and that key is not stored anywhere
but your own configuration — a database restored without it is permanently
unreadable.
Set backup.enabled, backup.image and backup.pvcName, and store
KEY_ENCRYPTION_KEY somewhere that survives this cluster.
{{- end }}
@@ -72,6 +72,8 @@ both read it.
value: {{ .Values.server.env.proxyAdvertiseHost | quote }}
- name: PROXY_LISTEN_HOST
value: {{ .Values.server.env.proxyListenHost | quote }}
- name: TRUSTED_PROXIES
value: {{ .Values.server.env.trustedProxies | quote }}
{{- if eq .Values.server.env.deploymentType "cloud" }}
- name: VANTAGE_DEPLOYMENT
value: "cloud"
@@ -83,3 +85,18 @@ both read it.
fieldRef:
fieldPath: status.podIP
{{- end -}}
{{/*
vantage.backup.env renders the environment vantagectl needs.
It reads the SAME values the server does rather than taking its own, because a
backup that connected to a different database, or stamped a fingerprint of a
different key, than the deployment it is backing up would be worse than no
backup: it would look like one.
*/}}
{{- define "vantage.backup.env" -}}
- name: MONGO_URI
value: {{ tpl .Values.server.env.mongoUri . | quote }}
- name: KEY_ENCRYPTION_KEY
value: {{ .Values.server.env.keyEncryptionKey | quote }}
{{- end -}}
@@ -0,0 +1,56 @@
{{- if .Values.backup.enabled }}
{{- if not .Values.backup.pvcName }}
{{- fail "backup.enabled requires backup.pvcName: a backup needs somewhere durable to land, and the chart cannot guess where that is" }}
{{- end }}
{{- if not .Values.backup.image }}
{{- fail "backup.enabled requires backup.image: the vantagectl image to run" }}
{{- end }}
apiVersion: batch/v1
kind: CronJob
metadata:
name: {{ include "vantage.fullname" . }}-backup
labels:
{{- include "vantage.labels" . | nindent 4 }}
app.kubernetes.io/component: backup
spec:
schedule: {{ .Values.backup.schedule | quote }}
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: {{ .Values.backup.successfulJobsHistoryLimit }}
failedJobsHistoryLimit: {{ .Values.backup.failedJobsHistoryLimit }}
jobTemplate:
spec:
backoffLimit: 2
template:
metadata:
labels:
{{- include "vantage.labels" . | nindent 12 }}
app.kubernetes.io/component: backup
spec:
restartPolicy: Never
containers:
- name: vantagectl
image: {{ .Values.backup.image | quote }}
args:
- backup
- --out
- /backups
{{- with .Values.backup.exclude }}
- --exclude
- {{ join "," . | quote }}
{{- end }}
env:
# Referenced, never redeclared. A backup job holding its own
# copy of KEY_ENCRYPTION_KEY is a second place for it to be
# wrong, and the fingerprint it stamps would then be a
# fingerprint of the wrong key.
{{- include "vantage.backup.env" . | nindent 16 }}
volumeMounts:
- name: backups
mountPath: /backups
resources:
{{- toYaml .Values.backup.resources | nindent 16 }}
volumes:
- name: backups
persistentVolumeClaim:
claimName: {{ .Values.backup.pvcName | quote }}
{{- end }}
+10 -9
View File
@@ -2,16 +2,14 @@
{{/*
Two hostnames, because the two audiences arrive over different protocols.
Browsers reach the web host. What answers there depends on the path: with
ingress.api.enabled, /api and /auth go straight to the server and everything
else goes to `web`. Without it, everything goes to `web`, which proxies those
prefixes onward itself (web/next.config.ts).
Browsers reach the web host, and the path decides what answers: /api, /auth,
/public, /install* and /update* go to the server, everything else to `web`.
Both work. Routing at the edge is one hop shorter and is what the Nginx Proxy
Manager deployment in front of the Docker install already does, so leaving it
off changes the shape of the request path between the two deployments. It is
still off by default, because turning it on where `web` is the only thing with
a public certificate would strand /api behind a route nobody can reach.
That split is not optional and ingress.api.enabled defaults to true. `web`
proxies nothing — it holds no address for the server at all — so with these
paths absent the UI loads and every request it makes 404s against Next. The
setting remains a value only so an installation terminating in front of this
ingress can route the prefixes itself; it must be routed somewhere.
The web host is normally a wildcard — `*.vantage.example.com` — because that is
the per-tenant instance namespace; APP_ROOT_LABEL resolves the instance from the
@@ -35,6 +33,9 @@ its HTTP port too.
{{- if and .Values.ingress.api.enabled (not $apiPaths) }}
{{- fail "ingress.api.enabled requires at least one path in ingress.api.paths" }}
{{- end }}
{{- if not .Values.ingress.api.enabled }}
{{- fail "ingress.api.enabled=false leaves /api, /auth and /public unrouted: web proxies nothing. Route those prefixes to the server at your own terminator, or leave this enabled." }}
{{- end }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
+4 -6
View File
@@ -38,12 +38,10 @@ spec:
image: "{{ .Values.web.image.repository }}:{{ .Values.web.image.tag }}"
ports:
- containerPort: {{ .Values.web.service.port }}
env:
- name: API_URL
value: {{ tpl .Values.web.env.apiUrl . | quote }}
# /healthz is served by this Next process; /api is rewritten to the
# server, so a probe there would report the backend's health and keep
# passing while this pod was wedged.
# /healthz is served by this Next process. /api never reaches this
# pod at all — the ingress routes it to the server — so there is no
# backend address to configure and no probe here that could report
# the backend's health by accident.
startupProbe:
httpGet:
path: /healthz
+27 -3
View File
@@ -63,6 +63,7 @@ server:
appRootLabel: vantage
proxyAdvertiseHost: "{{ .Release.Name }}-server"
proxyListenHost: "0.0.0.0"
trustedProxies: "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
persistence:
enabled: false
size: 1Gi
@@ -78,8 +79,6 @@ web:
service:
type: ClusterIP
port: 3000
env:
apiUrl: "http://{{ .Release.Name }}-server:8080"
ingress:
enabled: false
@@ -89,11 +88,14 @@ ingress:
web:
host: ""
extraHosts: []
# Not optional: web proxies nothing, so these prefixes reach the server
# only through this ingress. Turning it off serves the UI with a dead API.
api:
enabled: false
enabled: true
paths:
- /api/
- /auth/
- /public/
- /update
- /install
- /update.ps1
@@ -109,3 +111,25 @@ ingress:
certResolver: ""
imagePullSecrets: []
# Scheduled backups.
#
# Off by default, deliberately. A backup with nowhere durable to land is a
# false sense of safety, and the chart cannot know where that is — pvcName
# must name a volume you have decided will outlive the cluster.
#
# There is no restore manifest here on purpose: a restore is an operator
# decision with a confirmation attached, and must never be something a
# `helm upgrade` can trigger. Run one as a `kubectl run` Job with
# --confirm-db.
backup:
enabled: false
schedule: "0 2 * * *"
image: ""
pvcName: ""
# Collections to leave out. Recorded in each archive's manifest, so an
# archive can never claim to be complete when it is not.
exclude: []
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3
resources: {}
+5 -2
View File
@@ -47,6 +47,7 @@ services:
KEY_ENCRYPTION_KEY: ${KEY_ENCRYPTION_KEY:-}
GUACD_ADDR: guacd:4822
PROXY_ADVERTISE_HOST: server
TRUSTED_PROXIES: ${TRUSTED_PROXIES:-10.0.0.0/8,172.16.0.0/12,192.168.0.0/16}
depends_on:
redis:
condition: service_healthy
@@ -59,8 +60,10 @@ services:
restart: unless-stopped
ports:
- 3000:3000
environment:
API_URL: ${API_URL:-http://server:8080}
# No API_URL: web proxies nothing. The reverse proxy in front of this
# deployment must route /api, /auth, /public, /install*, /update* to
# server:8080 and everything else to web:3000. Reaching web:3000
# directly serves the UI and every API call 404s.
depends_on:
- server
volumes:
@@ -0,0 +1,641 @@
<title>Vantage Status Pages</title>
<style>
:root{
color-scheme: dark;
/* Vantage web/ dark tokens, copied verbatim from web/app/globals.css.
This mockup commits to one theme because web/ does. */
--ground:#071628; --panel:#0d2138; --panel-2:#102842; --well:#04101f;
--ink:#e4ecf6; --ink-2:#9fb3ca; --ink-3:#71879f;
--rule:#1e3855; --rule-soft:#172c44;
--accent:#5b9be8; --accent-hover:#7fb2f0; --accent-ink:#04101f;
--up:#4fb484; --pend:#d6a63f; --down:#e2705a; --logo:#7fb2f0;
--shadow:0 1px 0 rgba(0,0,0,.35), 0 20px 44px -26px rgba(0,0,0,.85);
--sans: ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
--mono: ui-monospace, "Cascadia Mono", "SF Mono", "JetBrains Mono", Menlo, Consolas, monospace;
--r:4px;
}
*{box-sizing:border-box;margin:0;padding:0}
body{background:var(--ground);color:var(--ink);font-family:var(--sans);-webkit-font-smoothing:antialiased;line-height:1.5}
a{color:inherit}
:focus-visible{outline:2px solid var(--accent);outline-offset:2px}
.page{max-width:1180px;margin:0 auto;padding:48px 24px 96px;display:flex;flex-direction:column;gap:56px}
.lede h1{font-size:1.6rem;font-weight:800;letter-spacing:-.035em;text-wrap:balance}
.lede p{color:var(--ink-2);font-size:.9rem;max-width:65ch;margin-top:8px}
.cap{font-family:var(--mono);font-size:.68rem;text-transform:uppercase;letter-spacing:.1em;color:var(--ink-2)}
.board{display:flex;flex-direction:column;gap:10px}
.board__head{display:flex;align-items:baseline;justify-content:space-between;gap:16px;flex-wrap:wrap}
.board__route{font-family:var(--mono);font-size:.7rem;color:var(--ink-3)}
.frame{border:1px solid var(--rule);border-radius:var(--r);background:var(--ground);box-shadow:var(--shadow);overflow:hidden}
/* address strip — shows the URL scheme being approved */
.addr{display:flex;align-items:center;gap:10px;background:var(--well);border-bottom:1px solid var(--rule);padding:9px 14px}
.addr__dots{display:flex;gap:5px}
.addr__dots i{width:8px;height:8px;border-radius:999px;background:var(--rule);display:block}
.addr__url{font-family:var(--mono);font-size:.72rem;color:var(--ink-2);overflow-x:auto;white-space:nowrap}
.addr__url b{color:var(--ink);font-weight:600}
.addr__tag{margin-left:auto;font-family:var(--mono);font-size:.62rem;text-transform:uppercase;letter-spacing:.1em;color:var(--ink-3);border:1px solid var(--rule);border-radius:999px;padding:2px 8px;white-space:nowrap}
/* ---------- public status page ---------- */
.pub{padding:40px 28px 32px}
.pub__inner{max-width:720px;margin:0 auto;display:flex;flex-direction:column;gap:28px}
.pub__head{display:flex;align-items:center;gap:14px}
.mark{width:38px;height:38px;border-radius:var(--r);background:var(--panel-2);border:1px solid var(--rule);display:grid;place-items:center;color:var(--logo);font-family:var(--mono);font-weight:700;font-size:.85rem;flex-shrink:0}
.pub__head h2{font-size:1.35rem;font-weight:800;letter-spacing:-.03em}
.pub__head p{color:var(--ink-2);font-size:.85rem;margin-top:2px}
.overall{display:flex;align-items:center;gap:11px;border:1px solid;border-radius:var(--r);padding:14px 16px;font-weight:600;font-size:.95rem}
.overall--down{background:rgba(226,112,90,.10);border-color:rgba(226,112,90,.30);color:var(--down)}
.glyph{width:16px;height:16px;flex-shrink:0}
.banner{border:1px solid var(--rule);background:var(--panel);border-radius:var(--r);padding:12px 14px;font-size:.85rem;color:var(--ink-2);display:flex;gap:10px}
.banner b{color:var(--ink);font-weight:600}
.group{display:flex;flex-direction:column;gap:10px}
.group > .cap{padding-left:2px}
.card{border:1px solid var(--rule);background:var(--panel);border-radius:var(--r)}
.rows > * + *{border-top:1px solid var(--rule-soft)}
.comp{padding:16px}
.comp__top{display:flex;align-items:center;justify-content:space-between;gap:16px;margin-bottom:10px}
.comp__name{font-weight:600;font-size:.92rem}
.state{display:inline-flex;align-items:center;gap:7px;font-size:.78rem;color:var(--ink-2);white-space:nowrap}
.dot{width:7px;height:7px;border-radius:999px;display:block;flex-shrink:0}
.dot--up{background:var(--up)} .dot--down{background:var(--down)}
.dot--maint{background:var(--accent)} .dot--pend{background:var(--pend)}
.dot--none{background:var(--rule)}
.bar{display:flex;gap:2px;overflow-x:auto;padding-bottom:2px}
.bar span{height:26px;width:3px;border-radius:999px;flex:0 0 auto;background:var(--rule)}
.bar .up{background:var(--up)} .bar .down{background:var(--down)}
.bar .maint{background:var(--accent)} .bar .none{background:var(--rule)}
.scale{display:flex;justify-content:space-between;margin-top:7px;font-size:.7rem;color:var(--ink-3)}
.scale b{color:var(--ink-2);font-weight:600;font-variant-numeric:tabular-nums}
.inc{padding:14px 16px}
.inc__top{display:flex;align-items:baseline;justify-content:space-between;gap:14px}
.inc__title{font-weight:600;font-size:.92rem}
.inc__meta{font-size:.75rem;color:var(--ink-3);margin-top:3px}
.inc__affects{font-size:.75rem;color:var(--ink-2);margin-top:5px}
.pill{font-family:var(--mono);font-size:.62rem;text-transform:uppercase;letter-spacing:.1em;border-radius:999px;padding:3px 9px;border:1px solid;white-space:nowrap}
.pill--inv{color:var(--down);border-color:rgba(226,112,90,.35);background:rgba(226,112,90,.10)}
.pill--mon{color:var(--pend);border-color:rgba(214,166,63,.35);background:rgba(214,166,63,.10)}
.pill--res{color:var(--up);border-color:rgba(79,180,132,.35);background:rgba(79,180,132,.10)}
.pill--sch{color:var(--accent);border-color:rgba(91,155,232,.35);background:rgba(91,155,232,.10)}
.pill--draft{color:var(--ink-2);border-color:var(--rule);background:var(--panel-2)}
.pill--live{color:var(--up);border-color:rgba(79,180,132,.35);background:rgba(79,180,132,.10)}
.timeline{margin-top:12px;border-left:1px solid var(--rule);padding-left:14px;display:flex;flex-direction:column;gap:12px}
.tl__head{display:flex;align-items:baseline;gap:9px}
.tl__st{font-family:var(--mono);font-size:.62rem;text-transform:uppercase;letter-spacing:.1em;color:var(--ink-2)}
.tl__at{font-size:.7rem;color:var(--ink-3);font-variant-numeric:tabular-nums}
.tl__body{font-size:.85rem;margin-top:3px;color:var(--ink)}
.pub__foot{text-align:center;font-size:.72rem;color:var(--ink-3);padding-top:6px}
/* ---------- editor ---------- */
.app{display:grid;grid-template-columns:236px 1fr;min-height:660px}
.side{background:var(--panel);border-right:1px solid var(--rule);display:flex;flex-direction:column}
.side__brand{height:64px;display:flex;align-items:center;gap:12px;padding:0 20px;border-bottom:1px solid var(--rule);flex-shrink:0}
.side__brand .mark{width:32px;height:32px;font-size:.78rem}
.side__brand b{font-size:1rem;font-weight:800;letter-spacing:-.035em;display:block;line-height:1.2}
.side__nav{padding:16px 12px;display:flex;flex-direction:column;gap:16px}
.navgrp + .navgrp{border-top:1px solid var(--rule);padding-top:16px}
.navgrp > .cap{padding:0 12px 6px}
.navgrp ul{list-style:none;display:flex;flex-direction:column;gap:4px}
.navgrp a{position:relative;display:flex;align-items:center;gap:12px;border-radius:var(--r);padding:9px 12px;font-size:.85rem;font-weight:500;color:var(--ink-2);text-decoration:none}
.navgrp a:hover{background:var(--panel-2);color:var(--ink)}
.navgrp a.on{background:var(--panel-2);color:var(--ink);font-weight:600}
.navgrp a.on::before{content:"";position:absolute;left:0;top:4px;bottom:4px;width:2px;border-radius:999px;background:var(--accent)}
.navgrp svg{width:16px;height:16px;flex-shrink:0;opacity:.9}
.main{padding:26px 28px 36px;display:flex;flex-direction:column;gap:22px;min-width:0}
.back{font-size:.78rem;color:var(--ink-2);text-decoration:none;display:inline-flex;gap:6px;align-items:center}
.back:hover{color:var(--ink)}
.phead{display:flex;align-items:flex-start;justify-content:space-between;gap:20px;flex-wrap:wrap}
.phead h2{font-size:1.3rem;font-weight:800;letter-spacing:-.03em}
.record{display:flex;align-items:center;gap:8px;margin-top:6px}
.record code{font-family:var(--mono);font-size:.72rem;color:var(--ink-2);background:var(--well);border:1px solid var(--rule);border-radius:var(--r);padding:3px 8px}
.copy{background:none;border:0;color:var(--ink-3);cursor:pointer;font-size:.72rem;font-family:var(--mono)}
.copy:hover{color:var(--accent)}
.acts{display:flex;gap:9px;flex-wrap:wrap}
.btn{font-size:.82rem;font-weight:600;border-radius:var(--r);padding:8px 14px;border:1px solid var(--rule);background:var(--panel);color:var(--ink);cursor:pointer;text-decoration:none;display:inline-flex;align-items:center;gap:7px}
.btn:hover{background:var(--panel-2)}
.btn--p{background:var(--accent);border-color:var(--accent);color:var(--accent-ink)}
.btn--p:hover{background:var(--accent-hover)}
.panel{border:1px solid var(--rule);background:var(--panel);border-radius:var(--r)}
.panel__head{display:flex;align-items:center;justify-content:space-between;gap:14px;padding:13px 16px;border-bottom:1px solid var(--rule)}
.panel__head h3{font-size:.95rem;font-weight:700}
.panel__head p{font-size:.76rem;color:var(--ink-3);margin-top:2px}
.panel__body{padding:16px;display:flex;flex-direction:column;gap:16px}
.fields{display:grid;grid-template-columns:repeat(auto-fit,minmax(230px,1fr));gap:14px}
.field{display:flex;flex-direction:column;gap:6px;min-width:0}
.field > label{font-size:.76rem;font-weight:600;color:var(--ink-2)}
.field .hint{font-size:.72rem;color:var(--ink-3)}
.in{background:var(--well);border:1px solid var(--rule);border-radius:var(--r);padding:8px 11px;font:inherit;font-size:.85rem;color:var(--ink);width:100%}
.in::placeholder{color:var(--ink-3)}
.in:focus{outline:2px solid var(--accent);outline-offset:-1px;border-color:var(--accent)}
.in--mono{font-family:var(--mono);font-size:.8rem}
.toggle{display:flex;align-items:center;justify-content:space-between;gap:16px;background:var(--panel-2);border:1px solid var(--rule);border-radius:var(--r);padding:12px 14px}
.toggle p{font-size:.76rem;color:var(--ink-3);margin-top:3px;max-width:52ch}
.toggle b{font-size:.85rem}
.sw{width:38px;height:21px;border-radius:999px;background:var(--up);border:0;position:relative;cursor:pointer;flex-shrink:0}
.sw::after{content:"";position:absolute;top:2px;left:19px;width:17px;height:17px;border-radius:999px;background:var(--accent-ink)}
.sw[aria-checked="false"]{background:var(--rule)}
.sw[aria-checked="false"]::after{left:2px;background:var(--ink-3)}
.sect{border:1px solid var(--rule);border-radius:var(--r);background:var(--panel-2)}
.sect__head{display:flex;align-items:center;gap:10px;padding:10px 12px;border-bottom:1px solid var(--rule)}
.sect__head .in{max-width:220px}
.sect__head .rm{margin-left:auto}
.rm{background:none;border:0;color:var(--ink-3);font-size:.75rem;cursor:pointer;font-family:var(--mono)}
.rm:hover{color:var(--down)}
.entry{display:grid;grid-template-columns:1fr 1fr auto;gap:12px;align-items:center;padding:11px 12px}
.entry + .entry{border-top:1px solid var(--rule-soft)}
.entry__mon{display:flex;flex-direction:column;gap:2px;min-width:0}
.entry__mon b{font-size:.84rem;font-weight:600}
.entry__mon span{font-family:var(--mono);font-size:.68rem;color:var(--ink-3)}
.adds{display:flex;gap:9px;flex-wrap:wrap;padding:0 12px 12px}
.inc-row{display:flex;align-items:flex-start;justify-content:space-between;gap:14px;padding:13px 14px}
.inc-row + .inc-row{border-top:1px solid var(--rule-soft)}
.inc-row__l{min-width:0}
.inc-row__l b{font-size:.88rem;font-weight:600;display:block}
.inc-row__l span{font-size:.74rem;color:var(--ink-3)}
.inc-row__r{display:flex;align-items:center;gap:9px;flex-shrink:0}
.notes{border-top:1px solid var(--rule);padding-top:14px;display:flex;flex-direction:column;gap:7px}
.notes li{font-size:.82rem;color:var(--ink-2);display:flex;gap:10px;list-style:none}
.notes li b{color:var(--ink);font-weight:600}
.notes .k{font-family:var(--mono);font-size:.66rem;text-transform:uppercase;letter-spacing:.1em;color:var(--ink-3);flex:0 0 76px;padding-top:2px}
@media (max-width:820px){
.app{grid-template-columns:1fr}
.side{display:none}
.entry{grid-template-columns:1fr}
.page{padding:32px 16px 64px}
}
</style>
<div class="page">
<header class="lede">
<p class="cap" style="margin-bottom:10px">Vantage · status pages · mockup for approval</p>
<h1>Two screens: what the public sees, and what the operator edits</h1>
<p>Drawn with the real <code style="font-family:var(--mono);font-size:.85em">web/</code> dark tokens and the existing sidebar idioms, so what gets approved here is what gets built. The public page is shown mid-incident rather than all-green, because that is the state it exists for.</p>
</header>
<!-- ================= PUBLIC ================= -->
<section class="board">
<div class="board__head">
<p class="cap">1 · Public status page</p>
<p class="board__route">web/app/status/[pageId]/page.tsx · no auth, no sidebar</p>
</div>
<div class="frame">
<div class="addr">
<span class="addr__dots"><i></i><i></i><i></i></span>
<span class="addr__url">https://acme.vantage.example.com<b>/status/api</b></span>
<span class="addr__tag">signed out</span>
</div>
<div class="pub">
<div class="pub__inner">
<div class="pub__head">
<div class="mark">AC</div>
<div>
<h2>Acme Platform Status</h2>
<p>Live availability for the Acme API and dashboard.</p>
</div>
</div>
<div class="overall overall--down">
<svg class="glyph" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.6" aria-hidden="true">
<circle cx="8" cy="8" r="6.4"/><path d="M8 4.8v3.6M8 11.1h.01" stroke-linecap="round"/>
</svg>
Service disruption
</div>
<div class="banner">
<svg class="glyph" viewBox="0 0 16 16" fill="none" stroke="var(--accent)" stroke-width="1.6" aria-hidden="true" style="margin-top:2px">
<circle cx="8" cy="8" r="6.4"/><path d="M8 7.4v3.8M8 5.1h.01" stroke-linecap="round"/>
</svg>
<span><b>Europe region only.</b> US and APAC are unaffected. Follow this page for updates.</span>
</div>
<div class="group">
<p class="cap">Active</p>
<div class="card">
<article class="inc">
<div class="inc__top">
<div>
<p class="inc__title">Elevated error rates on database writes</p>
<p class="inc__meta">Started 24 Aug 2026, 09:12 UTC</p>
</div>
<span class="pill pill--mon">monitoring</span>
</div>
<p class="inc__affects">Affects Primary database, Public API</p>
<div class="timeline">
<div>
<div class="tl__head"><span class="tl__st">monitoring</span><span class="tl__at">11:40 UTC</span></div>
<p class="tl__body">Failover completed. Write latency is back to normal and we are watching for recurrence before calling this resolved.</p>
</div>
<div>
<div class="tl__head"><span class="tl__st">identified</span><span class="tl__at">09:48 UTC</span></div>
<p class="tl__body">A failing disk on the primary database node is causing write timeouts. Failover to the standby node is in progress.</p>
</div>
<div>
<div class="tl__head"><span class="tl__st">investigating</span><span class="tl__at">09:15 UTC</span></div>
<p class="tl__body">We are investigating a rise in write errors affecting the API.</p>
</div>
</div>
</article>
</div>
</div>
<div class="group">
<p class="cap">Scheduled maintenance</p>
<div class="card">
<article class="inc">
<div class="inc__top">
<div>
<p class="inc__title">Object storage capacity upgrade</p>
<p class="inc__meta">31 Aug 2026, 02:00 04:00 UTC</p>
</div>
<span class="pill pill--sch">scheduled</span>
</div>
<p class="inc__affects">Affects Object storage</p>
</article>
</div>
</div>
<div class="group">
<p class="cap">API</p>
<div class="card rows">
<div class="comp" data-bar="api" data-state="down">
<div class="comp__top">
<span class="comp__name">Public API</span>
<span class="state"><i class="dot dot--down"></i>Down</span>
</div>
<div class="bar"></div>
<div class="scale"><span>90 days ago</span><span><b>99.81%</b> uptime</span><span>Today</span></div>
</div>
<div class="comp" data-bar="hooks" data-state="up">
<div class="comp__top">
<span class="comp__name">Webhook delivery</span>
<span class="state"><i class="dot dot--up"></i>Operational</span>
</div>
<div class="bar"></div>
<div class="scale"><span>90 days ago</span><span><b>99.99%</b> uptime</span><span>Today</span></div>
</div>
</div>
</div>
<div class="group">
<p class="cap">Web</p>
<div class="card rows">
<div class="comp" data-bar="dash" data-state="up">
<div class="comp__top">
<span class="comp__name">Dashboard</span>
<span class="state"><i class="dot dot--up"></i>Operational</span>
</div>
<div class="bar"></div>
<div class="scale"><span>90 days ago</span><span><b>99.97%</b> uptime</span><span>Today</span></div>
</div>
</div>
</div>
<div class="group">
<p class="cap">Data</p>
<div class="card rows">
<div class="comp" data-bar="db" data-state="down">
<div class="comp__top">
<span class="comp__name">Primary database</span>
<span class="state"><i class="dot dot--down"></i>Down</span>
</div>
<div class="bar"></div>
<div class="scale"><span>90 days ago</span><span><b>99.62%</b> uptime</span><span>Today</span></div>
</div>
<div class="comp" data-bar="obj" data-state="maint">
<div class="comp__top">
<span class="comp__name">Object storage</span>
<span class="state"><i class="dot dot--maint"></i>Maintenance</span>
</div>
<div class="bar"></div>
<div class="scale"><span>90 days ago</span><span><b>99.94%</b> uptime</span><span>Today</span></div>
</div>
<div class="comp" data-bar="new" data-state="up">
<div class="comp__top">
<span class="comp__name">Search index</span>
<span class="state"><i class="dot dot--up"></i>Operational</span>
</div>
<div class="bar"></div>
<div class="scale"><span>90 days ago</span><span><b>100.00%</b> uptime</span><span>Today</span></div>
</div>
</div>
</div>
<div class="group">
<p class="cap">Past incidents</p>
<div class="card rows">
<article class="inc">
<div class="inc__top">
<div>
<p class="inc__title">Public API unavailable</p>
<p class="inc__meta">2 Aug 2026, 14:02 UTC — resolved 14:19 UTC</p>
</div>
<span class="pill pill--res">resolved</span>
</div>
<p class="inc__affects">Affects Public API</p>
</article>
<article class="inc">
<div class="inc__top">
<div>
<p class="inc__title">Slow dashboard loads in Europe</p>
<p class="inc__meta">17 Jul 2026, 08:30 UTC — resolved 10:05 UTC</p>
</div>
<span class="pill pill--res">resolved</span>
</div>
<p class="inc__affects">Affects Dashboard</p>
</article>
</div>
</div>
<p class="pub__foot">Updated 24 Aug 2026, 11:58 UTC · refreshes every 60 seconds</p>
</div>
</div>
</div>
<ul class="notes">
<li><span class="k">Redacted</span><span>No target URL, host, port or failure text anywhere on this page. <b>Search index</b> shows the no-data tail as grey cells rather than claiming 100% for days before it existed.</span></li>
<li><span class="k">Maintenance</span><span><b>Object storage</b> reads as Maintenance, not Down — but its uptime figure is untouched. The window changes how it is drawn, never what the numbers say.</span></li>
<li><span class="k">Colour</span><span>Every state carries a word and a shape as well as a hue. The page is readable with colour vision differences and in greyscale print.</span></li>
</ul>
</section>
<!-- ================= EDITOR ================= -->
<section class="board">
<div class="board__head">
<p class="cap">2 · Status page editor</p>
<p class="board__route">web/app/(app)/status-pages/[pageId]/page.tsx · owner or admin</p>
</div>
<div class="frame">
<div class="app">
<aside class="side">
<div class="side__brand">
<div class="mark">V</div>
<div>
<b>Vantage</b>
<span class="cap">Acme Ltd</span>
</div>
</div>
<nav class="side__nav">
<div class="navgrp">
<p class="cap">Fleet</p>
<ul>
<li><a href="#"><svg viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5"><rect x="2" y="3" width="12" height="4" rx="1"/><rect x="2" y="9" width="12" height="4" rx="1"/></svg>Servers</a></li>
<li><a href="#"><svg viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5"><rect x="2.5" y="2.5" width="11" height="11" rx="1.5"/><path d="M6 6h4v4H6z"/></svg>Workloads</a></li>
<li><a href="#"><svg viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round"><path d="M1.5 8.5h3l2-4 3 7 2-3h3"/></svg>Monitors</a></li>
</ul>
</div>
<div class="navgrp">
<p class="cap">Access</p>
<ul>
<li><a href="#"><svg viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5"><circle cx="5.5" cy="8" r="3"/><path d="M8.5 8h6M12 8v2.5"/></svg>SSH Keys</a></li>
<li><a href="#"><svg viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5"><rect x="3" y="7" width="10" height="6.5" rx="1.5"/><path d="M5.5 7V5a2.5 2.5 0 015 0v2"/></svg>Secrets</a></li>
</ul>
</div>
<div class="navgrp">
<p class="cap">Instance</p>
<ul>
<li><a href="#" class="on"><svg viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5"><rect x="2" y="3" width="12" height="10" rx="1.5"/><path d="M4.5 10.5v-2M8 10.5v-4M11.5 10.5v-3" stroke-linecap="round"/></svg>Status Pages</a></li>
<li><a href="#"><svg viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5"><path d="M3 3h10v10H3z"/><path d="M5.5 6.5h5M5.5 9.5h3"/></svg>Audit Log</a></li>
<li><a href="#"><svg viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5"><circle cx="8" cy="8" r="2.2"/><path d="M8 1.8v1.6M8 12.6v1.6M14.2 8h-1.6M3.4 8H1.8"/></svg>Settings</a></li>
</ul>
</div>
</nav>
</aside>
<div class="main">
<a class="back" href="#">
<svg class="glyph" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round"><path d="M9.5 3.5L5 8l4.5 4.5"/></svg>
All status pages
</a>
<div class="phead">
<div>
<h2>Acme Platform Status</h2>
<div class="record">
<code>acme.vantage.example.com/status/api</code>
<button class="copy" type="button">copy</button>
</div>
</div>
<div class="acts">
<a class="btn" href="#">
<svg class="glyph" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round"><path d="M6.5 3.5h6v6M12.5 3.5L7 9"/><path d="M11 10.5v2h-8v-8h2"/></svg>
View page
</a>
<button class="btn btn--p" type="button">Save changes</button>
</div>
</div>
<div class="panel">
<div class="panel__head">
<div>
<h3>Details</h3>
<p>What visitors see at the top of the page.</p>
</div>
<span class="pill pill--live">published</span>
</div>
<div class="panel__body">
<div class="toggle">
<div>
<b>Published</b>
<p>Anyone with the link can read this page. Unpublished pages return not found, so you can compose before announcing.</p>
</div>
<button class="sw" type="button" role="switch" aria-checked="true" aria-label="Published"></button>
</div>
<div class="fields">
<div class="field">
<label for="f-title">Title</label>
<input class="in" id="f-title" value="Acme Platform Status">
</div>
<div class="field">
<label for="f-id">Page address</label>
<input class="in in--mono" id="f-id" value="api" disabled>
<span class="hint">Fixed once created — the link is already out there.</span>
</div>
<div class="field">
<label for="f-desc">Description</label>
<input class="in" id="f-desc" value="Live availability for the Acme API and dashboard.">
</div>
<div class="field">
<label for="f-logo">Logo URL</label>
<input class="in in--mono" id="f-logo" placeholder="https://acme.example.com/logo.svg">
</div>
</div>
<div class="field">
<label for="f-ban">Notice</label>
<input class="in" id="f-ban" value="Europe region only. US and APAC are unaffected. Follow this page for updates.">
<span class="hint">Shown above everything else. Clear it to remove the notice.</span>
</div>
</div>
</div>
<div class="panel">
<div class="panel__head">
<div>
<h3>Components</h3>
<p>Monitors grouped for the public page. Grouping here is separate from the groups on Monitors.</p>
</div>
<button class="btn" type="button">Add section</button>
</div>
<div class="panel__body">
<div class="sect">
<div class="sect__head">
<input class="in" value="API" aria-label="Section name">
<button class="rm" type="button">remove section</button>
</div>
<div class="entry">
<div class="entry__mon">
<b>prod-api-eu-health</b>
<span>http · every 30s</span>
</div>
<input class="in" value="Public API" aria-label="Public name for prod-api-eu-health">
<button class="rm" type="button">remove</button>
</div>
<div class="entry">
<div class="entry__mon">
<b>hooks-dispatch-probe</b>
<span>http · every 60s</span>
</div>
<input class="in" value="Webhook delivery" aria-label="Public name for hooks-dispatch-probe">
<button class="rm" type="button">remove</button>
</div>
<div class="adds"><button class="btn" type="button">Add monitor</button></div>
</div>
<div class="sect">
<div class="sect__head">
<input class="in" value="Data" aria-label="Section name">
<button class="rm" type="button">remove section</button>
</div>
<div class="entry">
<div class="entry__mon">
<b>pg-primary-10-0-0-5</b>
<span>tcp · every 30s</span>
</div>
<input class="in" value="Primary database" aria-label="Public name for pg-primary-10-0-0-5">
<button class="rm" type="button">remove</button>
</div>
<div class="entry">
<div class="entry__mon">
<b>minio-gw</b>
<span>http · every 60s</span>
</div>
<input class="in" placeholder="minio-gw" aria-label="Public name for minio-gw">
<button class="rm" type="button">remove</button>
</div>
<div class="adds"><button class="btn" type="button">Add monitor</button></div>
</div>
</div>
</div>
<div class="panel">
<div class="panel__head">
<div>
<h3>Incidents</h3>
<p>Written by you. Outages Vantage detects appear on the page automatically.</p>
</div>
<div class="acts">
<button class="btn" type="button">Schedule maintenance</button>
<button class="btn btn--p" type="button">Open incident</button>
</div>
</div>
<div>
<div class="inc-row">
<div class="inc-row__l">
<b>Elevated error rates on database writes</b>
<span>Opened 09:12 UTC · 3 updates · affects Primary database, Public API</span>
</div>
<div class="inc-row__r">
<span class="pill pill--mon">monitoring</span>
<button class="btn" type="button">Post update</button>
</div>
</div>
<div class="inc-row">
<div class="inc-row__l">
<b>Object storage capacity upgrade</b>
<span>31 Aug, 02:0004:00 UTC · affects Object storage</span>
</div>
<div class="inc-row__r">
<span class="pill pill--sch">scheduled</span>
<button class="btn" type="button">Edit</button>
</div>
</div>
<div class="inc-row">
<div class="inc-row__l">
<b>Slow dashboard loads in Europe</b>
<span>17 Jul · resolved after 1h 35m · affects Dashboard</span>
</div>
<div class="inc-row__r">
<span class="pill pill--res">resolved</span>
<button class="btn" type="button">Edit</button>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<ul class="notes">
<li><span class="k">Naming</span><span>The monitor's own identifier stays visible on the left; the <b>public name</b> is a separate field beside it. An empty field falls back to the identifier, which the placeholder shows — so publishing an internal name is always a visible choice.</span></li>
<li><span class="k">Address</span><span>The page address is fixed after creation and the record line carries the whole URL, click to copy. It is what gets pasted into a support article.</span></li>
<li><span class="k">Copy</span><span>Buttons name the outcome: <b>Open incident</b>, <b>Post update</b>, <b>Schedule maintenance</b> — the same words the public timeline then shows.</span></li>
</ul>
</section>
</div>
<script>
// 90 daily cells per component. Seeded rather than random so the mockup is
// stable between reloads and reviewers are looking at the same picture.
const PATTERNS = {
api: { downs: [2, 22], maint: [], noData: 0 },
hooks: { downs: [], maint: [], noData: 0 },
dash: { downs: [38], maint: [], noData: 0 },
db: { downs: [0, 1, 12, 13, 47], maint: [], noData: 0 },
obj: { downs: [61], maint: [0], noData: 0 },
new: { downs: [], maint: [], noData: 61 }
};
document.querySelectorAll(".comp").forEach((comp) => {
const p = PATTERNS[comp.dataset.bar];
const bar = comp.querySelector(".bar");
const frag = document.createDocumentFragment();
for (let i = 89; i >= 0; i--) {
const cell = document.createElement("span");
let cls = "up";
if (i >= 90 - p.noData) cls = "none";
else if (p.maint.includes(i)) cls = "maint";
else if (p.downs.includes(i)) cls = "down";
cell.className = cls;
cell.title = cls === "none" ? "no data" : cls === "maint" ? "maintenance" : cls === "down" ? "outage" : "operational";
frag.appendChild(cell);
}
bar.appendChild(frag);
});
</script>
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,286 @@
# Public status pages
Date: 2026-08-24
## Goal
Let an operator publish one or more public status pages from a Vantage
instance, at `<slug>.vantage.<tld>/status/<page-id>`, showing the state of any
monitors they choose, plus incidents and maintenance windows they author by
hand. The pages are completely public: no session, no token, no login.
Out of scope, deliberately:
- **Custom domains** (`status.customer.com`). Needs certificate provisioning and
a host-to-page lookup that bypasses `hostSlug` entirely. Its own sub-project.
- **Per-page themes.** `web/` is locked dark by design and a public page is not
the place to break that.
- **Subscriber notifications.** Email or webhook on incident updates is a
notification subsystem, and one already exists for monitors; wiring the two
together is a separate decision.
- **SLA reporting.** Uptime percentages are shown; contractual SLA calculation
with credits and exclusions is a different product.
## Current state
Everything needed to draw a status page already exists and is already scoped by
instance:
| Data | Where |
| --- | --- |
| Monitor identity and live state | `models.Monitor`, `Monitor.State` |
| Outage records | `models.Incident`, opened when a monitor flips down |
| Hourly uptime history | `models.Rollup` (`monitor_rollups`) |
| Sub-hour history | `models.MonitorSample`, TTL-expired |
| Instance from hostname | `auth.InstanceFromHost`, 60s cached |
Three things do not exist: any concept of a page, any operator-authored
incident, and any unauthenticated read path. The third is the constraint that
shapes the rest — every route under `/api` carries `auth.Middleware`,
`RequireScopes`, `RateLimitTokens` and `RequireActiveLicense` by virtue of where
it is mounted, and `AssertScopeMapComplete` fails boot on an `/api` route with
no scope entry.
## Approach
Two new collections hold the page and the authored incidents. A single
assembly function reads them alongside the existing monitor data and emits a
purpose-built public struct. The public route is mounted outside `/api`, is
cached in Redis, and is rate limited per client address.
The redaction boundary is the assembly function, and it is the security
property of this whole feature.
## Data model
Both collections carry `instance_id` and both must be added to
`services.ScopedCollections`, or their rows outlive a deleted instance.
### `status_pages`
One document per page. It is read whole, always, so its structure is embedded
rather than joined: one page is one Mongo read is one cache fill.
```
_id, instance_id
page_id // operator-chosen slug, [a-z0-9-], 3-40 chars
title, description, logo_url
published bool
banner { enabled, level, text }
sections [ { name, entries: [ { monitor_id, display_name } ] } ]
created_at, updated_at
```
Unique index on `(instance_id, page_id)`. The slug is operator-chosen rather
than random because it is a URL handed to customers and printed on support
pages; a random identifier would be unguessable and unmemorable in equal
measure.
`published` exists so a page can be composed before anyone sees it. An
unpublished page answers the same 404 as a page that does not exist — a
distinct 403 would confirm it exists.
Sections are page-local and unrelated to `Monitor.Group`, which is a display
label on the authenticated monitors list. One monitor may appear under "API" on
the customer page and "Edge" on the partner page, under two different display
names. That is the point of the override: a monitor's internal name is often
not a name you want published.
The banner is three fields on the page rather than a collection, because it is
one string with no lifecycle.
### `status_incidents`
Manual incidents and maintenance windows share one shape, because they share a
timeline, an impact and a set of affected components; splitting them into two
collections would duplicate all three.
```
_id, instance_id, incident_id
page_ids []string // which pages show it
kind "incident" | "maintenance"
title
impact // none | minor | major | critical
affected_monitors []string // monitor_ids
status // incident: investigating | identified | monitoring | resolved
// maintenance: scheduled | in_progress | completed
scheduled_start, scheduled_end // maintenance only
updates [ { at, status, body, author } ]
started_at, resolved_at, created_at, updated_at
```
Updates are embedded for the same reason sections are: they are few, and they
are never read apart from their incident.
`page_ids` is explicit rather than derived from `affected_monitors`. Deriving it
would be less to fill in, but adding a monitor to a page later would
retroactively republish old incidents to a new audience. An operator publishing
to customers chooses that audience.
### Auto-incidents are derived, never copied
The existing `incidents` collection remains the only writer for
monitor-detected outages. The public snapshot derives them at assembly time:
filter to the monitors on the page, last 90 days, render as display name, start,
end and duration.
`Incident.Cause` is dropped. It is where `dial tcp 10.0.0.5:5432: connect
refused` lives.
Copying auto-incidents into `status_incidents` would be a second writer for the
same fact, arriving by a different route with its own opportunity to disagree —
the same argument that keeps `RefreshWorkloadsCmd` from returning workloads
inline.
### Maintenance does not rewrite uptime
During a maintenance window, affected components render as "under maintenance"
rather than down. The uptime percentage and the history bar still come from the
rollups, unmodified.
Rollups are the durable record. Bending them so a page looks better is a lie
pointed the other way, and the operator who later asks "what was our actual
availability" gets an answer that was edited for publication.
## The redaction boundary
`services.BuildStatusSnapshot(instanceID, pageID)` is the only function that
reads `monitors`, `incidents`, `monitor_rollups` and `status_incidents` on
behalf of an anonymous caller, and it emits a purpose-built struct.
**`models.Monitor` is never marshalled to a public caller.** Target URL, host,
port, method, keyword, `state.message`, `state.cert_expiry_at` and
`channel_ids` all stay behind the boundary. A field added to `Monitor` next year
is private by default rather than published by accident.
What the snapshot contains, per entry: display name, current status, uptime
percentage over the last 90 days, and a 90-day history bar of one cell per day.
A cell is up, down, under maintenance, or no-data — `no-data` for days before
the monitor existed, which is a distinct thing from a day it was down. No
latency, no addresses, no failure text.
## Public read path
```
GET /public/status/:pageId
```
Mounted on the gin root, not under `apiGroup`. Putting it under `/api` would
require exempting it from authentication, scope enforcement, token rate
limiting and the licence gate — four holes, each one something a later change
can widen. Outside `/api` it needs none of them.
The instance is resolved from the request host through `auth.InstanceFromHost`.
A host with no instance label, an unknown slug, an unknown page and an
unpublished page all answer **404**, identically.
### The feature gate answers 200, not 403
Status pages are gated by a new `license.FeatureStatusPages = "status_pages"`,
on both the authoring routes and the public read.
The public side checks inline rather than through `RequireFeature`, which
aborts with a 403 JSON body. A public page needs to render an explanation:
```json
{ "available": false, "reason": "feature_unavailable", "title": "Acme Status" }
```
`reason` is `feature_unavailable` when the tier does not include the feature and
`licence_inactive` when the licence has lapsed. The title is included so the
page does not look broken; nothing else is.
**This is not only a server change.** The feature must be added to admin's
`plans` rows per `(deployment, tier)`, or every instance reads it as absent and
the feature ships dark.
### Cache
Redis key `vantage:status:<instance_id>:<page_id>` holds the assembled JSON with
a 30-second TTL. N visitors cost one Mongo read regardless of traffic.
Authoring writes delete the key, so an operator posting an incident update sees
it immediately rather than wondering for half a minute whether it saved.
Redis rather than Next ISR because with `replicaCount > 1` each `web` pod would
cache separately and two visitors would see different states during an incident.
### Rate limit
Per client address, one-minute fixed window, 120 requests, 429 with
`Retry-After` — the same shape as `RateLimitTokens`, including its most
important property: **when Redis is unavailable, allow rather than deny.** A
status page must survive the outage it exists to report.
### Trusted proxies
Nothing calls `r.SetTrustedProxies`, so gin trusts every proxy and
`c.ClientIP()` takes `X-Forwarded-For` verbatim. That is spoofable per request,
which makes a per-address limiter decorative.
This has not mattered so far because `ClientIP()` is only used for audit
strings. It matters now, so this work adds a trusted-proxy configuration and
sets it at boot. Without it the rate limit is theatre.
## Authoring API
Under `/api`, owner or admin, behind `RequireFeature("status_pages")`, every
mutation audited:
```
GET,POST /status-pages
GET,PUT,DELETE /status-pages/:pageId
GET,POST /status-pages/:pageId/incidents
PUT,DELETE /status-pages/:pageId/incidents/:incidentId
POST /status-pages/:pageId/incidents/:incidentId/updates
```
This adds a ninth scope resource, `status:read` and `status:write`. The entries
are required, not optional: `AssertScopeMapComplete` fails boot on an `/api`
route with no scope entry, which is exactly the safeguard working.
Handlers need `@…` annotations and `openapi.json` must be regenerated and
committed — `server-deploy.yml` runs `git diff --exit-code` against the
committed copy, so a handler whose annotation drifted fails CI.
## Frontend
`web/app/status/[pageId]/page.tsx`, **outside the `(app)` route group**, so it
inherits no sidebar, no session fetch and no auth redirect. Server-rendered
against the Go endpoint, with a client refresh every 60 seconds.
`web/next.config.ts` gains a `/public/:path*` rewrite so that client refresh
reaches the server.
The page stays dark, like the rest of `web/`, and carries no hex values — the
existing token palette covers every state it needs.
Authoring UI at `/status-pages` inside `(app)`, in the **Instance** sidebar
group. It is `adminOnly`, and since the whole group is, a member sees the group
disappear entirely rather than a labelled section with nothing under it.
## Testing
The snapshot tests are the ones that matter, because they are the redaction
boundary made executable:
- `BuildStatusSnapshot` output contains no target URL or host, no
`state.message`, no `incident.cause`, no `channel_ids`, no latency.
- A monitor on no page never appears in any page's snapshot.
- An unpublished page and an unknown page both 404.
- Feature absent and licence inactive both return 200 with `available: false`
and the matching `reason`.
- A cache hit performs no Mongo read; an authoring write invalidates the key.
- Slug validation: character set, length, uniqueness within an instance.
- Maintenance window renders the component as under maintenance while leaving
the uptime percentage untouched.
## Migration and rollout
No migration is needed — both collections are new and absent means empty. Index
builders follow the `EnsureWorkflowIndexes` precedent and warn rather than being
fatal: a missing index on a small collection degrades to a scan, which is no
reason to refuse to serve the fleet.
The feature ships dark until the `status_pages` feature is added to the plan
rows in admin.
@@ -0,0 +1,330 @@
# Control plane backup and restore
Date: 2026-09-07
Status: approved, ready for implementation planning
## Problem
Vantage has no backup story. A self-hosted deployment holds its entire state in
MongoDB and encrypts the sensitive half of it — SSH private keys, key
passphrases, vault secrets, OIDC client secrets, RDP and VNC credentials — with
AES-256-GCM under a single 32-byte key supplied as the `KEY_ENCRYPTION_KEY`
environment variable.
That key is a bare value. It carries no identifier, is not wrapped, and is not
recorded anywhere alongside the data it protects. Restoring a database without
it produces a control plane whose every secret is permanently unreadable, and
nothing in the product tells an operator this before it happens.
`mongodump` exists and operators can use it, but it says nothing about the
encryption key, so the most common way to lose everything is to hold a perfectly
good database dump and no key.
## Goal
A standalone command-line tool that backs up and restores a whole Vantage
deployment, and that makes the key relationship impossible to get wrong by
accident.
Explicitly not a goal: point-in-time recovery, incremental backups, built-in
storage backends, encryption of the archive itself, per-tenant export, and
backups scheduled from inside the server. Each is a separate decision and
several are better served by tools the operator already has.
## Design
### Scope of a backup
One backup covers one MongoDB database: every collection in it, whether or not
that collection is tenant-scoped. A deployment-level disaster recovery tool that
skipped `migrations` or `vulndb_meta` would restore a database the server
refuses to boot against.
Collections are enumerated live with `ListCollectionNames` rather than read from
a hardcoded list. This is the opposite choice to `services.ScopedCollections`,
and deliberately so: that list can afford to be hand-maintained because
`AssertNoScopedCollectionMissed` fails boot when it drifts. A backup tool has no
such assertion available, so a second hand-maintained registry would drift
silently and the first symptom would be a restore missing a collection nobody
noticed was added.
`--exclude` accepts collection names for the volume-heavy ones —
`workflow_log_lines`, `monitor_samples`, `audit_logs`. Whatever is excluded is
recorded in the manifest, so an archive can never claim to be complete when it
is not.
Redis is not backed up. It holds sessions only; losing it logs everyone out and
nothing else, which is already the documented behaviour. The restore output says
so explicitly rather than leaving an operator to wonder.
### Where the code lives
Two units.
`shared/backup/` holds the logic: dump, restore, manifest construction, archive
reading and writing, and key fingerprinting. It depends on the MongoDB driver
and the standard library, and on no CLI framework. Keeping it in `shared/` and
free of cobra is what lets `server` import it later if backups scheduled from
inside the control plane are ever built, without pulling a command-line parser
into the server binary.
`vantagectl/` is a new module in `go.work`, importing `shared`. It holds the
cobra command tree and nothing else. A separate module rather than a package
under `shared/` because adding cobra to `shared/go.mod` would put cobra and
pflag into the module graph of `server`, `admin` and `sitesvc`, none of which
use them. Binaries are unaffected — Go links only what is imported — but three
`go.sum` files would grow and three CI builds would fetch a dependency they do
not need. `agent/` is already a separate module for the same reason.
The tool imports nothing from `server/`. No `db.Col()`, no `services`, no config
loader, and it never dials the REST or gRPC API. It needs only network reach to
MongoDB, a database name, and `KEY_ENCRYPTION_KEY` in its own environment. This
is what lets it run against a control plane that is down, half-migrated, or was
deleted an hour ago — which is the only condition under which anyone runs a
restore.
### Dump implementation
The dump is written against the MongoDB driver, not by shelling out to
`mongodump`.
Two reasons. `server`'s runtime image is `scratch` and carries no shell and no
mongo tools, so a wrapper would depend on a matching `mongodump` version being
installed on whatever host runs the tool. And the manifest must be written by
the same process that read the documents, or the fingerprint and per-collection
checksums are claims about data the writer never saw.
The cost is that BSON round-tripping is ours to get right. Documents are written
as raw BSON exactly as the driver returns them, without an intermediate map, so
`ObjectId`, `Decimal128`, `DateTime`, binary subtypes and nulls survive
unchanged. A round-trip test asserting byte-equal BSON is the guard.
### Archive format
A gzipped tar named `vantage-backup-<db>-<RFC3339>.tar.gz`:
```
manifest.json
collections/<name>.bson concatenated raw BSON documents
indexes/<name>.json index specifications
```
`manifest.json` carries:
| Field | Purpose |
| --- | --- |
| `format_version` | Currently `1`. Restore refuses an unknown version rather than guessing at it |
| `created_at` | RFC3339, UTC |
| `vantage_version` | Build stamp of the tool that wrote the archive |
| `hostname` | Provenance; which machine produced this |
| `mongo_db` | Source database name |
| `mongo_server_version` | Restore warns on a major version gap |
| `key_fingerprint` | `sha256` of the raw 32 key bytes, hex, or `null`. Never the key |
| `collections[]` | Per collection: name, document count, uncompressed bytes, `sha256` of the `.bson` member |
| `excluded[]` | Collection names passed to `--exclude` |
Per-collection checksums mean a truncated or corrupted archive is detected
before a single document is written, rather than halfway through a restore.
### Key custody
The key never enters the archive. The archive is exactly as sensitive as a
`mongodump` of the same database, and no more.
What the archive carries is `sha256` of the raw key bytes. A hash of the key
proves identity without being a hint at the value, which is what allows an
operator to answer "will this archive restore into this deployment" without
holding both in front of them.
Backup refuses to run when `KEY_ENCRYPTION_KEY` is unset or malformed. An
archive full of ciphertext whose key was never recorded is worse than no archive
at all, because it looks like a backup. `--allow-no-key` exists for a deployment
that genuinely stores no encrypted material; it stamps `key_fingerprint: null`,
which restore then reports loudly rather than treating as a match.
Restore compares the archive's fingerprint against the key in the current
environment:
- Fingerprints match: proceed.
- Fingerprints differ: refuse, printing both.
- Archive has a fingerprint, environment has no key: refuse.
- `--ignore-key-mismatch`: proceed, having first printed exactly which
collections hold ciphertext that will be undecryptable — `keys`, `secrets`,
`auth_providers`, `console_sessions`, `settings`.
### Restore semantics
The order is fixed:
1. Read `manifest.json` and check `format_version`.
2. Verify every archive member against its manifest checksum. Nothing is written
before this passes.
3. Apply the fingerprint rules above.
4. Inspect the target: `ListCollectionNames` and document counts. A non-empty
database is refused, printing what was found. `--force` proceeds.
5. Per collection: under `--force`, drop it first; then bulk-insert in batches
of 1000 with `ordered=false`.
6. Replay index specifications from `indexes/<name>.json`, skipping `_id_`.
7. Print a summary: collection, documents restored, indexes created.
Restore is not idempotent, and says so. A second run without `--force` is
refused because step 4 now finds data. A restore interrupted during step 5
leaves a partial database that the next run refuses to touch — correct, because
the alternative is a silent merge. There are no merge or upsert semantics at
all: merging two control planes reconciles nothing and produces a fleet that
half works, and upserting by `_id` resurrects rows deleted since the backup,
which for revoked keys and deleted users is a security regression wearing the
costume of a convenience.
Index replay is fatal per collection when a unique index fails to build, and a
warning when a non-unique one does. A unique index that cannot be created means
the restored data violates it, and the unique indexes here — `(instance_id,
email)`, instance slug, settings instance, the ESO token hash — are
tenant-isolation properties rather than optimisations. The failure names the
offending index.
### Destructive confirmation
Restore under `--force` requires a typed confirmation when stdin is a TTY.
When stdin is not a TTY — a Kubernetes Job, a CI step, a cron entry — the
confirmation comes from `--confirm-db <name>`, whose value must equal the
resolved target database name or restore refuses. Naming the database in the
argument means a copy-pasted restore command carries its intended target with
it and cannot destroy a different one.
A dynamic flag name containing the database name was considered and rejected:
cobra registers flags before parsing, and the target database is not known at
registration time.
### Command surface
```
vantagectl root; prints help
├── backup --out DIR|- --exclude a,b --allow-no-key
├── restore ARCHIVE --force --confirm-db NAME --ignore-key-mismatch
├── inspect ARCHIVE
└── verify ARCHIVE
```
Persistent flags on the root command, so every subcommand accepts them and they
are documented once: `--mongo-uri` (env `MONGO_URI`) and `--db` (env `MONGO_DB`,
falling back to the URI path). There is no `--log-level`: the tool's entire
output is what it is telling the operator, and a level that could hide a key
warning is worth not having.
Environment fallback is wired with an explicit `Changed` check on each flag
rather than through viper. Viper is a configuration-file and remote-config
system; this tool reads no configuration file, and pulling it in to call
`os.Getenv` would make the largest dependency in the binary the one doing the
smallest job.
`inspect` prints the manifest — when the archive was made, by what version,
which collections it holds, how many documents, what was excluded, and the key
fingerprint — and contacts no database. It is what an operator runs to find out
whether an archive they have found is worth anything.
`verify` adds a live check: whether the archive's fingerprint matches the key in
the current environment, and — when `--mongo-uri` is given — whether that key
actually decrypts the target database. The second half is a probe: read one
ciphertext field from `secrets`, `keys` or `auth_providers` and attempt to open
it. A fingerprint comparison proves two archives agree; only a probe proves the
key in hand opens the data in front of you. `verify` is the command that
distinguishes "we have backups" from "we have backups that will restore", and
the documentation recommends running it on a schedule.
The probe needs AES-256-GCM open, which today lives in
`server/internal/services/crypto.go` and cannot be imported from another module.
Rather than copy it — the exact hazard `CLAUDE.md` names around mirrored token
blocks and `web/lib/targets.ts` — the primitives move to a new `shared/cryptobox`
package, and `services/crypto.go` becomes a thin delegation that keeps its
existing unexported function names and its `KEY_ENCRYPTION_KEY` lookup. One
implementation of the cipher, two callers.
`--out -` streams the tarball to stdout, so piping into `aws s3 cp -`, `restic`
or `age` covers storage and archive encryption without the tool growing backends
of its own.
### Distribution
Three ways to run it, because the deployments that need it run Docker Compose,
Kubernetes, or neither.
**Loose binary.** A new `.gitea/workflows/vantagectl-release.yml`, triggered on
`vantagectl/v*` tags, shaped like `agent-release.yml`. Builds `linux/amd64`,
`linux/arm64`, `darwin/arm64` and `windows/amd64` with `CGO_ENABLED=0`, writes
`checksums.txt`, and creates a Gitea release.
**Container image.** `vantagectl/Dockerfile` — the repo's convention is a
Dockerfile per module built from the repository root, because every Go module
depends on `shared` through a replace directive — produces a `scratch`
image holding the static binary and an explicitly copied `/tmp`, which the
archive is staged in before compression — the same omission that silently
disabled `vulnsched` on a scratch image. Pushed by `server-deploy.yml` as an
eighth image.
```bash
docker run --rm --network vantage_default \
-e MONGO_URI -e MONGO_DB -e KEY_ENCRYPTION_KEY \
-v /backups:/out \
gitea.hostxtra.co.uk/mrhid6/vantagectl backup --out /out
```
**Kubernetes.** The chart gains `backup.enabled`, defaulting to **false**,
rendering a `CronJob` that runs the same image and mounts the existing MongoDB
and `KEY_ENCRYPTION_KEY` secrets by reference rather than re-declaring them.
Output goes to a PVC named in values. The default is off because a backup with
nowhere durable to land is a false sense of safety and the chart cannot know
where that is; `NOTES.txt` says so on install.
Restore in Kubernetes is the same image run as a one-shot `Job`. The chart ships
no restore manifest: a restore is an operator decision with a confirmation
attached to it, and must never be something a `helm upgrade` can trigger.
`server-deploy.yml`'s rebuild trigger table gains a `vantagectl` row —
`vantagectl/`, `shared/`, `go.work` — which makes `shared/` fan out to four Go
images rather than three. That table is already called out in `CLAUDE.md` as a
place where a missed entry ships a stale image.
## Testing
`shared/backup` is tested against a real MongoDB, via `testcontainers-go` if the
module graph tolerates it and otherwise behind a `MONGO_TEST_URI` environment
variable that skips when unset.
Required cases:
- Round trip: seed one document of every awkward BSON type — `ObjectId`,
`Decimal128`, `DateTime`, binary, null, nested arrays — back up, restore into
a second database, assert byte-equal BSON.
- A single corrupted byte in a `.bson` member causes restore to refuse before
writing anything.
- Fingerprint mismatch is refused; `--ignore-key-mismatch` proceeds and names
the ciphertext-bearing collections.
- A non-empty target is refused; `--force` replaces it.
- An excluded collection is absent from the archive and named in the manifest.
- A unique index that cannot be built aborts the restore, naming the index.
Fingerprint computation is a pure function and is tested without a database.
## Documentation
`docsite/docs/operations/backup-and-restore.md`, covering:
- What `KEY_ENCRYPTION_KEY` is, that it is not in the backup, and that losing it
is unrecoverable. This comes first on the page, not as a note at the bottom.
- The three run modes above, each as a command that can be copied.
- A restore drill: restore into a scratch database and run `verify`, because an
untested backup is a hypothesis.
- What is not covered: Redis sessions, the vulnerability database (re-pulled
automatically), and agent state on managed servers — agents reconnect on their
own and `servers.agent_token_hash` is in the backup, so no re-enrolment is
needed.
`CLAUDE.md` gains a section describing the tool, since a new module, a new
image, a new workflow and a new chart toggle are each something that drifts
quietly.
## Open questions
None. Every decision above was settled during design.
@@ -96,8 +96,16 @@ rather than run in a half-prepared state.
## 4. Put a proxy in front
Point your reverse proxy at `web` on port `3000` and terminate TLS there. The
web app reaches the API internally, so there is no need to publish port `8080`.
Terminate TLS at your reverse proxy and route **one hostname to two backends**:
| Path | Backend |
| -------------------------------------------------------------------------------- | ------------- |
| `/api`, `/auth`, `/public`, `/install`, `/install.ps1`, `/update`, `/update.ps1` | `server:8080` |
| everything else | `web:3000` |
Both rules are required. The web app forwards nothing to the API, so a proxy
that sends the whole hostname to `web:3000` serves the interface and answers
`404` to every request it makes — starting with the login form.
Agents connect to port `9090`. Vantage does not terminate TLS itself, so put
that port behind your proxy too, with a certificate valid for the name in
@@ -28,13 +28,14 @@ entitlement.
## Features
Three features are enabled per instance rather than bundled into a tier:
Four features are enabled per instance rather than bundled into a tier:
| Feature | What it enables |
| ---------------------- | ------------------------------------------------------------------------------- |
| Browser console | The [browser console](../vantage/browser-console.md) |
| Single sign-on | [Sign-in through your identity provider](../vantage/settings.md#single-sign-on) |
| Vulnerability scanning | [Package vulnerability scanning](../vantage/vulnerabilities.md) |
| Status pages | [Public status pages](../vantage/status-pages.md) |
No tier includes them by default; you enable them on the instances that need
them.
@@ -0,0 +1,233 @@
---
id: backup-and-restore
title: Backup and restore
sidebar_label: Backup and restore
---
`vantagectl` is a separate command-line tool that backs up and restores the
MongoDB database behind a Vantage control plane. It talks to MongoDB directly,
never to the Vantage API, so it works against a control plane that is down,
half-migrated, or gone — exactly the situation a backup tool has to survive.
For the store-level overview — what holds what, and why the database alone is
not a backup — see [Backups](./backups.md). This page covers the tool.
:::danger The key comes first
Vantage encrypts SSH private keys, key passphrases, vault secrets, SSO client
secrets and console credentials with `KEY_ENCRYPTION_KEY`. **It is not in your
backup, and it is not recoverable.** A database restored without it is
permanently unreadable — not degraded, not partially readable, unreadable.
Store it wherever you store the credentials you could not rebuild: a password
manager, a secrets vault outside this control plane, a piece of paper in a
safe. Anywhere but next to the archive.
:::
## What a backup holds
Every collection in the database, the index definitions each one needs to be
useful again, and a SHA-256 **fingerprint** of `KEY_ENCRYPTION_KEY` — never the
key itself. The fingerprint is what lets a later `restore` or `verify` tell you
that the key you are holding is the wrong one, before it writes a database
nobody can read.
## What it does not hold
- **Redis sessions.** Everyone signs in again after a restore, which is already
true whenever Redis itself restarts.
- **The vulnerability database.** It is re-pulled automatically on next boot.
- **Agent state on managed servers.** Nothing needs re-enrolling: agents
reconnect on their own, because `servers.agent_token_hash` — the thing an
agent authenticates with — is itself in the backup.
:::note Pin the version
The image is published on each `vantagectl/v*` release and tagged with that
version; `:latest` also moves. Pin a version in anything scheduled. A restore
is easier to reason about when you can say which build produced the archive and
which one read it back.
:::
## Taking a backup
The loose binary:
```bash
export MONGO_URI=mongodb://localhost:27017
export MONGO_DB=vantage
export KEY_ENCRYPTION_KEY=<your 64-char hex key>
vantagectl backup --out /backups
```
The container:
```bash
docker run --rm \
-e MONGO_URI=mongodb://mongo:27017 \
-e MONGO_DB=vantage \
-e KEY_ENCRYPTION_KEY=<your 64-char hex key> \
-v /backups:/backups \
gitea.hostxtra.co.uk/mrhid6/vantage/vantagectl:0.1.0 backup --out /backups
```
Kubernetes, as a scheduled `CronJob` the Helm chart can render for you:
```yaml
backup:
enabled: true
schedule: "0 2 * * *"
image: "gitea.hostxtra.co.uk/mrhid6/vantage/vantagectl:0.1.0"
pvcName: "vantage-backups"
```
`backup.enabled` defaults to `false`, and the chart refuses to render if it is
turned on without both `backup.image` and `backup.pvcName` — a backup needs a
known image and somewhere durable to land, and guessing at either is worse than
refusing to start. `backup.exclude` names collections to leave out (recorded in
the archive's manifest, so an archive never claims to be complete when it is
not), and `backup.successfulJobsHistoryLimit` / `backup.failedJobsHistoryLimit`
/ `backup.resources` behave exactly as they do on any other `CronJob`.
`backup` refuses to run without `KEY_ENCRYPTION_KEY` set in the environment,
unless you pass `--allow-no-key` — for a deployment that genuinely stores no
encrypted data. Everywhere else, treat the refusal as the tool doing its job.
## Where to put the archive
`--out -` streams the tarball to stdout instead of writing a file, and every
line of progress output goes to stderr — so piping the archive into something
else is always safe, nothing progress-related lands in the stream.
Into `restic`:
```bash
vantagectl backup --out - | restic backup --stdin --stdin-filename vantage.tar.gz
```
Into S3:
```bash
vantagectl backup --out - | aws s3 cp - s3://my-backups/vantage-$(date +%F).tar.gz
```
An archive is as sensitive as a raw database dump — it carries every SSH key
assignment, every secret group, every session-adjacent setting, in a form the
right `KEY_ENCRYPTION_KEY` can decrypt. Whatever you pipe it into should
encrypt it at rest; `vantagectl` itself does not.
## Checking a backup is real
```bash
vantagectl verify /backups/vantage-backup-vantage-20260907T020000Z.tar.gz \
--mongo-uri mongodb://localhost:27017 --db vantage
```
Each line of output answers a different question:
- **`Archive`** — every member's checksum still matches; the tarball has not
been truncated or corrupted.
- **`Archive key`** / **`Your key`** — the fingerprint stored in the archive
next to the fingerprint of the `KEY_ENCRYPTION_KEY` in your environment.
- **`Key match`** — whether those two fingerprints agree.
- **`Live probe`** — given `--mongo-uri`, `verify` goes one step further and
decrypts a real ciphertext value from that database with the key you hold.
A fingerprint match proves two archives agree about a key; only the probe
proves the key in your hand actually reads the data.
`verify` exits non-zero the moment anything above is wrong, which is what makes
it worth putting on a schedule — a backup job that "succeeded" last night is
not the same claim as a backup that will actually restore.
## Looking inside an archive
`inspect` prints an archive's manifest and touches no database at all — no
`--mongo-uri`, no key. It is what to run against an archive of unknown origin,
before deciding whether it is the one you want:
```bash
vantagectl inspect /backups/vantage-backup-vantage-20260907T020000Z.tar.gz
```
It reports when the archive was taken and on which host, the Vantage and
MongoDB versions behind it, the database it came from, the key fingerprint (or
that it carries none), every collection with its document count and size, and
anything `--exclude` left out. Opening the archive verifies every member's
checksum on the way, so a corrupt archive fails here too.
Reach for `verify` instead when the question is whether the key you hold opens
it; reach for `inspect` when the question is what it is.
## Restoring
`restore` expects the target database to be empty. Pointed at one that already
holds data, it refuses outright: there are no merge semantics, because merging
two control planes reconciles nothing and upserting old data over new would
resurrect revoked keys and deleted users.
```bash
vantagectl restore /backups/vantage-backup-vantage-20260907T020000Z.tar.gz \
--mongo-uri mongodb://localhost:27017 --db vantage_restore
```
To overwrite a database that is not empty, add `--force`, which drops each
collection named in the archive before loading it. `--force` always needs a
second assurance, in one of two forms:
- `--confirm-db NAME`, naming the target exactly. A mismatch is refused. This
works everywhere — on a terminal and in a Kubernetes Job, a CI step or a cron
entry alike — and is the form to script.
- Nothing, on a terminal: `--force` alone prompts you to type the target
database's name back, a deliberate pause before something destructive.
Without a terminal and without `--confirm-db`, `--force` is refused: there is
nobody there to prompt. Naming the database in the command itself means a
copy-pasted invocation carries its intended target with it and cannot destroy a
different one by accident.
`--force` drops only the collections the archive carries. Anything else already
in the target is left alone and named in a warning, so an archive taken with
`--exclude workflow_log_lines` restored over a live database tells you the old
log lines are still there, joined to freshly restored runs. Dropping them
instead would delete data you never asked to delete.
`restore` also refuses when the archive's key fingerprint does not match the
`KEY_ENCRYPTION_KEY` in your environment — see "When the key is wrong" below.
## The restore drill
An untested backup is a hypothesis, not a backup. Rehearse the whole path,
monthly:
1. Restore last night's archive into a scratch database:
```bash
vantagectl restore /backups/vantage-backup-vantage-<date>.tar.gz \
--mongo-uri mongodb://localhost:27017 --db vantage_drill
```
2. Run `verify` against the result to confirm the data that landed is actually
readable with your current key:
```bash
vantagectl verify /backups/vantage-backup-vantage-<date>.tar.gz \
--mongo-uri mongodb://localhost:27017 --db vantage_drill
```
3. Drop the scratch database. It served its purpose.
The failure this catches is not "the archive is corrupt" — `verify` alone
catches that. It is "the archive is fine but nobody can actually stand a
control plane back up from it," which only a real restore proves.
## When the key is wrong
If `restore` finds the archive's key fingerprint does not match the
`KEY_ENCRYPTION_KEY` you are running with, it stops. Passing
`--ignore-key-mismatch` proceeds anyway, but says plainly which collections
will come back with ciphertext nobody can read:
- `keys` — SSH private keys and passphrases
- `secrets` — the vault
- `auth_providers` — OIDC/SSO client secrets
- `console_sessions` — RDP/VNC credentials
There is no way to recover that ciphertext afterwards. If you have reached
this point, the right key was lost along with the chance to read those rows —
the fix is to re-enter each of them by hand (re-upload SSH keys, re-save vault
secrets, reconfigure SSO), not to keep searching for a way to decrypt what is
already in the database.
+27 -3
View File
@@ -24,7 +24,23 @@ values is permanently unreadable.
Store the key somewhere other than the server it protects.
:::
## Backing up MongoDB
:::info Use `vantagectl`
[**Backup and restore**](./backup-and-restore.md) is the supported way to take
and restore a backup. It writes an archive that carries a fingerprint of
`KEY_ENCRYPTION_KEY` — never the key — so a restore taken with the wrong key
**refuses** rather than silently producing a database whose secrets nobody can
read. It also checksums every archive member before writing anything, and
refuses to restore into a database that already holds data. A plain
`mongodump` does none of that: it records nothing about which key the data was
encrypted under, so a restore from one succeeds even when the key is wrong and
the failure only shows up later, as unreadable secrets.
The rest of this page, past the table above, describes the `mongodump` /
`mongorestore` fallback for an operator who does not have `vantagectl`
available. Prefer the linked page.
:::
## Backing up MongoDB (fallback, without `vantagectl`)
With the bundled Mongo container:
@@ -33,6 +49,13 @@ docker compose exec -T mongo mongodump --archive --gzip --db vantage \
> /backups/vantage-$(date +%F).archive.gz
```
:::warning
This archive records nothing about which `KEY_ENCRYPTION_KEY` it was taken
under. Restoring it with the wrong key produces a database that looks intact
and is not — every secret in it is silently unreadable until something tries
to decrypt one.
:::
Restoring:
```bash
@@ -69,12 +92,13 @@ What it does **not** do is reconcile the world. After a restore:
| What | When |
| ----------------- | ----------------------------------------------------- |
| MongoDB dump | Nightly, retained per your policy |
| Backup | Nightly, retained per your policy |
| Environment file | On change, held in a password manager or secret store |
| Restore rehearsal | Occasionally, into a throwaway host |
Rehearse a restore now and again. It is the step most often skipped, and the one
that finds the problems.
that finds the problems. See [Backup and restore](./backup-and-restore.md) for
the drill, and for `verify`, which checks a backup is real without a restore.
## Cloud instances
@@ -25,6 +25,7 @@ it is absent.
| `VANTAGE_LICENSE` | no | | A licence supplied at startup, so an automated install does not have to paste one in |
| `VANTAGE_TRIVY_DB_REF` | no | `ghcr.io/aquasecurity/trivy-db:2` | Where the vulnerability database is pulled from. Point it at a mirror for an air-gapped install |
| `VANTAGE_VULNDB_DISABLED` | no | | `true` switches [vulnerability scanning](../vantage/vulnerabilities.md) off entirely. Findings already stored are still served, and still shown as stale |
| `TRUSTED_PROXIES` | no | `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16` | Comma-separated CIDRs or addresses of proxies allowed to set `X-Forwarded-For`. The shipped Docker Compose and Helm chart default to the private RFC1918 ranges, which covers Nginx Proxy Manager on the Docker bridge network and Traefik on a Kubernetes pod CIDR. An operator whose proxy sits on a public address must set this themselves, or every visitor behind it shares one address for rate-limiting purposes. Unset entirely (outside those shipped defaults) trusts none, so the client address is the direct peer. **On a LAN-only install, narrow this to your proxy's address.** The RFC1918 default trusts every private range, so a client on 192.168.0.0/16 reaching the server directly is itself a "trusted proxy" and can put whatever it likes in `X-Forwarded-For` — and, on the public status route, in `X-Forwarded-Host`. Behind a proxy on a public address, or with no proxy at all, that is not reachable; on a flat LAN it is |
:::danger `KEY_ENCRYPTION_KEY` has no recovery path
It encrypts SSH private keys, vault secrets, OIDC client secrets and console
+17 -5
View File
@@ -9,7 +9,7 @@ sidebar_label: Ports and networking
| Port | Service | Who connects | Expose publicly |
| ------- | ----------- | -------------------------------- | --------------- |
| `3000` | web | Browsers, via your reverse proxy | Yes, behind TLS |
| `8080` | server API | The web app | No, firewall it |
| `8080` | server API | Your reverse proxy | Not directly — proxied |
| `9090` | server gRPC | Agents | **Yes** |
| `4822` | guacd | The server | No, firewall it |
| `27017` | MongoDB | The server | No |
@@ -20,8 +20,8 @@ sidebar_label: Ports and networking
```mermaid
flowchart LR
B["Browser"] -->|HTTPS| P["Reverse proxy"]
P --> W["web :3000"]
W --> S["server :8080"]
P -->|"everything else"| W["web :3000"]
P -->|"/api /auth /public /install* /update*"| S["server :8080"]
A["Agent on a managed server"] -->|"gRPC/TLS :9090, outbound"| S
S --> G["guacd :4822"]
G -->|"relayed over the :9090 stream"| A
@@ -77,8 +77,20 @@ On a private network you can skip TLS instead, by setting `tls: false` in each
## Reverse proxy notes
- Point the proxy at `web:3000`. The web app reaches the API internally, so
`8080` does not need publishing.
- **The proxy routes two backends on one hostname**, and both are required:
| Path | Backend |
| ------------------------------------------------------------- | ------------- |
| `/api`, `/auth`, `/public`, `/install`, `/install.ps1`, `/update`, `/update.ps1` | `server:8080` |
| everything else | `web:3000` |
The web app forwards nothing to the API. Sending the whole hostname to
`web:3000` loads the interface and every request it makes answers `404`
including the login form.
- Both backends must be the **same** hostname and certificate. The browser
calls `/api` relative to the page it is on, and the session cookie is
host-only.
- The console uses a **WebSocket** at `/api/console/tunnel`. A proxy that does
not forward upgrade headers breaks the console and nothing else.
- Workflow log streaming is a long-lived response. A short proxy read timeout
+80
View File
@@ -20,6 +20,13 @@ or is not 64 hex characters.
## Nobody can sign in
**Every request 404s and the interface loads fine.** Your reverse proxy sends
the whole hostname to `web:3000`. `/api`, `/auth`, `/public`, `/install*` and
`/update*` belong to `server:8080` and the web app forwards nothing — see
[Ports and networking](./ports-and-networking.md#reverse-proxy-notes). The
tell is `curl -si https://<your-host>/auth/bootstrap-status` returning HTML
with `x-powered-by: Next.js` instead of JSON.
**`/setup` appears when users already exist.** The server is pointed at a
different database than you think. Check the database name in `MONGO_URI`,
which is taken from the end of the URI.
@@ -95,6 +102,53 @@ instantaneous.
- The keyword no longer appears in the response body.
- Retries are `0`, so a single dropped packet flips the state.
### The check gets a 403, 429 or a CAPTCHA page
The endpoint is fine and answers a browser normally, but the monitor records a
status it never sees by hand. Something between Vantage and the service is
blocking automated traffic: a CDN, a WAF, a bot-protection product, a reverse
proxy rule, or a rate limiter. The response usually comes from that layer and
never reaches the origin at all, so nothing appears in the application's own
logs.
Two things make it hard to spot. The check runs from the control plane's or the
agent's address rather than yours, and those addresses are often datacenter
ranges that bot protection scores badly. And a browser test proves nothing,
because a browser is exactly what the blocking layer is willing to serve.
Every HTTP check Vantage makes identifies itself:
```
User-Agent: Vantage-Monitor/1.0 (+https://vantage.hostxtra.co.uk)
```
That string is the hook to allow the check through. In whichever product is
doing the blocking, add a rule that skips bot protection, managed rules and rate
limiting for requests carrying it — Cloudflare, AWS WAF, Azure Front Door,
Akamai, Fastly, Imperva, Sucuri, ModSecurity, nginx and HAProxy all match on a
request header. The shape of the rule is the same everywhere:
> If the host is *yours*, the path is *the one being monitored*, and the
> User-Agent contains `Vantage-Monitor`, then skip the protection.
Three details are worth getting right:
- **Match on `contains`, not equality.** The version in the string moves. An
exact match breaks silently on an upgrade, and the symptom is a monitor that
goes down on deploy day.
- **Keep the rule narrow.** Scope it to the specific host and path being
monitored. A User-Agent is not a secret — anyone can send it — so a rule that
skips protection site-wide on that string alone is a bypass you have
published.
- **Allow the source address too, where you can.** Combining the User-Agent with
the checker's IP is stronger than either alone. Find the address in your
blocking product's own event log; it is whichever client IP was blocked on the
monitored path.
If the endpoint genuinely needs authentication rather than an exception, monitor
a purpose-built health path that does not, and leave the protected paths
protected.
## Notifications are not arriving
Use the channel **Test** button. It goes through the real delivery path, so a
@@ -113,6 +167,32 @@ needs `host`, `port`, `from` and `to`, and Telegram needs both `token` and
| Instance degraded despite a valid-looking licence | It expired more than a few days ago. Pasting a new one still works, which is how you recover |
| Cannot enrol another server | The server allowance is reached. Raise it in HQ or remove one |
## A status page 404s or shows no data
**404, and it should be published.** Check the **Published** toggle on the
page's editor — an unpublished page answers *not found* for everyone,
including you, with no session exemption. Also check the host: the public URL
is `<your-instance>.vantage.<yourdomain>/status/<page-id>`, the same
per-instance subdomain everything else in Vantage uses. A wrong or missing
subdomain resolves to no instance at all, which is also a 404.
Third possibility: `/public` is not routed to the server. Check with
`curl -si https://<your-instance>.vantage.<yourdomain>/public/status/<page-id>`
— JSON is correct, HTML carrying `x-powered-by: Next.js` means the proxy sent
that prefix to the web app.
**Loads, but shows an explanation instead of components.** This is not a
fault — it is the page working as designed. It means either the licence has
lapsed (a self-hosted instance past its grace period, or a cloud instance
between billing events) or the current tier does not include the **Status
pages** feature. Fix the licence or the plan and the same link starts serving
data again with no republish needed.
**One component reads `Unknown`.** The monitor behind it was deleted while
still listed on the page. Nothing is checking it any more, so the page says so
rather than showing a stale up or down. Remove the component from the page,
or point it at a replacement monitor, in the page's editor.
## HQ portal problems
The portal is a hosted service, so problems with it are ours to fix rather than
@@ -64,6 +64,15 @@ Posts the alert as message content.
Port `465` uses implicit TLS; anything else uses STARTTLS.
### Credentials are never read back
The SMTP `password`, the Telegram `token` and the webhook, Slack and Discord
`url`s come back from `GET /api/channels` as `••••••••` — a webhook URL is the
authorisation to post to that channel, so it is treated as a credential like
the rest. Writing that value back unchanged keeps the stored one, which is what
lets you rename a channel without retyping its password. Anything else you send
is written as given, so clearing the field clears the credential.
Alert emails look like the rest of the mail Vantage sends you.
## The message
+125
View File
@@ -0,0 +1,125 @@
---
id: status-pages
title: Status pages
sidebar_label: Status pages
---
A status page is a public page reporting a chosen set of monitors as up-front
components, with a 90-day history and an uptime percentage per component. It
needs no session and no token to read — anyone with the link can open it,
which is the point: it is what you hand a customer instead of an incident
email.
Requires the **Status pages** licence feature. If the licence lapses, or the
tier does not include the feature, the page keeps serving — it renders an
explanation rather than data or a broken page, so a customer who follows an
old link never sees an error.
## Creating a page
From **Status pages**, choose a page id and a title. The id is 340 characters
of lowercase letters, digits and `-`, starting and ending with a letter or
digit. It becomes part of the public URL:
```
https://<your-vantage-address>/status/<page-id>
```
On **Vantage Cloud** that address is your instance's own subdomain, so the page
is at `https://<your-instance>.vantage.hostxtra.co.uk/status/<page-id>`.
On a **self-hosted** install it is whatever address you reach Vantage on —
`https://vantage.acme.com/status/<page-id>`, or an IP and port on a LAN
install. A self-hosted install serves exactly one Vantage instance, so no
subdomain is needed to say which one you mean. The **Copy** control next to the
page address in the editor gives you the exact URL for your install, which is
the one to hand out.
**The page id cannot be changed after creation.** Once you have shared the
link, changing the id would break it, so pick something you would still be
happy with in a year — `platform`, `api`, a customer's own name for a
dedicated page.
## Draft versus published
A new page starts unpublished. Unpublished pages answer *not found* to
anyone who requests them, including you, from a browser without a session —
so you can build out the components and copy before announcing it. Toggle
**Published** when it is ready. Un-publishing later takes it back to *not
found* rather than deleting anything.
**Delete page**, in the editor header, is the only way to correct a page id you
regret — the id is fixed once created. It takes the page, its sections and its
authored incidents with it; monitors and their history are untouched. If you
only want the page off the internet, un-publish it instead.
## Sections and components
A page is organised into **sections** — arbitrary groupings such as "API" or
"Region: EU" — each holding one or more **components**. A component is a
monitor plus a **display name** you choose for this page.
The display name is never the monitor's own name unless you type it in. An
internal monitor name ("prod-db-primary-eu1") is rarely what you want a
customer reading; give it whatever name makes sense to them, and change it
for a different page without touching the monitor.
If a monitor listed on a page is later deleted, its component still appears —
reading `Unknown` rather than up or down, because nothing is checking it any
more and claiming otherwise would be a false claim of health.
## What a visitor sees
- Component name, current state (up / down / under maintenance / pending /
unknown) and a 90-day uptime percentage. **Pending** is a monitor that has
been added but has not produced a result yet; **unknown** is one nothing is
checking any more.
- A 90-day history bar per component.
- Any active incidents, upcoming maintenance, and a rolling history of both.
- An optional banner across the top of the page, for anything you want said
regardless of component state. It is one notice with one appearance — there
are no severity levels to choose between.
A visitor never sees a target URL, host or port, the check's expected status
or keyword, latency, a certificate expiry date, failure text, or which
notification channel is attached. That is a deliberate boundary, not an
oversight: nothing that would tell a stranger how your infrastructure is
reachable is on this page.
## Incidents and maintenance
Two kinds of entries appear on a page's timeline:
- **Automatic** — a monitor going down opens an incident on any page that
lists it, with no action from you. These appear the moment the monitor's
state changes and close the moment it recovers.
- **Authored** — an incident or maintenance window you create by hand, with
its own title, impact and a set of affected components you choose. You
post updates to it (Investigating → Identified → Monitoring → Resolved) as
the situation develops, and each update is timestamped and kept on the
page's history.
An authored incident is attached to one or more pages explicitly when you
create it — it does not follow a monitor onto every page that monitor happens
to be listed on.
### Scheduling maintenance
A maintenance window has a scheduled start and end (the end must be after the
start) and moves through Scheduled → In progress → Completed. While a window
is in progress and its affected components are within the scheduled time,
those components are drawn as "under maintenance" instead of up or down.
**Maintenance changes how a day is drawn, never the uptime number itself.**
The 90-day percentage is computed from what actually happened — a component
that stayed up throughout a maintenance window still shows as up in its
history, it is only the live status pill that reads "under maintenance" for
the duration.
## Delay before an update appears
A visitor's read of a page is cached for up to 30 seconds, so posting an
update or flipping Published does not necessarily change what a visitor sees
instantly — though most authoring actions invalidate that cache immediately,
so in practice it usually shows within a second or two. If a change genuinely
does not appear, reloading after 30 seconds always will.
+2 -1
View File
@@ -29,6 +29,7 @@ const sidebars: SidebarsConfig = {
"vantage/vulnerabilities",
"vantage/workloads",
"vantage/notification-channels",
"vantage/status-pages",
"vantage/secrets",
"vantage/browser-console",
"vantage/audit-log",
@@ -48,7 +49,7 @@ const sidebars: SidebarsConfig = {
{
type: "category",
label: "Operations",
items: ["operations/upgrading", "operations/backups", "operations/agent-updates"],
items: ["operations/upgrading", "operations/backups", "operations/backup-and-restore", "operations/agent-updates"],
},
],
};
+1
View File
@@ -6,4 +6,5 @@ use (
./server
./shared
./sitesvc
./vantagectl
)
+22 -4
View File
@@ -51,10 +51,10 @@ import (
// @name Authorization
// @description An API token, sent as "Bearer vt_…". Scoped and optionally expiring.
// @securityDefinitions.apikey esoAuth
// @in header
// @name Authorization
// @description The External Secrets read token, rotated under Settings. It reaches /api/secrets/{group}/values and nothing else. It is a different credential from an API token, and the two must never be substituted for one another.
// @securityDefinitions.apikey esoAuth
// @in header
// @name Authorization
// @description The External Secrets read token, rotated under Settings. It reaches /api/secrets/{group}/values and nothing else. It is a different credential from an API token, and the two must never be substituted for one another.
func main() {
mongoURI := getEnv("MONGO_URI", "mongodb://localhost:27017")
@@ -162,6 +162,10 @@ func runSchemaSetup() {
log.Printf("warning: failed to ensure workflow indexes: %v", err)
}
if err := services.EnsureMonitorSampleIndexes(); err != nil {
log.Printf("warning: failed to ensure monitor sample indexes: %v", err)
}
if err := services.EnsureVulnIndexes(); err != nil {
log.Printf("warning: failed to ensure vuln indexes: %v", err)
}
@@ -170,6 +174,10 @@ func runSchemaSetup() {
log.Printf("warning: failed to ensure workload indexes: %v", err)
}
if err := services.EnsureStatusPageIndexes(); err != nil {
log.Printf("warning: failed to ensure status page indexes: %v", err)
}
if err := services.EnsureAuditIndexes(); err != nil {
log.Printf("warning: failed to ensure audit indexes: %v", err)
}
@@ -255,9 +263,19 @@ func serve() {
})
r := gin.New()
// Without this gin trusts every proxy and ClientIP() is whatever the
// caller wrote in X-Forwarded-For. That was survivable while ClientIP()
// only produced audit strings; the public status limiter makes it load
// bearing. Empty means trust nobody, which is correct for a direct
// exposure and wrong behind a proxy — hence the explicit setting.
if err := r.SetTrustedProxies(api.TrustedProxies()); err != nil {
log.Fatalf("trusted proxies: %v", err)
}
r.Use(gin.Recovery())
r.Use(gin.LoggerWithConfig(gin.LoggerConfig{SkipPaths: []string{"/api/console/tunnel"}}))
r.Use(corsMiddleware())
services.SetStatusRedis(auth.Redis())
api.RegisterRoutes(r)
if err := api.AssertScopeMapComplete(r); err != nil {
+9 -2
View File
@@ -34,7 +34,14 @@ func listChannels(c *gin.Context) {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, channels)
// Redacted here rather than in the service: the dispatchers read the same
// documents and need the real credentials, so the masking belongs to the
// boundary that hands them to a client.
out := make([]models.NotificationChannel, 0, len(channels))
for _, ch := range channels {
out = append(out, ch.Redacted())
}
c.JSON(http.StatusOK, out)
}
// createChannel godoc
@@ -69,7 +76,7 @@ func createChannel(c *gin.Context) {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusCreated, created)
c.JSON(http.StatusCreated, created.Redacted())
}
// updateChannel godoc
File diff suppressed because it is too large Load Diff
+6
View File
@@ -47,6 +47,10 @@ func RegisterRoutes(r *gin.Engine) {
r.GET("/auth/oidc/:providerId/callback", auth.HandleSSOCallback)
r.GET("/auth/providers", auth.HandleListPublicProviders)
// Completely public: no session, no token, no licence gate. Mounted here
// rather than under /api precisely so that none of those apply.
r.GET("/public/status/:pageId", RateLimitPublicStatus(), getPublicStatusPage)
apiGroup := r.Group("/api")
apiGroup.Use(auth.Middleware())
// Scope enforcement sits between authentication and the licence gate, and
@@ -162,6 +166,8 @@ func RegisterRoutes(r *gin.Engine) {
apiGroup.POST("/servers/:id/workloads/refresh", refreshServerWorkloads)
apiGroup.POST("/servers/:id/workloads/:wid/action", auth.RequireRole("owner", "admin"), controlWorkload)
apiGroup.GET("/servers/:id/workloads/:wid/logs", auth.RequireRole("owner", "admin"), getWorkloadLogs)
registerStatusPageRoutes(apiGroup)
}
}
+50 -1
View File
@@ -2,6 +2,7 @@ package api
import (
"net/http"
"strconv"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/auth"
@@ -19,6 +20,7 @@ func registerMonitorRoutes(g *gin.RouterGroup) {
g.DELETE("/monitors/:id", deleteMonitor)
g.GET("/monitors/:id/incidents", getMonitorIncidents)
g.GET("/monitors/:id/uptime", getMonitorUptime)
g.GET("/monitors/:id/samples", getMonitorSamples)
}
// listMonitors godoc
@@ -111,7 +113,7 @@ func getMonitor(c *gin.Context) {
// @Accept json
// @Produce json
// @Param id path string true "Monitor ID"
// @Param body body object{name=string,type=string,target=models.MonitorTarget,interval_sec=int,runner=string,retries=int,enabled=bool,channel_ids=[]string} true "Fields to update"
// @Param body body object{name=string,group=string,type=string,target=models.MonitorTarget,interval_sec=int,runner=string,retries=int,enabled=bool,channel_ids=[]string} true "Fields to update"
// @Success 204
// @Failure 400 {object} ErrorResponse
// @Failure 500 {object} ErrorResponse
@@ -121,6 +123,7 @@ func getMonitor(c *gin.Context) {
func updateMonitor(c *gin.Context) {
var body struct {
Name *string `json:"name"`
Group *string `json:"group"`
Type *string `json:"type"`
Target *models.MonitorTarget `json:"target"`
IntervalSec *int `json:"interval_sec"`
@@ -137,6 +140,9 @@ func updateMonitor(c *gin.Context) {
if body.Name != nil {
upd["name"] = *body.Name
}
if body.Group != nil {
upd["group"] = *body.Group
}
if body.Type != nil {
upd["type"] = *body.Type
}
@@ -217,6 +223,49 @@ func getMonitorIncidents(c *gin.Context) {
c.JSON(http.StatusOK, incidents)
}
// getMonitorSamples godoc
//
// @Summary Get a monitor's individual check results
// @Description Raw check results for the last `minutes` minutes, oldest first. Samples expire after 48 hours; use the uptime rollups for longer ranges.
// @Tags monitors
// @Produce json
// @Param id path string true "Monitor ID"
// @Param minutes query int false "Window in minutes (default 60, max 2880)"
// @Success 200 {array} models.MonitorSample
// @Failure 404 {object} ErrorResponse
// @Failure 500 {object} ErrorResponse
// @Security cookieAuth
// @Security bearerAuth
// @Router /monitors/{id}/samples [get]
func getMonitorSamples(c *gin.Context) {
m, err := services.GetMonitor(auth.InstanceID(c), c.Param("id"))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
if m == nil {
c.JSON(http.StatusNotFound, gin.H{"error": "monitor not found"})
return
}
// Clamped rather than rejected: the window is a view setting, and the only
// honest answer past the TTL is the shorter window anyway.
minutes := 60
if raw := c.Query("minutes"); raw != "" {
if n, convErr := strconv.Atoi(raw); convErr == nil && n > 0 {
minutes = n
}
}
if max := int(services.MonitorSampleTTL.Minutes()); minutes > max {
minutes = max
}
samples, err := services.MonitorSamples(auth.InstanceID(c), c.Param("id"), time.Now().Add(-time.Duration(minutes)*time.Minute))
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusOK, samples)
}
// getMonitorUptime godoc
//
// @Summary Get a monitor's uptime rollups
+155
View File
@@ -0,0 +1,155 @@
package api
import (
"errors"
"log"
"net/http"
"strconv"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/auth"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/services"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/license"
"github.com/gin-gonic/gin"
)
// publicStatusRateLimit is per client address per minute. Generous enough that
// a busy page during an outage is unaffected, small enough that scanning for
// page ids is not free.
const publicStatusRateLimit = 120
// RateLimitPublicStatus counts requests per client address in a one-minute
// fixed window, exactly as RateLimitTokens does — including the part that
// matters most: when Redis is unavailable it allows rather than denies. A
// status page must survive the outage it exists to report.
func RateLimitPublicStatus() gin.HandlerFunc {
return func(c *gin.Context) {
rdb := auth.Redis()
if rdb == nil {
c.Next()
return
}
window := time.Now().UTC().Unix() / 60
key := "vantage:statusrl:" + c.ClientIP() + ":" + strconv.FormatInt(window, 10)
count, err := rdb.Incr(c.Request.Context(), key).Result()
if err != nil {
c.Next()
return
}
if count == 1 {
rdb.Expire(c.Request.Context(), key, 2*time.Minute)
}
if count > publicStatusRateLimit {
c.Header("Retry-After", "60")
c.AbortWithStatusJSON(http.StatusTooManyRequests, gin.H{
"error": "too many requests",
"code": "rate_limited",
})
return
}
c.Next()
}
}
// getPublicStatusPage is the only unauthenticated read of monitor data in the
// product.
//
// It is mounted on the gin root rather than under /api on purpose: /api
// carries auth.Middleware, RequireScopes, RateLimitTokens and
// RequireActiveLicense by virtue of where it is mounted, and a public route
// there would need four exemptions, each one a hole a later change can widen.
//
// Unknown host, unknown page and unpublished page all answer the same 404.
//
// It carries no @Router annotation deliberately. openapi.json declares a
// single server of "/api", so a @Router of /public/status/{pageId} would be
// published as /api/public/status/{pageId} — a path that does not exist, and
// which would sit behind auth.Middleware if it did. The real address is:
//
// GET {scheme}://{instance-host}/public/status/{pageId}
//
// on the gin root, unauthenticated, rate limited per client address.
//
// @Summary Public status page
// @Tags status
// @Produce json
// @Param pageId path string true "Status page id"
// @Success 200 {object} services.StatusSnapshot
// @Failure 404 {object} ErrorResponse
// @Failure 429 {object} ErrorResponse
func getPublicStatusPage(c *gin.Context) {
pageID := c.Param("pageId")
inst, ok := publicStatusInstance(c)
if !ok {
// Every 404 on this route is indistinguishable to the caller by
// design, so the log is the only place the three reasons are told
// apart. It carries no monitor data and no page contents.
log.Printf("public status: 404 page=%q reason=no_instance", pageID)
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
snap, err := services.PublicStatusSnapshot(inst.InstanceID, pageID)
if errors.Is(err, services.ErrPageNotFound) {
log.Printf("public status: 404 page=%q instance=%s reason=page_missing_or_unpublished", pageID, inst.InstanceID)
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
return
}
if err != nil {
log.Printf("public status: 500 page=%q instance=%s: %v", pageID, inst.InstanceID, err)
c.JSON(http.StatusInternalServerError, gin.H{"error": "internal error"})
return
}
// Public and cacheable, but only briefly: an intermediary holding this for
// minutes would show a resolved incident as ongoing.
c.Header("Cache-Control", "public, max-age=30")
c.JSON(http.StatusOK, snap)
}
// publicStatusInstance resolves which instance a public request is for.
//
// The browser never reaches this handler directly: the request arrives from
// the Next server, which forwards the visitor's host in X-Forwarded-Host
// because the Host header cannot be set on a fetch (undici drops it silently,
// as a forbidden header name). That makes X-Forwarded-Host a tenant selector,
// so it is honoured only when the machine that opened the connection is one of
// the configured trusted proxies.
//
// When the resulting host names no slug at all — vantage.acme.com,
// status.acme.com, a bare IP — and the deployment is not cloud, the single
// instance of that install is used. A self-hosted install has exactly one, and
// without this every self-hosted status page 404s forever. More than one is a
// refusal rather than a guess.
func publicStatusInstance(c *gin.Context) (*models.Instance, bool) {
// Host resolution is where this route fails silently: an untrusted peer
// means X-Forwarded-Host is ignored and the request host is the Go
// service's own name, which names no slug. Log the inputs and the branch
// taken, so the 404 says which of the four it was.
host := c.Request.Host
xfh := firstForwarded(c.GetHeader("X-Forwarded-Host"))
trusted := trustedPeer(c)
if trusted && xfh != "" {
host = xfh
}
log.Printf("public status: resolve peer=%s trusted=%t request_host=%q x_forwarded_host=%q using_host=%q slug=%q",
c.RemoteIP(), trusted, c.Request.Host, xfh, host, auth.HostSlug(host))
if inst, ok := auth.InstanceForHost(host); ok {
return inst, true
}
if slug := auth.HostSlug(host); slug != "" {
// The host named an instance and that instance does not exist.
log.Printf("public status: no instance for slug=%q (host=%q)", slug, host)
return nil, false
}
if services.DeploymentMode() == license.DeploymentCloud {
log.Printf("public status: host %q names no slug and deployment is cloud, refusing to guess", host)
return nil, false
}
inst, ok := auth.SoleInstance()
if !ok {
log.Printf("public status: host %q names no slug and this deployment has no single instance", host)
}
return inst, ok
}
+14
View File
@@ -100,6 +100,7 @@ var routeScopes = map[string]string{
"DELETE /api/monitors/:id": "monitors:write",
"GET /api/monitors/:id/incidents": "monitors:read",
"GET /api/monitors/:id/uptime": "monitors:read",
"GET /api/monitors/:id/samples": "monitors:read",
// Channel routes, registered by registerChannelRoutes. Channels exist to
// serve alerts, so they share the monitors scope rather than getting their
@@ -155,6 +156,19 @@ var routeScopes = map[string]string{
"GET /api/openapi.json": "settings:read",
"GET /api/docs": "settings:read",
"GET /api/docs/scalar.js": "settings:read",
// Status pages. Reading is status:read even though the pages themselves
// are public, because these routes read the unpublished ones too.
"GET /api/status-pages": "status:read",
"POST /api/status-pages": "status:write",
"GET /api/status-pages/:pageId": "status:read",
"PUT /api/status-pages/:pageId": "status:write",
"DELETE /api/status-pages/:pageId": "status:write",
"GET /api/status-pages/:pageId/incidents": "status:read",
"POST /api/status-pages/:pageId/incidents": "status:write",
"PUT /api/status-pages/:pageId/incidents/:incidentId": "status:write",
"DELETE /api/status-pages/:pageId/incidents/:incidentId": "status:write",
"POST /api/status-pages/:pageId/incidents/:incidentId/updates": "status:write",
}
// RequireScopes enforces routeScopes for token-authenticated requests and does
+312
View File
@@ -0,0 +1,312 @@
package api
import (
"errors"
"net/http"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/auth"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/services"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/license"
"github.com/gin-gonic/gin"
)
func registerStatusPageRoutes(g *gin.RouterGroup) {
// Owner or admin throughout: publishing a page is speaking to the public
// in the instance's name. The feature gate sits alongside the role gate so
// authoring and serving are gated by the same licence feature.
sp := g.Group("/status-pages")
sp.Use(auth.RequireRole("owner", "admin"), RequireFeature(license.FeatureStatusPages))
sp.GET("", listStatusPages)
sp.POST("", createStatusPage)
sp.GET("/:pageId", getStatusPage)
sp.PUT("/:pageId", updateStatusPage)
sp.DELETE("/:pageId", deleteStatusPage)
sp.GET("/:pageId/incidents", listStatusIncidents)
sp.POST("/:pageId/incidents", createStatusIncident)
sp.PUT("/:pageId/incidents/:incidentId", updateStatusIncident)
sp.DELETE("/:pageId/incidents/:incidentId", deleteStatusIncident)
sp.POST("/:pageId/incidents/:incidentId/updates", appendStatusIncidentUpdate)
}
// statusPageError maps the service errors onto codes once, so ten handlers do
// not each invent their own. services.ErrPageInvalid covers every validation
// failure in the status page and incident services — a missing title or an
// invalid incident status is a 400, not a 500.
func statusPageError(c *gin.Context, err error) {
switch {
case errors.Is(err, services.ErrPageNotFound), errors.Is(err, services.ErrIncidentNotFound):
c.JSON(http.StatusNotFound, gin.H{"error": err.Error()})
case errors.Is(err, services.ErrPageIDTaken):
c.JSON(http.StatusConflict, gin.H{"error": err.Error()})
case errors.Is(err, services.ErrInvalidPageID), errors.Is(err, services.ErrPageInvalid):
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
default:
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
}
}
// listStatusPages godoc
//
// @Summary List status pages
// @Tags status-pages
// @Produce json
// @Success 200 {array} models.StatusPage
// @Failure 500 {object} ErrorResponse
// @Security cookieAuth
// @Security bearerAuth
// @Router /status-pages [get]
func listStatusPages(c *gin.Context) {
pages, err := services.ListStatusPages(auth.InstanceID(c))
if err != nil {
statusPageError(c, err)
return
}
c.JSON(http.StatusOK, pages)
}
// createStatusPage godoc
//
// @Summary Create a status page
// @Tags status-pages
// @Accept json
// @Produce json
// @Param body body models.StatusPage true "Status page"
// @Success 201 {object} models.StatusPage
// @Failure 400 {object} ErrorResponse
// @Failure 409 {object} ErrorResponse
// @Security cookieAuth
// @Security bearerAuth
// @Router /status-pages [post]
func createStatusPage(c *gin.Context) {
var p models.StatusPage
if err := c.ShouldBindJSON(&p); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
created, err := services.CreateStatusPage(auth.InstanceID(c), &p)
if err != nil {
statusPageError(c, err)
return
}
services.LogEvent(auth.InstanceID(c), "status_page_created", actorFromCtx(c), "", "",
"Status page '"+created.PageID+"' created")
c.JSON(http.StatusCreated, created)
}
// getStatusPage godoc
//
// @Summary Get a status page
// @Tags status-pages
// @Produce json
// @Param pageId path string true "Page id"
// @Success 200 {object} models.StatusPage
// @Failure 404 {object} ErrorResponse
// @Security cookieAuth
// @Security bearerAuth
// @Router /status-pages/{pageId} [get]
func getStatusPage(c *gin.Context) {
page, err := services.GetStatusPage(auth.InstanceID(c), c.Param("pageId"))
if err != nil {
statusPageError(c, err)
return
}
c.JSON(http.StatusOK, page)
}
// updateStatusPage godoc
//
// @Summary Update a status page
// @Tags status-pages
// @Accept json
// @Produce json
// @Param pageId path string true "Page id"
// @Param body body models.StatusPage true "Status page"
// @Success 200 {object} models.StatusPage
// @Failure 404 {object} ErrorResponse
// @Security cookieAuth
// @Security bearerAuth
// @Router /status-pages/{pageId} [put]
func updateStatusPage(c *gin.Context) {
var p models.StatusPage
if err := c.ShouldBindJSON(&p); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
updated, err := services.UpdateStatusPage(auth.InstanceID(c), c.Param("pageId"), &p)
if err != nil {
statusPageError(c, err)
return
}
services.LogEvent(auth.InstanceID(c), "status_page_updated", actorFromCtx(c), "", "",
"Status page '"+updated.PageID+"' updated")
c.JSON(http.StatusOK, updated)
}
// deleteStatusPage godoc
//
// @Summary Delete a status page
// @Tags status-pages
// @Produce json
// @Param pageId path string true "Page id"
// @Success 204 "No Content"
// @Failure 404 {object} ErrorResponse
// @Security cookieAuth
// @Security bearerAuth
// @Router /status-pages/{pageId} [delete]
func deleteStatusPage(c *gin.Context) {
if err := services.DeleteStatusPage(auth.InstanceID(c), c.Param("pageId")); err != nil {
statusPageError(c, err)
return
}
services.LogEvent(auth.InstanceID(c), "status_page_deleted", actorFromCtx(c), "", "",
"Status page '"+c.Param("pageId")+"' deleted")
c.Status(http.StatusNoContent)
}
// listStatusIncidents godoc
//
// @Summary List authored incidents for a status page
// @Tags status-pages
// @Produce json
// @Param pageId path string true "Page id"
// @Success 200 {array} models.StatusIncident
// @Security cookieAuth
// @Security bearerAuth
// @Router /status-pages/{pageId}/incidents [get]
func listStatusIncidents(c *gin.Context) {
incs, err := services.ListStatusIncidents(auth.InstanceID(c), c.Param("pageId"))
if err != nil {
statusPageError(c, err)
return
}
c.JSON(http.StatusOK, incs)
}
// createStatusIncident godoc
//
// @Summary Create an incident or maintenance window
// @Tags status-pages
// @Accept json
// @Produce json
// @Param pageId path string true "Page id"
// @Param body body models.StatusIncident true "Incident"
// @Success 201 {object} models.StatusIncident
// @Failure 400 {object} ErrorResponse
// @Security cookieAuth
// @Security bearerAuth
// @Router /status-pages/{pageId}/incidents [post]
func createStatusIncident(c *gin.Context) {
var inc models.StatusIncident
if err := c.ShouldBindJSON(&inc); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
// The page in the path is always one of the pages the incident names, so
// creating from a page cannot produce an incident that page never shows.
if !contains(inc.PageIDs, c.Param("pageId")) {
inc.PageIDs = append(inc.PageIDs, c.Param("pageId"))
}
created, err := services.CreateStatusIncident(auth.InstanceID(c), &inc)
if err != nil {
statusPageError(c, err)
return
}
services.LogEvent(auth.InstanceID(c), "status_incident_created", actorFromCtx(c), "", "",
"Status "+created.Kind+" '"+created.Title+"' created")
c.JSON(http.StatusCreated, created)
}
func contains(list []string, want string) bool {
for _, v := range list {
if v == want {
return true
}
}
return false
}
// updateStatusIncident godoc
//
// @Summary Update an incident or maintenance window
// @Tags status-pages
// @Accept json
// @Produce json
// @Param pageId path string true "Page id"
// @Param incidentId path string true "Incident id"
// @Param body body models.StatusIncident true "Incident"
// @Success 200 {object} models.StatusIncident
// @Failure 404 {object} ErrorResponse
// @Security cookieAuth
// @Security bearerAuth
// @Router /status-pages/{pageId}/incidents/{incidentId} [put]
func updateStatusIncident(c *gin.Context) {
var inc models.StatusIncident
if err := c.ShouldBindJSON(&inc); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
updated, err := services.UpdateStatusIncident(auth.InstanceID(c), c.Param("incidentId"), &inc)
if err != nil {
statusPageError(c, err)
return
}
services.LogEvent(auth.InstanceID(c), "status_incident_updated", actorFromCtx(c), "", "",
"Status "+updated.Kind+" '"+updated.Title+"' updated")
c.JSON(http.StatusOK, updated)
}
// deleteStatusIncident godoc
//
// @Summary Delete an incident or maintenance window
// @Tags status-pages
// @Produce json
// @Param pageId path string true "Page id"
// @Param incidentId path string true "Incident id"
// @Success 204 "No Content"
// @Failure 404 {object} ErrorResponse
// @Security cookieAuth
// @Security bearerAuth
// @Router /status-pages/{pageId}/incidents/{incidentId} [delete]
func deleteStatusIncident(c *gin.Context) {
if err := services.DeleteStatusIncident(auth.InstanceID(c), c.Param("incidentId")); err != nil {
statusPageError(c, err)
return
}
services.LogEvent(auth.InstanceID(c), "status_incident_deleted", actorFromCtx(c), "", "",
"Status incident '"+c.Param("incidentId")+"' deleted")
c.Status(http.StatusNoContent)
}
// appendStatusIncidentUpdate godoc
//
// @Summary Post an update to an incident
// @Tags status-pages
// @Accept json
// @Produce json
// @Param pageId path string true "Page id"
// @Param incidentId path string true "Incident id"
// @Param body body StatusIncidentUpdateRequest true "Update"
// @Success 200 {object} models.StatusIncident
// @Failure 400 {object} ErrorResponse
// @Failure 404 {object} ErrorResponse
// @Security cookieAuth
// @Security bearerAuth
// @Router /status-pages/{pageId}/incidents/{incidentId}/updates [post]
func appendStatusIncidentUpdate(c *gin.Context) {
var body StatusIncidentUpdateRequest
if err := c.ShouldBindJSON(&body); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
updated, err := services.AppendStatusIncidentUpdate(
auth.InstanceID(c), c.Param("incidentId"), body.Status, body.Body, actorFromCtx(c))
if err != nil {
statusPageError(c, err)
return
}
services.LogEvent(auth.InstanceID(c), "status_incident_update_posted", actorFromCtx(c), "", "",
"Update posted to '"+updated.Title+"' ("+body.Status+")")
c.JSON(http.StatusOK, updated)
}
+84
View File
@@ -0,0 +1,84 @@
package api
import (
"net"
"os"
"strings"
"sync"
"github.com/gin-gonic/gin"
)
// TrustedProxies reads TRUSTED_PROXIES, a comma-separated list of CIDRs or
// addresses. Unset means trust none: ClientIP() is then the peer address,
// which is right for a direct exposure and means every request behind an
// un-configured proxy shares one address for rate limiting. That is a visible
// failure (one client limited) rather than an invisible one (no limit at all).
//
// This lives here rather than in main.go because the string has two consumers:
// gin's own SetTrustedProxies, which main.go calls with it, and trustedPeer
// below, which the public status page uses to decide whether to believe an
// X-Forwarded-Host. One variable, one parser.
func TrustedProxies() []string {
v := strings.TrimSpace(os.Getenv("TRUSTED_PROXIES"))
if v == "" {
return nil
}
out := []string{}
for _, p := range strings.Split(v, ",") {
if p = strings.TrimSpace(p); p != "" {
out = append(out, p)
}
}
return out
}
var (
trustedNetsOnce sync.Once
trustedNets []*net.IPNet
)
func parsedTrustedNets() []*net.IPNet {
trustedNetsOnce.Do(func() {
for _, entry := range TrustedProxies() {
if _, n, err := net.ParseCIDR(entry); err == nil {
trustedNets = append(trustedNets, n)
continue
}
// A bare address is a /32 or /128.
if ip := net.ParseIP(entry); ip != nil {
bits := 32
if ip.To4() == nil {
bits = 128
}
trustedNets = append(trustedNets, &net.IPNet{IP: ip, Mask: net.CIDRMask(bits, bits)})
}
}
})
return trustedNets
}
// trustedPeer reports whether the immediate peer is one of the configured
// proxies.
//
// It deliberately uses RemoteIP() rather than ClientIP(): ClientIP() is the
// reconstructed *client* address, which is derived from the very headers this
// function exists to decide whether to believe. X-Forwarded-Host selects a
// tenant on the public status route, so it is only honoured when the machine
// that actually opened the connection is trusted to have set it.
func trustedPeer(c *gin.Context) bool {
nets := parsedTrustedNets()
if len(nets) == 0 {
return false
}
ip := net.ParseIP(c.RemoteIP())
if ip == nil {
return false
}
for _, n := range nets {
if n.Contains(ip) {
return true
}
}
return false
}
+9
View File
@@ -237,3 +237,12 @@ type WorkloadLogsResponse struct {
Text string `json:"text"`
Truncated bool `json:"truncated"`
}
// --- status pages ---
// StatusIncidentUpdateRequest is one post to an incident's timeline. The author
// is taken from the session, never from the body.
type StatusIncidentUpdateRequest struct {
Status string `json:"status" binding:"required"`
Body string `json:"body" binding:"required"`
}
+67 -10
View File
@@ -23,6 +23,10 @@ var (
const instanceCacheTTL = 60 * time.Second
// soleInstanceCacheKey cannot collide with a slug: a slug is [a-z0-9-] and can
// never contain a NUL.
const soleInstanceCacheKey = "\x00sole"
func appRootLabel() string {
if v := os.Getenv("APP_ROOT_LABEL"); v != "" {
return strings.ToLower(v)
@@ -50,25 +54,78 @@ func hostSlug(host string) string {
return parts[0]
}
// HostSlug exposes the slug rules to callers outside this package that need to
// distinguish "this host names no instance at all" from "this host names an
// instance that does not exist". It is a thin wrapper rather than a second
// implementation on purpose.
func HostSlug(host string) string { return hostSlug(host) }
// InstanceFromHost resolves the instance named by the request's own Host
// header. Callers that must resolve a host from somewhere else — the public
// status page reads a trusted X-Forwarded-Host — use InstanceForHost so the
// slug rules and the 60s cache stay single-implementation.
func InstanceFromHost(c *gin.Context) (*models.Instance, bool) {
slug := hostSlug(c.Request.Host)
return InstanceForHost(c.Request.Host)
}
// InstanceForHost is InstanceFromHost with the host supplied explicitly.
func InstanceForHost(host string) (*models.Instance, bool) {
slug := hostSlug(host)
if slug == "" {
return nil, false
}
instanceCacheMu.Lock()
if e, ok := instanceCache[slug]; ok && time.Since(e.at) < instanceCacheTTL {
instanceCacheMu.Unlock()
return e.instance, e.instance != nil
if inst, hit := cachedInstanceFor(slug); hit {
return inst, inst != nil
}
instanceCacheMu.Unlock()
inst, err := services.GetInstanceBySlug(slug)
if err != nil || inst == nil {
// Negative entries are cached too. Without them an unknown but
// well-formed host costs a Mongo query per anonymous request, which
// the public status page exposes to the open internet — and the
// round trip is itself a timing oracle separating "no such instance"
// from "instance exists, page does not".
storeInstance(slug, nil)
return nil, false
}
instanceCacheMu.Lock()
instanceCache[slug] = cachedInstance{instance: inst, at: time.Now()}
instanceCacheMu.Unlock()
storeInstance(slug, inst)
return inst, true
}
// SoleInstance resolves the one instance of a deployment that has exactly one.
// It is how a self-hosted install serves a host that names no slug at all —
// vantage.acme.com, status.acme.com, or a bare address. It reuses the same
// count-then-read that bootstrap uses, and refuses rather than guessing when
// more than one instance exists.
func SoleInstance() (*models.Instance, bool) {
if inst, hit := cachedInstanceFor(soleInstanceCacheKey); hit {
return inst, inst != nil
}
n, err := services.CountInstances()
if err != nil || n != 1 {
storeInstance(soleInstanceCacheKey, nil)
return nil, false
}
inst, err := services.FirstInstance()
if err != nil || inst == nil {
storeInstance(soleInstanceCacheKey, nil)
return nil, false
}
storeInstance(soleInstanceCacheKey, inst)
return inst, true
}
func cachedInstanceFor(key string) (*models.Instance, bool) {
instanceCacheMu.Lock()
defer instanceCacheMu.Unlock()
if e, ok := instanceCache[key]; ok && time.Since(e.at) < instanceCacheTTL {
return e.instance, true
}
return nil, false
}
func storeInstance(key string, inst *models.Instance) {
instanceCacheMu.Lock()
instanceCache[key] = cachedInstance{instance: inst, at: time.Now()}
instanceCacheMu.Unlock()
}
+5
View File
@@ -17,6 +17,10 @@ const (
TypeTCP = "tcp"
TypeICMP = "icmp"
TypeTLS = "tls"
// UserAgent identifies Vantage monitor traffic so a WAF rule can single it
// out. Match on a prefix, not equality: the version moves.
UserAgent = "Vantage-Monitor/1.0 (+https://vantage.hostxtra.co.uk)"
)
type Spec struct {
@@ -80,6 +84,7 @@ func runHTTP(ctx context.Context, s Spec) Result {
if err != nil {
return Result{Message: err.Error()}
}
req.Header.Set("User-Agent", UserAgent)
resp, err := client.Do(req)
if err != nil {
return Result{LatencyMs: msSince(start), Message: err.Error()}
+44
View File
@@ -14,6 +14,28 @@ const (
ChannelTelegram = "telegram"
)
// RedactedSecret is what a channel's secret config values read as over the API.
// It is a sentinel and not merely a mask: a client may write it straight back,
// and the value it stood for is preserved. See NotificationChannel.Redacted.
const RedactedSecret = "••••••••"
// channelSecretKeys names, per channel type, the config entries that are
// credentials rather than settings. A Slack or Discord webhook URL is on this
// list because possession of the URL *is* the authorisation to post to that
// channel — there is nothing else to steal.
var channelSecretKeys = map[string][]string{
ChannelWebhook: {"url"},
ChannelSlack: {"url"},
ChannelDiscord: {"url"},
ChannelTelegram: {"token"},
ChannelSMTP: {"password"},
}
// ChannelSecretKeys reports which config keys of a channel type are secret.
func ChannelSecretKeys(channelType string) []string {
return channelSecretKeys[channelType]
}
type NotificationChannel struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
InstanceID string `bson:"instance_id" json:"instance_id"`
@@ -24,3 +46,25 @@ type NotificationChannel struct {
Enabled bool `bson:"enabled" json:"enabled"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
}
// Redacted returns a copy with every secret config value replaced by
// RedactedSecret, for handing to a client. Nothing internal uses it: the
// dispatchers read the stored document through GetChannel/GetChannels, so the
// redaction is a property of the API boundary and cannot break delivery.
//
// A set-but-secret key keeps its key, so a caller can still tell configured
// from absent; an empty value is left empty rather than being dressed up as a
// credential that is not there.
func (c NotificationChannel) Redacted() NotificationChannel {
out := c
out.Config = make(map[string]string, len(c.Config))
for k, v := range c.Config {
out.Config[k] = v
}
for _, k := range ChannelSecretKeys(c.Type) {
if out.Config[k] != "" {
out.Config[k] = RedactedSecret
}
}
return out
}
+23 -4
View File
@@ -43,10 +43,14 @@ type MonitorState struct {
}
type Monitor struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
InstanceID string `bson:"instance_id" json:"instance_id"`
MonitorID string `bson:"monitor_id" json:"monitor_id"`
Name string `bson:"name" json:"name"`
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
InstanceID string `bson:"instance_id" json:"instance_id"`
MonitorID string `bson:"monitor_id" json:"monitor_id"`
Name string `bson:"name" json:"name"`
// Group is a display-only label. It buckets rows on the monitors page and
// has no effect on scheduling, alerting or scope; an empty group means the
// monitor is listed on its own under "Ungrouped".
Group string `bson:"group,omitempty" json:"group,omitempty"`
Type string `bson:"type" json:"type"`
Target MonitorTarget `bson:"target" json:"target"`
IntervalSec int `bson:"interval_sec" json:"interval_sec"`
@@ -67,6 +71,21 @@ type Incident struct {
Cause string `bson:"cause,omitempty" json:"cause,omitempty"`
}
// MonitorSample is one check result, kept only long enough to draw the
// sub-hour views of the history chart. Rollup remains the durable record: a
// sample expires by TTL, a rollup does not.
//
// It carries no message. The failure text is on the incident, and a document
// per check is the one place in this schema where a few bytes multiply by the
// check rate.
type MonitorSample struct {
InstanceID string `bson:"instance_id" json:"instance_id"`
MonitorID string `bson:"monitor_id" json:"monitor_id"`
At time.Time `bson:"at" json:"at"`
Up bool `bson:"up" json:"up"`
LatencyMs int `bson:"latency_ms" json:"latency_ms"`
}
type Rollup struct {
InstanceID string `bson:"instance_id" json:"instance_id"`
MonitorID string `bson:"monitor_id" json:"monitor_id"`
+110
View File
@@ -0,0 +1,110 @@
package models
import (
"time"
"go.mongodb.org/mongo-driver/v2/bson"
)
// A status page entry kind. Incidents and maintenance share one document
// because they share a timeline, an impact and a set of affected components.
const (
StatusKindIncident = "incident"
StatusKindMaintenance = "maintenance"
)
const (
ImpactNone = "none"
ImpactMinor = "minor"
ImpactMajor = "major"
ImpactCritical = "critical"
)
// Incident statuses.
const (
IncidentInvestigating = "investigating"
IncidentIdentified = "identified"
IncidentMonitoring = "monitoring"
IncidentResolved = "resolved"
)
// Maintenance statuses.
const (
MaintenanceScheduled = "scheduled"
MaintenanceInProgress = "in_progress"
MaintenanceCompleted = "completed"
)
// StatusPageEntry names one monitor on one page.
//
// DisplayName overrides the monitor's own name for this page only. A monitor's
// internal name is frequently not a name anybody wants published, and the same
// monitor may need different words on a customer page and a partner page.
type StatusPageEntry struct {
MonitorID string `bson:"monitor_id" json:"monitor_id"`
DisplayName string `bson:"display_name,omitempty" json:"display_name,omitempty"`
}
// StatusPageSection is page-local and unrelated to Monitor.Group, which labels
// rows on the authenticated monitors list.
type StatusPageSection struct {
Name string `bson:"name" json:"name"`
Entries []StatusPageEntry `bson:"entries" json:"entries"`
}
// StatusPageBanner is three fields on the page rather than a collection,
// because it is one string with no lifecycle.
type StatusPageBanner struct {
Enabled bool `bson:"enabled" json:"enabled"`
Level string `bson:"level,omitempty" json:"level,omitempty"`
Text string `bson:"text,omitempty" json:"text,omitempty"`
}
// StatusPage is read whole, always, which is why its structure is embedded
// rather than joined: one page is one read is one cache fill.
type StatusPage struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
InstanceID string `bson:"instance_id" json:"instance_id"`
PageID string `bson:"page_id" json:"page_id"`
Title string `bson:"title" json:"title"`
Description string `bson:"description,omitempty" json:"description,omitempty"`
LogoURL string `bson:"logo_url,omitempty" json:"logo_url,omitempty"`
Published bool `bson:"published" json:"published"`
Banner StatusPageBanner `bson:"banner" json:"banner"`
Sections []StatusPageSection `bson:"sections" json:"sections"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
}
type StatusIncidentUpdate struct {
At time.Time `bson:"at" json:"at"`
Status string `bson:"status" json:"status"`
Body string `bson:"body" json:"body"`
Author string `bson:"author" json:"author"`
}
// StatusIncident is operator-authored. Monitor-detected outages stay in the
// incidents collection and are derived at assembly time; copying them here
// would be a second writer for the same fact.
//
// PageIDs is explicit rather than derived from AffectedMonitors: deriving it
// would mean adding a monitor to a page retroactively republishes old
// incidents to a new audience.
type StatusIncident struct {
ID bson.ObjectID `bson:"_id,omitempty" json:"_id,omitempty"`
InstanceID string `bson:"instance_id" json:"instance_id"`
IncidentID string `bson:"incident_id" json:"incident_id"`
PageIDs []string `bson:"page_ids" json:"page_ids"`
Kind string `bson:"kind" json:"kind"`
Title string `bson:"title" json:"title"`
Impact string `bson:"impact" json:"impact"`
AffectedMonitors []string `bson:"affected_monitors,omitempty" json:"affected_monitors,omitempty"`
Status string `bson:"status" json:"status"`
ScheduledStart *time.Time `bson:"scheduled_start,omitempty" json:"scheduled_start,omitempty"`
ScheduledEnd *time.Time `bson:"scheduled_end,omitempty" json:"scheduled_end,omitempty"`
Updates []StatusIncidentUpdate `bson:"updates" json:"updates"`
StartedAt time.Time `bson:"started_at" json:"started_at"`
ResolvedAt *time.Time `bson:"resolved_at,omitempty" json:"resolved_at,omitempty"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
UpdatedAt time.Time `bson:"updated_at" json:"updated_at"`
}
+45
View File
@@ -90,12 +90,57 @@ func CreateChannel(instanceID string, ch *models.NotificationChannel) (*models.N
}
func UpdateChannel(instanceID, channelID string, upd bson.M) error {
if cfg, ok := upd["config"].(map[string]string); ok {
merged, err := mergeChannelSecrets(instanceID, channelID, upd, cfg)
if err != nil {
return err
}
upd["config"] = merged
}
ctx, cancel := monCtx()
defer cancel()
_, err := db.Col("notification_channels").UpdateOne(ctx, bson.M{"channel_id": channelID, "instance_id": instanceID}, bson.M{"$set": upd})
return err
}
// mergeChannelSecrets resolves models.RedactedSecret back to what it stood for.
//
// The API hands out a sentinel rather than the credential, and the UI's edit
// form round-trips whatever it was given, so an ordinary "rename this channel"
// save arrives carrying the sentinel in place of the password. Writing it
// through would replace the credential with eight bullet characters and break
// delivery on the next alert. A value that is not the sentinel is written
// verbatim — including the empty string, which is how a credential is cleared.
func mergeChannelSecrets(instanceID, channelID string, upd bson.M, cfg map[string]string) (map[string]string, error) {
stored, err := GetChannel(instanceID, channelID)
if err != nil {
return nil, err
}
if stored == nil {
return cfg, nil
}
// The secret keys are the ones of the type being saved, which the same
// request may be changing.
channelType := stored.Type
if t, ok := upd["type"].(string); ok && t != "" {
channelType = t
}
out := make(map[string]string, len(cfg))
for k, v := range cfg {
out[k] = v
}
for _, k := range models.ChannelSecretKeys(channelType) {
if out[k] == models.RedactedSecret {
if prev, ok := stored.Config[k]; ok {
out[k] = prev
} else {
delete(out, k)
}
}
}
return out, nil
}
func DeleteChannel(instanceID, channelID string) error {
ctx, cancel := monCtx()
defer cancel()
+8 -40
View File
@@ -1,22 +1,23 @@
package services
import (
"crypto/aes"
"crypto/cipher"
"crypto/rand"
"encoding/hex"
"fmt"
"io"
"os"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/cryptobox"
)
// encryptionKey reads KEY_ENCRYPTION_KEY. The cipher itself lives in
// shared/cryptobox so vantagectl's verify probe uses the same implementation
// rather than a second copy that can drift.
func encryptionKey() ([]byte, error) {
raw := os.Getenv("KEY_ENCRYPTION_KEY")
if raw == "" {
return nil, fmt.Errorf("KEY_ENCRYPTION_KEY is not set")
}
key, err := hex.DecodeString(raw)
if err != nil || len(key) != 32 {
if err != nil || len(key) != cryptobox.KeySize {
return nil, fmt.Errorf("KEY_ENCRYPTION_KEY must be a 64-character hex string (32 bytes)")
}
return key, nil
@@ -27,20 +28,7 @@ func encryptString(plaintext string) (string, error) {
if err != nil {
return "", err
}
block, err := aes.NewCipher(key)
if err != nil {
return "", err
}
gcm, err := cipher.NewGCM(block)
if err != nil {
return "", err
}
nonce := make([]byte, gcm.NonceSize())
if _, err := io.ReadFull(rand.Reader, nonce); err != nil {
return "", err
}
sealed := gcm.Seal(nonce, nonce, []byte(plaintext), nil)
return hex.EncodeToString(sealed), nil
return cryptobox.Seal(key, plaintext)
}
func decryptString(ciphertextHex string) (string, error) {
@@ -48,27 +36,7 @@ func decryptString(ciphertextHex string) (string, error) {
if err != nil {
return "", err
}
data, err := hex.DecodeString(ciphertextHex)
if err != nil {
return "", fmt.Errorf("invalid ciphertext encoding")
}
block, err := aes.NewCipher(key)
if err != nil {
return "", err
}
gcm, err := cipher.NewGCM(block)
if err != nil {
return "", err
}
nonceSize := gcm.NonceSize()
if len(data) < nonceSize {
return "", fmt.Errorf("ciphertext too short")
}
plaintext, err := gcm.Open(nil, data[:nonceSize], data[nonceSize:], nil)
if err != nil {
return "", fmt.Errorf("decryption failed")
}
return string(plaintext), nil
return cryptobox.Open(key, ciphertextHex)
}
func encryptPrivateKey(plaintext string) (string, error) { return encryptString(plaintext) }
@@ -36,6 +36,7 @@ var ScopedCollections = []string{
"monitors",
"incidents",
"monitor_rollups",
"monitor_samples",
"notification_channels",
"console_sessions",
"audit_logs",
@@ -45,6 +46,8 @@ var ScopedCollections = []string{
"vuln_alert_rules",
"api_tokens",
"server_workloads",
"status_pages",
"status_incidents",
}
// collectionRenames maps the two collections whose names change. Ordered so the
+70
View File
@@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"log"
"strings"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/checker"
@@ -21,6 +22,22 @@ func monCtx() (context.Context, context.CancelFunc) {
return context.WithTimeout(context.Background(), 5*time.Second)
}
// MaxMonitorGroupLen bounds the display-only group label. It is a heading on
// the monitors page, not an identifier, so the cap is about the layout rather
// than storage.
const MaxMonitorGroupLen = 48
// normaliseGroup collapses the ways two people write the same group. Grouping
// is by exact string, so " Production " and "Production" must not become two
// headings.
func normaliseGroup(g string) (string, error) {
g = strings.Join(strings.Fields(g), " ")
if len([]rune(g)) > MaxMonitorGroupLen {
return "", fmt.Errorf("group must be %d characters or fewer", MaxMonitorGroupLen)
}
return g, nil
}
func SpecFor(m *models.Monitor) checker.Spec {
return checker.Spec{
Type: m.Type,
@@ -126,6 +143,11 @@ func CreateMonitor(instanceID string, m *models.Monitor) (*models.Monitor, error
if err := validateRunner(instanceID, m.Runner); err != nil {
return nil, err
}
group, err := normaliseGroup(m.Group)
if err != nil {
return nil, err
}
m.Group = group
m.InstanceID = instanceID
m.MonitorID = uuid.NewString()
m.CreatedAt = time.Now()
@@ -158,6 +180,17 @@ func UpdateMonitor(instanceID, monitorID string, upd bson.M) error {
return err
}
}
if raw, present := upd["group"]; present {
g, ok := raw.(string)
if !ok {
return fmt.Errorf("group must be a string")
}
group, err := normaliseGroup(g)
if err != nil {
return err
}
upd["group"] = group
}
if raw, present := upd["runner"]; present {
runner, ok := raw.(string)
if !ok {
@@ -188,6 +221,7 @@ func DeleteMonitor(instanceID, monitorID string) error {
}
db.Col("incidents").DeleteMany(ctx, bson.M{"monitor_id": monitorID, "instance_id": instanceID})
db.Col("monitor_rollups").DeleteMany(ctx, bson.M{"monitor_id": monitorID, "instance_id": instanceID})
db.Col("monitor_samples").DeleteMany(ctx, bson.M{"monitor_id": monitorID, "instance_id": instanceID})
return nil
}
@@ -225,6 +259,31 @@ func UptimeRollups(instanceID, monitorID string, since time.Time) ([]models.Roll
return out, nil
}
// MaxMonitorSamples bounds one range read. At the 10s floor, 48h is 17,280
// checks; the chart buckets them anyway, so a cap costs nothing visible and
// stops one monitor pulling a megabyte of JSON per poll.
const MaxMonitorSamples = 6000
// MonitorSamples returns individual check results since a point in time,
// oldest first. Samples older than MonitorSampleTTL have expired, so an early
// `since` silently returns a shorter window rather than an error — the caller
// draws the gap.
func MonitorSamples(instanceID, monitorID string, since time.Time) ([]models.MonitorSample, error) {
ctx, cancel := monCtx()
defer cancel()
cur, err := db.Col("monitor_samples").Find(ctx,
bson.M{"monitor_id": monitorID, "instance_id": instanceID, "at": bson.M{"$gte": since}},
options.Find().SetSort(bson.M{"at": 1}).SetLimit(MaxMonitorSamples))
if err != nil {
return nil, err
}
var out []models.MonitorSample
if err := cur.All(ctx, &out); err != nil {
return nil, err
}
return out, nil
}
func IngestResult(instanceID, runner, monitorID string, res checker.Result) error {
if instanceID == "" {
return errors.New("instance id required")
@@ -295,6 +354,17 @@ func ingestResult(instanceID, runner, monitorID string, res checker.Result) erro
up = 1
}
/* The sample is the same result at full resolution, expiring by TTL. It is
written next to the rollup rather than instead of it: the rollup is what
survives, the sample is what the sub-hour views read. */
db.Col("monitor_samples").InsertOne(ctx, models.MonitorSample{
InstanceID: m.InstanceID,
MonitorID: monitorID,
At: now,
Up: res.Up,
LatencyMs: res.LatencyMs,
})
db.Col("monitor_rollups").UpdateOne(ctx,
bson.M{"monitor_id": monitorID, "period_start": bucket},
bson.M{
@@ -0,0 +1,42 @@
package services
import (
"context"
"log"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/db"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
// MonitorSampleTTL is how long an individual check result is kept.
//
// It matches the longest range the chart draws from samples rather than the
// longest range it draws at all: 24h and 48h come from the hourly rollups,
// which are permanent. Keeping samples past the window that reads them would
// only grow the collection.
const MonitorSampleTTL = 48 * time.Hour
// EnsureMonitorSampleIndexes declares the sample range index and its TTL.
//
// Warn rather than fatal, like the other history indexes — but note the TTL is
// not an optimisation: without it nothing ever removes a sample, and the
// collection grows at the fleet's total check rate forever. A boot that logs
// this warning needs following up.
func EnsureMonitorSampleIndexes() error {
ctx := context.Background()
idx := []mongo.IndexModel{
// Every read is a range scan over this key.
{Keys: bson.D{{Key: "instance_id", Value: 1}, {Key: "monitor_id", Value: 1}, {Key: "at", Value: 1}}},
// Expiry is Mongo's job: a sweeper would be another leader-scoped loop
// doing what the server already does for free.
{Keys: bson.D{{Key: "at", Value: 1}}, Options: options.Index().SetExpireAfterSeconds(int32(MonitorSampleTTL.Seconds()))},
}
if _, err := db.Col("monitor_samples").Indexes().CreateMany(ctx, idx); err != nil {
log.Printf("warning: monitor_samples indexes: %v", err)
}
return nil
}
+1
View File
@@ -26,6 +26,7 @@ var ScopeResources = []string{
"vulns",
"workloads",
"settings",
"status",
}
const (
+323
View File
@@ -0,0 +1,323 @@
package services
import (
"errors"
"fmt"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/db"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
"github.com/google/uuid"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
var ErrIncidentNotFound = errors.New("status incident not found")
var incidentStatuses = map[string]bool{
models.IncidentInvestigating: true,
models.IncidentIdentified: true,
models.IncidentMonitoring: true,
models.IncidentResolved: true,
}
var maintenanceStatuses = map[string]bool{
models.MaintenanceScheduled: true,
models.MaintenanceInProgress: true,
models.MaintenanceCompleted: true,
}
var impacts = map[string]bool{
models.ImpactNone: true, models.ImpactMinor: true,
models.ImpactMajor: true, models.ImpactCritical: true,
}
func validateIncident(inc *models.StatusIncident) error {
if inc.Title == "" {
return fmt.Errorf("%w: title is required", ErrPageInvalid)
}
if len(inc.PageIDs) == 0 {
return fmt.Errorf("%w: at least one page is required", ErrPageInvalid)
}
switch inc.Kind {
case models.StatusKindIncident:
if !incidentStatuses[inc.Status] {
return fmt.Errorf("%w: invalid incident status %q", ErrPageInvalid, inc.Status)
}
case models.StatusKindMaintenance:
if !maintenanceStatuses[inc.Status] {
return fmt.Errorf("%w: invalid maintenance status %q", ErrPageInvalid, inc.Status)
}
if inc.ScheduledStart == nil || inc.ScheduledEnd == nil {
return fmt.Errorf("%w: maintenance needs a scheduled start and end", ErrPageInvalid)
}
if !inc.ScheduledEnd.After(*inc.ScheduledStart) {
return fmt.Errorf("%w: maintenance must end after it starts", ErrPageInvalid)
}
default:
return fmt.Errorf("%w: invalid kind %q", ErrPageInvalid, inc.Kind)
}
if inc.Impact == "" {
inc.Impact = models.ImpactNone
}
if !impacts[inc.Impact] {
return fmt.Errorf("%w: invalid impact %q", ErrPageInvalid, inc.Impact)
}
return nil
}
// checkAffectedOnPages refuses an incident naming a component none of its pages
// carries.
//
// An incident's affected components are the page's own components, not the
// fleet's monitors: publishing "api-gateway is degraded" on a page that never
// listed api-gateway names a machine to the public that the page deliberately
// does not, which is the same leak assembleSnapshot's redaction boundary exists
// to prevent — reached from the authoring side instead of the read side.
//
// It is a separate pass rather than part of validateIncident because it reads
// the database, and validateIncident is a pure function of the document. The
// UI only offers the page's components, but as elsewhere the API is the
// boundary and the UI is the courtesy.
//
// A monitor dropped from the page AFTER an incident named it makes the next
// edit of that incident fail, and that is intended: the fix is one unchecked
// box, and the alternative is a page quietly publishing a component it no
// longer has.
func checkAffectedOnPages(instanceID string, inc *models.StatusIncident) error {
if len(inc.AffectedMonitors) == 0 {
return nil
}
ctx, cancel := spCtx()
defer cancel()
cur, err := db.Col("status_pages").Find(ctx, bson.M{
"instance_id": instanceID,
"page_id": bson.M{"$in": inc.PageIDs},
})
if err != nil {
return err
}
var pages []models.StatusPage
if err := cur.All(ctx, &pages); err != nil {
return err
}
onPage := map[string]bool{}
for _, p := range pages {
for _, sec := range p.Sections {
for _, e := range sec.Entries {
onPage[e.MonitorID] = true
}
}
}
for _, id := range inc.AffectedMonitors {
if !onPage[id] {
return fmt.Errorf("%w: %s is not a component of this status page; add it to the page first, or leave it out of the incident",
ErrPageInvalid, id)
}
}
return nil
}
func CreateStatusIncident(instanceID string, inc *models.StatusIncident) (*models.StatusIncident, error) {
if err := validateIncident(inc); err != nil {
return nil, err
}
if err := checkAffectedOnPages(instanceID, inc); err != nil {
return nil, err
}
inc.ID = bson.ObjectID{}
inc.InstanceID = instanceID
inc.IncidentID = uuid.NewString()
now := time.Now()
inc.CreatedAt = now
inc.UpdatedAt = now
if inc.StartedAt.IsZero() {
if inc.Kind == models.StatusKindMaintenance && inc.ScheduledStart != nil {
inc.StartedAt = *inc.ScheduledStart
} else {
inc.StartedAt = now
}
}
if inc.Updates == nil {
inc.Updates = []models.StatusIncidentUpdate{}
}
ctx, cancel := spCtx()
defer cancel()
if _, err := db.Col("status_incidents").InsertOne(ctx, inc); err != nil {
return nil, err
}
invalidatePages(instanceID, inc.PageIDs)
return inc, nil
}
func invalidatePages(instanceID string, pageIDs []string) {
for _, p := range pageIDs {
InvalidateStatusCache(instanceID, p)
}
}
func getIncident(instanceID, incidentID string) (*models.StatusIncident, error) {
ctx, cancel := spCtx()
defer cancel()
var inc models.StatusIncident
err := db.Col("status_incidents").
FindOne(ctx, bson.M{"instance_id": instanceID, "incident_id": incidentID}).
Decode(&inc)
if errors.Is(err, mongo.ErrNoDocuments) {
return nil, ErrIncidentNotFound
}
if err != nil {
return nil, err
}
return &inc, nil
}
func ListStatusIncidents(instanceID, pageID string) ([]models.StatusIncident, error) {
filter := bson.M{"instance_id": instanceID}
if pageID != "" {
filter["page_ids"] = pageID
}
ctx, cancel := spCtx()
defer cancel()
cur, err := db.Col("status_incidents").Find(ctx, filter,
options.Find().SetSort(bson.M{"started_at": -1}))
if err != nil {
return nil, err
}
out := []models.StatusIncident{}
if err := cur.All(ctx, &out); err != nil {
return nil, err
}
return out, nil
}
// ListStatusIncidentsForPage is the public read's query: one page, bounded by
// the history window, so a five-year-old instance does not assemble five years
// of incidents on every cache miss.
func ListStatusIncidentsForPage(instanceID, pageID string, since time.Time) ([]models.StatusIncident, error) {
ctx, cancel := spCtx()
defer cancel()
cur, err := db.Col("status_incidents").Find(ctx, bson.M{
"instance_id": instanceID,
"page_ids": pageID,
"$or": []bson.M{
{"started_at": bson.M{"$gte": since}},
{"resolved_at": nil},
{"status": models.MaintenanceScheduled},
},
}, options.Find().SetSort(bson.M{"started_at": -1}))
if err != nil {
return nil, err
}
out := []models.StatusIncident{}
if err := cur.All(ctx, &out); err != nil {
return nil, err
}
return out, nil
}
func UpdateStatusIncident(instanceID, incidentID string, inc *models.StatusIncident) (*models.StatusIncident, error) {
existing, err := getIncident(instanceID, incidentID)
if err != nil {
return nil, err
}
inc.Kind = existing.Kind // kind is fixed at creation
if err := validateIncident(inc); err != nil {
return nil, err
}
if err := checkAffectedOnPages(instanceID, inc); err != nil {
return nil, err
}
set := bson.M{
"page_ids": inc.PageIDs,
"title": inc.Title,
"impact": inc.Impact,
"affected_monitors": inc.AffectedMonitors,
"status": inc.Status,
"scheduled_start": inc.ScheduledStart,
"scheduled_end": inc.ScheduledEnd,
"updated_at": time.Now(),
}
if inc.Status == models.IncidentResolved || inc.Status == models.MaintenanceCompleted {
if existing.ResolvedAt == nil {
now := time.Now()
set["resolved_at"] = now
}
} else {
// Reopening clears it, so a mistakenly resolved incident does not keep
// a resolution time it no longer has.
set["resolved_at"] = nil
}
ctx, cancel := spCtx()
defer cancel()
if _, err := db.Col("status_incidents").UpdateOne(ctx,
bson.M{"instance_id": instanceID, "incident_id": incidentID},
bson.M{"$set": set}); err != nil {
return nil, err
}
// Both old and new page sets, or a page the incident was just removed from
// keeps showing it for up to 30 seconds.
invalidatePages(instanceID, existing.PageIDs)
invalidatePages(instanceID, inc.PageIDs)
return getIncident(instanceID, incidentID)
}
func AppendStatusIncidentUpdate(instanceID, incidentID, status, body, author string) (*models.StatusIncident, error) {
existing, err := getIncident(instanceID, incidentID)
if err != nil {
return nil, err
}
if body == "" {
return nil, fmt.Errorf("%w: update body is required", ErrPageInvalid)
}
valid := incidentStatuses
if existing.Kind == models.StatusKindMaintenance {
valid = maintenanceStatuses
}
if !valid[status] {
return nil, fmt.Errorf("%w: invalid status %q for a %s", ErrPageInvalid, status, existing.Kind)
}
upd := models.StatusIncidentUpdate{At: time.Now(), Status: status, Body: body, Author: author}
set := bson.M{"status": status, "updated_at": upd.At}
if status == models.IncidentResolved || status == models.MaintenanceCompleted {
set["resolved_at"] = upd.At
} else {
// Reopening via an appended update must clear a previously-set
// resolved_at the same way UpdateStatusIncident does — otherwise a
// resolved incident reopened to "monitoring" keeps a stale resolved_at
// and silently drops off ListStatusIncidentsForPage once started_at
// ages past the since cutoff, because none of its $or clauses match.
set["resolved_at"] = nil
}
ctx, cancel := spCtx()
defer cancel()
if _, err := db.Col("status_incidents").UpdateOne(ctx,
bson.M{"instance_id": instanceID, "incident_id": incidentID},
bson.M{"$push": bson.M{"updates": upd}, "$set": set}); err != nil {
return nil, err
}
invalidatePages(instanceID, existing.PageIDs)
return getIncident(instanceID, incidentID)
}
func DeleteStatusIncident(instanceID, incidentID string) error {
existing, err := getIncident(instanceID, incidentID)
if err != nil {
return err
}
ctx, cancel := spCtx()
defer cancel()
if _, err := db.Col("status_incidents").DeleteOne(ctx,
bson.M{"instance_id": instanceID, "incident_id": incidentID}); err != nil {
return err
}
invalidatePages(instanceID, existing.PageIDs)
return nil
}
+232
View File
@@ -0,0 +1,232 @@
package services
import (
"context"
"errors"
"fmt"
"log"
"regexp"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/db"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
"github.com/redis/go-redis/v9"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
// ErrInvalidPageID is returned for any page id that would not be safe or
// pleasant in a URL handed to a customer.
var ErrInvalidPageID = errors.New("page id must be 3-40 characters of a-z, 0-9 and -, starting and ending alphanumeric")
// The slug is operator-chosen rather than random because it is printed on
// support pages and typed by people. First and last characters are
// alphanumeric so a page id never reads as a flag or a trailing separator.
var pageIDRe = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{1,38}[a-z0-9]$`)
func ValidatePageID(id string) error {
if !pageIDRe.MatchString(id) {
return ErrInvalidPageID
}
return nil
}
func statusCacheKey(instanceID, pageID string) string {
return "vantage:status:" + instanceID + ":" + pageID
}
func spCtx() (context.Context, context.CancelFunc) {
return context.WithTimeout(context.Background(), 5*time.Second)
}
// EnsureStatusPageIndexes follows EnsureWorkflowIndexes rather than
// EnsureAuthIndexes: the unique page_id index is a correctness property, but a
// missing secondary index on a small collection degrades to a scan, which is no
// reason to refuse to serve the fleet. main.go warns rather than exiting.
//
// All three are attempted and the failures joined, rather than returning on
// the first. The three are independent, and two of them are uniqueness
// constraints — bailing out on the status_pages index meant a transient
// failure there silently left status_incidents with no unique
// (instance_id, incident_id) index at all.
func EnsureStatusPageIndexes() error {
ctx, cancel := spCtx()
defer cancel()
var errs []error
if _, err := db.Col("status_pages").Indexes().CreateOne(ctx, mongo.IndexModel{
Keys: bson.D{{Key: "instance_id", Value: 1}, {Key: "page_id", Value: 1}},
Options: options.Index().SetUnique(true),
}); err != nil {
errs = append(errs, fmt.Errorf("status_pages (instance_id, page_id): %w", err))
}
if _, err := db.Col("status_incidents").Indexes().CreateOne(ctx, mongo.IndexModel{
Keys: bson.D{{Key: "instance_id", Value: 1}, {Key: "incident_id", Value: 1}},
Options: options.Index().SetUnique(true),
}); err != nil {
errs = append(errs, fmt.Errorf("status_incidents (instance_id, incident_id): %w", err))
}
// The public read filters by page and orders by recency, and it is the
// only query on this collection that runs on every visit.
if _, err := db.Col("status_incidents").Indexes().CreateOne(ctx, mongo.IndexModel{
Keys: bson.D{{Key: "instance_id", Value: 1}, {Key: "page_ids", Value: 1}, {Key: "started_at", Value: -1}},
}); err != nil {
errs = append(errs, fmt.Errorf("status_incidents (instance_id, page_ids, started_at): %w", err))
}
return errors.Join(errs...)
}
var (
ErrPageNotFound = errors.New("status page not found")
ErrPageIDTaken = errors.New("that page id is already in use")
// ErrPageInvalid is the sentinel for validation failures on a page or
// incident body — anything the caller can fix by sending a different
// request. statusPageError maps it to 400; wrap it rather than returning a
// bare error, or a bad request answers 500.
ErrPageInvalid = errors.New("status page request invalid")
)
// statusRedis is set by main.go at boot. It is nil in any process that has not
// set it, and every use below treats nil as "no cache" rather than an error.
var statusRedis *redis.Client
func SetStatusRedis(c *redis.Client) { statusRedis = c }
// InvalidateStatusCache drops the assembled snapshot so an operator posting an
// incident update sees it immediately rather than wondering for half a minute
// whether it saved. Best effort: a stale entry expires in 30s anyway, and a
// Redis error here must not fail the write that already succeeded.
func InvalidateStatusCache(instanceID, pageID string) {
if statusRedis == nil {
return
}
ctx, cancel := spCtx()
defer cancel()
if err := statusRedis.Del(ctx, statusCacheKey(instanceID, pageID)).Err(); err != nil {
log.Printf("status cache invalidate %s/%s: %v", instanceID, pageID, err)
}
}
func ListStatusPages(instanceID string) ([]models.StatusPage, error) {
ctx, cancel := spCtx()
defer cancel()
cur, err := db.Col("status_pages").Find(ctx,
bson.M{"instance_id": instanceID},
options.Find().SetSort(bson.M{"created_at": 1}))
if err != nil {
return nil, err
}
pages := []models.StatusPage{}
if err := cur.All(ctx, &pages); err != nil {
return nil, err
}
return pages, nil
}
func GetStatusPage(instanceID, pageID string) (*models.StatusPage, error) {
ctx, cancel := spCtx()
defer cancel()
var p models.StatusPage
err := db.Col("status_pages").
FindOne(ctx, bson.M{"instance_id": instanceID, "page_id": pageID}).
Decode(&p)
if errors.Is(err, mongo.ErrNoDocuments) {
return nil, ErrPageNotFound
}
if err != nil {
return nil, err
}
return &p, nil
}
func CreateStatusPage(instanceID string, p *models.StatusPage) (*models.StatusPage, error) {
if err := ValidatePageID(p.PageID); err != nil {
return nil, err
}
if p.Title == "" {
return nil, fmt.Errorf("%w: title is required", ErrPageInvalid)
}
p.ID = bson.ObjectID{}
p.InstanceID = instanceID
p.CreatedAt = time.Now()
p.UpdatedAt = p.CreatedAt
if p.Sections == nil {
p.Sections = []models.StatusPageSection{}
}
ctx, cancel := spCtx()
defer cancel()
res, err := db.Col("status_pages").InsertOne(ctx, p)
if mongo.IsDuplicateKeyError(err) {
// The unique index is what settles a race between two people reaching
// for one page id; a pre-check alone would not.
return nil, ErrPageIDTaken
}
if err != nil {
return nil, err
}
if oid, ok := res.InsertedID.(bson.ObjectID); ok {
p.ID = oid
}
return p, nil
}
// UpdateStatusPage replaces the whole page document except its identity and
// creation time. Last-write-wins over one small document beats merge semantics
// between two people editing one page, the same call PUT /servers/:id/tags
// already makes.
//
// The page id itself is immutable: it is a URL that has been handed out.
func UpdateStatusPage(instanceID, pageID string, p *models.StatusPage) (*models.StatusPage, error) {
if p.Title == "" {
return nil, fmt.Errorf("%w: title is required", ErrPageInvalid)
}
if p.Sections == nil {
p.Sections = []models.StatusPageSection{}
}
ctx, cancel := spCtx()
defer cancel()
res, err := db.Col("status_pages").UpdateOne(ctx,
bson.M{"instance_id": instanceID, "page_id": pageID},
bson.M{"$set": bson.M{
"title": p.Title,
"description": p.Description,
"logo_url": p.LogoURL,
"published": p.Published,
"banner": p.Banner,
"sections": p.Sections,
"updated_at": time.Now(),
}})
if err != nil {
return nil, err
}
if res.MatchedCount == 0 {
return nil, ErrPageNotFound
}
InvalidateStatusCache(instanceID, pageID)
return GetStatusPage(instanceID, pageID)
}
func DeleteStatusPage(instanceID, pageID string) error {
ctx, cancel := spCtx()
defer cancel()
res, err := db.Col("status_pages").DeleteOne(ctx,
bson.M{"instance_id": instanceID, "page_id": pageID})
if err != nil {
return err
}
if res.DeletedCount == 0 {
return ErrPageNotFound
}
// Authored incidents keep their page_ids entry. A page deleted by mistake
// and recreated with the same id gets its incident history back, and an id
// that is never reused costs two bytes in an array.
InvalidateStatusCache(instanceID, pageID)
return nil
}
@@ -0,0 +1,63 @@
package services
import (
"strings"
"testing"
)
// A new instance-scoped collection that is not in ScopedCollections leaves its
// rows behind when the instance is deleted. This is the cheapest possible
// guard against the omission.
func TestStatusCollectionsAreScoped(t *testing.T) {
want := []string{"status_pages", "status_incidents"}
for _, name := range want {
found := false
for _, got := range ScopedCollections {
if got == name {
found = true
break
}
}
if !found {
t.Errorf("ScopedCollections is missing %q", name)
}
}
}
func TestValidatePageID(t *testing.T) {
valid := []string{"api", "prod-eu", "status2", "a1b", strings.Repeat("a", 40)}
for _, s := range valid {
if err := ValidatePageID(s); err != nil {
t.Errorf("ValidatePageID(%q) = %v, want nil", s, err)
}
}
invalid := []string{
"", // empty
"ab", // too short
strings.Repeat("a", 41), // too long
"-api", // leading hyphen
"api-", // trailing hyphen
"API", // uppercase
"my page", // space
"api_v2", // underscore
"api/v2", // path separator
"..", // dots
}
for _, s := range invalid {
if err := ValidatePageID(s); err == nil {
t.Errorf("ValidatePageID(%q) = nil, want error", s)
}
}
}
func TestStatusCacheKeyIsScopedByInstance(t *testing.T) {
a := statusCacheKey("inst-a", "api")
b := statusCacheKey("inst-b", "api")
if a == b {
t.Fatalf("two instances share a cache key: %q", a)
}
if a != "vantage:status:inst-a:api" {
t.Fatalf("statusCacheKey = %q, want vantage:status:inst-a:api", a)
}
}
+594
View File
@@ -0,0 +1,594 @@
package services
import (
"encoding/json"
"log"
"sort"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/db"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/license"
"go.mongodb.org/mongo-driver/v2/bson"
)
// HistoryDays is the width of the public history bar. It is also the window the
// public uptime percentage is computed over.
const HistoryDays = 90
// Day cell and component states. "maintenance" and "no_data" exist only here:
// a monitor has no such states, and conflating no_data with down would report
// a component as broken for every day before it was created.
const (
PublicUp = "up"
PublicDown = "down"
PublicMaintenance = "maintenance"
PublicNoData = "no_data"
PublicPending = "pending"
PublicDegraded = "degraded"
)
type PublicDay struct {
Date string `json:"date"`
State string `json:"state"`
Uptime float64 `json:"uptime"`
}
// PublicComponent is everything an anonymous caller learns about a monitor.
//
// Deliberately absent, and it must stay that way: the target URL, host and
// port, the expected status and keyword, State.Message, State.CertExpiryAt,
// latency, the runner, and the notification channel ids.
type PublicComponent struct {
Name string `json:"name"`
Status string `json:"status"`
Uptime90d float64 `json:"uptime_90d"`
Days []PublicDay `json:"days"`
}
type PublicSection struct {
Name string `json:"name"`
Components []PublicComponent `json:"components"`
}
type PublicIncidentUpdate struct {
At time.Time `json:"at"`
Status string `json:"status"`
Body string `json:"body"`
}
// PublicIncident covers both authored incidents and derived monitor outages.
// A derived one carries no updates and no impact — and never a cause, which is
// where internal hostnames live.
type PublicIncident struct {
ID string `json:"id"`
Kind string `json:"kind"`
Title string `json:"title"`
Impact string `json:"impact,omitempty"`
Status string `json:"status"`
Affected []string `json:"affected,omitempty"`
StartedAt time.Time `json:"started_at"`
ResolvedAt *time.Time `json:"resolved_at,omitempty"`
ScheduledStart *time.Time `json:"scheduled_start,omitempty"`
ScheduledEnd *time.Time `json:"scheduled_end,omitempty"`
Updates []PublicIncidentUpdate `json:"updates,omitempty"`
}
type PublicBanner struct {
Level string `json:"level"`
Text string `json:"text"`
}
// StatusSnapshot is the entire public API surface of this feature.
type StatusSnapshot struct {
Available bool `json:"available"`
Reason string `json:"reason,omitempty"`
Title string `json:"title"`
Description string `json:"description,omitempty"`
LogoURL string `json:"logo_url,omitempty"`
Banner *PublicBanner `json:"banner,omitempty"`
Overall string `json:"overall"`
Sections []PublicSection `json:"sections"`
ActiveIncidents []PublicIncident `json:"active_incidents"`
UpcomingMaintenance []PublicIncident `json:"upcoming_maintenance"`
History []PublicIncident `json:"history"`
GeneratedAt time.Time `json:"generated_at"`
}
// snapshotInput is everything assembleSnapshot needs, already read. Keeping the
// assembly pure is what makes the redaction boundary testable without a
// database.
type snapshotInput struct {
Page models.StatusPage
Monitors map[string]models.Monitor
Rollups map[string][]models.Rollup
AutoIncidents []models.Incident
Authored []models.StatusIncident
Now time.Time
}
func assembleSnapshot(in snapshotInput) StatusSnapshot {
snap := StatusSnapshot{
Available: true,
Title: in.Page.Title,
Description: in.Page.Description,
LogoURL: in.Page.LogoURL,
Sections: []PublicSection{},
ActiveIncidents: []PublicIncident{},
UpcomingMaintenance: []PublicIncident{},
History: []PublicIncident{},
GeneratedAt: in.Now,
}
if in.Page.Banner.Enabled && in.Page.Banner.Text != "" {
snap.Banner = &PublicBanner{Level: in.Page.Banner.Level, Text: in.Page.Banner.Text}
}
authored := authoredForPage(in.Page.PageID, in.Authored)
underMaintenance := maintenanceMonitors(authored, in.Now)
// names maps monitor id to the name this page publishes, so incidents can
// name their affected components without reaching back into models.Monitor.
names := map[string]string{}
for _, sec := range in.Page.Sections {
out := PublicSection{Name: sec.Name, Components: []PublicComponent{}}
for _, entry := range sec.Entries {
mon, known := in.Monitors[entry.MonitorID]
name := publicName(entry, mon.MonitorID)
names[entry.MonitorID] = name
// Uptime is computed from the days as reported by rollups, before
// any maintenance repaint — a no_data day must never be counted as
// zero uptime just because it is later redrawn as "maintenance".
days := buildDays(in.Rollups[entry.MonitorID], in.Now)
comp := PublicComponent{
Name: name,
}
comp.Uptime90d = uptimeFromDays(days)
comp.Days = applyMaintenanceRepaint(days, underMaintenance[entry.MonitorID])
comp.Status = componentStatus(mon, known, underMaintenance[entry.MonitorID], in.Now)
out.Components = append(out.Components, comp)
}
snap.Sections = append(snap.Sections, out)
}
for _, inc := range authored {
p := publicFromAuthored(inc, names)
switch {
case inc.Kind == models.StatusKindMaintenance && inc.Status == models.MaintenanceScheduled:
snap.UpcomingMaintenance = append(snap.UpcomingMaintenance, p)
case isOpen(inc):
snap.ActiveIncidents = append(snap.ActiveIncidents, p)
default:
snap.History = append(snap.History, p)
}
}
for _, inc := range in.AutoIncidents {
name, onPage := names[inc.MonitorID]
if !onPage {
continue
}
if inc.StartedAt.Before(in.Now.AddDate(0, 0, -HistoryDays)) {
continue
}
derived := PublicIncident{
ID: inc.IncidentID,
Kind: models.StatusKindIncident,
Title: name + " unavailable",
Status: autoStatus(inc),
Affected: []string{name},
StartedAt: inc.StartedAt,
ResolvedAt: inc.ResolvedAt,
}
// An outage that has not recovered is happening now. Filing it under
// History while the component pill reads Down and Overall reads down
// told the reader the disruption was over.
if inc.ResolvedAt == nil {
snap.ActiveIncidents = append(snap.ActiveIncidents, derived)
} else {
snap.History = append(snap.History, derived)
}
}
sort.Slice(snap.History, func(i, j int) bool {
return snap.History[i].StartedAt.After(snap.History[j].StartedAt)
})
sort.Slice(snap.ActiveIncidents, func(i, j int) bool {
return snap.ActiveIncidents[i].StartedAt.After(snap.ActiveIncidents[j].StartedAt)
})
sort.Slice(snap.UpcomingMaintenance, func(i, j int) bool {
return snap.UpcomingMaintenance[i].StartedAt.Before(snap.UpcomingMaintenance[j].StartedAt)
})
snap.Overall = overallState(snap.Sections)
return snap
}
// publicName never falls back to the monitor's own name. An operator who has
// not chosen a public name has not consented to publishing the internal one.
func publicName(entry models.StatusPageEntry, monitorID string) string {
if entry.DisplayName != "" {
return entry.DisplayName
}
if monitorID != "" {
return monitorID
}
return entry.MonitorID
}
func authoredForPage(pageID string, all []models.StatusIncident) []models.StatusIncident {
out := []models.StatusIncident{}
for _, inc := range all {
for _, p := range inc.PageIDs {
if p == pageID {
out = append(out, inc)
break
}
}
}
return out
}
func maintenanceMonitors(authored []models.StatusIncident, now time.Time) map[string]bool {
out := map[string]bool{}
for _, inc := range authored {
if inc.Kind != models.StatusKindMaintenance || inc.Status != models.MaintenanceInProgress {
continue
}
if inc.ScheduledStart != nil && now.Before(*inc.ScheduledStart) {
continue
}
if inc.ScheduledEnd != nil && now.After(*inc.ScheduledEnd) {
continue
}
for _, m := range inc.AffectedMonitors {
out[m] = true
}
}
return out
}
func isOpen(inc models.StatusIncident) bool {
if inc.Kind == models.StatusKindMaintenance {
return inc.Status == models.MaintenanceInProgress
}
return inc.Status != models.IncidentResolved
}
func autoStatus(inc models.Incident) string {
if inc.ResolvedAt != nil {
return models.IncidentResolved
}
return models.IncidentInvestigating
}
func publicFromAuthored(inc models.StatusIncident, names map[string]string) PublicIncident {
p := PublicIncident{
ID: inc.IncidentID,
Kind: inc.Kind,
Title: inc.Title,
Impact: inc.Impact,
Status: inc.Status,
StartedAt: inc.StartedAt,
ResolvedAt: inc.ResolvedAt,
ScheduledStart: inc.ScheduledStart,
ScheduledEnd: inc.ScheduledEnd,
}
for _, m := range inc.AffectedMonitors {
// A monitor not on this page contributes nothing: publishing the raw
// id would name a component the reader cannot see.
if name, ok := names[m]; ok {
p.Affected = append(p.Affected, name)
}
}
for _, u := range inc.Updates {
p.Updates = append(p.Updates, PublicIncidentUpdate{At: u.At, Status: u.Status, Body: u.Body})
}
return p
}
// buildDays produces exactly HistoryDays cells, oldest first, ending today.
// It carries no maintenance state: a maintenance repaint is a display concern
// applied afterwards by applyMaintenanceRepaint, once uptimeFromDays has
// already read the true no_data/up/down state of each day. Folding the
// repaint in here would let a today cell with no rollups yet flip from
// no_data to maintenance before its uptime contribution was decided, and
// uptimeFromDays skips no_data days by their State — so that day would stop
// being skipped and start counting as a zero.
func buildDays(rollups []models.Rollup, now time.Time) []PublicDay {
type bucket struct{ checks, up int }
byDay := map[string]*bucket{}
for _, r := range rollups {
key := r.PeriodStart.UTC().Format("2006-01-02")
b, ok := byDay[key]
if !ok {
b = &bucket{}
byDay[key] = b
}
b.checks += r.Checks
b.up += r.UpCount
}
today := now.UTC().Truncate(24 * time.Hour)
days := make([]PublicDay, 0, HistoryDays)
for i := HistoryDays - 1; i >= 0; i-- {
d := today.AddDate(0, 0, -i)
key := d.Format("2006-01-02")
day := PublicDay{Date: key, State: PublicNoData}
if b, ok := byDay[key]; ok && b.checks > 0 {
day.Uptime = float64(b.up) / float64(b.checks) * 100
if day.Uptime >= 99.9 {
day.State = PublicUp
} else {
day.State = PublicDown
}
}
days = append(days, day)
}
return days
}
// applyMaintenanceRepaint redraws today's cell as "maintenance" for display,
// after uptimeFromDays has already computed the component's Uptime90d from
// the unpainted days. It never touches Uptime, and it must run after that
// computation, not before: repainting first would turn a today cell with no
// rollups yet from no_data (skipped) into maintenance (a 0% day counted in
// the average), and repainting a day that DOES have rollups must still leave
// that day's real up/down contribution in the average — maintenance changes
// how a day is drawn, never what the numbers say.
func applyMaintenanceRepaint(days []PublicDay, inMaintenance bool) []PublicDay {
if inMaintenance && len(days) > 0 {
days[len(days)-1].State = PublicMaintenance
}
return days
}
// uptimeFromDays ignores no_data days rather than counting them as zero. A
// component created last week is not 92% available.
func uptimeFromDays(days []PublicDay) float64 {
var sum float64
var n int
for _, d := range days {
if d.State == PublicNoData {
continue
}
sum += d.Uptime
n++
}
if n == 0 {
return 0
}
return sum / float64(n)
}
func componentStatus(mon models.Monitor, known, inMaintenance bool, now time.Time) string {
if inMaintenance {
return PublicMaintenance
}
if !known {
// The monitor was deleted while still listed on a page. Saying "up"
// would be a claim nothing is checking.
return PublicNoData
}
switch mon.State.Status {
case models.StatusUp:
return PublicUp
case models.StatusDown:
return PublicDown
default:
return PublicPending
}
}
const statusCacheTTL = 30 * time.Second
// unavailableSnapshot returns a StatusSnapshot with Available: false and all
// four list fields initialized to empty slices rather than nil, ensuring
// consistent JSON serialization across the available and unavailable paths.
func unavailableSnapshot(reason, title string) *StatusSnapshot {
return &StatusSnapshot{
Available: false,
Reason: reason,
Title: title,
Sections: []PublicSection{},
ActiveIncidents: []PublicIncident{},
UpcomingMaintenance: []PublicIncident{},
History: []PublicIncident{},
}
}
// PublicStatusSnapshot is the whole public read path.
//
// A missing page, an unpublished page and a page belonging to another instance
// all return ErrPageNotFound, identically. A distinct error for "exists but
// unpublished" would confirm it exists.
func PublicStatusSnapshot(instanceID, pageID string) (*StatusSnapshot, error) {
if err := ValidatePageID(pageID); err != nil {
log.Printf("public status: page id %q is not a valid id: %v", pageID, err)
return nil, ErrPageNotFound
}
if snap := cachedSnapshot(instanceID, pageID); snap != nil {
return snap, nil
}
page, err := GetStatusPage(instanceID, pageID)
if err != nil {
log.Printf("public status: instance=%s page=%q lookup: %v", instanceID, pageID, err)
return nil, err
}
if !page.Published {
log.Printf("public status: instance=%s page=%q exists but published=false", instanceID, pageID)
return nil, ErrPageNotFound
}
// The licence check answers 200 with available:false rather than 403,
// because the reader is a member of the public who can do nothing about it
// and deserves an explanation rather than a browser error.
st := GetLicenseState(instanceID)
if !st.Active() {
return unavailableSnapshot("licence_inactive", page.Title), nil
}
if !st.Feature(license.FeatureStatusPages) {
return unavailableSnapshot("feature_unavailable", page.Title), nil
}
in, err := loadSnapshotInput(instanceID, *page)
if err != nil {
return nil, err
}
snap := assembleSnapshot(in)
storeSnapshot(instanceID, pageID, snap)
return &snap, nil
}
func loadSnapshotInput(instanceID string, page models.StatusPage) (snapshotInput, error) {
in := snapshotInput{
Page: page,
Monitors: map[string]models.Monitor{},
Rollups: map[string][]models.Rollup{},
Now: time.Now().UTC(),
}
since := in.Now.AddDate(0, 0, -HistoryDays)
ids := []string{}
for _, sec := range page.Sections {
for _, e := range sec.Entries {
ids = append(ids, e.MonitorID)
}
}
if len(ids) == 0 {
// A page with no components still renders: title, banner and any
// authored incidents. Skipping the monitor queries avoids three
// unbounded $in lookups on an empty list.
authored, err := ListStatusIncidentsForPage(instanceID, page.PageID, since)
if err != nil {
return in, err
}
in.Authored = authored
return in, nil
}
ctx, cancel := spCtx()
defer cancel()
cur, err := db.Col("monitors").Find(ctx, bson.M{
"instance_id": instanceID,
"monitor_id": bson.M{"$in": ids},
})
if err != nil {
return in, err
}
mons := []models.Monitor{}
if err := cur.All(ctx, &mons); err != nil {
return in, err
}
for _, m := range mons {
in.Monitors[m.MonitorID] = m
}
rc, err := db.Col("monitor_rollups").Find(ctx, bson.M{
"instance_id": instanceID,
"monitor_id": bson.M{"$in": ids},
"period_start": bson.M{"$gte": since},
})
if err != nil {
return in, err
}
rollups := []models.Rollup{}
if err := rc.All(ctx, &rollups); err != nil {
return in, err
}
for _, r := range rollups {
in.Rollups[r.MonitorID] = append(in.Rollups[r.MonitorID], r)
}
ic, err := db.Col("incidents").Find(ctx, bson.M{
"instance_id": instanceID,
"monitor_id": bson.M{"$in": ids},
"started_at": bson.M{"$gte": since},
})
if err != nil {
return in, err
}
auto := []models.Incident{}
if err := ic.All(ctx, &auto); err != nil {
return in, err
}
in.AutoIncidents = auto
authored, err := ListStatusIncidentsForPage(instanceID, page.PageID, since)
if err != nil {
return in, err
}
in.Authored = authored
return in, nil
}
// A cache miss on Redis is a cache miss, never an error: the status page must
// survive the outage it exists to report.
func cachedSnapshot(instanceID, pageID string) *StatusSnapshot {
if statusRedis == nil {
return nil
}
ctx, cancel := spCtx()
defer cancel()
raw, err := statusRedis.Get(ctx, statusCacheKey(instanceID, pageID)).Bytes()
if err != nil || len(raw) == 0 {
return nil
}
var snap StatusSnapshot
if err := json.Unmarshal(raw, &snap); err != nil {
return nil
}
return &snap
}
func storeSnapshot(instanceID, pageID string, snap StatusSnapshot) {
if statusRedis == nil {
return
}
raw, err := json.Marshal(snap)
if err != nil {
return
}
ctx, cancel := spCtx()
defer cancel()
if err := statusRedis.Set(ctx, statusCacheKey(instanceID, pageID), raw, statusCacheTTL).Err(); err != nil {
log.Printf("status cache store %s/%s: %v", instanceID, pageID, err)
}
}
func overallState(sections []PublicSection) string {
worst := PublicUp
anyDown, anyMaint, anyOther := false, false, false
counted := 0
for _, s := range sections {
for _, c := range s.Components {
counted++
switch c.Status {
case PublicDown:
anyDown = true
case PublicMaintenance:
anyMaint = true
case PublicPending, PublicNoData:
anyOther = true
}
}
}
switch {
case counted == 0:
// Nothing is being reported, so nothing is known. "All systems
// operational" over zero components is a claim of health made from no
// evidence at all; PublicNoData is what the view renders as
// "Status unknown".
worst = PublicNoData
case anyDown:
worst = PublicDown
case anyMaint:
worst = PublicMaintenance
case anyOther:
worst = PublicDegraded
}
return worst
}
@@ -0,0 +1,280 @@
package services
import (
"encoding/json"
"strings"
"testing"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/models"
)
func testInput(now time.Time) snapshotInput {
return snapshotInput{
Page: models.StatusPage{
PageID: "api",
Title: "Acme Status",
Published: true,
Sections: []models.StatusPageSection{{
Name: "API",
Entries: []models.StatusPageEntry{
{MonitorID: "mon-1", DisplayName: "Public API"},
{MonitorID: "mon-2"},
},
}},
},
Monitors: map[string]models.Monitor{
"mon-1": {
MonitorID: "mon-1",
Name: "prod-api-internal",
Type: models.MonitorHTTP,
ChannelIDs: []string{"chan-123"},
Target: models.MonitorTarget{
URL: "https://internal.example.com/health",
Keyword: "SECRETKEYWORD",
},
State: models.MonitorState{
Status: models.StatusUp,
Message: "dial tcp 10.0.0.5:5432: connect refused",
},
},
"mon-2": {
MonitorID: "mon-2",
Name: "db-primary",
Type: models.MonitorTCP,
Target: models.MonitorTarget{Host: "10.0.0.5", Port: 5432},
State: models.MonitorState{Status: models.StatusDown},
},
},
Rollups: map[string][]models.Rollup{},
AutoIncidents: []models.Incident{},
Authored: []models.StatusIncident{},
Now: now,
}
}
// The snapshot is the only thing that reaches an anonymous caller. If any of
// these strings can be found in its JSON, the boundary has a hole in it.
func TestAssembleSnapshotRedactsMonitorInternals(t *testing.T) {
in := testInput(time.Date(2026, 8, 24, 12, 0, 0, 0, time.UTC))
in.AutoIncidents = []models.Incident{{
IncidentID: "inc-1",
MonitorID: "mon-2",
StartedAt: in.Now.Add(-2 * time.Hour),
Cause: "dial tcp 10.0.0.5:5432: connect refused",
}}
b, err := json.Marshal(assembleSnapshot(in))
if err != nil {
t.Fatalf("marshal: %v", err)
}
got := string(b)
leaks := []string{
"internal.example.com",
"10.0.0.5",
"connect refused",
"chan-123",
"SECRETKEYWORD",
"prod-api-internal",
"db-primary",
"5432",
}
for _, leak := range leaks {
if strings.Contains(got, leak) {
t.Errorf("snapshot leaked %q\nfull snapshot: %s", leak, got)
}
}
}
func TestAssembleSnapshotUsesDisplayNameThenMonitorName(t *testing.T) {
in := testInput(time.Date(2026, 8, 24, 12, 0, 0, 0, time.UTC))
// mon-2 has no DisplayName override, and its monitor name ("db-primary")
// is in TestAssembleSnapshotRedactsMonitorInternals's leak list, so an
// un-overridden entry must fall back to something safe. It falls back to
// the monitor id, never the internal name.
snap := assembleSnapshot(in)
comps := snap.Sections[0].Components
if comps[0].Name != "Public API" {
t.Errorf("component 0 name = %q, want %q", comps[0].Name, "Public API")
}
if comps[1].Name != "mon-2" {
t.Errorf("component 1 name = %q, want the monitor id as fallback", comps[1].Name)
}
}
func TestAssembleSnapshotHistoryIs90DaysWithNoDataForMissingRollups(t *testing.T) {
now := time.Date(2026, 8, 24, 12, 0, 0, 0, time.UTC)
in := testInput(now)
in.Rollups = map[string][]models.Rollup{
"mon-1": {
{MonitorID: "mon-1", PeriodStart: now.Add(-24 * time.Hour), Checks: 60, UpCount: 60},
{MonitorID: "mon-1", PeriodStart: now.Add(-48 * time.Hour), Checks: 60, UpCount: 30},
},
}
snap := assembleSnapshot(in)
days := snap.Sections[0].Components[0].Days
if len(days) != 90 {
t.Fatalf("len(days) = %d, want 90", len(days))
}
if days[89].Date != "2026-08-24" {
t.Errorf("last day = %q, want 2026-08-24", days[89].Date)
}
if days[88].State != "up" {
t.Errorf("yesterday state = %q, want up", days[88].State)
}
if days[87].State != "down" {
t.Errorf("two days ago state = %q, want down (50%% up)", days[87].State)
}
if days[0].State != "no_data" {
t.Errorf("oldest day state = %q, want no_data", days[0].State)
}
}
func TestAssembleSnapshotMaintenanceDoesNotChangeUptime(t *testing.T) {
now := time.Date(2026, 8, 24, 12, 0, 0, 0, time.UTC)
in := testInput(now)
in.Rollups = map[string][]models.Rollup{
"mon-2": {{MonitorID: "mon-2", PeriodStart: now, Checks: 100, UpCount: 50}},
}
start := now.Add(-time.Hour)
end := now.Add(time.Hour)
in.Authored = []models.StatusIncident{{
IncidentID: "mnt-1",
PageIDs: []string{"api"},
Kind: models.StatusKindMaintenance,
Title: "Database upgrade",
Status: models.MaintenanceInProgress,
AffectedMonitors: []string{"mon-2"},
ScheduledStart: &start,
ScheduledEnd: &end,
StartedAt: start,
}}
snap := assembleSnapshot(in)
comp := snap.Sections[0].Components[1]
if comp.Status != "maintenance" {
t.Errorf("status = %q, want maintenance", comp.Status)
}
// Rollups are the durable record. A maintenance window changes how the
// component is drawn, never what the numbers say.
if comp.Uptime90d != 50 {
t.Errorf("uptime = %v, want 50 (unmodified by the window)", comp.Uptime90d)
}
}
// TestAssembleSnapshotMaintenanceRepaintDoesNotCountNoDataAsZero guards
// against the maintenance repaint corrupting Uptime90d for a component whose
// today rollup has not landed yet — an in-progress maintenance window on a
// young component, or one that simply started before today's hourly rollup
// was written. Repainting today's no_data cell to "maintenance" must never
// make uptimeFromDays stop skipping it: doing so would turn a component with
// one good day of history from 100% into 50%.
func TestAssembleSnapshotMaintenanceRepaintDoesNotCountNoDataAsZero(t *testing.T) {
now := time.Date(2026, 8, 24, 12, 0, 0, 0, time.UTC)
in := testInput(now)
in.Rollups = map[string][]models.Rollup{
// Only yesterday has a rollup, fully up. Today has none.
"mon-2": {{MonitorID: "mon-2", PeriodStart: now.Add(-24 * time.Hour), Checks: 60, UpCount: 60}},
}
start := now.Add(-time.Hour)
end := now.Add(time.Hour)
in.Authored = []models.StatusIncident{{
IncidentID: "mnt-2",
PageIDs: []string{"api"},
Kind: models.StatusKindMaintenance,
Title: "Database upgrade",
Status: models.MaintenanceInProgress,
AffectedMonitors: []string{"mon-2"},
ScheduledStart: &start,
ScheduledEnd: &end,
StartedAt: start,
}}
snap := assembleSnapshot(in)
comp := snap.Sections[0].Components[1]
if comp.Status != "maintenance" {
t.Errorf("status = %q, want maintenance", comp.Status)
}
// Today's cell is redrawn as maintenance for display...
if comp.Days[89].State != "maintenance" {
t.Errorf("today state = %q, want maintenance", comp.Days[89].State)
}
// ...but it carries no rollup, so it must not have been averaged in as a
// zero. The only day with data was 100% up, so the 90-day figure is 100,
// not (100+0)/2 = 50.
if comp.Uptime90d != 100 {
t.Errorf("uptime = %v, want 100 (today's no_data must not count as zero)", comp.Uptime90d)
}
}
func TestAssembleSnapshotOnlyIncludesAuthoredIncidentsForThisPage(t *testing.T) {
now := time.Date(2026, 8, 24, 12, 0, 0, 0, time.UTC)
in := testInput(now)
in.Authored = []models.StatusIncident{
{IncidentID: "mine", PageIDs: []string{"api"}, Kind: models.StatusKindIncident,
Title: "Mine", Status: models.IncidentInvestigating, StartedAt: now},
{IncidentID: "theirs", PageIDs: []string{"partners"}, Kind: models.StatusKindIncident,
Title: "Theirs", Status: models.IncidentInvestigating, StartedAt: now},
}
snap := assembleSnapshot(in)
if len(snap.ActiveIncidents) != 1 || snap.ActiveIncidents[0].Title != "Mine" {
t.Fatalf("active incidents = %+v, want only the one naming this page", snap.ActiveIncidents)
}
}
// A derived outage that has not recovered is happening now. It used to be
// appended to History unconditionally, so a live outage was reported under
// "Past incidents" while the component pill next to it read Down.
func TestAssembleSnapshotDerivedIncidentActiveUntilResolved(t *testing.T) {
now := time.Date(2026, 8, 24, 12, 0, 0, 0, time.UTC)
resolved := now.Add(-30 * time.Minute)
in := testInput(now)
in.AutoIncidents = []models.Incident{
{
IncidentID: "inc-open",
MonitorID: "mon-2",
StartedAt: now.Add(-2 * time.Hour),
},
{
IncidentID: "inc-closed",
MonitorID: "mon-1",
StartedAt: now.Add(-3 * time.Hour),
ResolvedAt: &resolved,
},
}
snap := assembleSnapshot(in)
if len(snap.ActiveIncidents) != 1 || snap.ActiveIncidents[0].ID != "inc-open" {
t.Fatalf("unresolved incident should be active, got %+v", snap.ActiveIncidents)
}
if snap.ActiveIncidents[0].ResolvedAt != nil {
t.Errorf("active incident carries a resolved_at: %v", snap.ActiveIncidents[0].ResolvedAt)
}
if len(snap.History) != 1 || snap.History[0].ID != "inc-closed" {
t.Fatalf("resolved incident should be history, got %+v", snap.History)
}
if snap.History[0].ResolvedAt == nil {
t.Errorf("history entry lost its resolved_at")
}
}
// A page with no components knows nothing, and claiming "all systems
// operational" from no evidence is the one answer it must not give.
func TestAssembleSnapshotEmptyPageIsNotOperational(t *testing.T) {
now := time.Date(2026, 8, 24, 12, 0, 0, 0, time.UTC)
in := testInput(now)
in.Page.Sections = nil
if got := assembleSnapshot(in).Overall; got != PublicNoData {
t.Errorf("overall for a page with no components = %q, want %q", got, PublicNoData)
}
in = testInput(now)
in.Page.Sections = []models.StatusPageSection{{Name: "API"}}
if got := assembleSnapshot(in).Overall; got != PublicNoData {
t.Errorf("overall for a page with an empty section = %q, want %q", got, PublicNoData)
}
}
+253
View File
@@ -0,0 +1,253 @@
package backup
import (
"archive/tar"
"compress/gzip"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path"
"path/filepath"
"strings"
"time"
)
// ErrChecksum is returned when an archive member does not match the checksum
// the manifest recorded for it.
var ErrChecksum = errors.New("archive member failed its checksum")
// Writer streams a tar.gz. Members are written in the order they are produced
// and the manifest goes last, because its per-collection checksums are only
// known once every collection has been written.
type Writer struct {
gz *gzip.Writer
tar *tar.Writer
}
// NewWriter starts an archive on out. out may be a file or stdout; nothing here
// seeks.
func NewWriter(out io.Writer) *Writer {
gz := gzip.NewWriter(out)
return &Writer{gz: gz, tar: tar.NewWriter(gz)}
}
func (w *Writer) writeMember(name string, body []byte) error {
h := &tar.Header{
Name: name,
Mode: 0o600,
Size: int64(len(body)),
ModTime: time.Now().UTC(),
Typeflag: tar.TypeReg,
}
if err := w.tar.WriteHeader(h); err != nil {
return fmt.Errorf("write header %s: %w", name, err)
}
if _, err := w.tar.Write(body); err != nil {
return fmt.Errorf("write %s: %w", name, err)
}
return nil
}
// WriteCollection writes the concatenated raw BSON of one collection and
// returns the manifest entry describing it.
func (w *Writer) WriteCollection(name string, docs [][]byte) (CollectionEntry, error) {
var body []byte
for _, d := range docs {
body = append(body, d...)
}
sum := sha256.Sum256(body)
entry := CollectionEntry{
Name: name,
Documents: int64(len(docs)),
Bytes: int64(len(body)),
SHA256: hex.EncodeToString(sum[:]),
}
if err := w.writeMember(collectionMember(name), body); err != nil {
return CollectionEntry{}, err
}
return entry, nil
}
// WriteIndexes writes a collection's index specifications verbatim.
func (w *Writer) WriteIndexes(name string, specsJSON []byte) error {
return w.writeMember(indexMember(name), specsJSON)
}
// Close writes the manifest and finishes the archive.
func (w *Writer) Close(m Manifest) error {
raw, err := json.MarshalIndent(m, "", " ")
if err != nil {
return fmt.Errorf("marshal manifest: %w", err)
}
if err := w.writeMember(ManifestName, raw); err != nil {
return err
}
if err := w.tar.Close(); err != nil {
return err
}
return w.gz.Close()
}
func collectionMember(name string) string { return "collections/" + name + ".bson" }
func indexMember(name string) string { return "indexes/" + name + ".json" }
// Reader is an opened archive.
//
// Open extracts to a temporary directory rather than streaming, because gzip
// offers no random access and the manifest — which carries the checksums every
// other member is judged against — is written last. Verifying before writing a
// single document to the target is worth one pass over local disk. This is why
// the container image needs a /tmp.
type Reader struct {
dir string
manifest Manifest
}
// Open extracts, verifies and returns the archive at path. The caller must
// Close it.
func Open(archivePath string) (*Reader, error) {
dir, err := os.MkdirTemp("", "vantage-restore-*")
if err != nil {
return nil, fmt.Errorf("temp dir: %w", err)
}
r := &Reader{dir: dir}
if err := r.extract(archivePath); err != nil {
r.Close()
return nil, err
}
if err := r.loadManifest(); err != nil {
r.Close()
return nil, err
}
if err := r.verifyMembers(); err != nil {
r.Close()
return nil, err
}
return r, nil
}
func (r *Reader) extract(archivePath string) error {
f, err := os.Open(archivePath)
if err != nil {
return fmt.Errorf("open archive: %w", err)
}
defer f.Close()
gz, err := gzip.NewReader(f)
if err != nil {
return fmt.Errorf("archive is not gzip: %w", err)
}
defer gz.Close()
tr := tar.NewReader(gz)
for {
h, err := tr.Next()
if err == io.EOF {
return nil
}
if err != nil {
return fmt.Errorf("read archive: %w", err)
}
if h.Typeflag != tar.TypeReg {
continue
}
dest, err := safeJoin(r.dir, h.Name)
if err != nil {
return err
}
if err := os.MkdirAll(filepath.Dir(dest), 0o700); err != nil {
return fmt.Errorf("mkdir: %w", err)
}
out, err := os.OpenFile(dest, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o600)
if err != nil {
return fmt.Errorf("create %s: %w", h.Name, err)
}
if _, err := io.Copy(out, tr); err != nil {
out.Close()
return fmt.Errorf("extract %s: %w", h.Name, err)
}
if err := out.Close(); err != nil {
return err
}
}
}
// safeJoin refuses a member name that escapes the extraction directory. An
// archive is operator-supplied input and may not be one we wrote.
func safeJoin(dir, name string) (string, error) {
clean := path.Clean("/" + name)
dest := filepath.Join(dir, filepath.FromSlash(strings.TrimPrefix(clean, "/")))
if !strings.HasPrefix(dest, filepath.Clean(dir)+string(os.PathSeparator)) {
return "", fmt.Errorf("archive member %q escapes the extraction directory", name)
}
return dest, nil
}
func (r *Reader) loadManifest() error {
raw, err := os.ReadFile(filepath.Join(r.dir, ManifestName))
if err != nil {
return fmt.Errorf("archive has no %s: %w", ManifestName, err)
}
if err := json.Unmarshal(raw, &r.manifest); err != nil {
return fmt.Errorf("parse %s: %w", ManifestName, err)
}
return r.manifest.Check()
}
func (r *Reader) verifyMembers() error {
for _, c := range r.manifest.Collections {
f, err := r.OpenCollection(c.Name)
if err != nil {
return fmt.Errorf("%w: %s is named in the manifest but absent from the archive",
ErrChecksum, c.Name)
}
h := sha256.New()
n, err := io.Copy(h, f)
f.Close()
if err != nil {
return fmt.Errorf("read %s: %w", c.Name, err)
}
if n != c.Bytes {
return fmt.Errorf("%w: %s is %d bytes, manifest says %d", ErrChecksum, c.Name, n, c.Bytes)
}
if got := hex.EncodeToString(h.Sum(nil)); got != c.SHA256 {
return fmt.Errorf("%w: %s checksum %s, manifest says %s", ErrChecksum, c.Name, got, c.SHA256)
}
}
return nil
}
// Manifest returns the verified manifest.
func (r *Reader) Manifest() Manifest { return r.manifest }
// OpenCollection returns the raw BSON stream for one collection.
func (r *Reader) OpenCollection(name string) (io.ReadCloser, error) {
p, err := safeJoin(r.dir, collectionMember(name))
if err != nil {
return nil, err
}
return os.Open(p)
}
// IndexesJSON returns a collection's index specifications, or nil when the
// archive holds none. A collection with no indexes beyond _id_ is ordinary and
// is not an error.
func (r *Reader) IndexesJSON(name string) ([]byte, error) {
p, err := safeJoin(r.dir, indexMember(name))
if err != nil {
return nil, err
}
raw, err := os.ReadFile(p)
if os.IsNotExist(err) {
return nil, nil
}
return raw, err
}
// Close removes the extraction directory.
func (r *Reader) Close() error { return os.RemoveAll(r.dir) }
+216
View File
@@ -0,0 +1,216 @@
package backup
import (
"archive/tar"
"bytes"
"compress/gzip"
"errors"
"io"
"os"
"path/filepath"
"testing"
"time"
)
// writeSampleArchive builds a two-collection archive on disk and returns its path.
func writeSampleArchive(t *testing.T) string {
t.Helper()
path := filepath.Join(t.TempDir(), "sample.tar.gz")
f, err := os.Create(path)
if err != nil {
t.Fatalf("create: %v", err)
}
defer f.Close()
w := NewWriter(f)
servers, err := w.WriteCollection("servers", [][]byte{[]byte("one"), []byte("two")})
if err != nil {
t.Fatalf("WriteCollection: %v", err)
}
if err := w.WriteIndexes("servers", []byte(`[{"name":"idx"}]`)); err != nil {
t.Fatalf("WriteIndexes: %v", err)
}
keys, err := w.WriteCollection("keys", [][]byte{[]byte("k")})
if err != nil {
t.Fatalf("WriteCollection: %v", err)
}
if err := w.Close(Manifest{
FormatVersion: FormatVersion,
CreatedAt: time.Now().UTC(),
MongoDB: "vantage",
Collections: []CollectionEntry{servers, keys},
}); err != nil {
t.Fatalf("Close: %v", err)
}
return path
}
func TestWriterRecordsCountsAndChecksums(t *testing.T) {
var buf bytes.Buffer
w := NewWriter(&buf)
e, err := w.WriteCollection("servers", [][]byte{[]byte("one"), []byte("two")})
if err != nil {
t.Fatalf("WriteCollection: %v", err)
}
if e.Name != "servers" {
t.Fatalf("name %q", e.Name)
}
if e.Documents != 2 {
t.Fatalf("documents %d, want 2", e.Documents)
}
if e.Bytes != 6 {
t.Fatalf("bytes %d, want 6", e.Bytes)
}
if len(e.SHA256) != 64 {
t.Fatalf("sha256 %q is not 64 hex chars", e.SHA256)
}
}
func TestRoundTrip(t *testing.T) {
r, err := Open(writeSampleArchive(t))
if err != nil {
t.Fatalf("Open: %v", err)
}
defer r.Close()
if r.Manifest().MongoDB != "vantage" {
t.Fatalf("manifest not read back: %+v", r.Manifest())
}
rc, err := r.OpenCollection("servers")
if err != nil {
t.Fatalf("OpenCollection: %v", err)
}
defer rc.Close()
got, err := io.ReadAll(rc)
if err != nil {
t.Fatalf("read: %v", err)
}
if string(got) != "onetwo" {
t.Fatalf("got %q, want %q", got, "onetwo")
}
idx, err := r.IndexesJSON("servers")
if err != nil {
t.Fatalf("IndexesJSON: %v", err)
}
if string(idx) != `[{"name":"idx"}]` {
t.Fatalf("indexes round-tripped as %q", idx)
}
}
func TestIndexesJSONAbsentIsEmptyNotError(t *testing.T) {
r, err := Open(writeSampleArchive(t))
if err != nil {
t.Fatalf("Open: %v", err)
}
defer r.Close()
idx, err := r.IndexesJSON("keys")
if err != nil {
t.Fatalf("a collection with no index member must not error: %v", err)
}
if len(idx) != 0 {
t.Fatalf("want empty, got %q", idx)
}
}
func TestOpenRejectsCorruptedMember(t *testing.T) {
path := writeSampleArchive(t)
// Rewrite the archive with one byte of a collection member flipped, leaving
// the manifest's checksum describing the original.
corrupt := filepath.Join(t.TempDir(), "corrupt.tar.gz")
rewriteFlippingCollectionByte(t, path, corrupt, "servers")
if _, err := Open(corrupt); !errors.Is(err, ErrChecksum) {
t.Fatalf("got %v, want ErrChecksum", err)
}
}
func TestOpenRejectsUnknownFormatVersion(t *testing.T) {
path := filepath.Join(t.TempDir(), "future.tar.gz")
f, err := os.Create(path)
if err != nil {
t.Fatalf("create: %v", err)
}
w := NewWriter(f)
if err := w.Close(Manifest{FormatVersion: 99}); err != nil {
t.Fatalf("Close: %v", err)
}
f.Close()
if _, err := Open(path); !errors.Is(err, ErrUnknownFormat) {
t.Fatalf("got %v, want ErrUnknownFormat", err)
}
}
func TestCloseRemovesTempDir(t *testing.T) {
r, err := Open(writeSampleArchive(t))
if err != nil {
t.Fatalf("Open: %v", err)
}
dir := r.dir
if _, err := os.Stat(dir); err != nil {
t.Fatalf("temp dir missing while open: %v", err)
}
if err := r.Close(); err != nil {
t.Fatalf("Close: %v", err)
}
if _, err := os.Stat(dir); !os.IsNotExist(err) {
t.Fatalf("temp dir %s survived Close", dir)
}
}
// rewriteFlippingCollectionByte copies an archive, flipping one byte inside the
// named collection's .bson member so its content no longer matches the checksum
// the manifest recorded.
func rewriteFlippingCollectionByte(t *testing.T, src, dst, collection string) {
t.Helper()
in, err := os.Open(src)
if err != nil {
t.Fatalf("open src: %v", err)
}
defer in.Close()
gz, err := gzip.NewReader(in)
if err != nil {
t.Fatalf("gzip: %v", err)
}
defer gz.Close()
out, err := os.Create(dst)
if err != nil {
t.Fatalf("create dst: %v", err)
}
defer out.Close()
gw := gzip.NewWriter(out)
defer gw.Close()
tw := tar.NewWriter(gw)
defer tw.Close()
tr := tar.NewReader(gz)
target := "collections/" + collection + ".bson"
for {
h, err := tr.Next()
if err == io.EOF {
return
}
if err != nil {
t.Fatalf("tar next: %v", err)
}
body, err := io.ReadAll(tr)
if err != nil {
t.Fatalf("read member: %v", err)
}
if h.Name == target && len(body) > 0 {
body[0] ^= 0xFF
}
h.Size = int64(len(body))
if err := tw.WriteHeader(h); err != nil {
t.Fatalf("write header: %v", err)
}
if _, err := tw.Write(body); err != nil {
t.Fatalf("write body: %v", err)
}
}
}
+184
View File
@@ -0,0 +1,184 @@
package backup
import (
"context"
"encoding/json"
"fmt"
"io"
"os"
"sort"
"time"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
)
// DumpOptions configures one backup.
type DumpOptions struct {
Client *mongo.Client
Database string
Exclude []string
// KeyHex is KEY_ENCRYPTION_KEY. It is fingerprinted and discarded; it is
// never written to the archive.
KeyHex string
// AllowNoKey permits a backup of a deployment that stores no encrypted
// material. The manifest then records a null fingerprint, which restore
// reports rather than treating as a match.
AllowNoKey bool
VantageVersion string
Out io.Writer
}
// Dump writes a complete archive of one database to opt.Out.
//
// Collections are enumerated live rather than read from a list. A backup tool
// has no equivalent of AssertNoScopedCollectionMissed to catch a hardcoded list
// drifting, and the first symptom of that drift would be a restore silently
// missing a collection added since the list was written.
func Dump(ctx context.Context, opt DumpOptions) (Manifest, error) {
fingerprint, err := dumpFingerprint(opt)
if err != nil {
return Manifest{}, err
}
db := opt.Client.Database(opt.Database)
names, err := db.ListCollectionNames(ctx, bson.M{})
if err != nil {
return Manifest{}, fmt.Errorf("list collections: %w", err)
}
sort.Strings(names)
excluded := map[string]bool{}
for _, e := range opt.Exclude {
excluded[e] = true
}
serverVersion, err := mongoServerVersion(ctx, opt.Client)
if err != nil {
return Manifest{}, err
}
hostname, err := os.Hostname()
if err != nil {
hostname = "unknown"
}
w := NewWriter(opt.Out)
entries := make([]CollectionEntry, 0, len(names))
for _, name := range names {
if excluded[name] {
continue
}
entry, err := dumpCollection(ctx, w, db, name)
if err != nil {
return Manifest{}, err
}
entries = append(entries, entry)
}
m := Manifest{
FormatVersion: FormatVersion,
CreatedAt: time.Now().UTC(),
VantageVersion: opt.VantageVersion,
Hostname: hostname,
MongoDB: opt.Database,
MongoServerVersion: serverVersion,
KeyFingerprint: fingerprint,
Collections: entries,
Excluded: append([]string{}, opt.Exclude...),
}
if err := w.Close(m); err != nil {
return Manifest{}, err
}
return m, nil
}
// dumpFingerprint applies the key policy before any output is produced. An
// archive of ciphertext whose key was never recorded is worse than no archive,
// because it looks like a backup.
func dumpFingerprint(opt DumpOptions) (*string, error) {
if opt.KeyHex == "" {
if opt.AllowNoKey {
return nil, nil
}
return nil, fmt.Errorf("%w: pass --allow-no-key only if this deployment stores no encrypted data", ErrNoKey)
}
fp, err := FingerprintHex(opt.KeyHex)
if err != nil {
return nil, err
}
return &fp, nil
}
func dumpCollection(ctx context.Context, w *Writer, db *mongo.Database, name string) (CollectionEntry, error) {
cur, err := db.Collection(name).Find(ctx, bson.M{})
if err != nil {
return CollectionEntry{}, fmt.Errorf("find %s: %w", name, err)
}
defer cur.Close(ctx)
var docs [][]byte
for cur.Next(ctx) {
// cur.Current is only valid until the next Next, and it is written to
// the archive verbatim rather than through a map, so every BSON type
// survives exactly as the server stored it.
docs = append(docs, append([]byte(nil), cur.Current...))
}
if err := cur.Err(); err != nil {
return CollectionEntry{}, fmt.Errorf("iterate %s: %w", name, err)
}
entry, err := w.WriteCollection(name, docs)
if err != nil {
return CollectionEntry{}, err
}
if err := dumpIndexes(ctx, w, db, name); err != nil {
return CollectionEntry{}, err
}
return entry, nil
}
func dumpIndexes(ctx context.Context, w *Writer, db *mongo.Database, name string) error {
cur, err := db.Collection(name).Indexes().List(ctx)
if err != nil {
return fmt.Errorf("list indexes on %s: %w", name, err)
}
defer cur.Close(ctx)
// The specs are read as raw BSON and re-encoded as extended JSON, one
// element per index, so key order and every option the server reported —
// partialFilterExpression, collation, weights and the rest — survive
// verbatim. Decoding into bson.M would lose compound key order, and
// reconstructing an index from a hand-picked set of options would drop
// whatever was not picked.
var specs []bson.Raw
if err := cur.All(ctx, &specs); err != nil {
return fmt.Errorf("read indexes on %s: %w", name, err)
}
encoded := make([]json.RawMessage, 0, len(specs))
for _, spec := range specs {
ej, err := bson.MarshalExtJSON(spec, false, false)
if err != nil {
return fmt.Errorf("encode indexes on %s: %w", name, err)
}
encoded = append(encoded, ej)
}
raw, err := json.Marshal(encoded)
if err != nil {
return fmt.Errorf("encode indexes on %s: %w", name, err)
}
return w.WriteIndexes(name, raw)
}
func mongoServerVersion(ctx context.Context, client *mongo.Client) (string, error) {
var res struct {
Version string `bson:"version"`
}
err := client.Database("admin").RunCommand(ctx, bson.D{{Key: "buildInfo", Value: 1}}).Decode(&res)
if err != nil {
return "", fmt.Errorf("buildInfo: %w", err)
}
return res.Version, nil
}
+206
View File
@@ -0,0 +1,206 @@
package backup
import (
"bytes"
"context"
"errors"
"io"
"os"
"path/filepath"
"testing"
"time"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
)
func seed(t *testing.T, client *mongo.Client, dbName string) {
t.Helper()
ctx := context.Background()
db := client.Database(dbName)
if _, err := db.Collection("servers").InsertMany(ctx, []any{
bson.M{"_id": bson.NewObjectID(), "name": "alpha", "instance_id": "i1"},
bson.M{"_id": bson.NewObjectID(), "name": "beta", "instance_id": "i1"},
}); err != nil {
t.Fatalf("insert servers: %v", err)
}
if _, err := db.Collection("audit_logs").InsertOne(ctx, bson.M{"action": "login"}); err != nil {
t.Fatalf("insert audit_logs: %v", err)
}
}
func dumpToFile(t *testing.T, opt DumpOptions) (string, Manifest) {
t.Helper()
path := filepath.Join(t.TempDir(), "out.tar.gz")
f, err := os.Create(path)
if err != nil {
t.Fatalf("create: %v", err)
}
opt.Out = f
m, err := Dump(context.Background(), opt)
if cerr := f.Close(); cerr != nil {
t.Fatalf("close: %v", cerr)
}
if err != nil {
t.Fatalf("Dump: %v", err)
}
return path, m
}
func TestDumpEnumeratesEveryCollection(t *testing.T) {
client, dbName := testDB(t)
seed(t, client, dbName)
_, m := dumpToFile(t, DumpOptions{
Client: client, Database: dbName, KeyHex: validKeyHex, VantageVersion: "test",
})
if _, ok := m.Collection("servers"); !ok {
t.Fatal("servers missing from the manifest")
}
if _, ok := m.Collection("audit_logs"); !ok {
t.Fatal("audit_logs missing; enumeration must not filter by a hardcoded list")
}
servers, _ := m.Collection("servers")
if servers.Documents != 2 {
t.Fatalf("servers documents %d, want 2", servers.Documents)
}
if m.MongoDB != dbName {
t.Fatalf("manifest database %q, want %q", m.MongoDB, dbName)
}
if m.MongoServerVersion == "" {
t.Fatal("manifest records no MongoDB server version")
}
if m.Hostname == "" {
t.Fatal("manifest records no hostname")
}
}
func TestDumpRecordsKeyFingerprint(t *testing.T) {
client, dbName := testDB(t)
seed(t, client, dbName)
_, m := dumpToFile(t, DumpOptions{Client: client, Database: dbName, KeyHex: validKeyHex})
want, err := FingerprintHex(validKeyHex)
if err != nil {
t.Fatalf("FingerprintHex: %v", err)
}
if m.KeyFingerprint == nil || *m.KeyFingerprint != want {
t.Fatalf("fingerprint %v, want %s", m.KeyFingerprint, want)
}
}
func TestDumpRefusesWithoutAKey(t *testing.T) {
client, dbName := testDB(t)
seed(t, client, dbName)
var buf bytes.Buffer
_, err := Dump(context.Background(), DumpOptions{
Client: client, Database: dbName, Out: &buf,
})
if !errors.Is(err, ErrNoKey) {
t.Fatalf("got %v, want ErrNoKey", err)
}
if buf.Len() != 0 {
t.Fatal("refusal must happen before anything is written")
}
}
func TestDumpAllowNoKeyStampsNull(t *testing.T) {
client, dbName := testDB(t)
seed(t, client, dbName)
_, m := dumpToFile(t, DumpOptions{Client: client, Database: dbName, AllowNoKey: true})
if m.KeyFingerprint != nil {
t.Fatalf("want a null fingerprint, got %v", *m.KeyFingerprint)
}
}
func TestDumpRejectsMalformedKey(t *testing.T) {
client, dbName := testDB(t)
var buf bytes.Buffer
_, err := Dump(context.Background(), DumpOptions{
Client: client, Database: dbName, KeyHex: "nonsense", Out: &buf,
})
if !errors.Is(err, ErrBadKey) {
t.Fatalf("got %v, want ErrBadKey", err)
}
}
func TestDumpExcludeIsRecordedAndOmitted(t *testing.T) {
client, dbName := testDB(t)
seed(t, client, dbName)
path, m := dumpToFile(t, DumpOptions{
Client: client, Database: dbName, KeyHex: validKeyHex,
Exclude: []string{"audit_logs"},
})
if _, ok := m.Collection("audit_logs"); ok {
t.Fatal("excluded collection is in the manifest's collection list")
}
if len(m.Excluded) != 1 || m.Excluded[0] != "audit_logs" {
t.Fatalf("excluded recorded as %v", m.Excluded)
}
r, err := Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
defer r.Close()
if _, err := r.OpenCollection("audit_logs"); err == nil {
t.Fatal("excluded collection is present in the archive")
}
}
func TestDumpPreservesAwkwardBSONTypes(t *testing.T) {
client, dbName := testDB(t)
ctx := context.Background()
dec, err := bson.ParseDecimal128("1234.5678")
if err != nil {
t.Fatalf("ParseDecimal128: %v", err)
}
doc := bson.M{
"_id": bson.NewObjectID(),
"decimal": dec,
"when": bson.NewDateTimeFromTime(mustTime(t)),
"binary": bson.Binary{Subtype: 0x00, Data: []byte{0x01, 0x02, 0x03}},
"nothing": nil,
"nested": bson.A{bson.M{"deep": bson.A{1, 2, 3}}},
}
if _, err := client.Database(dbName).Collection("odd").InsertOne(ctx, doc); err != nil {
t.Fatalf("insert: %v", err)
}
path, _ := dumpToFile(t, DumpOptions{Client: client, Database: dbName, KeyHex: validKeyHex})
original, err := client.Database(dbName).Collection("odd").FindOne(ctx, bson.M{}).Raw()
if err != nil {
t.Fatalf("read back: %v", err)
}
r, err := Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
defer r.Close()
rc, err := r.OpenCollection("odd")
if err != nil {
t.Fatalf("OpenCollection: %v", err)
}
defer rc.Close()
archived, err := io.ReadAll(rc)
if err != nil {
t.Fatalf("read: %v", err)
}
if !bytes.Equal(archived, []byte(original)) {
t.Fatal("archived BSON differs from what the driver returned")
}
}
func mustTime(t *testing.T) time.Time {
t.Helper()
return time.Date(2026, 9, 7, 12, 0, 0, 0, time.UTC)
}
+57
View File
@@ -0,0 +1,57 @@
// Package backup dumps and restores a whole Vantage MongoDB database.
//
// The archive never contains KEY_ENCRYPTION_KEY. It contains a fingerprint of
// it, which is enough to answer "will this archive restore into this
// deployment" and is not a hint at the value.
package backup
import (
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/cryptobox"
)
// ErrNoKey is returned when no key was supplied at all. It is distinct from
// ErrBadKey because the operator remedies are different: one is "set the
// variable", the other is "the value you set is wrong".
var ErrNoKey = errors.New("KEY_ENCRYPTION_KEY is not set")
// ErrBadKey is returned when a key was supplied but is not 64 hex characters.
var ErrBadKey = errors.New("KEY_ENCRYPTION_KEY must be a 64-character hex string (32 bytes)")
// ParseKey decodes the hex form used by KEY_ENCRYPTION_KEY.
func ParseKey(hexKey string) ([]byte, error) {
if hexKey == "" {
return nil, ErrNoKey
}
key, err := hex.DecodeString(hexKey)
if err != nil {
return nil, fmt.Errorf("%w: not hexadecimal", ErrBadKey)
}
if len(key) != cryptobox.KeySize {
return nil, fmt.Errorf("%w: decoded to %d bytes", ErrBadKey, len(key))
}
return key, nil
}
// Fingerprint is the SHA-256 of the raw key bytes, hex encoded.
//
// Of the raw bytes rather than of the hex string, so an operator who writes the
// key in uppercase in one deployment and lowercase in another still gets one
// fingerprint for one key.
func Fingerprint(key []byte) string {
sum := sha256.Sum256(key)
return hex.EncodeToString(sum[:])
}
// FingerprintHex parses and fingerprints in one step.
func FingerprintHex(hexKey string) (string, error) {
key, err := ParseKey(hexKey)
if err != nil {
return "", err
}
return Fingerprint(key), nil
}
+61
View File
@@ -0,0 +1,61 @@
package backup
import (
"errors"
"strings"
"testing"
)
const validKeyHex = "0000000000000000000000000000000000000000000000000000000000000001"
func TestFingerprintIsStableAndNotTheKey(t *testing.T) {
fp, err := FingerprintHex(validKeyHex)
if err != nil {
t.Fatalf("FingerprintHex: %v", err)
}
if len(fp) != 64 {
t.Fatalf("fingerprint is %d chars, want 64", len(fp))
}
if strings.EqualFold(fp, validKeyHex) {
t.Fatal("fingerprint equals the key")
}
again, err := FingerprintHex(validKeyHex)
if err != nil {
t.Fatalf("FingerprintHex: %v", err)
}
if fp != again {
t.Fatal("fingerprint is not stable across calls")
}
}
func TestFingerprintDiffersPerKey(t *testing.T) {
other := "0000000000000000000000000000000000000000000000000000000000000002"
a, err := FingerprintHex(validKeyHex)
if err != nil {
t.Fatalf("FingerprintHex: %v", err)
}
b, err := FingerprintHex(other)
if err != nil {
t.Fatalf("FingerprintHex: %v", err)
}
if a == b {
t.Fatal("two different keys produced the same fingerprint")
}
}
func TestParseKeyRejections(t *testing.T) {
if _, err := ParseKey(""); !errors.Is(err, ErrNoKey) {
t.Fatalf("empty key: got %v, want ErrNoKey", err)
}
for _, bad := range []string{"zz", validKeyHex[:62], validKeyHex + "00"} {
if _, err := ParseKey(bad); !errors.Is(err, ErrBadKey) {
t.Fatalf("key %q: got %v, want ErrBadKey", bad, err)
}
}
}
func TestParseKeyAcceptsUppercase(t *testing.T) {
if _, err := ParseKey(strings.ToUpper(validKeyHex)); err != nil {
t.Fatalf("uppercase hex rejected: %v", err)
}
}
+102
View File
@@ -0,0 +1,102 @@
package backup
import (
"errors"
"fmt"
"strings"
"time"
)
// FormatVersion is the archive format this build reads and writes. Restore
// refuses anything else rather than guessing at a layout it does not know.
const FormatVersion = 1
// ManifestName is the archive member holding the manifest.
const ManifestName = "manifest.json"
// ErrUnknownFormat is returned for an archive this build cannot read.
var ErrUnknownFormat = errors.New("unsupported archive format version")
// ErrBadCollectionName is returned for a manifest naming a collection that
// cannot safely be used as a path component.
var ErrBadCollectionName = errors.New("manifest names an unusable collection")
// CollectionEntry describes one collection in the archive. Bytes and SHA256
// cover the uncompressed .bson member, which is what restore verifies before
// writing anything.
type CollectionEntry struct {
Name string `json:"name"`
Documents int64 `json:"documents"`
Bytes int64 `json:"bytes"`
SHA256 string `json:"sha256"`
}
// Manifest is the archive's index and its provenance.
//
// KeyFingerprint is a pointer so "this archive recorded no key" is a distinct
// state from "this archive recorded the empty string". A null here is a real
// condition an operator must be told about, not a default.
type Manifest struct {
FormatVersion int `json:"format_version"`
CreatedAt time.Time `json:"created_at"`
VantageVersion string `json:"vantage_version"`
Hostname string `json:"hostname"`
MongoDB string `json:"mongo_db"`
MongoServerVersion string `json:"mongo_server_version"`
KeyFingerprint *string `json:"key_fingerprint"`
Collections []CollectionEntry `json:"collections"`
Excluded []string `json:"excluded"`
}
// Check validates what can be validated without reading the rest of the archive.
func (m Manifest) Check() error {
if m.FormatVersion != FormatVersion {
return fmt.Errorf("%w: archive is version %d, this build reads version %d",
ErrUnknownFormat, m.FormatVersion, FormatVersion)
}
// Collection names become path components inside the extraction directory,
// and an archive is operator-supplied input that may not be one we wrote.
for _, c := range m.Collections {
if err := checkCollectionName(c.Name); err != nil {
return err
}
}
return nil
}
// checkCollectionName refuses a name that could escape a directory when joined
// as a path component.
func checkCollectionName(name string) error {
switch {
case name == "":
return fmt.Errorf("%w: a collection entry has no name", ErrBadCollectionName)
case name == "." || name == "..":
return fmt.Errorf("%w: %q", ErrBadCollectionName, name)
case strings.ContainsAny(name, "/\\"), strings.Contains(name, ".."):
return fmt.Errorf("%w: %q", ErrBadCollectionName, name)
}
return nil
}
// Collection looks up one entry by name.
func (m Manifest) Collection(name string) (CollectionEntry, bool) {
for _, c := range m.Collections {
if c.Name == name {
return c, true
}
}
return CollectionEntry{}, false
}
// CiphertextCollections names the collections holding AES-GCM ciphertext.
//
// It exists to be printed. When a restore proceeds under a key that does not
// match the archive, this is the list of what will be unreadable afterwards,
// and an operator deserves to see it before the write rather than discover it
// a week later.
//
// settings is not in the list: it holds no encrypted material. Its ESO read
// token is a SHA-256 hash, not ciphertext.
func CiphertextCollections() []string {
return []string{"keys", "secrets", "auth_providers", "console_sessions"}
}
+103
View File
@@ -0,0 +1,103 @@
package backup
import (
"encoding/json"
"errors"
"strings"
"testing"
"time"
)
func TestManifestJSONShape(t *testing.T) {
fp := "abc"
m := Manifest{
FormatVersion: FormatVersion,
CreatedAt: time.Date(2026, 9, 7, 12, 0, 0, 0, time.UTC),
VantageVersion: "dev",
Hostname: "box",
MongoDB: "vantage",
MongoServerVersion: "7.0.5",
KeyFingerprint: &fp,
Collections: []CollectionEntry{{Name: "servers", Documents: 3, Bytes: 120, SHA256: "dead"}},
Excluded: []string{"audit_logs"},
}
raw, err := json.Marshal(m)
if err != nil {
t.Fatalf("marshal: %v", err)
}
for _, want := range []string{
`"format_version":1`, `"created_at":"2026-09-07T12:00:00Z"`,
`"key_fingerprint":"abc"`, `"mongo_server_version":"7.0.5"`,
`"excluded":["audit_logs"]`,
} {
if !strings.Contains(string(raw), want) {
t.Fatalf("manifest JSON missing %s\ngot: %s", want, raw)
}
}
}
func TestManifestNullFingerprint(t *testing.T) {
raw, err := json.Marshal(Manifest{FormatVersion: FormatVersion})
if err != nil {
t.Fatalf("marshal: %v", err)
}
if !strings.Contains(string(raw), `"key_fingerprint":null`) {
t.Fatalf("absent key must marshal as null, got: %s", raw)
}
}
func TestManifestCheckRejectsOtherVersions(t *testing.T) {
if err := (Manifest{FormatVersion: FormatVersion}).Check(); err != nil {
t.Fatalf("current version rejected: %v", err)
}
for _, v := range []int{0, 2, 99} {
if err := (Manifest{FormatVersion: v}).Check(); !errors.Is(err, ErrUnknownFormat) {
t.Fatalf("version %d: got %v, want ErrUnknownFormat", v, err)
}
}
}
func TestManifestCollectionLookup(t *testing.T) {
m := Manifest{Collections: []CollectionEntry{{Name: "keys", Documents: 1}}}
if _, ok := m.Collection("keys"); !ok {
t.Fatal("known collection not found")
}
if _, ok := m.Collection("nope"); ok {
t.Fatal("unknown collection reported as found")
}
}
func TestCiphertextCollections(t *testing.T) {
got := CiphertextCollections()
want := []string{"keys", "secrets", "auth_providers", "console_sessions"}
if len(got) != len(want) {
t.Fatalf("got %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("got %v, want %v", got, want)
}
}
}
// TestManifestRefusesUnusableCollectionNames covers names being used as path
// components inside the extraction directory. An archive is operator-supplied
// input and may not be one we wrote.
func TestManifestRefusesUnusableCollectionNames(t *testing.T) {
for _, name := range []string{"", ".", "..", "../etc/passwd", "a/b", "a..b"} {
m := Manifest{
FormatVersion: FormatVersion,
Collections: []CollectionEntry{{Name: name}},
}
if err := m.Check(); !errors.Is(err, ErrBadCollectionName) {
t.Fatalf("collection name %q was accepted (err %v)", name, err)
}
}
m := Manifest{
FormatVersion: FormatVersion,
Collections: []CollectionEntry{{Name: "workflow_log_lines"}},
}
if err := m.Check(); err != nil {
t.Fatalf("an ordinary collection name was refused: %v", err)
}
}
+44
View File
@@ -0,0 +1,44 @@
package backup
import (
"context"
"fmt"
"os"
"testing"
"time"
"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
// testDB connects to the MongoDB named by MONGO_TEST_URI and returns a client
// plus a database name unique to this test, dropped when the test ends.
//
// Skips rather than fails when the variable is unset: these tests need a real
// server, and a developer without one should still be able to run the rest of
// the suite.
func testDB(t *testing.T) (*mongo.Client, string) {
t.Helper()
uri := os.Getenv("MONGO_TEST_URI")
if uri == "" {
t.Skip("MONGO_TEST_URI is not set; skipping tests that need MongoDB")
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
client, err := mongo.Connect(options.Client().ApplyURI(uri))
if err != nil {
t.Fatalf("connect: %v", err)
}
if err := client.Ping(ctx, nil); err != nil {
t.Fatalf("ping: %v", err)
}
name := fmt.Sprintf("vantage_test_%d", time.Now().UnixNano())
t.Cleanup(func() {
c, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
_ = client.Database(name).Drop(c)
_ = client.Disconnect(c)
})
return client, name
}
+393
View File
@@ -0,0 +1,393 @@
package backup
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"sort"
"strings"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
// BatchSize is how many documents are inserted per bulk write.
const BatchSize = 1000
// ErrTargetNotEmpty is returned when the target database already holds data and
// Force was not set.
var ErrTargetNotEmpty = errors.New("target database is not empty")
// ErrKeyMismatch is returned when the archive's key fingerprint does not match
// the key supplied.
var ErrKeyMismatch = errors.New("KEY_ENCRYPTION_KEY does not match the archive")
// ErrIndexBuild is returned when a unique index in the archive cannot be built
// on the restored data.
var ErrIndexBuild = errors.New("index could not be built on the restored data")
// RestoredCollection is what one collection's restore produced.
type RestoredCollection struct {
Name string
Documents int64
Indexes int
}
// RestoreResult is the summary a caller prints.
type RestoreResult struct {
Collections []RestoredCollection
}
// RestoreOptions configures one restore.
type RestoreOptions struct {
Client *mongo.Client
Database string
Archive *Reader
// Force drops each collection in the archive before loading it. Without it
// a non-empty target is refused.
Force bool
KeyHex string
// IgnoreKeyMismatch proceeds past a fingerprint mismatch, having first
// warned which collections will hold unreadable ciphertext afterwards.
IgnoreKeyMismatch bool
// Warn receives operator-facing warnings. A nil Warn discards them.
Warn func(string)
}
func (o RestoreOptions) warn(format string, args ...any) {
if o.Warn != nil {
o.Warn(fmt.Sprintf(format, args...))
}
}
// Restore loads an archive into a database.
//
// The order is fixed and every check that can refuse does so before the first
// write: format, checksums (done by Open), key policy, then target inspection.
// A restore that has begun writing and then fails leaves a partial database
// which the next run refuses to touch, which is correct — the alternative is a
// silent merge, and merging two control planes reconciles nothing.
func Restore(ctx context.Context, opt RestoreOptions) (RestoreResult, error) {
m := opt.Archive.Manifest()
if err := checkKey(m, opt); err != nil {
return RestoreResult{}, err
}
if err := checkTarget(ctx, opt); err != nil {
return RestoreResult{}, err
}
if err := warnLeftovers(ctx, opt, m); err != nil {
return RestoreResult{}, err
}
if len(m.Excluded) > 0 {
opt.warn("this archive excluded %s; those collections will be empty after the restore",
strings.Join(m.Excluded, ", "))
}
opt.warn("Redis is not restored. Sessions are the only state it holds, so everyone signs in again.")
res := RestoreResult{}
if err := warnVersionGap(ctx, opt, m); err != nil {
return res, err
}
for _, entry := range m.Collections {
rc, err := restoreCollection(ctx, opt, entry)
if err != nil {
return res, err
}
res.Collections = append(res.Collections, rc)
}
return res, nil
}
// checkKey applies the fingerprint policy.
func checkKey(m Manifest, opt RestoreOptions) error {
if m.KeyFingerprint == nil {
opt.warn("this archive carries no key fingerprint, so nothing here proves your " +
"KEY_ENCRYPTION_KEY opens its ciphertext")
return nil
}
if opt.KeyHex == "" {
return fmt.Errorf("%w: the archive records a key fingerprint, so a key is required "+
"(pass --ignore-key-mismatch only if you accept unreadable secrets)", ErrNoKey)
}
got, err := FingerprintHex(opt.KeyHex)
if err != nil {
return err
}
if got == *m.KeyFingerprint {
return nil
}
if !opt.IgnoreKeyMismatch {
return fmt.Errorf("%w: archive fingerprint %s, your key fingerprints as %s",
ErrKeyMismatch, *m.KeyFingerprint, got)
}
opt.warn("proceeding past a key mismatch: ciphertext in %s will be permanently unreadable",
strings.Join(CiphertextCollections(), ", "))
return nil
}
// checkTarget refuses a non-empty database unless Force was set.
func checkTarget(ctx context.Context, opt RestoreOptions) error {
db := opt.Client.Database(opt.Database)
names, err := db.ListCollectionNames(ctx, bson.M{})
if err != nil {
return fmt.Errorf("inspect target: %w", err)
}
if len(names) == 0 || opt.Force {
return nil
}
sort.Strings(names)
var found []string
for _, n := range names {
count, err := db.Collection(n).CountDocuments(ctx, bson.M{})
if err != nil {
return fmt.Errorf("count %s: %w", n, err)
}
found = append(found, fmt.Sprintf("%s (%d)", n, count))
}
return fmt.Errorf("%w: %s holds %s", ErrTargetNotEmpty, opt.Database, strings.Join(found, ", "))
}
// warnLeftovers names the collections already in the target that the archive
// does not carry.
//
// They are named rather than dropped. A --force restore of an archive taken
// with --exclude workflow_log_lines leaves the old logs joined to restored
// runs, which the operator must know; but dropping a collection the archive
// never mentioned would delete data nobody asked to delete, and there is no
// way back from that.
func warnLeftovers(ctx context.Context, opt RestoreOptions, m Manifest) error {
if !opt.Force {
// Without Force the target was already proven empty.
return nil
}
names, err := opt.Client.Database(opt.Database).ListCollectionNames(ctx, bson.M{})
if err != nil {
return fmt.Errorf("inspect target: %w", err)
}
inArchive := map[string]bool{}
for _, c := range m.Collections {
inArchive[c.Name] = true
}
var leftover []string
for _, n := range names {
if !inArchive[n] {
leftover = append(leftover, n)
}
}
if len(leftover) == 0 {
return nil
}
sort.Strings(leftover)
opt.warn("this archive does not carry %s, which already exist in %s and are left "+
"untouched: their contents will sit alongside the restored data",
strings.Join(leftover, ", "), opt.Database)
return nil
}
// warnVersionGap reports a major version difference between the server that
// produced the archive and the one receiving it. It warns rather than refuses:
// restoring across a major version is a normal part of an upgrade, and a tool
// that refused would be blocking the migration it exists to make safe.
func warnVersionGap(ctx context.Context, opt RestoreOptions, m Manifest) error {
if m.MongoServerVersion == "" {
return nil
}
target, err := mongoServerVersion(ctx, opt.Client)
if err != nil {
return err
}
if majorOf(m.MongoServerVersion) != majorOf(target) {
opt.warn("this archive came from MongoDB %s and you are restoring onto %s",
m.MongoServerVersion, target)
}
return nil
}
func majorOf(version string) string {
if i := strings.IndexByte(version, '.'); i >= 0 {
return version[:i]
}
return version
}
func restoreCollection(ctx context.Context, opt RestoreOptions, entry CollectionEntry) (RestoredCollection, error) {
coll := opt.Client.Database(opt.Database).Collection(entry.Name)
if opt.Force {
if err := coll.Drop(ctx); err != nil {
return RestoredCollection{}, fmt.Errorf("drop %s: %w", entry.Name, err)
}
}
written, err := insertDocuments(ctx, opt, coll, entry)
if err != nil {
return RestoredCollection{}, err
}
indexes, err := replayIndexes(ctx, opt, coll, entry.Name)
if err != nil {
return RestoredCollection{}, err
}
return RestoredCollection{Name: entry.Name, Documents: written, Indexes: indexes}, nil
}
func insertDocuments(ctx context.Context, opt RestoreOptions, coll *mongo.Collection, entry CollectionEntry) (int64, error) {
rc, err := opt.Archive.OpenCollection(entry.Name)
if err != nil {
return 0, fmt.Errorf("open %s in archive: %w", entry.Name, err)
}
defer rc.Close()
raw, err := io.ReadAll(rc)
if err != nil {
return 0, fmt.Errorf("read %s: %w", entry.Name, err)
}
var written int64
batch := make([]any, 0, BatchSize)
flush := func() error {
if len(batch) == 0 {
return nil
}
if _, err := coll.InsertMany(ctx, batch, options.InsertMany().SetOrdered(false)); err != nil {
return fmt.Errorf("insert into %s: %w", entry.Name, err)
}
written += int64(len(batch))
batch = batch[:0]
return nil
}
for len(raw) > 0 {
doc, rest, err := splitBSON(raw)
if err != nil {
return 0, fmt.Errorf("%s: %w", entry.Name, err)
}
batch = append(batch, doc)
raw = rest
if len(batch) == BatchSize {
if err := flush(); err != nil {
return 0, err
}
}
}
if err := flush(); err != nil {
return 0, err
}
return written, nil
}
// splitBSON peels one document off the front of a concatenated BSON stream. A
// BSON document declares its own length in its first four bytes.
func splitBSON(raw []byte) (bson.Raw, []byte, error) {
if len(raw) < 4 {
return nil, nil, fmt.Errorf("truncated BSON: %d trailing bytes", len(raw))
}
n := int(int32(raw[0]) | int32(raw[1])<<8 | int32(raw[2])<<16 | int32(raw[3])<<24)
if n < 5 || n > len(raw) {
return nil, nil, fmt.Errorf("BSON document declares length %d with %d bytes remaining", n, len(raw))
}
return bson.Raw(raw[:n]), raw[n:], nil
}
// replayIndexes recreates the archived indexes.
//
// The specs are handed to the createIndexes command exactly as the source
// server reported them, rather than reconstructed into a mongo.IndexModel from
// a hand-picked set of options. Reconstruction dropped every option nobody had
// thought to pick — partialFilterExpression above all, which this codebase
// relies on for partial unique indexes, and which replayed as a full unique
// index fails on any real database. It also lost compound key order, which is
// significant.
//
// A unique index that will not build means the restored data violates it, and
// the unique indexes here — (instance_id, email), instance slug, settings
// instance, the ESO token hash — are tenant-isolation properties rather than
// optimisations. That aborts. A non-unique index failing is a performance
// problem and warns.
func replayIndexes(ctx context.Context, opt RestoreOptions, coll *mongo.Collection, name string) (int, error) {
raw, err := opt.Archive.IndexesJSON(name)
if err != nil {
return 0, fmt.Errorf("read index specs for %s: %w", name, err)
}
if len(raw) == 0 {
return 0, nil
}
var encoded []json.RawMessage
if err := json.Unmarshal(raw, &encoded); err != nil {
return 0, fmt.Errorf("parse index specs for %s: %w", name, err)
}
db := coll.Database()
created := 0
for _, ej := range encoded {
spec, indexName, unique, ok, err := indexSpecFrom(ej)
if err != nil {
return created, fmt.Errorf("parse index specs for %s: %w", name, err)
}
if !ok {
continue
}
cmd := bson.D{
{Key: "createIndexes", Value: name},
{Key: "indexes", Value: bson.A{spec}},
}
if err := db.RunCommand(ctx, cmd).Err(); err != nil {
if unique {
return created, fmt.Errorf("%w: %s on %s: %v", ErrIndexBuild, indexName, name, err)
}
opt.warn("index %s on %s was not created: %v", indexName, name, err)
continue
}
created++
}
return created, nil
}
// droppedIndexSpecFields are the fields the server reports on an existing index
// but rejects when creating one. Everything else is passed through untouched.
var droppedIndexSpecFields = map[string]bool{"v": true, "ns": true}
// indexSpecFrom decodes one archived extended-JSON index specification into an
// ordered bson.D suitable for createIndexes.
//
// The _id_ index is skipped: MongoDB creates it itself and refuses an explicit
// attempt to create it.
func indexSpecFrom(ej []byte) (bson.D, string, bool, bool, error) {
var d bson.D
if err := bson.UnmarshalExtJSON(ej, false, &d); err != nil {
return nil, "", false, false, err
}
out := make(bson.D, 0, len(d))
var name string
unique := false
hasKey := false
for _, e := range d {
switch e.Key {
case "name":
name, _ = e.Value.(string)
case "unique":
if u, ok := e.Value.(bool); ok {
unique = u
}
case "key":
hasKey = true
}
if droppedIndexSpecFields[e.Key] {
continue
}
out = append(out, e)
}
if name == "_id_" || !hasKey {
return nil, name, false, false, nil
}
return out, name, unique, true, nil
}
+439
View File
@@ -0,0 +1,439 @@
package backup
import (
"context"
"errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
// archiveOf seeds a database, dumps it, and returns an opened Reader.
func archiveOf(t *testing.T, client *mongo.Client, keyHex string, allowNoKey bool) *Reader {
t.Helper()
_, srcDB := testDB(t)
seed(t, client, srcDB)
path, _ := dumpToFile(t, DumpOptions{
Client: client, Database: srcDB, KeyHex: keyHex, AllowNoKey: allowNoKey,
})
r, err := Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
t.Cleanup(func() { r.Close() })
return r
}
func countIn(t *testing.T, client *mongo.Client, dbName, coll string) int64 {
t.Helper()
n, err := client.Database(dbName).Collection(coll).CountDocuments(context.Background(), bson.M{})
if err != nil {
t.Fatalf("count %s: %v", coll, err)
}
return n
}
func TestRestoreIntoEmptyDatabase(t *testing.T) {
client, _ := testDB(t)
archive := archiveOf(t, client, validKeyHex, false)
_, target := testDB(t)
res, err := Restore(context.Background(), RestoreOptions{
Client: client, Database: target, Archive: archive, KeyHex: validKeyHex,
})
if err != nil {
t.Fatalf("Restore: %v", err)
}
if countIn(t, client, target, "servers") != 2 {
t.Fatal("servers not restored")
}
if len(res.Collections) == 0 {
t.Fatal("result reports no collections")
}
}
func TestRestoreRefusesNonEmptyTarget(t *testing.T) {
client, _ := testDB(t)
archive := archiveOf(t, client, validKeyHex, false)
_, target := testDB(t)
if _, err := client.Database(target).Collection("servers").
InsertOne(context.Background(), bson.M{"name": "existing"}); err != nil {
t.Fatalf("seed target: %v", err)
}
_, err := Restore(context.Background(), RestoreOptions{
Client: client, Database: target, Archive: archive, KeyHex: validKeyHex,
})
if !errors.Is(err, ErrTargetNotEmpty) {
t.Fatalf("got %v, want ErrTargetNotEmpty", err)
}
if countIn(t, client, target, "servers") != 1 {
t.Fatal("a refused restore modified the target")
}
}
func TestRestoreForceReplaces(t *testing.T) {
client, _ := testDB(t)
archive := archiveOf(t, client, validKeyHex, false)
_, target := testDB(t)
if _, err := client.Database(target).Collection("servers").
InsertOne(context.Background(), bson.M{"name": "existing"}); err != nil {
t.Fatalf("seed target: %v", err)
}
if _, err := Restore(context.Background(), RestoreOptions{
Client: client, Database: target, Archive: archive, KeyHex: validKeyHex, Force: true,
}); err != nil {
t.Fatalf("Restore --force: %v", err)
}
if got := countIn(t, client, target, "servers"); got != 2 {
t.Fatalf("servers has %d documents, want 2; force must drop, not merge", got)
}
n, err := client.Database(target).Collection("servers").
CountDocuments(context.Background(), bson.M{"name": "existing"})
if err != nil {
t.Fatalf("count: %v", err)
}
if n != 0 {
t.Fatal("the pre-existing document survived --force")
}
}
func TestRestoreRefusesKeyMismatch(t *testing.T) {
client, _ := testDB(t)
archive := archiveOf(t, client, validKeyHex, false)
_, target := testDB(t)
other := "0000000000000000000000000000000000000000000000000000000000000002"
_, err := Restore(context.Background(), RestoreOptions{
Client: client, Database: target, Archive: archive, KeyHex: other,
})
if !errors.Is(err, ErrKeyMismatch) {
t.Fatalf("got %v, want ErrKeyMismatch", err)
}
names, err := client.Database(target).ListCollectionNames(context.Background(), bson.M{})
if err != nil {
t.Fatalf("list: %v", err)
}
if len(names) != 0 {
t.Fatalf("a refused restore wrote %v", names)
}
}
func TestRestoreRefusesWhenArchiveHasKeyAndEnvironmentDoesNot(t *testing.T) {
client, _ := testDB(t)
archive := archiveOf(t, client, validKeyHex, false)
_, target := testDB(t)
_, err := Restore(context.Background(), RestoreOptions{
Client: client, Database: target, Archive: archive,
})
if !errors.Is(err, ErrNoKey) {
t.Fatalf("got %v, want ErrNoKey", err)
}
}
func TestRestoreIgnoreKeyMismatchWarnsAndProceeds(t *testing.T) {
client, _ := testDB(t)
archive := archiveOf(t, client, validKeyHex, false)
_, target := testDB(t)
other := "0000000000000000000000000000000000000000000000000000000000000002"
var warnings []string
if _, err := Restore(context.Background(), RestoreOptions{
Client: client, Database: target, Archive: archive,
KeyHex: other, IgnoreKeyMismatch: true,
Warn: func(s string) { warnings = append(warnings, s) },
}); err != nil {
t.Fatalf("Restore: %v", err)
}
joined := strings.Join(warnings, "\n")
for _, name := range CiphertextCollections() {
if !strings.Contains(joined, name) {
t.Fatalf("warning does not name %s\ngot:\n%s", name, joined)
}
}
if countIn(t, client, target, "servers") != 2 {
t.Fatal("restore did not proceed")
}
}
func TestRestoreNullFingerprintIsReportedNotAssumed(t *testing.T) {
client, _ := testDB(t)
archive := archiveOf(t, client, "", true)
_, target := testDB(t)
var warnings []string
if _, err := Restore(context.Background(), RestoreOptions{
Client: client, Database: target, Archive: archive, KeyHex: validKeyHex,
Warn: func(s string) { warnings = append(warnings, s) },
}); err != nil {
t.Fatalf("Restore: %v", err)
}
if !strings.Contains(strings.Join(warnings, "\n"), "no key fingerprint") {
t.Fatalf("a null fingerprint must be reported, got: %v", warnings)
}
}
func TestRestoreReplaysIndexes(t *testing.T) {
client, _ := testDB(t)
ctx := context.Background()
_, srcDB := testDB(t)
seed(t, client, srcDB)
if _, err := client.Database(srcDB).Collection("servers").Indexes().
CreateOne(ctx, mongo.IndexModel{Keys: bson.D{{Key: "name", Value: 1}}}); err != nil {
t.Fatalf("create index: %v", err)
}
path, _ := dumpToFile(t, DumpOptions{Client: client, Database: srcDB, KeyHex: validKeyHex})
archive, err := Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
defer archive.Close()
_, target := testDB(t)
res, err := Restore(ctx, RestoreOptions{
Client: client, Database: target, Archive: archive, KeyHex: validKeyHex,
})
if err != nil {
t.Fatalf("Restore: %v", err)
}
cur, err := client.Database(target).Collection("servers").Indexes().List(ctx)
if err != nil {
t.Fatalf("list indexes: %v", err)
}
var specs []bson.M
if err := cur.All(ctx, &specs); err != nil {
t.Fatalf("read indexes: %v", err)
}
found := false
for _, s := range specs {
if s["name"] == "name_1" {
found = true
}
}
if !found {
t.Fatalf("index name_1 not replayed; got %v", specs)
}
for _, c := range res.Collections {
if c.Name == "servers" && c.Indexes < 1 {
t.Fatal("result reports no indexes created for servers")
}
}
}
// TestRestoreAbortsWhenAUniqueIndexCannotBuild replaces the brief's
// TestRestoreAbortsOnUniqueIndexViolation per ruling 1: creating an index on
// the target also creates the collection, so that version's assertion
// (err == nil) would have passed on the wrong error (ErrTargetNotEmpty), and
// Force: true does not rescue it because the drop removes the index before
// replayIndexes runs. This version builds a hand-made archive whose data and
// index specification directly contradict each other, which is the actual
// shape of a corrupted archive that replayIndexes must refuse to load.
func TestRestoreAbortsWhenAUniqueIndexCannotBuild(t *testing.T) {
client, _ := testDB(t)
ctx := context.Background()
// Built by hand rather than dumped: two documents that collide on email
// alongside an index specification declaring email unique. No live database
// would let those coexist, which is exactly the point — this is the shape
// of a corrupted or hand-edited archive, and restore must refuse rather
// than load the rows and leave the index missing.
a, err := bson.Marshal(bson.M{"email": "a@example.com"})
if err != nil {
t.Fatalf("marshal: %v", err)
}
b, err := bson.Marshal(bson.M{"email": "a@example.com"})
if err != nil {
t.Fatalf("marshal: %v", err)
}
path := filepath.Join(t.TempDir(), "dupes.tar.gz")
f, err := os.Create(path)
if err != nil {
t.Fatalf("create: %v", err)
}
w := NewWriter(f)
entry, err := w.WriteCollection("users", [][]byte{a, b})
if err != nil {
t.Fatalf("WriteCollection: %v", err)
}
if err := w.WriteIndexes("users",
[]byte(`[{"name":"email_1","key":{"email":1},"unique":true}]`)); err != nil {
t.Fatalf("WriteIndexes: %v", err)
}
fp, err := FingerprintHex(validKeyHex)
if err != nil {
t.Fatalf("FingerprintHex: %v", err)
}
if err := w.Close(Manifest{
FormatVersion: FormatVersion,
CreatedAt: time.Now().UTC(),
MongoDB: "handmade",
KeyFingerprint: &fp,
Collections: []CollectionEntry{entry},
}); err != nil {
t.Fatalf("Close: %v", err)
}
if err := f.Close(); err != nil {
t.Fatalf("close: %v", err)
}
archive, err := Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
defer archive.Close()
_, target := testDB(t)
if _, err := Restore(ctx, RestoreOptions{
Client: client, Database: target, Archive: archive, KeyHex: validKeyHex,
}); !errors.Is(err, ErrIndexBuild) {
t.Fatalf("got %v, want ErrIndexBuild", err)
} else if !strings.Contains(err.Error(), "email_1") {
t.Fatalf("the error must name the offending index, got: %v", err)
}
}
func TestRestoreSameVersionDoesNotWarnAboutIt(t *testing.T) {
client, _ := testDB(t)
archive := archiveOf(t, client, validKeyHex, false)
_, target := testDB(t)
var warnings []string
if _, err := Restore(context.Background(), RestoreOptions{
Client: client, Database: target, Archive: archive, KeyHex: validKeyHex,
Warn: func(s string) { warnings = append(warnings, s) },
}); err != nil {
t.Fatalf("Restore: %v", err)
}
for _, w := range warnings {
if strings.Contains(w, "you are restoring onto") {
t.Fatalf("same-version restore warned about a version gap: %s", w)
}
}
}
func TestCompoundIndexKeyOrderIsPreserved(t *testing.T) {
spec, name, unique, ok, err := indexSpecFrom([]byte(
`{"v":2,"key":{"b":1,"a":1},"name":"b_1_a_1","ns":"db.c"}`))
if err != nil {
t.Fatalf("indexSpecFrom: %v", err)
}
if !ok {
t.Fatal("spec rejected")
}
if unique {
t.Fatal("index reported as unique")
}
if name != "b_1_a_1" {
t.Fatalf("name %q", name)
}
var keys bson.D
for _, e := range spec {
switch e.Key {
case "v", "ns":
t.Fatalf("%q must be stripped before createIndexes, got %v", e.Key, spec)
case "key":
d, isD := e.Value.(bson.D)
if !isD {
t.Fatalf("key is %T, want bson.D", e.Value)
}
keys = d
}
}
// Compound index key order is significant, so it is carried through
// verbatim rather than reconstructed from an unordered map.
if len(keys) != 2 || keys[0].Key != "b" || keys[1].Key != "a" {
t.Fatalf("key order not preserved, got %v", keys)
}
}
func TestIdIndexIsSkipped(t *testing.T) {
_, name, _, ok, err := indexSpecFrom([]byte(`{"v":2,"key":{"_id":1},"name":"_id_"}`))
if err != nil {
t.Fatalf("indexSpecFrom: %v", err)
}
if ok {
t.Fatal("_id_ must be skipped; MongoDB creates it itself")
}
if name != "_id_" {
t.Fatalf("name %q", name)
}
}
// TestRestoreReplaysPartialUniqueIndex is the regression guard for the defect
// that made a restore abort on any real database: a partial unique index —
// this codebase has them on workflow_steps and settings — replayed as a full
// unique index hits duplicate keys, and a failing unique index is fatal.
func TestRestoreReplaysPartialUniqueIndex(t *testing.T) {
client, _ := testDB(t)
ctx := context.Background()
_, srcDB := testDB(t)
seed(t, client, srcDB)
coll := client.Database(srcDB).Collection("workflow_steps")
docs := []any{
bson.M{"instance_id": "i1", "slug": "same", "source": "default"},
bson.M{"instance_id": "i1", "slug": "same", "source": "custom"},
bson.M{"instance_id": "i1", "slug": "same", "source": "custom"},
}
if _, err := coll.InsertMany(ctx, docs); err != nil {
t.Fatalf("insert: %v", err)
}
if _, err := coll.Indexes().CreateOne(ctx, mongo.IndexModel{
Keys: bson.D{{Key: "instance_id", Value: 1}, {Key: "slug", Value: 1}},
Options: options.Index().SetName("default_step_slug").SetUnique(true).
SetPartialFilterExpression(bson.M{"source": "default"}),
}); err != nil {
t.Fatalf("create partial index: %v", err)
}
path, _ := dumpToFile(t, DumpOptions{Client: client, Database: srcDB, KeyHex: validKeyHex})
archive, err := Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
defer archive.Close()
_, target := testDB(t)
if _, err := Restore(ctx, RestoreOptions{
Client: client, Database: target, Archive: archive, KeyHex: validKeyHex,
}); err != nil {
t.Fatalf("Restore: %v", err)
}
cur, err := client.Database(target).Collection("workflow_steps").Indexes().List(ctx)
if err != nil {
t.Fatalf("list indexes: %v", err)
}
var specs []bson.M
if err := cur.All(ctx, &specs); err != nil {
t.Fatalf("read indexes: %v", err)
}
for _, s := range specs {
if s["name"] != "default_step_slug" {
continue
}
if s["unique"] != true {
t.Fatalf("index lost its uniqueness: %v", s)
}
if s["partialFilterExpression"] == nil {
t.Fatalf("partialFilterExpression was dropped: %v", s)
}
keys, isD := s["key"].(bson.D)
if isD && (len(keys) != 2 || keys[0].Key != "instance_id" || keys[1].Key != "slug") {
t.Fatalf("compound key order not preserved: %v", keys)
}
return
}
t.Fatalf("partial unique index not replayed; got %v", specs)
}
+197
View File
@@ -0,0 +1,197 @@
package backup
import (
"context"
"fmt"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/cryptobox"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
)
// VerifyOptions configures a verification. Client and Database are optional;
// supplying them turns on the live probe.
type VerifyOptions struct {
Archive *Reader
KeyHex string
Client *mongo.Client
Database string
}
// VerifyReport is what verify found.
type VerifyReport struct {
ArchiveFingerprint *string
KeyFingerprint *string
KeyMatchesArchive bool
// ProbeAttempted is false when no client was supplied, and also when the
// database holds no ciphertext to probe.
ProbeAttempted bool
ProbeCollection string
ProbeDecrypted bool
Problems []string
}
// OK reports whether this archive is usable with the key in hand.
func (r VerifyReport) OK() bool { return len(r.Problems) == 0 }
func (r *VerifyReport) problem(format string, args ...any) {
r.Problems = append(r.Problems, fmt.Sprintf(format, args...))
}
// Verify checks an already-opened archive against the key in hand and, when a
// client is supplied, against a live database.
//
// Open has already verified every member's checksum, so integrity is not
// rechecked here. What this adds is the question an operator actually has:
// will the key I hold open the data this archive carries. A fingerprint
// comparison proves two archives agree; only the probe proves the key opens
// real ciphertext.
func Verify(ctx context.Context, opt VerifyOptions) (VerifyReport, error) {
m := opt.Archive.Manifest()
rep := VerifyReport{ArchiveFingerprint: m.KeyFingerprint}
if opt.KeyHex != "" {
fp, err := FingerprintHex(opt.KeyHex)
if err != nil {
return rep, err
}
rep.KeyFingerprint = &fp
}
switch {
case m.KeyFingerprint == nil && rep.KeyFingerprint == nil:
rep.problem("neither the archive nor this environment names a key; nothing here " +
"proves the archive's ciphertext can ever be read")
case m.KeyFingerprint == nil:
rep.problem("the archive carries no key fingerprint, so it cannot be matched " +
"against the key you hold")
case rep.KeyFingerprint == nil:
rep.problem("KEY_ENCRYPTION_KEY is not set, so the archive's fingerprint %s "+
"cannot be checked against anything", *m.KeyFingerprint)
case *m.KeyFingerprint == *rep.KeyFingerprint:
rep.KeyMatchesArchive = true
default:
rep.problem("key mismatch: archive fingerprint %s, your key fingerprints as %s",
*m.KeyFingerprint, *rep.KeyFingerprint)
}
if opt.Client == nil || opt.Database == "" || opt.KeyHex == "" {
return rep, nil
}
if err := probe(ctx, opt, &rep); err != nil {
return rep, err
}
return rep, nil
}
// probe reads one ciphertext field from the live database and tries to open it.
func probe(ctx context.Context, opt VerifyOptions, rep *VerifyReport) error {
key, err := ParseKey(opt.KeyHex)
if err != nil {
return err
}
for _, coll := range CiphertextCollections() {
ciphertext, ok, err := findCiphertext(ctx, opt.Client.Database(opt.Database), coll)
if err != nil {
return err
}
if !ok {
continue
}
rep.ProbeAttempted = true
rep.ProbeCollection = coll
if _, err := cryptobox.Open(key, ciphertext); err != nil {
rep.problem("the key in hand does not decrypt live ciphertext in %s", coll)
return nil
}
rep.ProbeDecrypted = true
return nil
}
// No ciphertext anywhere is an ordinary state — a deployment that has
// stored no secrets, keys or SSO configuration yet — and is not a failure.
return nil
}
// ciphertextFields names, per collection, the fields that hold hex ciphertext.
// A value is a candidate only if it is a hex string long enough to carry a GCM
// nonce and tag, which is what keeps this from probing a plaintext field.
//
// This map MIRRORS BY HAND the bson tags in server/internal/models, which this
// package cannot import: shared/ is a separate module and models is under
// server/internal. It must change in the same commit as any rename of the
// fields below — the same mirrored-constant hazard as web/lib/targets.ts and
// services.MaxWorkloadLogLines. The sources are:
//
// keys — models/key.go: private_key_enc, passphrase_enc
// secrets — models/secret.go: encrypted_value
// auth_providers — models/auth_provider.go: client_secret_enc
// console_sessions — models/console_session.go: rdp_user_enc, rdp_pass_enc
//
// settings is deliberately absent: it holds no ciphertext at all. The ESO read
// token is stored as a SHA-256 hash, which no key opens.
var ciphertextFields = map[string][]string{
"keys": {"private_key_enc", "passphrase_enc"},
"secrets": {"encrypted_value"},
"auth_providers": {"client_secret_enc"},
"console_sessions": {"rdp_user_enc", "rdp_pass_enc"},
}
func findCiphertext(ctx context.Context, db *mongo.Database, coll string) (string, bool, error) {
fields, ok := ciphertextFields[coll]
if !ok {
return "", false, nil
}
cur, err := db.Collection(coll).Find(ctx, bson.M{})
if err != nil {
return "", false, fmt.Errorf("probe %s: %w", coll, err)
}
defer cur.Close(ctx)
for cur.Next(ctx) {
var doc bson.M
if err := cur.Decode(&doc); err != nil {
return "", false, fmt.Errorf("probe %s: %w", coll, err)
}
for _, f := range fields {
if v, ok := looksLikeCiphertext(doc[f]); ok {
return v, true, nil
}
}
}
return "", false, cur.Err()
}
// looksLikeCiphertext accepts a hex string long enough to be a sealed value. It
// descends into a sub-document so a field that holds a map of sealed values is
// still reachable.
func looksLikeCiphertext(v any) (string, bool) {
switch t := v.(type) {
case string:
// 12-byte nonce plus a 16-byte tag is 56 hex characters before any
// plaintext at all, so anything shorter is not a sealed value.
if len(t) < 56 || !isHex(t) {
return "", false
}
return t, true
case bson.M:
for _, inner := range t {
if s, ok := looksLikeCiphertext(inner); ok {
return s, true
}
}
}
return "", false
}
func isHex(s string) bool {
for _, c := range s {
switch {
case c >= '0' && c <= '9', c >= 'a' && c <= 'f', c >= 'A' && c <= 'F':
default:
return false
}
}
return true
}
+156
View File
@@ -0,0 +1,156 @@
package backup
import (
"context"
"testing"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/cryptobox"
"go.mongodb.org/mongo-driver/v2/bson"
)
func TestVerifyMatchingKey(t *testing.T) {
client, _ := testDB(t)
archive := archiveOf(t, client, validKeyHex, false)
rep, err := Verify(context.Background(), VerifyOptions{Archive: archive, KeyHex: validKeyHex})
if err != nil {
t.Fatalf("Verify: %v", err)
}
if !rep.KeyMatchesArchive {
t.Fatal("matching key reported as a mismatch")
}
if rep.ProbeAttempted {
t.Fatal("probe ran with no client supplied")
}
if !rep.OK() {
t.Fatalf("report not OK: %v", rep.Problems)
}
}
func TestVerifyMismatchedKeyIsNotOK(t *testing.T) {
client, _ := testDB(t)
archive := archiveOf(t, client, validKeyHex, false)
other := "0000000000000000000000000000000000000000000000000000000000000002"
rep, err := Verify(context.Background(), VerifyOptions{Archive: archive, KeyHex: other})
if err != nil {
t.Fatalf("Verify: %v", err)
}
if rep.KeyMatchesArchive {
t.Fatal("mismatched key reported as matching")
}
if rep.OK() {
t.Fatal("a mismatch must not report OK")
}
}
func TestVerifyProbeDecryptsLiveCiphertext(t *testing.T) {
client, dbName := testDB(t)
ctx := context.Background()
key, err := ParseKey(validKeyHex)
if err != nil {
t.Fatalf("ParseKey: %v", err)
}
sealed, err := cryptobox.Seal(key, "s3cret")
if err != nil {
t.Fatalf("Seal: %v", err)
}
if _, err := client.Database(dbName).Collection("secrets").InsertOne(ctx, bson.M{
"instance_id": "i1",
"values": bson.M{"TOKEN": sealed},
}); err != nil {
t.Fatalf("insert: %v", err)
}
path, _ := dumpToFile(t, DumpOptions{Client: client, Database: dbName, KeyHex: validKeyHex})
archive, err := Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
defer archive.Close()
rep, err := Verify(ctx, VerifyOptions{
Archive: archive, KeyHex: validKeyHex, Client: client, Database: dbName,
})
if err != nil {
t.Fatalf("Verify: %v", err)
}
if !rep.ProbeAttempted {
t.Fatal("probe did not run with a client supplied")
}
if !rep.ProbeDecrypted {
t.Fatalf("probe failed to decrypt live ciphertext: %v", rep.Problems)
}
if rep.ProbeCollection != "secrets" {
t.Fatalf("probe collection %q, want secrets", rep.ProbeCollection)
}
if !rep.OK() {
t.Fatalf("report not OK: %v", rep.Problems)
}
}
func TestVerifyProbeFailsWithWrongKey(t *testing.T) {
client, dbName := testDB(t)
ctx := context.Background()
key, err := ParseKey(validKeyHex)
if err != nil {
t.Fatalf("ParseKey: %v", err)
}
sealed, err := cryptobox.Seal(key, "s3cret")
if err != nil {
t.Fatalf("Seal: %v", err)
}
if _, err := client.Database(dbName).Collection("secrets").InsertOne(ctx, bson.M{
"values": bson.M{"TOKEN": sealed},
}); err != nil {
t.Fatalf("insert: %v", err)
}
other := "0000000000000000000000000000000000000000000000000000000000000002"
path, _ := dumpToFile(t, DumpOptions{Client: client, Database: dbName, KeyHex: other})
archive, err := Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
defer archive.Close()
rep, err := Verify(ctx, VerifyOptions{
Archive: archive, KeyHex: other, Client: client, Database: dbName,
})
if err != nil {
t.Fatalf("Verify: %v", err)
}
if rep.ProbeDecrypted {
t.Fatal("probe decrypted with the wrong key")
}
if rep.OK() {
t.Fatal("a failed probe must not report OK")
}
}
func TestVerifyProbeAbsentCiphertextIsNotAFailure(t *testing.T) {
client, dbName := testDB(t)
seed(t, client, dbName)
path, _ := dumpToFile(t, DumpOptions{Client: client, Database: dbName, KeyHex: validKeyHex})
archive, err := Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
defer archive.Close()
rep, err := Verify(context.Background(), VerifyOptions{
Archive: archive, KeyHex: validKeyHex, Client: client, Database: dbName,
})
if err != nil {
t.Fatalf("Verify: %v", err)
}
if rep.ProbeAttempted {
t.Fatal("probe claims to have run against a database with no ciphertext")
}
if !rep.OK() {
t.Fatalf("a database storing no secrets must still verify: %v", rep.Problems)
}
}
+67
View File
@@ -0,0 +1,67 @@
// Package cryptobox is the AES-256-GCM primitive used for everything Vantage
// encrypts at rest: SSH private keys, key passphrases, vault secrets, OIDC
// client secrets and console credentials.
//
// It takes a raw key and reads no environment. Key sourcing belongs to the
// caller, because the two callers source it differently: the server reads
// KEY_ENCRYPTION_KEY at the point of use, while vantagectl is handed one.
package cryptobox
import (
"crypto/aes"
"crypto/cipher"
"crypto/rand"
"encoding/hex"
"fmt"
"io"
)
// KeySize is the only key length accepted. AES-256 by construction.
const KeySize = 32
func gcmFor(key []byte) (cipher.AEAD, error) {
if len(key) != KeySize {
return nil, fmt.Errorf("key must be %d bytes, got %d", KeySize, len(key))
}
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
return cipher.NewGCM(block)
}
// Seal encrypts plaintext and returns nonce||ciphertext, hex encoded.
func Seal(key []byte, plaintext string) (string, error) {
gcm, err := gcmFor(key)
if err != nil {
return "", err
}
nonce := make([]byte, gcm.NonceSize())
if _, err := io.ReadFull(rand.Reader, nonce); err != nil {
return "", err
}
return hex.EncodeToString(gcm.Seal(nonce, nonce, []byte(plaintext), nil)), nil
}
// Open reverses Seal. Every failure mode returns an error that does not
// distinguish a wrong key from corrupt data, because the caller cannot act on
// the difference and an oracle is worth avoiding for free.
func Open(key []byte, ciphertextHex string) (string, error) {
gcm, err := gcmFor(key)
if err != nil {
return "", err
}
data, err := hex.DecodeString(ciphertextHex)
if err != nil {
return "", fmt.Errorf("invalid ciphertext encoding")
}
n := gcm.NonceSize()
if len(data) < n {
return "", fmt.Errorf("ciphertext too short")
}
plaintext, err := gcm.Open(nil, data[:n], data[n:], nil)
if err != nil {
return "", fmt.Errorf("decryption failed")
}
return string(plaintext), nil
}
+79
View File
@@ -0,0 +1,79 @@
package cryptobox
import (
"bytes"
"crypto/rand"
"encoding/hex"
"testing"
)
func testKey(t *testing.T) []byte {
t.Helper()
k := make([]byte, KeySize)
if _, err := rand.Read(k); err != nil {
t.Fatalf("rand: %v", err)
}
return k
}
func TestSealOpenRoundTrip(t *testing.T) {
key := testKey(t)
sealed, err := Seal(key, "hunter2")
if err != nil {
t.Fatalf("Seal: %v", err)
}
if _, err := hex.DecodeString(sealed); err != nil {
t.Fatalf("Seal output is not hex: %v", err)
}
if bytes.Contains([]byte(sealed), []byte("hunter2")) {
t.Fatal("plaintext appears in ciphertext")
}
got, err := Open(key, sealed)
if err != nil {
t.Fatalf("Open: %v", err)
}
if got != "hunter2" {
t.Fatalf("got %q, want %q", got, "hunter2")
}
}
func TestSealIsNonDeterministic(t *testing.T) {
key := testKey(t)
a, err := Seal(key, "same")
if err != nil {
t.Fatalf("Seal: %v", err)
}
b, err := Seal(key, "same")
if err != nil {
t.Fatalf("Seal: %v", err)
}
if a == b {
t.Fatal("two seals of the same plaintext are identical; nonce is not random")
}
}
func TestOpenWrongKeyFails(t *testing.T) {
sealed, err := Seal(testKey(t), "secret")
if err != nil {
t.Fatalf("Seal: %v", err)
}
if _, err := Open(testKey(t), sealed); err == nil {
t.Fatal("Open with the wrong key succeeded")
}
}
func TestOpenRejectsBadInput(t *testing.T) {
key := testKey(t)
if _, err := Open(key, "not-hex"); err == nil {
t.Fatal("Open accepted non-hex input")
}
if _, err := Open(key, "abcd"); err == nil {
t.Fatal("Open accepted a ciphertext shorter than the nonce")
}
}
func TestWrongKeySizeRejected(t *testing.T) {
if _, err := Seal(make([]byte, 16), "x"); err == nil {
t.Fatal("Seal accepted a 16-byte key")
}
}
+5
View File
@@ -35,6 +35,11 @@ const (
// findings themselves. The gate is at collection, not display: an ungated
// instance stores no inventory, and storage is the expensive half.
FeatureVulnScanning = "vuln_scanning"
// FeatureStatusPages gates public status pages at both ends: authoring
// them, and serving them. Serving answers 200 with available:false rather
// than 403, because the page has to render an explanation to a member of
// the public who cannot do anything about it.
FeatureStatusPages = "status_pages"
)
// Support levels. Carried for display and enforced by nothing — there is no code
+36
View File
@@ -0,0 +1,36 @@
# Build stage
#
# Context is the repository root, not vantagectl/, because vantagectl depends on
# the shared module through a replace directive.
FROM golang:1.26 AS builder
WORKDIR /src
# Manifests first so the dependency layer caches independently of source edits.
COPY shared/go.mod shared/go.sum ./shared/
COPY vantagectl/go.mod vantagectl/go.sum ./vantagectl/
RUN cd vantagectl && go mod download
COPY shared/ ./shared/
COPY vantagectl/ ./vantagectl/
ARG VERSION=dev
RUN cd vantagectl && CGO_ENABLED=0 GOOS=linux go build \
-ldflags="-s -w -X main.Version=${VERSION}" -o /vantagectl .
# Staged so the scratch image below can have a /tmp. It cannot mkdir one
# itself — scratch has no shell.
RUN mkdir -p /staging/tmp && chmod 1777 /staging/tmp
# Runtime stage
FROM scratch
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
# restore extracts an archive here before verifying its checksums, and backup
# stages nothing but still inherits os.MkdirTemp's requirements. Without this
# every restore stops at "temp dir: stat /tmp: no such file or directory".
COPY --from=builder /staging/tmp /tmp
COPY --from=builder /vantagectl /vantagectl
ENTRYPOINT ["/vantagectl"]
+26
View File
@@ -0,0 +1,26 @@
module gitea.hostxtra.co.uk/mrhid6/vantage/vantagectl
go 1.26
replace gitea.hostxtra.co.uk/mrhid6/vantage/shared => ../shared
require (
gitea.hostxtra.co.uk/mrhid6/vantage/shared v0.0.0-00010101000000-000000000000
github.com/spf13/cobra v1.10.2
go.mongodb.org/mongo-driver/v2 v2.8.0
golang.org/x/term v0.45.0
)
require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/klauspost/compress v1.17.6 // indirect
github.com/spf13/pflag v1.0.9 // indirect
github.com/xdg-go/pbkdf2 v1.0.0 // indirect
github.com/xdg-go/scram v1.2.0 // indirect
github.com/xdg-go/stringprep v1.0.4 // indirect
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
golang.org/x/crypto v0.54.0 // indirect
golang.org/x/sync v0.22.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/text v0.40.0 // indirect
)
+60
View File
@@ -0,0 +1,60 @@
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/klauspost/compress v1.17.6 h1:60eq2E/jlfwQXtvZEeBUYADs+BwKBWURIY+Gj2eRGjI=
github.com/klauspost/compress v1.17.6/go.mod h1:/dCuZOvVtNoHsyb+cuJD3itjs3NbnF6KH9zAO4BDxPM=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/xdg-go/pbkdf2 v1.0.0 h1:Su7DPu48wXMwC3bs7MCNG+z4FhcyEuz5dlvchbq0B0c=
github.com/xdg-go/pbkdf2 v1.0.0/go.mod h1:jrpuAogTd400dnrH08LKmI/xc1MbPOebTwRqcT5RDeI=
github.com/xdg-go/scram v1.2.0 h1:bYKF2AEwG5rqd1BumT4gAnvwU/M9nBp2pTSxeZw7Wvs=
github.com/xdg-go/scram v1.2.0/go.mod h1:3dlrS0iBaWKYVt2ZfA4cj48umJZ+cAEbR6/SjLA88I8=
github.com/xdg-go/stringprep v1.0.4 h1:XLI/Ng3O1Atzq0oBs3TWm+5ZVgkq2aqdlvP9JtoZ6c8=
github.com/xdg-go/stringprep v1.0.4/go.mod h1:mPGuuIYwz7CmR2bT9j4GbQqutWS1zV24gijq1dTyGkM=
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 h1:ilQV1hzziu+LLM3zUTJ0trRztfwgjqKnBWNtSRkbmwM=
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78/go.mod h1:aL8wCCfTfSfmXjznFBSZNN13rSJjlIOI1fUNAtF7rmI=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
go.mongodb.org/mongo-driver/v2 v2.8.0 h1:CxWDGQYY8QQwNjAl/aq2sfWakdnWZynnqJ9F4DhHbP8=
go.mongodb.org/mongo-driver/v2 v2.8.0/go.mod h1:yOI9kBsufol30iFsl1slpdq1I0eHPzybRWdyYUs8K/0=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.45.0 h1:NwWyBmoJCbfTHpxrWoZ9C6/VxOf7ic219I8xZZFdrf0=
golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+156
View File
@@ -0,0 +1,156 @@
package cmd
import (
"context"
"fmt"
"io"
"os"
"path/filepath"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/backup"
"github.com/spf13/cobra"
)
func newBackupCmd() *cobra.Command {
var (
out string
exclude []string
allowNoKey bool
)
c := &cobra.Command{
Use: "backup",
Short: "Write an archive of the database",
Long: "backup writes every collection in the database to a gzipped tar\n" +
"archive, along with a fingerprint of KEY_ENCRYPTION_KEY.\n\n" +
"The key itself is never written. The fingerprint is what lets a later\n" +
"restore refuse rather than produce a database whose secrets nobody\n" +
"can read.\n\n" +
"Pass --out - to stream to stdout, which is how this composes with\n" +
"restic, age, or aws s3 cp -.",
Args: cobra.NoArgs,
RunE: func(c *cobra.Command, _ []string) error {
ctx := c.Context()
g, err := resolveGlobals(c)
if err != nil {
return err
}
client, err := connect(ctx, g)
if err != nil {
return err
}
defer client.Disconnect(context.Background())
w, dest, name, err := backupDestination(out, g.Database)
if err != nil {
return err
}
committed := false
defer func() {
if !committed {
dest.Cleanup()
}
}()
m, err := backup.Dump(ctx, backup.DumpOptions{
Client: client,
Database: g.Database,
Exclude: exclude,
KeyHex: g.KeyHex,
AllowNoKey: allowNoKey,
VantageVersion: c.Root().Version,
Out: w,
})
if err != nil {
return err
}
if err := dest.Commit(); err != nil {
return err
}
committed = true
// Progress goes to stderr so --out - stays a clean pipe.
var docs int64
for _, coll := range m.Collections {
docs += coll.Documents
}
fmt.Fprintf(c.ErrOrStderr(), "wrote %s: %d collections, %d documents\n",
name, len(m.Collections), docs)
if m.KeyFingerprint == nil {
fmt.Fprintln(c.ErrOrStderr(),
"warning: no key recorded; nothing in this archive proves its "+
"ciphertext can ever be read")
}
return nil
},
}
c.Flags().StringVar(&out, "out", ".", "directory to write the archive into, or - for stdout")
c.Flags().StringSliceVar(&exclude, "exclude", nil,
"collections to leave out, comma separated (recorded in the manifest)")
c.Flags().BoolVar(&allowNoKey, "allow-no-key", false,
"back up without KEY_ENCRYPTION_KEY set; only for a deployment storing no encrypted data")
return c
}
// backupDestination resolves --out to a writer, a closer and a name to print.
func backupDestination(out, database string) (io.Writer, destination, string, error) {
if out == "-" {
return os.Stdout, stdoutDestination{}, "stdout", nil
}
name := archiveName(database, time.Now().UTC())
path := filepath.Join(out, name)
// Written under a temporary name and renamed on success, the same
// discipline the agent uses for authorized_keys: a failed backup must not
// leave a partial file named exactly like a good archive.
tmp := path + ".partial"
f, err := os.OpenFile(tmp, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
if err != nil {
return nil, nil, "", fmt.Errorf("create %s: %w", tmp, err)
}
d := &fileDestination{f: f, tmp: tmp, final: path}
return f, d, path, nil
}
// fileDestination finishes a file-backed backup. Commit renames the temporary
// file into place; Cleanup removes it if Commit was never called.
type fileDestination struct {
f *os.File
tmp string
final string
}
func (d *fileDestination) Commit() error {
if err := d.f.Close(); err != nil {
return fmt.Errorf("close %s: %w", d.tmp, err)
}
if err := os.Rename(d.tmp, d.final); err != nil {
return fmt.Errorf("rename %s: %w", d.tmp, err)
}
return nil
}
func (d *fileDestination) Cleanup() {
d.f.Close()
os.Remove(d.tmp)
}
// destination is how the two --out modes finish. stdout commits by doing
// nothing; there is no partial file to clean up either.
type destination interface {
Commit() error
Cleanup()
}
type stdoutDestination struct{}
func (stdoutDestination) Commit() error { return nil }
func (stdoutDestination) Cleanup() {}
// archiveName is sortable and carries no colon, because an operator will copy
// these onto a Windows share sooner or later and a colon is not a legal
// filename character there.
func archiveName(database string, at time.Time) string {
return fmt.Sprintf("vantage-backup-%s-%s.tar.gz", database, at.Format("20060102T150405Z"))
}
+78
View File
@@ -0,0 +1,78 @@
package cmd
import (
"fmt"
"io"
"strings"
"text/tabwriter"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/backup"
"github.com/spf13/cobra"
)
func newInspectCmd() *cobra.Command {
return &cobra.Command{
Use: "inspect ARCHIVE",
Short: "Print an archive's manifest",
Long: "inspect reads an archive and prints what it holds. It contacts no\n" +
"database, so it is safe to run against an archive of unknown origin\n" +
"and is the fastest way to find out whether one is worth anything.",
Args: cobra.ExactArgs(1),
RunE: func(c *cobra.Command, args []string) error {
archive, err := backup.Open(args[0])
if err != nil {
return err
}
defer archive.Close()
renderManifest(c.OutOrStdout(), archive.Manifest())
return nil
},
}
}
// renderManifest prints a manifest for a human.
func renderManifest(w io.Writer, m backup.Manifest) {
fmt.Fprintf(w, "Created %s\n", m.CreatedAt.UTC().Format("2006-01-02 15:04:05 MST"))
fmt.Fprintf(w, "Database %s\n", m.MongoDB)
fmt.Fprintf(w, "MongoDB %s\n", m.MongoServerVersion)
fmt.Fprintf(w, "Written by vantagectl %s on %s\n", m.VantageVersion, m.Hostname)
fmt.Fprintf(w, "Format version %d\n", m.FormatVersion)
if m.KeyFingerprint == nil {
fmt.Fprintf(w, "Key none recorded — this archive cannot be checked "+
"against any KEY_ENCRYPTION_KEY\n")
} else {
fmt.Fprintf(w, "Key %s\n", *m.KeyFingerprint)
}
if len(m.Excluded) > 0 {
fmt.Fprintf(w, "Excluded %s\n", strings.Join(m.Excluded, ", "))
}
var docs, bytes int64
for _, c := range m.Collections {
docs += c.Documents
bytes += c.Bytes
}
fmt.Fprintf(w, "\n%d collections, %d documents, %s\n\n",
len(m.Collections), docs, humanBytes(bytes))
tw := tabwriter.NewWriter(w, 0, 0, 2, ' ', 0)
fmt.Fprintln(tw, "COLLECTION\tDOCUMENTS\tSIZE")
for _, c := range m.Collections {
fmt.Fprintf(tw, "%s\t%d\t%s\n", c.Name, c.Documents, humanBytes(c.Bytes))
}
tw.Flush()
}
func humanBytes(n int64) string {
const unit = 1024
if n < unit {
return fmt.Sprintf("%d B", n)
}
div, exp := int64(unit), 0
for v := n / unit; v >= unit; v /= unit {
div *= unit
exp++
}
return fmt.Sprintf("%.1f %cB", float64(n)/float64(div), "KMGTP"[exp])
}
+116
View File
@@ -0,0 +1,116 @@
package cmd
import (
"bytes"
"os"
"strings"
"testing"
"time"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/backup"
)
func TestArchiveNameIsSortableAndNamesTheDatabase(t *testing.T) {
at := time.Date(2026, 9, 7, 14, 30, 5, 0, time.UTC)
got := archiveName("vantage", at)
if !strings.HasPrefix(got, "vantage-backup-vantage-") {
t.Fatalf("name %q does not name the database", got)
}
if !strings.HasSuffix(got, ".tar.gz") {
t.Fatalf("name %q has the wrong extension", got)
}
if strings.ContainsAny(got, ":") {
t.Fatalf("name %q contains a colon, which Windows will not accept", got)
}
if !strings.Contains(got, "20260907") {
t.Fatalf("name %q does not carry a sortable date", got)
}
}
func TestRenderManifestShowsWhatMatters(t *testing.T) {
fp := "ab12"
m := backup.Manifest{
FormatVersion: backup.FormatVersion,
CreatedAt: time.Date(2026, 9, 7, 14, 0, 0, 0, time.UTC),
VantageVersion: "1.4.0",
Hostname: "ops-box",
MongoDB: "vantage",
MongoServerVersion: "7.0.5",
KeyFingerprint: &fp,
Collections: []backup.CollectionEntry{
{Name: "servers", Documents: 12, Bytes: 4096},
{Name: "keys", Documents: 3, Bytes: 900},
},
Excluded: []string{"audit_logs"},
}
var buf bytes.Buffer
renderManifest(&buf, m)
out := buf.String()
for _, want := range []string{
"vantage", "1.4.0", "ops-box", "7.0.5", "ab12",
"servers", "12", "keys", "audit_logs", "2026-09-07",
} {
if !strings.Contains(out, want) {
t.Fatalf("inspect output missing %q:\n%s", want, out)
}
}
}
func TestRenderManifestFlagsAMissingFingerprint(t *testing.T) {
var buf bytes.Buffer
renderManifest(&buf, backup.Manifest{FormatVersion: backup.FormatVersion})
out := buf.String()
if !strings.Contains(out, "none recorded") {
t.Fatalf("a null fingerprint must be called out, got:\n%s", out)
}
if !strings.Contains(out, "cannot be checked") {
t.Fatalf("a null fingerprint must explain the consequence, got:\n%s", out)
}
}
// TestBackupDestinationDoesNotLeaveAPartialArchive covers the failure path: a
// backup that errors must not leave a file named exactly like a good archive.
func TestBackupDestinationDoesNotLeaveAPartialArchive(t *testing.T) {
dir := t.TempDir()
w, dest, path, err := backupDestination(dir, "vantage")
if err != nil {
t.Fatalf("backupDestination: %v", err)
}
if _, err := w.Write([]byte("half an archive")); err != nil {
t.Fatalf("write: %v", err)
}
dest.Cleanup()
entries, err := os.ReadDir(dir)
if err != nil {
t.Fatalf("readdir: %v", err)
}
if len(entries) != 0 {
t.Fatalf("a failed backup left %v behind", entries)
}
if _, err := os.Stat(path); !os.IsNotExist(err) {
t.Fatalf("%s exists after a failed backup", path)
}
}
func TestBackupDestinationRenamesOnCommit(t *testing.T) {
dir := t.TempDir()
w, dest, path, err := backupDestination(dir, "vantage")
if err != nil {
t.Fatalf("backupDestination: %v", err)
}
if _, err := w.Write([]byte("a whole archive")); err != nil {
t.Fatalf("write: %v", err)
}
if err := dest.Commit(); err != nil {
t.Fatalf("Commit: %v", err)
}
if _, err := os.Stat(path); err != nil {
t.Fatalf("committed archive is not at %s: %v", path, err)
}
if _, err := os.Stat(path + ".partial"); !os.IsNotExist(err) {
t.Fatal("the temporary file was left behind")
}
}
+134
View File
@@ -0,0 +1,134 @@
package cmd
import (
"bufio"
"context"
"errors"
"fmt"
"io"
"os"
"strings"
"text/tabwriter"
"gitea.hostxtra.co.uk/mrhid6/vantage/shared/backup"
"github.com/spf13/cobra"
"golang.org/x/term"
)
// ErrNotConfirmed is returned when a destructive restore was not confirmed.
var ErrNotConfirmed = errors.New("restore not confirmed")
func newRestoreCmd() *cobra.Command {
var (
force bool
confirmDB string
ignoreKeyErr bool
)
c := &cobra.Command{
Use: "restore ARCHIVE",
Short: "Load an archive into a database",
Long: "restore loads an archive into a MongoDB database.\n\n" +
"The target is expected to be empty. A database that already holds data\n" +
"is refused unless --force is given, which drops each collection in the\n" +
"archive before loading it. There are no merge semantics: merging two\n" +
"control planes reconciles nothing, and upserting would resurrect\n" +
"revoked keys and deleted users.\n\n" +
"Restore does not touch Redis. Sessions are all it holds, so everyone\n" +
"signs in again.",
Args: cobra.ExactArgs(1),
RunE: func(c *cobra.Command, args []string) error {
ctx := c.Context()
g, err := resolveGlobals(c)
if err != nil {
return err
}
archive, err := backup.Open(args[0])
if err != nil {
return err
}
defer archive.Close()
if force {
isTTY := term.IsTerminal(int(os.Stdin.Fd()))
if err := confirmDestruction(c.InOrStdin(), c.OutOrStdout(), isTTY,
confirmDB, g.Database); err != nil {
return err
}
}
client, err := connect(ctx, g)
if err != nil {
return err
}
defer client.Disconnect(context.Background())
res, err := backup.Restore(ctx, backup.RestoreOptions{
Client: client,
Database: g.Database,
Archive: archive,
Force: force,
KeyHex: g.KeyHex,
IgnoreKeyMismatch: ignoreKeyErr,
Warn: func(s string) {
fmt.Fprintln(c.ErrOrStderr(), "warning:", s)
},
})
if err != nil {
return err
}
tw := tabwriter.NewWriter(c.OutOrStdout(), 0, 0, 2, ' ', 0)
fmt.Fprintln(tw, "COLLECTION\tDOCUMENTS\tINDEXES")
for _, coll := range res.Collections {
fmt.Fprintf(tw, "%s\t%d\t%d\n", coll.Name, coll.Documents, coll.Indexes)
}
tw.Flush()
fmt.Fprintf(c.OutOrStdout(), "\nrestored %d collections into %s\n",
len(res.Collections), g.Database)
return nil
},
}
c.Flags().BoolVar(&force, "force", false,
"drop each collection in the archive before loading it")
c.Flags().StringVar(&confirmDB, "confirm-db", "",
"name of the database being overwritten; required with --force when there is no terminal")
c.Flags().BoolVar(&ignoreKeyErr, "ignore-key-mismatch", false,
"restore even though KEY_ENCRYPTION_KEY does not match the archive")
return c
}
// confirmDestruction gates a --force restore.
//
// On a terminal the operator types the database name. Without one — a
// Kubernetes Job, a CI step, a cron entry — the same assurance comes from
// --confirm-db, whose value must equal the target. Naming the database in the
// argument means a copy-pasted command carries its intended target with it and
// cannot destroy a different one.
func confirmDestruction(in io.Reader, out io.Writer, isTTY bool, confirmDB, database string) error {
if confirmDB != "" {
if confirmDB != database {
return fmt.Errorf("%w: --confirm-db says %q but the target is %q",
ErrNotConfirmed, confirmDB, database)
}
return nil
}
if !isTTY {
return fmt.Errorf("%w: --force with no terminal needs --confirm-db %s",
ErrNotConfirmed, database)
}
fmt.Fprintf(out, "This drops every collection in the archive from %q and reloads it.\n", database)
fmt.Fprintf(out, "Type the database name to continue: ")
line, err := bufio.NewReader(in).ReadString('\n')
if err != nil && err != io.EOF {
return fmt.Errorf("%w: %v", ErrNotConfirmed, err)
}
if strings.TrimSpace(line) != database {
return fmt.Errorf("%w: that is not %q", ErrNotConfirmed, database)
}
return nil
}

Some files were not shown because too many files have changed in this diff Show More