refactor: stand up vantage-ctl as its own repository
vantagectl/ becomes the repository root, so the module is gitea.hostxtra.co.uk/vantage/vantage-ctl and the image is vantage/vantage-ctl. The command keeps the name vantagectl. Release tags become a bare v*: the vantagectl/ prefix existed to namespace one component inside a shared repository, and unlike the agent's agent/v* nothing reads this one programmatically.
This commit is contained in:
@@ -0,0 +1,136 @@
|
||||
name: Release
|
||||
|
||||
on:
|
||||
push:
|
||||
# A bare v* now this is its own repository. The old prefixed
|
||||
# vantagectl/v* tag existed to namespace one component's releases
|
||||
# inside a shared repository, and unlike the agent's agent/v* — which
|
||||
# the control plane greps release tag names for — nothing reads this
|
||||
# one programmatically. Releases before the move keep their prefixed
|
||||
# tags in the monorepo's release list.
|
||||
tags:
|
||||
- "v*"
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-docker
|
||||
container: node:26
|
||||
env:
|
||||
GOPRIVATE: gitea.hostxtra.co.uk/*
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
# vantage-shared is a private module, so the Go builds below cannot
|
||||
# resolve it without a credential.
|
||||
- name: Write the module fetch credential
|
||||
run: |
|
||||
umask 077
|
||||
printf 'machine gitea.hostxtra.co.uk\nlogin %s\npassword %s\n' \
|
||||
"${{ secrets.REGISTRY_USER }}" "${{ secrets.RELEASE_TOKEN }}" \
|
||||
> "$HOME/.netrc"
|
||||
|
||||
- name: Set up Go
|
||||
uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: "1.26"
|
||||
cache: true
|
||||
cache-dependency-path: go.sum
|
||||
|
||||
- name: Extract version
|
||||
id: version
|
||||
run: echo "VERSION=${GITHUB_REF_NAME}" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Test
|
||||
run: go test ./...
|
||||
|
||||
- name: Build
|
||||
env:
|
||||
VERSION: ${{ steps.version.outputs.VERSION }}
|
||||
run: |
|
||||
mkdir -p dist
|
||||
for target in linux/amd64 linux/arm64 darwin/arm64 windows/amd64; do
|
||||
goos="${target%/*}"
|
||||
goarch="${target#*/}"
|
||||
out="dist/vantagectl-${goos}-${goarch}"
|
||||
if [ "$goos" = "windows" ]; then out="${out}.exe"; fi
|
||||
CGO_ENABLED=0 GOOS="$goos" GOARCH="$goarch" go build \
|
||||
-ldflags="-s -w -X main.Version=${VERSION}" \
|
||||
-o "$out" .
|
||||
done
|
||||
|
||||
- name: Checksums
|
||||
working-directory: dist
|
||||
run: sha256sum vantagectl-* > checksums.txt
|
||||
|
||||
- name: Create release
|
||||
uses: https://gitea.com/actions/gitea-release-action@v1
|
||||
with:
|
||||
token: ${{ secrets.RELEASE_TOKEN }}
|
||||
files: |
|
||||
dist/vantagectl-linux-amd64
|
||||
dist/vantagectl-linux-arm64
|
||||
dist/vantagectl-darwin-arm64
|
||||
dist/vantagectl-windows-amd64.exe
|
||||
dist/checksums.txt
|
||||
|
||||
# The image is built here rather than in server-deploy.yml on every push to
|
||||
# main, because vantagectl is a released tool rather than a running service.
|
||||
# An operator restoring a database should be able to name the version they
|
||||
# ran; ":latest, rebuilt whenever main moved" cannot be named after the
|
||||
# fact. It is a separate job from the binaries because it needs a
|
||||
# docker-capable runner rather than a Go one, and it does not need the
|
||||
# binaries — the image builds from source in its own stage.
|
||||
image:
|
||||
runs-on: ubuntu-docker
|
||||
container: docker:dind
|
||||
steps:
|
||||
- name: Setup
|
||||
run: apk add --update nodejs npm git
|
||||
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Extract version
|
||||
id: version
|
||||
run: |
|
||||
# v0.1.0 for the binary stamp, 0.1.0 for the image tag: a
|
||||
# leading v is conventional on a git tag and unconventional on
|
||||
# a container tag.
|
||||
VERSION="${GITHUB_REF_NAME}"
|
||||
echo "VERSION=${VERSION}" >> $GITHUB_OUTPUT
|
||||
echo "IMAGE_TAG=${VERSION#v}" >> $GITHUB_OUTPUT
|
||||
|
||||
# The image build resolves the private vantage-shared module, so
|
||||
# it needs a credential of its own — this job does not share the
|
||||
# build job's filesystem.
|
||||
- name: Write the module fetch credential
|
||||
run: |
|
||||
umask 077
|
||||
printf 'machine gitea.hostxtra.co.uk\nlogin %s\npassword %s\n' \
|
||||
"${{ secrets.REGISTRY_USER }}" "${{ secrets.RELEASE_TOKEN }}" \
|
||||
> "$HOME/.netrc"
|
||||
|
||||
- name: Log in to registry
|
||||
run: |
|
||||
echo "${{ secrets.RELEASE_TOKEN }}" | \
|
||||
docker login ${{ vars.DOCKER_HOST }} \
|
||||
-u "${{ secrets.REGISTRY_USER }}" --password-stdin
|
||||
|
||||
- name: Build and push image
|
||||
env:
|
||||
VERSION: ${{ steps.version.outputs.VERSION }}
|
||||
IMAGE_TAG: ${{ steps.version.outputs.IMAGE_TAG }}
|
||||
run: |
|
||||
REPO="${{ vars.DOCKER_HOST }}/vantage/vantage-ctl"
|
||||
# VERSION is passed through so `vantagectl --version` inside
|
||||
# the image reports the tag it was built from rather than
|
||||
# "dev". The context is the repository root.
|
||||
docker build \
|
||||
--build-arg VERSION="${VERSION}" \
|
||||
--secret id=netrc,src="$HOME/.netrc" \
|
||||
-t "${REPO}:${IMAGE_TAG}" \
|
||||
-t "${REPO}:latest" \
|
||||
-f Dockerfile .
|
||||
docker push "${REPO}:${IMAGE_TAG}"
|
||||
docker push "${REPO}:latest"
|
||||
@@ -0,0 +1,4 @@
|
||||
dist
|
||||
.env
|
||||
*.tar.gz
|
||||
*.tar.gz.partial
|
||||
@@ -0,0 +1,132 @@
|
||||
# vantagectl (`vantage-ctl`)
|
||||
|
||||
The backup and restore CLI for a Vantage control plane. Extracted from the
|
||||
`vantage` monorepo with its history; the command is the repository root, so the
|
||||
module is `gitea.hostxtra.co.uk/vantage/vantage-ctl`.
|
||||
|
||||
**The command is still `vantagectl`.** Only the repository is `vantage-ctl` —
|
||||
the binary, the cobra `Use:` string and every runbook keep the name an operator
|
||||
actually types.
|
||||
|
||||
```
|
||||
vantage-ctl/
|
||||
├── main.go
|
||||
├── internal/cmd/ # argument parsing and operator-facing output only
|
||||
├── Dockerfile # scratch runtime, with an explicitly copied /tmp
|
||||
└── .gitea/workflows/release.yml
|
||||
```
|
||||
|
||||
## Why this is not a subcommand of the server
|
||||
|
||||
`server` imports the whole control-plane dependency graph, and `spf13/cobra` has
|
||||
no business in a process that also terminates gRPC streams and serves the REST
|
||||
API. More to the point: **`vantagectl` has to run when the control plane does
|
||||
not.** A backup or restore against a database with no server container alive is
|
||||
the normal case, not the exception, so it cannot be a mode of the binary whose
|
||||
failure is the reason you reached for it.
|
||||
|
||||
## Where the logic actually lives
|
||||
|
||||
Almost none of it is here. Dump, restore, verify, manifest and fingerprint are
|
||||
`shared/backup` in **`vantage-shared`**; `internal/cmd` holds argument parsing
|
||||
and the words an operator reads. That split is what would let the control plane
|
||||
import `shared/backup` later — a scheduled in-process backup, say — without a
|
||||
second implementation to keep in sync. `shared/cryptobox` is the same move one
|
||||
layer down: the single AES-256-GCM implementation, which the server delegates to
|
||||
rather than keeping its own copy that `backup` would have had to duplicate to
|
||||
decrypt a probe value during `verify`.
|
||||
|
||||
So **a fix to backup behaviour is usually a `vantage-shared` release plus a pin
|
||||
bump here**, and it is not live until this repository cuts a tag. That is
|
||||
deliberate — an operator restoring a database should be running a version they
|
||||
can name — but it does mean the fix is not in anybody's hands the moment it
|
||||
merges.
|
||||
|
||||
## The refusals, and why each exists
|
||||
|
||||
These are the load-bearing parts. They are refusals rather than warnings because
|
||||
each one guards something with no way back.
|
||||
|
||||
- **`backup` will not run without `KEY_ENCRYPTION_KEY`** unless `--allow-no-key`
|
||||
is passed. The archive stores a SHA-256 **fingerprint** of the key, never the
|
||||
key. An archive with no fingerprint cannot tell a later restore that the wrong
|
||||
key is in hand — it can only find that out when the data comes back as noise.
|
||||
- **`restore` refuses a non-empty target database, and there are no merge
|
||||
semantics.** No code path upserts an archive's documents over existing ones:
|
||||
merging two control planes reconciles nothing about which SSH keys are still
|
||||
valid or which users still exist, and an upsert would resurrect a revoked key
|
||||
or a deleted member from the older side.
|
||||
- **`--force` needs a second assurance**: `--confirm-db NAME` matching the
|
||||
target exactly, which works everywhere, or — on a terminal only, and only when
|
||||
`--confirm-db` was not given — the database name typed back at a prompt.
|
||||
`--confirm-db` is accepted on a terminal too, and is the stronger of the two:
|
||||
naming the target in the command itself means a copied command carries its
|
||||
intended target and cannot destroy a different one by accident. Without a
|
||||
terminal and without `--confirm-db`, `--force` is refused.
|
||||
- **`--force` drops only what the archive names.** Collections in the target
|
||||
that the archive does not carry are left alone and **named in a warning** — an
|
||||
archive taken with `--exclude workflow_log_lines`, restored over a live
|
||||
database, leaves the old lines joined to restored runs and the operator must
|
||||
be told. Dropping them instead would delete data nobody asked to delete.
|
||||
|
||||
## Three decisions that look like details and are not
|
||||
|
||||
- **Collections are enumerated live**, not read from a static list — the
|
||||
opposite of what instance-deletion purge does in the control plane. Purge must
|
||||
never miss a tenant-scoped collection, so it keeps a hand-maintained registry;
|
||||
a backup must never miss **any** collection, including the ones carrying no
|
||||
`instance_id` at all (`migrations`, `vulndb_meta`).
|
||||
- **Index specifications are replayed verbatim, never reconstructed.**
|
||||
`dumpIndexes` stores each spec as extended JSON over the raw BSON the server
|
||||
reported, and `replayIndexes` hands it back through `createIndexes` with only
|
||||
`v` and `ns` stripped and `_id_` skipped. Rebuilding a `mongo.IndexModel` from
|
||||
hand-picked options dropped `partialFilterExpression`, which the control plane
|
||||
relies on, so a partial unique index came back as a full one, failed on
|
||||
duplicate keys, and aborted the restore mid-write. Reconstructing the key
|
||||
document from JSON also loses compound key order, which is significant.
|
||||
- **A file-backed `backup` writes `<name>.tar.gz.partial` and renames on
|
||||
success**, the same discipline the agent uses for `authorized_keys`. A failed
|
||||
dump must not leave a partial file named exactly like a good archive.
|
||||
`--out -` is untouched: a broken pipe has no file to mislead anyone.
|
||||
|
||||
## The mirror that fails silently
|
||||
|
||||
**`backup.ciphertextFields` lives in `vantage-shared` and mirrors
|
||||
`server/internal/models` in the `vantage` repository by hand** — `shared/` cannot
|
||||
import it. The map naming each collection's `*_enc` fields (`keys`, `secrets`,
|
||||
`auth_providers`, `console_sessions`) must change in the same commit as any of
|
||||
those bson tags.
|
||||
|
||||
Wrong field names are **silent**: `verify`'s live probe simply finds no
|
||||
ciphertext and reports "this database stores no ciphertext yet", so the one gate
|
||||
that catches what a key fingerprint cannot becomes a no-op. `settings` is
|
||||
deliberately in neither that map nor `CiphertextCollections()` — its ESO read
|
||||
token is a SHA-256 hash, not ciphertext.
|
||||
|
||||
## Building and releasing
|
||||
|
||||
`GOPRIVATE=gitea.hostxtra.co.uk/*` plus a credential, since `vantage-shared` is
|
||||
private. CI writes a netrc from `REGISTRY_USER` + `RELEASE_TOKEN` in **both**
|
||||
jobs — they do not share a filesystem — and the image build takes it as a
|
||||
BuildKit secret rather than a build arg, which would survive in the builder
|
||||
layer's history. **`RELEASE_TOKEN` needs read access to the `vantage` org.**
|
||||
|
||||
Tags are a bare **`v*`** now this is its own repository. The old
|
||||
`vantagectl/v*` prefix namespaced one component's releases inside a shared
|
||||
repository; unlike the agent's `agent/v*` — which the control plane greps
|
||||
release tag names for — nothing reads this one programmatically. Releases from
|
||||
before the move keep their prefixed tags in the monorepo's release list.
|
||||
|
||||
```bash
|
||||
git tag v0.2.0 && git push origin v0.2.0
|
||||
```
|
||||
|
||||
Builds `linux/amd64`, `linux/arm64`, `darwin/arm64` and `windows/amd64`, writes
|
||||
`checksums.txt`, creates the Gitea release, and pushes
|
||||
`vantage/vantage-ctl:<version>` and `:latest`.
|
||||
|
||||
**The runtime stage is `scratch`, and it carries an explicitly copied `/tmp`.**
|
||||
`restore` extracts an archive to a temporary directory before verifying its
|
||||
checksums, and a scratch image has none for `os.MkdirTemp` to find. Without it
|
||||
every restore fails — the same omission costs the control plane only its
|
||||
vulnerability scanning, but here it is the whole tool.
|
||||
+8
-5
@@ -1,10 +1,13 @@
|
||||
# Build stage
|
||||
#
|
||||
# Context is vantagectl/ itself. It used to be the repository root, so that
|
||||
# shared/ could be copied in beside it; shared is now the private module
|
||||
# gitea.hostxtra.co.uk/vantage/vantage-shared, fetched like any other
|
||||
# dependency. The credential for it arrives as a BuildKit secret rather than a
|
||||
# build arg, which would be baked into this stage's layer history.
|
||||
# Context is the repository root.
|
||||
#
|
||||
# The command stays named vantagectl even though the repository is vantage-ctl:
|
||||
# it is what an operator types, and it is in every runbook. Only the paths moved.
|
||||
#
|
||||
# vantage-shared is a private module, so both steps below need a credential. It
|
||||
# arrives as a BuildKit secret rather than a build arg, which would be baked
|
||||
# into this stage's layer history.
|
||||
FROM golang:1.26 AS builder
|
||||
|
||||
WORKDIR /src
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
module gitea.hostxtra.co.uk/mrhid6/vantage/vantagectl
|
||||
module gitea.hostxtra.co.uk/vantage/vantage-ctl
|
||||
|
||||
go 1.26
|
||||
|
||||
|
||||
Reference in New Issue
Block a user