137 lines
7.3 KiB
Markdown
137 lines
7.3 KiB
Markdown
# vantagectl (`vantage-ctl`)
|
|
|
|
The backup and restore CLI for a Vantage control plane. Extracted from the
|
|
`vantage` monorepo with its history; the command is the repository root, so the
|
|
module is `gitea.hostxtra.co.uk/vantage/vantage-ctl`.
|
|
|
|
**The command is still `vantagectl`.** Only the repository is `vantage-ctl` -
|
|
the binary, the cobra `Use:` string and every runbook keep the name an operator
|
|
actually types.
|
|
|
|
```
|
|
vantage-ctl/
|
|
├── main.go
|
|
├── internal/cmd/ # argument parsing and operator-facing output only
|
|
├── Dockerfile # scratch runtime, with an explicitly copied /tmp
|
|
└── .gitea/workflows/release.yml
|
|
```
|
|
|
|
## Why this is not a subcommand of the server
|
|
|
|
`server` imports the whole control-plane dependency graph, and `spf13/cobra` has
|
|
no business in a process that also terminates gRPC streams and serves the REST
|
|
API. More to the point: **`vantagectl` has to run when the control plane does
|
|
not.** A backup or restore against a database with no server container alive is
|
|
the normal case, not the exception, so it cannot be a mode of the binary whose
|
|
failure is the reason you reached for it.
|
|
|
|
## Where the logic actually lives
|
|
|
|
Almost none of it is here. Dump, restore, verify, manifest and fingerprint are
|
|
`shared/backup` in **`vantage-shared`**; `internal/cmd` holds argument parsing
|
|
and the words an operator reads. That split is what would let the control plane
|
|
import `shared/backup` later - a scheduled in-process backup, say - without a
|
|
second implementation to keep in sync. `shared/cryptobox` is the same move one
|
|
layer down: the single AES-256-GCM implementation, which the server delegates to
|
|
rather than keeping its own copy that `backup` would have had to duplicate to
|
|
decrypt a probe value during `verify`.
|
|
|
|
So **a fix to backup behaviour is usually a `vantage-shared` release plus a pin
|
|
bump here**, and it is not live until this repository cuts a tag. That is
|
|
deliberate - an operator restoring a database should be running a version they
|
|
can name - but it does mean the fix is not in anybody's hands the moment it
|
|
merges.
|
|
|
|
## The refusals, and why each exists
|
|
|
|
These are the load-bearing parts. They are refusals rather than warnings because
|
|
each one guards something with no way back.
|
|
|
|
- **`backup` will not run without `KEY_ENCRYPTION_KEY`** unless `--allow-no-key`
|
|
is passed. The archive stores a SHA-256 **fingerprint** of the key, never the
|
|
key. An archive with no fingerprint cannot tell a later restore that the wrong
|
|
key is in hand - it can only find that out when the data comes back as noise.
|
|
- **`restore` refuses a non-empty target database, and there are no merge
|
|
semantics.** No code path upserts an archive's documents over existing ones:
|
|
merging two control planes reconciles nothing about which SSH keys are still
|
|
valid or which users still exist, and an upsert would resurrect a revoked key
|
|
or a deleted member from the older side.
|
|
- **`--force` needs a second assurance**: `--confirm-db NAME` matching the
|
|
target exactly, which works everywhere, or - on a terminal only, and only when
|
|
`--confirm-db` was not given - the database name typed back at a prompt.
|
|
`--confirm-db` is accepted on a terminal too, and is the stronger of the two:
|
|
naming the target in the command itself means a copied command carries its
|
|
intended target and cannot destroy a different one by accident. Without a
|
|
terminal and without `--confirm-db`, `--force` is refused.
|
|
- **`--force` drops only what the archive names.** Collections in the target
|
|
that the archive does not carry are left alone and **named in a warning** - an
|
|
archive taken with `--exclude workflow_log_lines`, restored over a live
|
|
database, leaves the old lines joined to restored runs and the operator must
|
|
be told. Dropping them instead would delete data nobody asked to delete.
|
|
|
|
## Three decisions that look like details and are not
|
|
|
|
- **Collections are enumerated live**, not read from a static list - the
|
|
opposite of what instance-deletion purge does in the control plane. Purge must
|
|
never miss a tenant-scoped collection, so it keeps a hand-maintained registry;
|
|
a backup must never miss **any** collection, including the ones carrying no
|
|
`instance_id` at all (`migrations`, `vulndb_meta`).
|
|
- **Index specifications are replayed verbatim, never reconstructed.**
|
|
`dumpIndexes` stores each spec as extended JSON over the raw BSON the server
|
|
reported, and `replayIndexes` hands it back through `createIndexes` with only
|
|
`v` and `ns` stripped and `_id_` skipped. Rebuilding a `mongo.IndexModel` from
|
|
hand-picked options dropped `partialFilterExpression`, which the control plane
|
|
relies on, so a partial unique index came back as a full one, failed on
|
|
duplicate keys, and aborted the restore mid-write. Reconstructing the key
|
|
document from JSON also loses compound key order, which is significant.
|
|
- **A file-backed `backup` writes `<name>.tar.gz.partial` and renames on
|
|
success**, the same discipline the agent uses for `authorized_keys`. A failed
|
|
dump must not leave a partial file named exactly like a good archive.
|
|
`--out -` is untouched: a broken pipe has no file to mislead anyone.
|
|
|
|
## The mirror that fails silently
|
|
|
|
**`backup.ciphertextFields` lives in `vantage-shared` and mirrors
|
|
`server/internal/models` in the `vantage` repository by hand** - `shared/` cannot
|
|
import it. The map naming each collection's `*_enc` fields (`keys`, `secrets`,
|
|
`auth_providers`, `console_sessions`) must change in the same commit as any of
|
|
those bson tags.
|
|
|
|
Wrong field names are **silent**: `verify`'s live probe simply finds no
|
|
ciphertext and reports "this database stores no ciphertext yet", so the one gate
|
|
that catches what a key fingerprint cannot becomes a no-op. `settings` is
|
|
deliberately in neither that map nor `CiphertextCollections()` - its ESO read
|
|
token is a SHA-256 hash, not ciphertext.
|
|
|
|
## Building and releasing
|
|
|
|
`GOPRIVATE=gitea.hostxtra.co.uk/*` plus a credential, since `vantage-shared` is
|
|
private. CI writes a netrc from `REGISTRY_USER` + `RELEASE_TOKEN` in **both**
|
|
jobs - they do not share a filesystem - and the image build takes it as a
|
|
BuildKit secret rather than a build arg, which would survive in the builder
|
|
layer's history. **`RELEASE_TOKEN` needs read access to the `vantage` org.**
|
|
|
|
Tags are a bare **`v*`** now this is its own repository. The old
|
|
`vantagectl/v*` prefix namespaced one component's releases inside a shared
|
|
repository; unlike the agent's `agent/v*` - which the control plane greps
|
|
release tag names for - nothing reads this one programmatically. Releases from
|
|
before the move keep their prefixed tags in the monorepo's release list.
|
|
|
|
```bash
|
|
git tag v0.2.0 && git push origin v0.2.0
|
|
```
|
|
|
|
Builds `linux/amd64`, `linux/arm64`, `darwin/arm64` and `windows/amd64`, writes
|
|
`checksums.txt`, creates the Gitea release, and pushes
|
|
`vantage/vantage-ctl:<version>` and `:latest`.
|
|
|
|
**The runtime stage is `scratch`, and it carries an explicitly copied `/tmp`.**
|
|
`restore` extracts an archive to a temporary directory before verifying its
|
|
checksums, and a scratch image has none for `os.MkdirTemp` to find. Without it
|
|
every restore fails - the same omission costs the control plane only its
|
|
vulnerability scanning, but here it is the whole tool.
|
|
|
|
## Writing style
|
|
|
|
Never use em dashes (the long dash character) anywhere: code, comments, UI copy, docs, commit messages. Use a plain hyphen ` - `, a comma, a colon, or split the sentence instead.
|