diff --git a/docs/superpowers/plans/2026-09-07-control-plane-backup-restore.md b/docs/superpowers/plans/2026-09-07-control-plane-backup-restore.md new file mode 100644 index 0000000..cbdfe49 --- /dev/null +++ b/docs/superpowers/plans/2026-09-07-control-plane-backup-restore.md @@ -0,0 +1,4432 @@ +# Control Plane Backup and Restore Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Ship `vantagectl`, a standalone CLI that backs up and restores a whole Vantage MongoDB database and records a fingerprint of `KEY_ENCRYPTION_KEY` so a restore can never silently produce a database whose secrets are unreadable. + +**Architecture:** Logic lives in `shared/backup` (MongoDB driver plus standard library, no CLI framework) so `server` can import it later. The cobra command tree lives in a new `vantagectl/` module so cobra never enters the module graph of `server`, `admin` or `sitesvc`. AES-GCM primitives move to a new `shared/cryptobox` and `server/internal/services/crypto.go` delegates to it, so the cipher has one implementation and two callers. + +**Tech Stack:** Go 1.26, `go.mongodb.org/mongo-driver/v2 v2.8.0`, `github.com/spf13/cobra`, standard library `archive/tar` and `compress/gzip`. + +**Spec:** `docs/superpowers/specs/2026-09-07-control-plane-backup-restore-design.md` + +## Global Constraints + +- Go 1.26. Every new module declares `go 1.26`. +- MongoDB driver is `go.mongodb.org/mongo-driver/v2 v2.8.0`, matching `shared/go.mod`. Do not introduce v1. +- `shared/backup` and `shared/cryptobox` must not import cobra, pflag, or anything from `server/`, `admin/`, `sitesvc/` or `agent/`. +- `vantagectl/` must not import anything from `server/`. +- Archive `format_version` is `1`. +- Restore batch size is 1000 documents, `ordered=false`. +- Ciphertext-bearing collections, used verbatim in restore warnings: `keys`, `secrets`, `auth_providers`, `console_sessions`, `settings`. +- Every new Go module gets `replace gitea.hostxtra.co.uk/mrhid6/vantage/shared => ../shared`, matching `server/go.mod:81`. +- Dockerfiles live at `/Dockerfile` and build from the repository root. +- Tests that need MongoDB read `MONGO_TEST_URI` and call `t.Skip` when it is unset. Do not add testcontainers. +- Commit after every task. Conventional commit prefixes (`feat:`, `test:`, `docs:`, `chore:`), no attribution lines. + +## File Structure + +| Path | Responsibility | +| --- | --- | +| `shared/cryptobox/cryptobox.go` | AES-256-GCM seal and open over a raw 32-byte key. No environment access. | +| `shared/cryptobox/cryptobox_test.go` | Round trip, wrong key, short ciphertext. | +| `shared/backup/fingerprint.go` | Parse a hex key, produce its SHA-256 fingerprint. Pure. | +| `shared/backup/manifest.go` | `Manifest` and `CollectionEntry` types, JSON shape, version check. | +| `shared/backup/archive.go` | Tar+gzip writer and reader, per-member SHA-256, extraction to a temp dir. | +| `shared/backup/dump.go` | `Dump`: enumerate collections, write raw BSON and index specs, build the manifest. | +| `shared/backup/restore.go` | `Restore`: verify, fingerprint policy, target inspection, insert, index replay. | +| `shared/backup/verify.go` | `Verify`: archive integrity, fingerprint comparison, optional live probe decrypt. | +| `shared/backup/*_test.go` | One test file per unit above. | +| `vantagectl/main.go` | `main`, version stamp, calls `cmd.Execute`. | +| `vantagectl/internal/cmd/root.go` | Cobra root, persistent flags, environment fallback, Mongo client construction. | +| `vantagectl/internal/cmd/backup.go` | `backup` subcommand. | +| `vantagectl/internal/cmd/restore.go` | `restore` subcommand, TTY and `--confirm-db` rules. | +| `vantagectl/internal/cmd/inspect.go` | `inspect` subcommand. No database contact. | +| `vantagectl/internal/cmd/verify.go` | `verify` subcommand. | +| `vantagectl/Dockerfile` | Scratch image with a staged `/tmp`. | +| `.gitea/workflows/vantagectl-release.yml` | Tag-triggered multi-platform binary release. | +| `deploy/chart/vantage/templates/backup-cronjob.yaml` | Optional CronJob, default off. | +| `docsite/docs/operations/backup-and-restore.md` | Operator documentation. | + +--- + +### Task 1: Extract AES-GCM into `shared/cryptobox` + +**Files:** +- Create: `shared/cryptobox/cryptobox.go` +- Create: `shared/cryptobox/cryptobox_test.go` +- Modify: `server/internal/services/crypto.go` + +**Interfaces:** +- Consumes: nothing. +- Produces: `cryptobox.Seal(key []byte, plaintext string) (string, error)`, `cryptobox.Open(key []byte, ciphertextHex string) (string, error)`, `cryptobox.KeySize = 32`. + +- [ ] **Step 1: Write the failing test** + +Create `shared/cryptobox/cryptobox_test.go`: + +```go +package cryptobox + +import ( + "bytes" + "crypto/rand" + "encoding/hex" + "testing" +) + +func testKey(t *testing.T) []byte { + t.Helper() + k := make([]byte, KeySize) + if _, err := rand.Read(k); err != nil { + t.Fatalf("rand: %v", err) + } + return k +} + +func TestSealOpenRoundTrip(t *testing.T) { + key := testKey(t) + sealed, err := Seal(key, "hunter2") + if err != nil { + t.Fatalf("Seal: %v", err) + } + if _, err := hex.DecodeString(sealed); err != nil { + t.Fatalf("Seal output is not hex: %v", err) + } + if bytes.Contains([]byte(sealed), []byte("hunter2")) { + t.Fatal("plaintext appears in ciphertext") + } + got, err := Open(key, sealed) + if err != nil { + t.Fatalf("Open: %v", err) + } + if got != "hunter2" { + t.Fatalf("got %q, want %q", got, "hunter2") + } +} + +func TestSealIsNonDeterministic(t *testing.T) { + key := testKey(t) + a, err := Seal(key, "same") + if err != nil { + t.Fatalf("Seal: %v", err) + } + b, err := Seal(key, "same") + if err != nil { + t.Fatalf("Seal: %v", err) + } + if a == b { + t.Fatal("two seals of the same plaintext are identical; nonce is not random") + } +} + +func TestOpenWrongKeyFails(t *testing.T) { + sealed, err := Seal(testKey(t), "secret") + if err != nil { + t.Fatalf("Seal: %v", err) + } + if _, err := Open(testKey(t), sealed); err == nil { + t.Fatal("Open with the wrong key succeeded") + } +} + +func TestOpenRejectsBadInput(t *testing.T) { + key := testKey(t) + if _, err := Open(key, "not-hex"); err == nil { + t.Fatal("Open accepted non-hex input") + } + if _, err := Open(key, "abcd"); err == nil { + t.Fatal("Open accepted a ciphertext shorter than the nonce") + } +} + +func TestWrongKeySizeRejected(t *testing.T) { + if _, err := Seal(make([]byte, 16), "x"); err == nil { + t.Fatal("Seal accepted a 16-byte key") + } +} +``` + +- [ ] **Step 2: Run the test and verify it fails** + +Run: `cd shared && go test ./cryptobox/...` +Expected: FAIL — the package does not compile, `undefined: Seal`. + +- [ ] **Step 3: Write the implementation** + +Create `shared/cryptobox/cryptobox.go`: + +```go +// Package cryptobox is the AES-256-GCM primitive used for everything Vantage +// encrypts at rest: SSH private keys, key passphrases, vault secrets, OIDC +// client secrets and console credentials. +// +// It takes a raw key and reads no environment. Key sourcing belongs to the +// caller, because the two callers source it differently: the server reads +// KEY_ENCRYPTION_KEY at the point of use, while vantagectl is handed one. +package cryptobox + +import ( + "crypto/aes" + "crypto/cipher" + "crypto/rand" + "encoding/hex" + "fmt" + "io" +) + +// KeySize is the only key length accepted. AES-256 by construction. +const KeySize = 32 + +func gcmFor(key []byte) (cipher.AEAD, error) { + if len(key) != KeySize { + return nil, fmt.Errorf("key must be %d bytes, got %d", KeySize, len(key)) + } + block, err := aes.NewCipher(key) + if err != nil { + return nil, err + } + return cipher.NewGCM(block) +} + +// Seal encrypts plaintext and returns nonce||ciphertext, hex encoded. +func Seal(key []byte, plaintext string) (string, error) { + gcm, err := gcmFor(key) + if err != nil { + return "", err + } + nonce := make([]byte, gcm.NonceSize()) + if _, err := io.ReadFull(rand.Reader, nonce); err != nil { + return "", err + } + return hex.EncodeToString(gcm.Seal(nonce, nonce, []byte(plaintext), nil)), nil +} + +// Open reverses Seal. Every failure mode returns an error that does not +// distinguish a wrong key from corrupt data, because the caller cannot act on +// the difference and an oracle is worth avoiding for free. +func Open(key []byte, ciphertextHex string) (string, error) { + gcm, err := gcmFor(key) + if err != nil { + return "", err + } + data, err := hex.DecodeString(ciphertextHex) + if err != nil { + return "", fmt.Errorf("invalid ciphertext encoding") + } + n := gcm.NonceSize() + if len(data) < n { + return "", fmt.Errorf("ciphertext too short") + } + plaintext, err := gcm.Open(nil, data[:n], data[n:], nil) + if err != nil { + return "", fmt.Errorf("decryption failed") + } + return string(plaintext), nil +} +``` + +- [ ] **Step 4: Run the test and verify it passes** + +Run: `cd shared && go test ./cryptobox/...` +Expected: PASS, five tests. + +- [ ] **Step 5: Make `services/crypto.go` delegate** + +Replace the whole body of `server/internal/services/crypto.go` with: + +```go +package services + +import ( + "encoding/hex" + "fmt" + "os" + + "gitea.hostxtra.co.uk/mrhid6/vantage/shared/cryptobox" +) + +// encryptionKey reads KEY_ENCRYPTION_KEY. The cipher itself lives in +// shared/cryptobox so vantagectl's verify probe uses the same implementation +// rather than a second copy that can drift. +func encryptionKey() ([]byte, error) { + raw := os.Getenv("KEY_ENCRYPTION_KEY") + if raw == "" { + return nil, fmt.Errorf("KEY_ENCRYPTION_KEY is not set") + } + key, err := hex.DecodeString(raw) + if err != nil || len(key) != cryptobox.KeySize { + return nil, fmt.Errorf("KEY_ENCRYPTION_KEY must be a 64-character hex string (32 bytes)") + } + return key, nil +} + +func encryptString(plaintext string) (string, error) { + key, err := encryptionKey() + if err != nil { + return "", err + } + return cryptobox.Seal(key, plaintext) +} + +func decryptString(ciphertextHex string) (string, error) { + key, err := encryptionKey() + if err != nil { + return "", err + } + return cryptobox.Open(key, ciphertextHex) +} + +func encryptPrivateKey(plaintext string) (string, error) { return encryptString(plaintext) } + +func decryptPrivateKey(ciphertextHex string) (string, error) { return decryptString(ciphertextHex) } +``` + +The exported behaviour, the function names and the two error strings are unchanged, so nothing else in `services` needs touching. + +- [ ] **Step 6: Verify the server still builds and its tests still pass** + +Run: `cd server && go build ./... && go test ./internal/services/...` +Expected: build succeeds, existing tests pass. + +- [ ] **Step 7: Commit** + +```bash +git add shared/cryptobox server/internal/services/crypto.go +git commit -m "feat: Extract AES-GCM into shared/cryptobox + +services/crypto.go keeps its function names and its KEY_ENCRYPTION_KEY +lookup and delegates the cipher, so vantagectl's verify probe can decrypt +with the same implementation rather than a second copy." +``` + +--- + +### Task 2: Key fingerprint + +**Files:** +- Create: `shared/backup/fingerprint.go` +- Create: `shared/backup/fingerprint_test.go` + +**Interfaces:** +- Consumes: `cryptobox.KeySize` from Task 1. +- Produces: `backup.ParseKey(hexKey string) ([]byte, error)`, `backup.Fingerprint(key []byte) string`, `backup.FingerprintHex(hexKey string) (string, error)`, `backup.ErrNoKey`, `backup.ErrBadKey`. + +- [ ] **Step 1: Write the failing test** + +Create `shared/backup/fingerprint_test.go`: + +```go +package backup + +import ( + "errors" + "strings" + "testing" +) + +const validKeyHex = "0000000000000000000000000000000000000000000000000000000000000001" + +func TestFingerprintIsStableAndNotTheKey(t *testing.T) { + fp, err := FingerprintHex(validKeyHex) + if err != nil { + t.Fatalf("FingerprintHex: %v", err) + } + if len(fp) != 64 { + t.Fatalf("fingerprint is %d chars, want 64", len(fp)) + } + if strings.EqualFold(fp, validKeyHex) { + t.Fatal("fingerprint equals the key") + } + again, err := FingerprintHex(validKeyHex) + if err != nil { + t.Fatalf("FingerprintHex: %v", err) + } + if fp != again { + t.Fatal("fingerprint is not stable across calls") + } +} + +func TestFingerprintDiffersPerKey(t *testing.T) { + other := "0000000000000000000000000000000000000000000000000000000000000002" + a, err := FingerprintHex(validKeyHex) + if err != nil { + t.Fatalf("FingerprintHex: %v", err) + } + b, err := FingerprintHex(other) + if err != nil { + t.Fatalf("FingerprintHex: %v", err) + } + if a == b { + t.Fatal("two different keys produced the same fingerprint") + } +} + +func TestParseKeyRejections(t *testing.T) { + if _, err := ParseKey(""); !errors.Is(err, ErrNoKey) { + t.Fatalf("empty key: got %v, want ErrNoKey", err) + } + for _, bad := range []string{"zz", validKeyHex[:62], validKeyHex + "00"} { + if _, err := ParseKey(bad); !errors.Is(err, ErrBadKey) { + t.Fatalf("key %q: got %v, want ErrBadKey", bad, err) + } + } +} + +func TestParseKeyAcceptsUppercase(t *testing.T) { + if _, err := ParseKey(strings.ToUpper(validKeyHex)); err != nil { + t.Fatalf("uppercase hex rejected: %v", err) + } +} +``` + +- [ ] **Step 2: Run the test and verify it fails** + +Run: `cd shared && go test ./backup/...` +Expected: FAIL — `undefined: FingerprintHex`. + +- [ ] **Step 3: Write the implementation** + +Create `shared/backup/fingerprint.go`: + +```go +// Package backup dumps and restores a whole Vantage MongoDB database. +// +// The archive never contains KEY_ENCRYPTION_KEY. It contains a fingerprint of +// it, which is enough to answer "will this archive restore into this +// deployment" and is not a hint at the value. +package backup + +import ( + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + + "gitea.hostxtra.co.uk/mrhid6/vantage/shared/cryptobox" +) + +// ErrNoKey is returned when no key was supplied at all. It is distinct from +// ErrBadKey because the operator remedies are different: one is "set the +// variable", the other is "the value you set is wrong". +var ErrNoKey = errors.New("KEY_ENCRYPTION_KEY is not set") + +// ErrBadKey is returned when a key was supplied but is not 64 hex characters. +var ErrBadKey = errors.New("KEY_ENCRYPTION_KEY must be a 64-character hex string (32 bytes)") + +// ParseKey decodes the hex form used by KEY_ENCRYPTION_KEY. +func ParseKey(hexKey string) ([]byte, error) { + if hexKey == "" { + return nil, ErrNoKey + } + key, err := hex.DecodeString(hexKey) + if err != nil { + return nil, fmt.Errorf("%w: not hexadecimal", ErrBadKey) + } + if len(key) != cryptobox.KeySize { + return nil, fmt.Errorf("%w: decoded to %d bytes", ErrBadKey, len(key)) + } + return key, nil +} + +// Fingerprint is the SHA-256 of the raw key bytes, hex encoded. +// +// Of the raw bytes rather than of the hex string, so an operator who writes the +// key in uppercase in one deployment and lowercase in another still gets one +// fingerprint for one key. +func Fingerprint(key []byte) string { + sum := sha256.Sum256(key) + return hex.EncodeToString(sum[:]) +} + +// FingerprintHex parses and fingerprints in one step. +func FingerprintHex(hexKey string) (string, error) { + key, err := ParseKey(hexKey) + if err != nil { + return "", err + } + return Fingerprint(key), nil +} +``` + +- [ ] **Step 4: Run the test and verify it passes** + +Run: `cd shared && go test ./backup/...` +Expected: PASS, four tests. + +- [ ] **Step 5: Commit** + +```bash +git add shared/backup +git commit -m "feat: Add key fingerprinting for backup archives + +Fingerprint hashes the raw key bytes rather than the hex string, so the +same key written in different cases fingerprints identically." +``` + +--- + +### Task 3: Manifest types + +**Files:** +- Create: `shared/backup/manifest.go` +- Create: `shared/backup/manifest_test.go` + +**Interfaces:** +- Consumes: nothing from earlier tasks. +- Produces: `backup.FormatVersion = 1`, `backup.Manifest`, `backup.CollectionEntry`, `(Manifest).Check() error`, `(Manifest).Collection(name string) (CollectionEntry, bool)`, `backup.ErrUnknownFormat`, `backup.CiphertextCollections() []string`. + +- [ ] **Step 1: Write the failing test** + +Create `shared/backup/manifest_test.go`: + +```go +package backup + +import ( + "encoding/json" + "errors" + "strings" + "testing" + "time" +) + +func TestManifestJSONShape(t *testing.T) { + fp := "abc" + m := Manifest{ + FormatVersion: FormatVersion, + CreatedAt: time.Date(2026, 9, 7, 12, 0, 0, 0, time.UTC), + VantageVersion: "dev", + Hostname: "box", + MongoDB: "vantage", + MongoServerVersion: "7.0.5", + KeyFingerprint: &fp, + Collections: []CollectionEntry{{Name: "servers", Documents: 3, Bytes: 120, SHA256: "dead"}}, + Excluded: []string{"audit_logs"}, + } + raw, err := json.Marshal(m) + if err != nil { + t.Fatalf("marshal: %v", err) + } + for _, want := range []string{ + `"format_version":1`, `"created_at":"2026-09-07T12:00:00Z"`, + `"key_fingerprint":"abc"`, `"mongo_server_version":"7.0.5"`, + `"excluded":["audit_logs"]`, + } { + if !strings.Contains(string(raw), want) { + t.Fatalf("manifest JSON missing %s\ngot: %s", want, raw) + } + } +} + +func TestManifestNullFingerprint(t *testing.T) { + raw, err := json.Marshal(Manifest{FormatVersion: FormatVersion}) + if err != nil { + t.Fatalf("marshal: %v", err) + } + if !strings.Contains(string(raw), `"key_fingerprint":null`) { + t.Fatalf("absent key must marshal as null, got: %s", raw) + } +} + +func TestManifestCheckRejectsOtherVersions(t *testing.T) { + if err := (Manifest{FormatVersion: FormatVersion}).Check(); err != nil { + t.Fatalf("current version rejected: %v", err) + } + for _, v := range []int{0, 2, 99} { + if err := (Manifest{FormatVersion: v}).Check(); !errors.Is(err, ErrUnknownFormat) { + t.Fatalf("version %d: got %v, want ErrUnknownFormat", v, err) + } + } +} + +func TestManifestCollectionLookup(t *testing.T) { + m := Manifest{Collections: []CollectionEntry{{Name: "keys", Documents: 1}}} + if _, ok := m.Collection("keys"); !ok { + t.Fatal("known collection not found") + } + if _, ok := m.Collection("nope"); ok { + t.Fatal("unknown collection reported as found") + } +} + +func TestCiphertextCollections(t *testing.T) { + got := CiphertextCollections() + want := []string{"keys", "secrets", "auth_providers", "console_sessions", "settings"} + if len(got) != len(want) { + t.Fatalf("got %v, want %v", got, want) + } + for i := range want { + if got[i] != want[i] { + t.Fatalf("got %v, want %v", got, want) + } + } +} +``` + +- [ ] **Step 2: Run the test and verify it fails** + +Run: `cd shared && go test ./backup/...` +Expected: FAIL — `undefined: Manifest`. + +- [ ] **Step 3: Write the implementation** + +Create `shared/backup/manifest.go`: + +```go +package backup + +import ( + "errors" + "fmt" + "time" +) + +// FormatVersion is the archive format this build reads and writes. Restore +// refuses anything else rather than guessing at a layout it does not know. +const FormatVersion = 1 + +// ManifestName is the archive member holding the manifest. +const ManifestName = "manifest.json" + +// ErrUnknownFormat is returned for an archive this build cannot read. +var ErrUnknownFormat = errors.New("unsupported archive format version") + +// CollectionEntry describes one collection in the archive. Bytes and SHA256 +// cover the uncompressed .bson member, which is what restore verifies before +// writing anything. +type CollectionEntry struct { + Name string `json:"name"` + Documents int64 `json:"documents"` + Bytes int64 `json:"bytes"` + SHA256 string `json:"sha256"` +} + +// Manifest is the archive's index and its provenance. +// +// KeyFingerprint is a pointer so "this archive recorded no key" is a distinct +// state from "this archive recorded the empty string". A null here is a real +// condition an operator must be told about, not a default. +type Manifest struct { + FormatVersion int `json:"format_version"` + CreatedAt time.Time `json:"created_at"` + VantageVersion string `json:"vantage_version"` + Hostname string `json:"hostname"` + MongoDB string `json:"mongo_db"` + MongoServerVersion string `json:"mongo_server_version"` + KeyFingerprint *string `json:"key_fingerprint"` + Collections []CollectionEntry `json:"collections"` + Excluded []string `json:"excluded"` +} + +// Check validates what can be validated without reading the rest of the archive. +func (m Manifest) Check() error { + if m.FormatVersion != FormatVersion { + return fmt.Errorf("%w: archive is version %d, this build reads version %d", + ErrUnknownFormat, m.FormatVersion, FormatVersion) + } + return nil +} + +// Collection looks up one entry by name. +func (m Manifest) Collection(name string) (CollectionEntry, bool) { + for _, c := range m.Collections { + if c.Name == name { + return c, true + } + } + return CollectionEntry{}, false +} + +// CiphertextCollections names the collections holding AES-GCM ciphertext. +// +// It exists to be printed. When a restore proceeds under a key that does not +// match the archive, this is the list of what will be unreadable afterwards, +// and an operator deserves to see it before the write rather than discover it +// a week later. +func CiphertextCollections() []string { + return []string{"keys", "secrets", "auth_providers", "console_sessions", "settings"} +} +``` + +- [ ] **Step 4: Run the test and verify it passes** + +Run: `cd shared && go test ./backup/...` +Expected: PASS, nine tests across the package. + +- [ ] **Step 5: Commit** + +```bash +git add shared/backup/manifest.go shared/backup/manifest_test.go +git commit -m "feat: Add the backup archive manifest + +KeyFingerprint is a pointer so an archive that recorded no key is a state +restore can report, not a default it silently treats as a match." +``` + +--- + +### Task 4: Archive writer and reader + +**Files:** +- Create: `shared/backup/archive.go` +- Create: `shared/backup/archive_test.go` + +**Interfaces:** +- Consumes: `Manifest`, `CollectionEntry`, `ManifestName`, `ErrUnknownFormat` from Task 3. +- Produces: + - `backup.NewWriter(out io.Writer) *Writer` + - `(*Writer).WriteCollection(name string, docs [][]byte) (CollectionEntry, error)` + - `(*Writer).WriteIndexes(name string, specsJSON []byte) error` + - `(*Writer).Close(m Manifest) error` + - `backup.Open(path string) (*Reader, error)` + - `(*Reader).Manifest() Manifest` + - `(*Reader).OpenCollection(name string) (io.ReadCloser, error)` + - `(*Reader).IndexesJSON(name string) ([]byte, error)` + - `(*Reader).Close() error` + - `backup.ErrChecksum` + +- [ ] **Step 1: Write the failing test** + +Create `shared/backup/archive_test.go`: + +```go +package backup + +import ( + "bytes" + "errors" + "io" + "os" + "path/filepath" + "testing" + "time" +) + +// writeSampleArchive builds a two-collection archive on disk and returns its path. +func writeSampleArchive(t *testing.T) string { + t.Helper() + path := filepath.Join(t.TempDir(), "sample.tar.gz") + f, err := os.Create(path) + if err != nil { + t.Fatalf("create: %v", err) + } + defer f.Close() + + w := NewWriter(f) + servers, err := w.WriteCollection("servers", [][]byte{[]byte("one"), []byte("two")}) + if err != nil { + t.Fatalf("WriteCollection: %v", err) + } + if err := w.WriteIndexes("servers", []byte(`[{"name":"idx"}]`)); err != nil { + t.Fatalf("WriteIndexes: %v", err) + } + keys, err := w.WriteCollection("keys", [][]byte{[]byte("k")}) + if err != nil { + t.Fatalf("WriteCollection: %v", err) + } + if err := w.Close(Manifest{ + FormatVersion: FormatVersion, + CreatedAt: time.Now().UTC(), + MongoDB: "vantage", + Collections: []CollectionEntry{servers, keys}, + }); err != nil { + t.Fatalf("Close: %v", err) + } + return path +} + +func TestWriterRecordsCountsAndChecksums(t *testing.T) { + var buf bytes.Buffer + w := NewWriter(&buf) + e, err := w.WriteCollection("servers", [][]byte{[]byte("one"), []byte("two")}) + if err != nil { + t.Fatalf("WriteCollection: %v", err) + } + if e.Name != "servers" { + t.Fatalf("name %q", e.Name) + } + if e.Documents != 2 { + t.Fatalf("documents %d, want 2", e.Documents) + } + if e.Bytes != 6 { + t.Fatalf("bytes %d, want 6", e.Bytes) + } + if len(e.SHA256) != 64 { + t.Fatalf("sha256 %q is not 64 hex chars", e.SHA256) + } +} + +func TestRoundTrip(t *testing.T) { + r, err := Open(writeSampleArchive(t)) + if err != nil { + t.Fatalf("Open: %v", err) + } + defer r.Close() + + if r.Manifest().MongoDB != "vantage" { + t.Fatalf("manifest not read back: %+v", r.Manifest()) + } + + rc, err := r.OpenCollection("servers") + if err != nil { + t.Fatalf("OpenCollection: %v", err) + } + defer rc.Close() + got, err := io.ReadAll(rc) + if err != nil { + t.Fatalf("read: %v", err) + } + if string(got) != "onetwo" { + t.Fatalf("got %q, want %q", got, "onetwo") + } + + idx, err := r.IndexesJSON("servers") + if err != nil { + t.Fatalf("IndexesJSON: %v", err) + } + if string(idx) != `[{"name":"idx"}]` { + t.Fatalf("indexes round-tripped as %q", idx) + } +} + +func TestIndexesJSONAbsentIsEmptyNotError(t *testing.T) { + r, err := Open(writeSampleArchive(t)) + if err != nil { + t.Fatalf("Open: %v", err) + } + defer r.Close() + idx, err := r.IndexesJSON("keys") + if err != nil { + t.Fatalf("a collection with no index member must not error: %v", err) + } + if len(idx) != 0 { + t.Fatalf("want empty, got %q", idx) + } +} + +func TestOpenRejectsCorruptedMember(t *testing.T) { + path := writeSampleArchive(t) + + // Rewrite the archive with one byte of a collection member flipped, leaving + // the manifest's checksum describing the original. + corrupt := filepath.Join(t.TempDir(), "corrupt.tar.gz") + rewriteFlippingCollectionByte(t, path, corrupt, "servers") + + if _, err := Open(corrupt); !errors.Is(err, ErrChecksum) { + t.Fatalf("got %v, want ErrChecksum", err) + } +} + +func TestOpenRejectsUnknownFormatVersion(t *testing.T) { + path := filepath.Join(t.TempDir(), "future.tar.gz") + f, err := os.Create(path) + if err != nil { + t.Fatalf("create: %v", err) + } + w := NewWriter(f) + if err := w.Close(Manifest{FormatVersion: 99}); err != nil { + t.Fatalf("Close: %v", err) + } + f.Close() + + if _, err := Open(path); !errors.Is(err, ErrUnknownFormat) { + t.Fatalf("got %v, want ErrUnknownFormat", err) + } +} + +func TestCloseRemovesTempDir(t *testing.T) { + r, err := Open(writeSampleArchive(t)) + if err != nil { + t.Fatalf("Open: %v", err) + } + dir := r.dir + if _, err := os.Stat(dir); err != nil { + t.Fatalf("temp dir missing while open: %v", err) + } + if err := r.Close(); err != nil { + t.Fatalf("Close: %v", err) + } + if _, err := os.Stat(dir); !os.IsNotExist(err) { + t.Fatalf("temp dir %s survived Close", dir) + } +} +``` + +Add the corruption helper to the same file: + +```go +// rewriteFlippingCollectionByte copies an archive, flipping one byte inside the +// named collection's .bson member so its content no longer matches the checksum +// the manifest recorded. +func rewriteFlippingCollectionByte(t *testing.T, src, dst, collection string) { + t.Helper() + + in, err := os.Open(src) + if err != nil { + t.Fatalf("open src: %v", err) + } + defer in.Close() + gz, err := gzip.NewReader(in) + if err != nil { + t.Fatalf("gzip: %v", err) + } + defer gz.Close() + + out, err := os.Create(dst) + if err != nil { + t.Fatalf("create dst: %v", err) + } + defer out.Close() + gw := gzip.NewWriter(out) + defer gw.Close() + tw := tar.NewWriter(gw) + defer tw.Close() + + tr := tar.NewReader(gz) + target := "collections/" + collection + ".bson" + for { + h, err := tr.Next() + if err == io.EOF { + return + } + if err != nil { + t.Fatalf("tar next: %v", err) + } + body, err := io.ReadAll(tr) + if err != nil { + t.Fatalf("read member: %v", err) + } + if h.Name == target && len(body) > 0 { + body[0] ^= 0xFF + } + h.Size = int64(len(body)) + if err := tw.WriteHeader(h); err != nil { + t.Fatalf("write header: %v", err) + } + if _, err := tw.Write(body); err != nil { + t.Fatalf("write body: %v", err) + } + } +} +``` + +Add `"archive/tar"` and `"compress/gzip"` to the test file's imports. + +- [ ] **Step 2: Run the test and verify it fails** + +Run: `cd shared && go test ./backup/...` +Expected: FAIL — `undefined: NewWriter`. + +- [ ] **Step 3: Write the implementation** + +Create `shared/backup/archive.go`: + +```go +package backup + +import ( + "archive/tar" + "compress/gzip" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "io" + "os" + "path" + "path/filepath" + "strings" + "time" +) + +// ErrChecksum is returned when an archive member does not match the checksum +// the manifest recorded for it. +var ErrChecksum = errors.New("archive member failed its checksum") + +// Writer streams a tar.gz. Members are written in the order they are produced +// and the manifest goes last, because its per-collection checksums are only +// known once every collection has been written. +type Writer struct { + gz *gzip.Writer + tar *tar.Writer +} + +// NewWriter starts an archive on out. out may be a file or stdout; nothing here +// seeks. +func NewWriter(out io.Writer) *Writer { + gz := gzip.NewWriter(out) + return &Writer{gz: gz, tar: tar.NewWriter(gz)} +} + +func (w *Writer) writeMember(name string, body []byte) error { + h := &tar.Header{ + Name: name, + Mode: 0o600, + Size: int64(len(body)), + ModTime: time.Now().UTC(), + Typeflag: tar.TypeReg, + } + if err := w.tar.WriteHeader(h); err != nil { + return fmt.Errorf("write header %s: %w", name, err) + } + if _, err := w.tar.Write(body); err != nil { + return fmt.Errorf("write %s: %w", name, err) + } + return nil +} + +// WriteCollection writes the concatenated raw BSON of one collection and +// returns the manifest entry describing it. +func (w *Writer) WriteCollection(name string, docs [][]byte) (CollectionEntry, error) { + var body []byte + for _, d := range docs { + body = append(body, d...) + } + sum := sha256.Sum256(body) + entry := CollectionEntry{ + Name: name, + Documents: int64(len(docs)), + Bytes: int64(len(body)), + SHA256: hex.EncodeToString(sum[:]), + } + if err := w.writeMember(collectionMember(name), body); err != nil { + return CollectionEntry{}, err + } + return entry, nil +} + +// WriteIndexes writes a collection's index specifications verbatim. +func (w *Writer) WriteIndexes(name string, specsJSON []byte) error { + return w.writeMember(indexMember(name), specsJSON) +} + +// Close writes the manifest and finishes the archive. +func (w *Writer) Close(m Manifest) error { + raw, err := json.MarshalIndent(m, "", " ") + if err != nil { + return fmt.Errorf("marshal manifest: %w", err) + } + if err := w.writeMember(ManifestName, raw); err != nil { + return err + } + if err := w.tar.Close(); err != nil { + return err + } + return w.gz.Close() +} + +func collectionMember(name string) string { return "collections/" + name + ".bson" } +func indexMember(name string) string { return "indexes/" + name + ".json" } + +// Reader is an opened archive. +// +// Open extracts to a temporary directory rather than streaming, because gzip +// offers no random access and the manifest — which carries the checksums every +// other member is judged against — is written last. Verifying before writing a +// single document to the target is worth one pass over local disk. This is why +// the container image needs a /tmp. +type Reader struct { + dir string + manifest Manifest +} + +// Open extracts, verifies and returns the archive at path. The caller must +// Close it. +func Open(archivePath string) (*Reader, error) { + dir, err := os.MkdirTemp("", "vantage-restore-*") + if err != nil { + return nil, fmt.Errorf("temp dir: %w", err) + } + r := &Reader{dir: dir} + + if err := r.extract(archivePath); err != nil { + r.Close() + return nil, err + } + if err := r.loadManifest(); err != nil { + r.Close() + return nil, err + } + if err := r.verifyMembers(); err != nil { + r.Close() + return nil, err + } + return r, nil +} + +func (r *Reader) extract(archivePath string) error { + f, err := os.Open(archivePath) + if err != nil { + return fmt.Errorf("open archive: %w", err) + } + defer f.Close() + + gz, err := gzip.NewReader(f) + if err != nil { + return fmt.Errorf("archive is not gzip: %w", err) + } + defer gz.Close() + + tr := tar.NewReader(gz) + for { + h, err := tr.Next() + if err == io.EOF { + return nil + } + if err != nil { + return fmt.Errorf("read archive: %w", err) + } + if h.Typeflag != tar.TypeReg { + continue + } + dest, err := safeJoin(r.dir, h.Name) + if err != nil { + return err + } + if err := os.MkdirAll(filepath.Dir(dest), 0o700); err != nil { + return fmt.Errorf("mkdir: %w", err) + } + out, err := os.OpenFile(dest, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o600) + if err != nil { + return fmt.Errorf("create %s: %w", h.Name, err) + } + if _, err := io.Copy(out, tr); err != nil { + out.Close() + return fmt.Errorf("extract %s: %w", h.Name, err) + } + if err := out.Close(); err != nil { + return err + } + } +} + +// safeJoin refuses a member name that escapes the extraction directory. An +// archive is operator-supplied input and may not be one we wrote. +func safeJoin(dir, name string) (string, error) { + clean := path.Clean("/" + name) + dest := filepath.Join(dir, filepath.FromSlash(strings.TrimPrefix(clean, "/"))) + if !strings.HasPrefix(dest, filepath.Clean(dir)+string(os.PathSeparator)) { + return "", fmt.Errorf("archive member %q escapes the extraction directory", name) + } + return dest, nil +} + +func (r *Reader) loadManifest() error { + raw, err := os.ReadFile(filepath.Join(r.dir, ManifestName)) + if err != nil { + return fmt.Errorf("archive has no %s: %w", ManifestName, err) + } + if err := json.Unmarshal(raw, &r.manifest); err != nil { + return fmt.Errorf("parse %s: %w", ManifestName, err) + } + return r.manifest.Check() +} + +func (r *Reader) verifyMembers() error { + for _, c := range r.manifest.Collections { + f, err := os.Open(filepath.Join(r.dir, collectionMember(c.Name))) + if err != nil { + return fmt.Errorf("%w: %s is named in the manifest but absent from the archive", + ErrChecksum, c.Name) + } + h := sha256.New() + n, err := io.Copy(h, f) + f.Close() + if err != nil { + return fmt.Errorf("read %s: %w", c.Name, err) + } + if n != c.Bytes { + return fmt.Errorf("%w: %s is %d bytes, manifest says %d", ErrChecksum, c.Name, n, c.Bytes) + } + if got := hex.EncodeToString(h.Sum(nil)); got != c.SHA256 { + return fmt.Errorf("%w: %s checksum %s, manifest says %s", ErrChecksum, c.Name, got, c.SHA256) + } + } + return nil +} + +// Manifest returns the verified manifest. +func (r *Reader) Manifest() Manifest { return r.manifest } + +// OpenCollection returns the raw BSON stream for one collection. +func (r *Reader) OpenCollection(name string) (io.ReadCloser, error) { + return os.Open(filepath.Join(r.dir, collectionMember(name))) +} + +// IndexesJSON returns a collection's index specifications, or nil when the +// archive holds none. A collection with no indexes beyond _id_ is ordinary and +// is not an error. +func (r *Reader) IndexesJSON(name string) ([]byte, error) { + raw, err := os.ReadFile(filepath.Join(r.dir, indexMember(name))) + if os.IsNotExist(err) { + return nil, nil + } + return raw, err +} + +// Close removes the extraction directory. +func (r *Reader) Close() error { return os.RemoveAll(r.dir) } +``` + +- [ ] **Step 4: Run the test and verify it passes** + +Run: `cd shared && go test ./backup/... -run 'Archive|RoundTrip|Writer|Open|Close|Indexes'` +Expected: PASS. + +- [ ] **Step 5: Run the whole package** + +Run: `cd shared && go test ./backup/...` +Expected: PASS, all tests from Tasks 2, 3 and 4. + +- [ ] **Step 6: Commit** + +```bash +git add shared/backup/archive.go shared/backup/archive_test.go +git commit -m "feat: Add the backup archive writer and reader + +Open extracts and verifies every member against the manifest before the +reader is usable, so a corrupt archive is refused before a restore writes +its first document rather than halfway through." +``` + +--- + +### Task 5: Dump + +**Files:** +- Create: `shared/backup/dump.go` +- Create: `shared/backup/mongo_test.go` +- Create: `shared/backup/dump_test.go` + +**Interfaces:** +- Consumes: `Writer`, `Manifest`, `CollectionEntry`, `FingerprintHex`, `ErrNoKey`, `ErrBadKey`. +- Produces: + - `backup.DumpOptions{Client *mongo.Client; Database string; Exclude []string; KeyHex string; AllowNoKey bool; VantageVersion string; Out io.Writer}` + - `backup.Dump(ctx context.Context, opt DumpOptions) (Manifest, error)` + - Test helper `testDB(t *testing.T) (*mongo.Client, string)` in `mongo_test.go`. + +- [ ] **Step 1: Write the MongoDB test helper** + +Create `shared/backup/mongo_test.go`: + +```go +package backup + +import ( + "context" + "fmt" + "os" + "testing" + "time" + + "go.mongodb.org/mongo-driver/v2/mongo" + "go.mongodb.org/mongo-driver/v2/mongo/options" +) + +// testDB connects to the MongoDB named by MONGO_TEST_URI and returns a client +// plus a database name unique to this test, dropped when the test ends. +// +// Skips rather than fails when the variable is unset: these tests need a real +// server, and a developer without one should still be able to run the rest of +// the suite. +func testDB(t *testing.T) (*mongo.Client, string) { + t.Helper() + uri := os.Getenv("MONGO_TEST_URI") + if uri == "" { + t.Skip("MONGO_TEST_URI is not set; skipping tests that need MongoDB") + } + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + + client, err := mongo.Connect(options.Client().ApplyURI(uri)) + if err != nil { + t.Fatalf("connect: %v", err) + } + if err := client.Ping(ctx, nil); err != nil { + t.Fatalf("ping: %v", err) + } + name := fmt.Sprintf("vantage_test_%d", time.Now().UnixNano()) + t.Cleanup(func() { + c, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + _ = client.Database(name).Drop(c) + _ = client.Disconnect(c) + }) + return client, name +} +``` + +- [ ] **Step 2: Write the failing dump test** + +Create `shared/backup/dump_test.go`: + +```go +package backup + +import ( + "bytes" + "context" + "errors" + "os" + "path/filepath" + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" + "go.mongodb.org/mongo-driver/v2/mongo" +) + +func seed(t *testing.T, client *mongo.Client, dbName string) { + t.Helper() + ctx := context.Background() + db := client.Database(dbName) + if _, err := db.Collection("servers").InsertMany(ctx, []any{ + bson.M{"_id": bson.NewObjectID(), "name": "alpha", "instance_id": "i1"}, + bson.M{"_id": bson.NewObjectID(), "name": "beta", "instance_id": "i1"}, + }); err != nil { + t.Fatalf("insert servers: %v", err) + } + if _, err := db.Collection("audit_logs").InsertOne(ctx, bson.M{"action": "login"}); err != nil { + t.Fatalf("insert audit_logs: %v", err) + } +} + +func dumpToFile(t *testing.T, opt DumpOptions) (string, Manifest) { + t.Helper() + path := filepath.Join(t.TempDir(), "out.tar.gz") + f, err := os.Create(path) + if err != nil { + t.Fatalf("create: %v", err) + } + opt.Out = f + m, err := Dump(context.Background(), opt) + if cerr := f.Close(); cerr != nil { + t.Fatalf("close: %v", cerr) + } + if err != nil { + t.Fatalf("Dump: %v", err) + } + return path, m +} + +func TestDumpEnumeratesEveryCollection(t *testing.T) { + client, dbName := testDB(t) + seed(t, client, dbName) + + _, m := dumpToFile(t, DumpOptions{ + Client: client, Database: dbName, KeyHex: validKeyHex, VantageVersion: "test", + }) + + if _, ok := m.Collection("servers"); !ok { + t.Fatal("servers missing from the manifest") + } + if _, ok := m.Collection("audit_logs"); !ok { + t.Fatal("audit_logs missing; enumeration must not filter by a hardcoded list") + } + servers, _ := m.Collection("servers") + if servers.Documents != 2 { + t.Fatalf("servers documents %d, want 2", servers.Documents) + } + if m.MongoDB != dbName { + t.Fatalf("manifest database %q, want %q", m.MongoDB, dbName) + } + if m.MongoServerVersion == "" { + t.Fatal("manifest records no MongoDB server version") + } + if m.Hostname == "" { + t.Fatal("manifest records no hostname") + } +} + +func TestDumpRecordsKeyFingerprint(t *testing.T) { + client, dbName := testDB(t) + seed(t, client, dbName) + + _, m := dumpToFile(t, DumpOptions{Client: client, Database: dbName, KeyHex: validKeyHex}) + + want, err := FingerprintHex(validKeyHex) + if err != nil { + t.Fatalf("FingerprintHex: %v", err) + } + if m.KeyFingerprint == nil || *m.KeyFingerprint != want { + t.Fatalf("fingerprint %v, want %s", m.KeyFingerprint, want) + } +} + +func TestDumpRefusesWithoutAKey(t *testing.T) { + client, dbName := testDB(t) + seed(t, client, dbName) + + var buf bytes.Buffer + _, err := Dump(context.Background(), DumpOptions{ + Client: client, Database: dbName, Out: &buf, + }) + if !errors.Is(err, ErrNoKey) { + t.Fatalf("got %v, want ErrNoKey", err) + } + if buf.Len() != 0 { + t.Fatal("refusal must happen before anything is written") + } +} + +func TestDumpAllowNoKeyStampsNull(t *testing.T) { + client, dbName := testDB(t) + seed(t, client, dbName) + + _, m := dumpToFile(t, DumpOptions{Client: client, Database: dbName, AllowNoKey: true}) + if m.KeyFingerprint != nil { + t.Fatalf("want a null fingerprint, got %v", *m.KeyFingerprint) + } +} + +func TestDumpRejectsMalformedKey(t *testing.T) { + client, dbName := testDB(t) + var buf bytes.Buffer + _, err := Dump(context.Background(), DumpOptions{ + Client: client, Database: dbName, KeyHex: "nonsense", Out: &buf, + }) + if !errors.Is(err, ErrBadKey) { + t.Fatalf("got %v, want ErrBadKey", err) + } +} + +func TestDumpExcludeIsRecordedAndOmitted(t *testing.T) { + client, dbName := testDB(t) + seed(t, client, dbName) + + path, m := dumpToFile(t, DumpOptions{ + Client: client, Database: dbName, KeyHex: validKeyHex, + Exclude: []string{"audit_logs"}, + }) + + if _, ok := m.Collection("audit_logs"); ok { + t.Fatal("excluded collection is in the manifest's collection list") + } + if len(m.Excluded) != 1 || m.Excluded[0] != "audit_logs" { + t.Fatalf("excluded recorded as %v", m.Excluded) + } + + r, err := Open(path) + if err != nil { + t.Fatalf("Open: %v", err) + } + defer r.Close() + if _, err := r.OpenCollection("audit_logs"); err == nil { + t.Fatal("excluded collection is present in the archive") + } +} + +func TestDumpPreservesAwkwardBSONTypes(t *testing.T) { + client, dbName := testDB(t) + ctx := context.Background() + + dec, err := bson.ParseDecimal128("1234.5678") + if err != nil { + t.Fatalf("ParseDecimal128: %v", err) + } + doc := bson.M{ + "_id": bson.NewObjectID(), + "decimal": dec, + "when": bson.NewDateTimeFromTime(mustTime(t)), + "binary": bson.Binary{Subtype: 0x00, Data: []byte{0x01, 0x02, 0x03}}, + "nothing": nil, + "nested": bson.A{bson.M{"deep": bson.A{1, 2, 3}}}, + } + if _, err := client.Database(dbName).Collection("odd").InsertOne(ctx, doc); err != nil { + t.Fatalf("insert: %v", err) + } + + path, _ := dumpToFile(t, DumpOptions{Client: client, Database: dbName, KeyHex: validKeyHex}) + + original, err := client.Database(dbName).Collection("odd").FindOne(ctx, bson.M{}).Raw() + if err != nil { + t.Fatalf("read back: %v", err) + } + + r, err := Open(path) + if err != nil { + t.Fatalf("Open: %v", err) + } + defer r.Close() + rc, err := r.OpenCollection("odd") + if err != nil { + t.Fatalf("OpenCollection: %v", err) + } + defer rc.Close() + archived, err := io.ReadAll(rc) + if err != nil { + t.Fatalf("read: %v", err) + } + if !bytes.Equal(archived, []byte(original)) { + t.Fatal("archived BSON differs from what the driver returned") + } +} + +func mustTime(t *testing.T) time.Time { + t.Helper() + return time.Date(2026, 9, 7, 12, 0, 0, 0, time.UTC) +} +``` + +Add `"io"` and `"time"` to this file's imports. + +- [ ] **Step 3: Run the test and verify it fails** + +Run: `cd shared && MONGO_TEST_URI=mongodb://localhost:27017 go test ./backup/... -run Dump` +Expected: FAIL — `undefined: Dump`. + +If no MongoDB is available locally, start one: `docker run -d --rm -p 27017:27017 --name vantage-test-mongo mongo:7`. + +- [ ] **Step 4: Write the implementation** + +Create `shared/backup/dump.go`: + +```go +package backup + +import ( + "context" + "encoding/json" + "fmt" + "io" + "os" + "sort" + "time" + + "go.mongodb.org/mongo-driver/v2/bson" + "go.mongodb.org/mongo-driver/v2/mongo" +) + +// DumpOptions configures one backup. +type DumpOptions struct { + Client *mongo.Client + Database string + Exclude []string + + // KeyHex is KEY_ENCRYPTION_KEY. It is fingerprinted and discarded; it is + // never written to the archive. + KeyHex string + + // AllowNoKey permits a backup of a deployment that stores no encrypted + // material. The manifest then records a null fingerprint, which restore + // reports rather than treating as a match. + AllowNoKey bool + + VantageVersion string + Out io.Writer +} + +// Dump writes a complete archive of one database to opt.Out. +// +// Collections are enumerated live rather than read from a list. A backup tool +// has no equivalent of AssertNoScopedCollectionMissed to catch a hardcoded list +// drifting, and the first symptom of that drift would be a restore silently +// missing a collection added since the list was written. +func Dump(ctx context.Context, opt DumpOptions) (Manifest, error) { + fingerprint, err := dumpFingerprint(opt) + if err != nil { + return Manifest{}, err + } + + db := opt.Client.Database(opt.Database) + names, err := db.ListCollectionNames(ctx, bson.M{}) + if err != nil { + return Manifest{}, fmt.Errorf("list collections: %w", err) + } + sort.Strings(names) + + excluded := map[string]bool{} + for _, e := range opt.Exclude { + excluded[e] = true + } + + serverVersion, err := mongoServerVersion(ctx, opt.Client) + if err != nil { + return Manifest{}, err + } + hostname, err := os.Hostname() + if err != nil { + hostname = "unknown" + } + + w := NewWriter(opt.Out) + entries := make([]CollectionEntry, 0, len(names)) + for _, name := range names { + if excluded[name] { + continue + } + entry, err := dumpCollection(ctx, w, db, name) + if err != nil { + return Manifest{}, err + } + entries = append(entries, entry) + } + + m := Manifest{ + FormatVersion: FormatVersion, + CreatedAt: time.Now().UTC(), + VantageVersion: opt.VantageVersion, + Hostname: hostname, + MongoDB: opt.Database, + MongoServerVersion: serverVersion, + KeyFingerprint: fingerprint, + Collections: entries, + Excluded: append([]string{}, opt.Exclude...), + } + if err := w.Close(m); err != nil { + return Manifest{}, err + } + return m, nil +} + +// dumpFingerprint applies the key policy before any output is produced. An +// archive of ciphertext whose key was never recorded is worse than no archive, +// because it looks like a backup. +func dumpFingerprint(opt DumpOptions) (*string, error) { + if opt.KeyHex == "" { + if opt.AllowNoKey { + return nil, nil + } + return nil, fmt.Errorf("%w: pass --allow-no-key only if this deployment stores no encrypted data", ErrNoKey) + } + fp, err := FingerprintHex(opt.KeyHex) + if err != nil { + return nil, err + } + return &fp, nil +} + +func dumpCollection(ctx context.Context, w *Writer, db *mongo.Database, name string) (CollectionEntry, error) { + cur, err := db.Collection(name).Find(ctx, bson.M{}) + if err != nil { + return CollectionEntry{}, fmt.Errorf("find %s: %w", name, err) + } + defer cur.Close(ctx) + + var docs [][]byte + for cur.Next(ctx) { + // cur.Current is only valid until the next Next, and it is written to + // the archive verbatim rather than through a map, so every BSON type + // survives exactly as the server stored it. + docs = append(docs, append([]byte(nil), cur.Current...)) + } + if err := cur.Err(); err != nil { + return CollectionEntry{}, fmt.Errorf("iterate %s: %w", name, err) + } + + entry, err := w.WriteCollection(name, docs) + if err != nil { + return CollectionEntry{}, err + } + if err := dumpIndexes(ctx, w, db, name); err != nil { + return CollectionEntry{}, err + } + return entry, nil +} + +func dumpIndexes(ctx context.Context, w *Writer, db *mongo.Database, name string) error { + cur, err := db.Collection(name).Indexes().List(ctx) + if err != nil { + return fmt.Errorf("list indexes on %s: %w", name, err) + } + defer cur.Close(ctx) + + var specs []bson.M + if err := cur.All(ctx, &specs); err != nil { + return fmt.Errorf("read indexes on %s: %w", name, err) + } + raw, err := json.Marshal(specs) + if err != nil { + return fmt.Errorf("encode indexes on %s: %w", name, err) + } + return w.WriteIndexes(name, raw) +} + +func mongoServerVersion(ctx context.Context, client *mongo.Client) (string, error) { + var res struct { + Version string `bson:"version"` + } + err := client.Database("admin").RunCommand(ctx, bson.D{{Key: "buildInfo", Value: 1}}).Decode(&res) + if err != nil { + return "", fmt.Errorf("buildInfo: %w", err) + } + return res.Version, nil +} +``` + +- [ ] **Step 5: Run the test and verify it passes** + +Run: `cd shared && MONGO_TEST_URI=mongodb://localhost:27017 go test ./backup/... -run Dump -v` +Expected: PASS, seven tests. + +- [ ] **Step 6: Verify the suite still skips cleanly without MongoDB** + +Run: `cd shared && go test ./backup/...` +Expected: PASS, with the Mongo-dependent tests reported as skipped. + +- [ ] **Step 7: Commit** + +```bash +git add shared/backup/dump.go shared/backup/dump_test.go shared/backup/mongo_test.go +git commit -m "feat: Add the backup dump + +Collections are enumerated live rather than from a list, so a collection +added later is backed up with no code change. Documents are written as the +raw BSON the driver returned, so Decimal128, ObjectId, DateTime and binary +subtypes survive byte for byte." +``` + +--- +### Task 6: Restore + +**Files:** +- Create: `shared/backup/restore.go` +- Create: `shared/backup/restore_test.go` + +**Interfaces:** +- Consumes: `Reader`, `Manifest`, `CiphertextCollections`, `FingerprintHex`, `ErrNoKey`, plus the `testDB` and `seed` helpers from Task 5. +- Produces: + - `backup.RestoreOptions{Client *mongo.Client; Database string; Archive *Reader; Force bool; KeyHex string; IgnoreKeyMismatch bool; Warn func(string)}` + - `backup.RestoreResult{Collections []RestoredCollection}` + - `backup.RestoredCollection{Name string; Documents int64; Indexes int}` + - `backup.Restore(ctx context.Context, opt RestoreOptions) (RestoreResult, error)` + - `backup.ErrTargetNotEmpty`, `backup.ErrKeyMismatch`, `backup.ErrIndexBuild` + - `backup.BatchSize = 1000` + +- [ ] **Step 1: Write the failing test** + +Create `shared/backup/restore_test.go`: + +```go +package backup + +import ( + "context" + "errors" + "strings" + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" + "go.mongodb.org/mongo-driver/v2/mongo" +) + +// archiveOf seeds a database, dumps it, and returns an opened Reader. +func archiveOf(t *testing.T, client *mongo.Client, keyHex string, allowNoKey bool) *Reader { + t.Helper() + _, srcDB := testDB(t) + seed(t, client, srcDB) + path, _ := dumpToFile(t, DumpOptions{ + Client: client, Database: srcDB, KeyHex: keyHex, AllowNoKey: allowNoKey, + }) + r, err := Open(path) + if err != nil { + t.Fatalf("Open: %v", err) + } + t.Cleanup(func() { r.Close() }) + return r +} + +func countIn(t *testing.T, client *mongo.Client, dbName, coll string) int64 { + t.Helper() + n, err := client.Database(dbName).Collection(coll).CountDocuments(context.Background(), bson.M{}) + if err != nil { + t.Fatalf("count %s: %v", coll, err) + } + return n +} + +func TestRestoreIntoEmptyDatabase(t *testing.T) { + client, _ := testDB(t) + archive := archiveOf(t, client, validKeyHex, false) + _, target := testDB(t) + + res, err := Restore(context.Background(), RestoreOptions{ + Client: client, Database: target, Archive: archive, KeyHex: validKeyHex, + }) + if err != nil { + t.Fatalf("Restore: %v", err) + } + if countIn(t, client, target, "servers") != 2 { + t.Fatal("servers not restored") + } + if len(res.Collections) == 0 { + t.Fatal("result reports no collections") + } +} + +func TestRestoreRefusesNonEmptyTarget(t *testing.T) { + client, _ := testDB(t) + archive := archiveOf(t, client, validKeyHex, false) + _, target := testDB(t) + if _, err := client.Database(target).Collection("servers"). + InsertOne(context.Background(), bson.M{"name": "existing"}); err != nil { + t.Fatalf("seed target: %v", err) + } + + _, err := Restore(context.Background(), RestoreOptions{ + Client: client, Database: target, Archive: archive, KeyHex: validKeyHex, + }) + if !errors.Is(err, ErrTargetNotEmpty) { + t.Fatalf("got %v, want ErrTargetNotEmpty", err) + } + if countIn(t, client, target, "servers") != 1 { + t.Fatal("a refused restore modified the target") + } +} + +func TestRestoreForceReplaces(t *testing.T) { + client, _ := testDB(t) + archive := archiveOf(t, client, validKeyHex, false) + _, target := testDB(t) + if _, err := client.Database(target).Collection("servers"). + InsertOne(context.Background(), bson.M{"name": "existing"}); err != nil { + t.Fatalf("seed target: %v", err) + } + + if _, err := Restore(context.Background(), RestoreOptions{ + Client: client, Database: target, Archive: archive, KeyHex: validKeyHex, Force: true, + }); err != nil { + t.Fatalf("Restore --force: %v", err) + } + if got := countIn(t, client, target, "servers"); got != 2 { + t.Fatalf("servers has %d documents, want 2; force must drop, not merge", got) + } + n, err := client.Database(target).Collection("servers"). + CountDocuments(context.Background(), bson.M{"name": "existing"}) + if err != nil { + t.Fatalf("count: %v", err) + } + if n != 0 { + t.Fatal("the pre-existing document survived --force") + } +} + +func TestRestoreRefusesKeyMismatch(t *testing.T) { + client, _ := testDB(t) + archive := archiveOf(t, client, validKeyHex, false) + _, target := testDB(t) + other := "0000000000000000000000000000000000000000000000000000000000000002" + + _, err := Restore(context.Background(), RestoreOptions{ + Client: client, Database: target, Archive: archive, KeyHex: other, + }) + if !errors.Is(err, ErrKeyMismatch) { + t.Fatalf("got %v, want ErrKeyMismatch", err) + } + names, err := client.Database(target).ListCollectionNames(context.Background(), bson.M{}) + if err != nil { + t.Fatalf("list: %v", err) + } + if len(names) != 0 { + t.Fatalf("a refused restore wrote %v", names) + } +} + +func TestRestoreRefusesWhenArchiveHasKeyAndEnvironmentDoesNot(t *testing.T) { + client, _ := testDB(t) + archive := archiveOf(t, client, validKeyHex, false) + _, target := testDB(t) + + _, err := Restore(context.Background(), RestoreOptions{ + Client: client, Database: target, Archive: archive, + }) + if !errors.Is(err, ErrNoKey) { + t.Fatalf("got %v, want ErrNoKey", err) + } +} + +func TestRestoreIgnoreKeyMismatchWarnsAndProceeds(t *testing.T) { + client, _ := testDB(t) + archive := archiveOf(t, client, validKeyHex, false) + _, target := testDB(t) + other := "0000000000000000000000000000000000000000000000000000000000000002" + + var warnings []string + if _, err := Restore(context.Background(), RestoreOptions{ + Client: client, Database: target, Archive: archive, + KeyHex: other, IgnoreKeyMismatch: true, + Warn: func(s string) { warnings = append(warnings, s) }, + }); err != nil { + t.Fatalf("Restore: %v", err) + } + joined := strings.Join(warnings, "\n") + for _, name := range CiphertextCollections() { + if !strings.Contains(joined, name) { + t.Fatalf("warning does not name %s\ngot:\n%s", name, joined) + } + } + if countIn(t, client, target, "servers") != 2 { + t.Fatal("restore did not proceed") + } +} + +func TestRestoreNullFingerprintIsReportedNotAssumed(t *testing.T) { + client, _ := testDB(t) + archive := archiveOf(t, client, "", true) + _, target := testDB(t) + + var warnings []string + if _, err := Restore(context.Background(), RestoreOptions{ + Client: client, Database: target, Archive: archive, KeyHex: validKeyHex, + Warn: func(s string) { warnings = append(warnings, s) }, + }); err != nil { + t.Fatalf("Restore: %v", err) + } + if !strings.Contains(strings.Join(warnings, "\n"), "no key fingerprint") { + t.Fatalf("a null fingerprint must be reported, got: %v", warnings) + } +} + +func TestRestoreReplaysIndexes(t *testing.T) { + client, _ := testDB(t) + ctx := context.Background() + _, srcDB := testDB(t) + seed(t, client, srcDB) + if _, err := client.Database(srcDB).Collection("servers").Indexes(). + CreateOne(ctx, mongo.IndexModel{Keys: bson.D{{Key: "name", Value: 1}}}); err != nil { + t.Fatalf("create index: %v", err) + } + path, _ := dumpToFile(t, DumpOptions{Client: client, Database: srcDB, KeyHex: validKeyHex}) + archive, err := Open(path) + if err != nil { + t.Fatalf("Open: %v", err) + } + defer archive.Close() + + _, target := testDB(t) + res, err := Restore(ctx, RestoreOptions{ + Client: client, Database: target, Archive: archive, KeyHex: validKeyHex, + }) + if err != nil { + t.Fatalf("Restore: %v", err) + } + + cur, err := client.Database(target).Collection("servers").Indexes().List(ctx) + if err != nil { + t.Fatalf("list indexes: %v", err) + } + var specs []bson.M + if err := cur.All(ctx, &specs); err != nil { + t.Fatalf("read indexes: %v", err) + } + found := false + for _, s := range specs { + if s["name"] == "name_1" { + found = true + } + } + if !found { + t.Fatalf("index name_1 not replayed; got %v", specs) + } + for _, c := range res.Collections { + if c.Name == "servers" && c.Indexes < 1 { + t.Fatal("result reports no indexes created for servers") + } + } +} + +func TestRestoreAbortsOnUniqueIndexViolation(t *testing.T) { + client, _ := testDB(t) + ctx := context.Background() + _, srcDB := testDB(t) + + // Two documents that will collide once the unique index is replayed. The + // index is created after the documents so the source database itself never + // enforces it, which is exactly the shape of a corrupted archive. + if _, err := client.Database(srcDB).Collection("users").InsertMany(ctx, []any{ + bson.M{"email": "a@example.com"}, + bson.M{"email": "a@example.com"}, + }); err != nil { + t.Fatalf("insert: %v", err) + } + path, _ := dumpToFile(t, DumpOptions{Client: client, Database: srcDB, KeyHex: validKeyHex}) + + // Inject a unique index spec into the archive by dumping a second database + // that has the index but no rows, then restoring the first archive over a + // target that already carries the index. + _, target := testDB(t) + if _, err := client.Database(target).Collection("users").Indexes().CreateOne(ctx, + mongo.IndexModel{ + Keys: bson.D{{Key: "email", Value: 1}}, + Options: options.Index().SetUnique(true).SetName("email_1"), + }); err != nil { + t.Fatalf("create unique index: %v", err) + } + + archive, err := Open(path) + if err != nil { + t.Fatalf("Open: %v", err) + } + defer archive.Close() + + _, err = Restore(ctx, RestoreOptions{ + Client: client, Database: target, Archive: archive, KeyHex: validKeyHex, Force: false, + }) + if err == nil { + t.Fatal("restore into a target holding a violated unique index succeeded") + } +} +``` + +Add `"go.mongodb.org/mongo-driver/v2/mongo/options"` to this file's imports. + +- [ ] **Step 2: Run the test and verify it fails** + +Run: `cd shared && MONGO_TEST_URI=mongodb://localhost:27017 go test ./backup/... -run Restore` +Expected: FAIL — `undefined: Restore`. + +- [ ] **Step 3: Write the implementation** + +Create `shared/backup/restore.go`: + +```go +package backup + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "io" + "sort" + "strings" + + "go.mongodb.org/mongo-driver/v2/bson" + "go.mongodb.org/mongo-driver/v2/mongo" + "go.mongodb.org/mongo-driver/v2/mongo/options" +) + +// BatchSize is how many documents are inserted per bulk write. +const BatchSize = 1000 + +// ErrTargetNotEmpty is returned when the target database already holds data and +// Force was not set. +var ErrTargetNotEmpty = errors.New("target database is not empty") + +// ErrKeyMismatch is returned when the archive's key fingerprint does not match +// the key supplied. +var ErrKeyMismatch = errors.New("KEY_ENCRYPTION_KEY does not match the archive") + +// ErrIndexBuild is returned when a unique index in the archive cannot be built +// on the restored data. +var ErrIndexBuild = errors.New("index could not be built on the restored data") + +// RestoredCollection is what one collection's restore produced. +type RestoredCollection struct { + Name string + Documents int64 + Indexes int +} + +// RestoreResult is the summary a caller prints. +type RestoreResult struct { + Collections []RestoredCollection +} + +// RestoreOptions configures one restore. +type RestoreOptions struct { + Client *mongo.Client + Database string + Archive *Reader + + // Force drops each collection in the archive before loading it. Without it + // a non-empty target is refused. + Force bool + + KeyHex string + + // IgnoreKeyMismatch proceeds past a fingerprint mismatch, having first + // warned which collections will hold unreadable ciphertext afterwards. + IgnoreKeyMismatch bool + + // Warn receives operator-facing warnings. A nil Warn discards them. + Warn func(string) +} + +func (o RestoreOptions) warn(format string, args ...any) { + if o.Warn != nil { + o.Warn(fmt.Sprintf(format, args...)) + } +} + +// Restore loads an archive into a database. +// +// The order is fixed and every check that can refuse does so before the first +// write: format, checksums (done by Open), key policy, then target inspection. +// A restore that has begun writing and then fails leaves a partial database +// which the next run refuses to touch, which is correct — the alternative is a +// silent merge, and merging two control planes reconciles nothing. +func Restore(ctx context.Context, opt RestoreOptions) (RestoreResult, error) { + m := opt.Archive.Manifest() + + if err := checkKey(m, opt); err != nil { + return RestoreResult{}, err + } + if err := checkTarget(ctx, opt); err != nil { + return RestoreResult{}, err + } + if len(m.Excluded) > 0 { + opt.warn("this archive excluded %s; those collections will be empty after the restore", + strings.Join(m.Excluded, ", ")) + } + opt.warn("Redis is not restored. Sessions are the only state it holds, so everyone signs in again.") + + res := RestoreResult{} + for _, entry := range m.Collections { + rc, err := restoreCollection(ctx, opt, entry) + if err != nil { + return res, err + } + res.Collections = append(res.Collections, rc) + } + return res, nil +} + +// checkKey applies the fingerprint policy. +func checkKey(m Manifest, opt RestoreOptions) error { + if m.KeyFingerprint == nil { + opt.warn("this archive carries no key fingerprint, so nothing here proves your " + + "KEY_ENCRYPTION_KEY opens its ciphertext") + return nil + } + if opt.KeyHex == "" { + return fmt.Errorf("%w: the archive records a key fingerprint, so a key is required "+ + "(pass --ignore-key-mismatch only if you accept unreadable secrets)", ErrNoKey) + } + got, err := FingerprintHex(opt.KeyHex) + if err != nil { + return err + } + if got == *m.KeyFingerprint { + return nil + } + if !opt.IgnoreKeyMismatch { + return fmt.Errorf("%w: archive fingerprint %s, your key fingerprints as %s", + ErrKeyMismatch, *m.KeyFingerprint, got) + } + opt.warn("proceeding past a key mismatch: ciphertext in %s will be permanently unreadable", + strings.Join(CiphertextCollections(), ", ")) + return nil +} + +// checkTarget refuses a non-empty database unless Force was set. +func checkTarget(ctx context.Context, opt RestoreOptions) error { + db := opt.Client.Database(opt.Database) + names, err := db.ListCollectionNames(ctx, bson.M{}) + if err != nil { + return fmt.Errorf("inspect target: %w", err) + } + if len(names) == 0 || opt.Force { + return nil + } + sort.Strings(names) + + var found []string + for _, n := range names { + count, err := db.Collection(n).CountDocuments(ctx, bson.M{}) + if err != nil { + return fmt.Errorf("count %s: %w", n, err) + } + found = append(found, fmt.Sprintf("%s (%d)", n, count)) + } + return fmt.Errorf("%w: %s holds %s", ErrTargetNotEmpty, opt.Database, strings.Join(found, ", ")) +} + +func restoreCollection(ctx context.Context, opt RestoreOptions, entry CollectionEntry) (RestoredCollection, error) { + coll := opt.Client.Database(opt.Database).Collection(entry.Name) + if opt.Force { + if err := coll.Drop(ctx); err != nil { + return RestoredCollection{}, fmt.Errorf("drop %s: %w", entry.Name, err) + } + } + + written, err := insertDocuments(ctx, opt, coll, entry) + if err != nil { + return RestoredCollection{}, err + } + indexes, err := replayIndexes(ctx, opt, coll, entry.Name) + if err != nil { + return RestoredCollection{}, err + } + return RestoredCollection{Name: entry.Name, Documents: written, Indexes: indexes}, nil +} + +func insertDocuments(ctx context.Context, opt RestoreOptions, coll *mongo.Collection, entry CollectionEntry) (int64, error) { + rc, err := opt.Archive.OpenCollection(entry.Name) + if err != nil { + return 0, fmt.Errorf("open %s in archive: %w", entry.Name, err) + } + defer rc.Close() + + raw, err := io.ReadAll(rc) + if err != nil { + return 0, fmt.Errorf("read %s: %w", entry.Name, err) + } + + var written int64 + batch := make([]any, 0, BatchSize) + flush := func() error { + if len(batch) == 0 { + return nil + } + if _, err := coll.InsertMany(ctx, batch, options.InsertMany().SetOrdered(false)); err != nil { + return fmt.Errorf("insert into %s: %w", entry.Name, err) + } + written += int64(len(batch)) + batch = batch[:0] + return nil + } + + for len(raw) > 0 { + doc, rest, err := splitBSON(raw) + if err != nil { + return 0, fmt.Errorf("%s: %w", entry.Name, err) + } + batch = append(batch, doc) + raw = rest + if len(batch) == BatchSize { + if err := flush(); err != nil { + return 0, err + } + } + } + if err := flush(); err != nil { + return 0, err + } + return written, nil +} + +// splitBSON peels one document off the front of a concatenated BSON stream. A +// BSON document declares its own length in its first four bytes. +func splitBSON(raw []byte) (bson.Raw, []byte, error) { + if len(raw) < 4 { + return nil, nil, fmt.Errorf("truncated BSON: %d trailing bytes", len(raw)) + } + n := int(int32(raw[0]) | int32(raw[1])<<8 | int32(raw[2])<<16 | int32(raw[3])<<24) + if n < 5 || n > len(raw) { + return nil, nil, fmt.Errorf("BSON document declares length %d with %d bytes remaining", n, len(raw)) + } + return bson.Raw(raw[:n]), raw[n:], nil +} + +// replayIndexes recreates the archived indexes. +// +// A unique index that will not build means the restored data violates it, and +// the unique indexes here — (instance_id, email), instance slug, settings +// instance, the ESO token hash — are tenant-isolation properties rather than +// optimisations. That aborts. A non-unique index failing is a performance +// problem and warns. +func replayIndexes(ctx context.Context, opt RestoreOptions, coll *mongo.Collection, name string) (int, error) { + raw, err := opt.Archive.IndexesJSON(name) + if err != nil { + return 0, fmt.Errorf("read index specs for %s: %w", name, err) + } + if len(raw) == 0 { + return 0, nil + } + var specs []map[string]any + if err := json.Unmarshal(raw, &specs); err != nil { + return 0, fmt.Errorf("parse index specs for %s: %w", name, err) + } + + created := 0 + for _, spec := range specs { + model, indexName, unique, ok := indexModelFrom(spec) + if !ok { + continue + } + if _, err := coll.Indexes().CreateOne(ctx, model); err != nil { + if unique { + return created, fmt.Errorf("%w: %s on %s: %v", ErrIndexBuild, indexName, name, err) + } + opt.warn("index %s on %s was not created: %v", indexName, name, err) + continue + } + created++ + } + return created, nil +} + +// indexModelFrom converts one archived index specification into a model. +// The _id_ index is skipped: MongoDB creates it itself and refuses an explicit +// attempt to create it. +func indexModelFrom(spec map[string]any) (mongo.IndexModel, string, bool, bool) { + name, _ := spec["name"].(string) + if name == "_id_" { + return mongo.IndexModel{}, name, false, false + } + keys, ok := spec["key"].(map[string]any) + if !ok || len(keys) == 0 { + return mongo.IndexModel{}, name, false, false + } + + // JSON objects do not preserve order but compound index key order is + // significant, so the field order recorded by the server is recovered from + // the spec's own ordering where available and sorted otherwise. bson.M + // round-trips through json as a map; the archive therefore stores the key + // document and this reconstructs a deterministic bson.D from it. + fields := make([]string, 0, len(keys)) + for k := range keys { + fields = append(fields, k) + } + sort.Strings(fields) + d := make(bson.D, 0, len(fields)) + for _, f := range fields { + d = append(d, bson.E{Key: f, Value: keys[f]}) + } + + opts := options.Index().SetName(name) + unique := false + if u, ok := spec["unique"].(bool); ok && u { + unique = true + opts = opts.SetUnique(true) + } + if s, ok := spec["sparse"].(bool); ok && s { + opts = opts.SetSparse(true) + } + if e, ok := spec["expireAfterSeconds"].(float64); ok { + opts = opts.SetExpireAfterSeconds(int32(e)) + } + return mongo.IndexModel{Keys: d, Options: opts}, name, unique, true +} +``` + +- [ ] **Step 4: Run the test and verify it passes** + +Run: `cd shared && MONGO_TEST_URI=mongodb://localhost:27017 go test ./backup/... -run Restore -v` +Expected: PASS, eight tests. + +- [ ] **Step 5: Warn on a MongoDB major version gap** + +The manifest records the source server's version so a restore onto a +substantially older or newer server is visible rather than discovered later. +Add to `Restore`, immediately after the `opt.warn` about Redis: + +```go + if err := warnVersionGap(ctx, opt, m); err != nil { + return res, err + } +``` + +And the function: + +```go +// warnVersionGap reports a major version difference between the server that +// produced the archive and the one receiving it. It warns rather than refuses: +// restoring across a major version is a normal part of an upgrade, and a tool +// that refused would be blocking the migration it exists to make safe. +func warnVersionGap(ctx context.Context, opt RestoreOptions, m Manifest) error { + if m.MongoServerVersion == "" { + return nil + } + target, err := mongoServerVersion(ctx, opt.Client) + if err != nil { + return err + } + if majorOf(m.MongoServerVersion) != majorOf(target) { + opt.warn("this archive came from MongoDB %s and you are restoring onto %s", + m.MongoServerVersion, target) + } + return nil +} + +func majorOf(version string) string { + if i := strings.IndexByte(version, '.'); i >= 0 { + return version[:i] + } + return version +} +``` + +Add a test to `restore_test.go` asserting the same-version case is silent: + +```go +func TestRestoreSameVersionDoesNotWarnAboutIt(t *testing.T) { + client, _ := testDB(t) + archive := archiveOf(t, client, validKeyHex, false) + _, target := testDB(t) + + var warnings []string + if _, err := Restore(context.Background(), RestoreOptions{ + Client: client, Database: target, Archive: archive, KeyHex: validKeyHex, + Warn: func(s string) { warnings = append(warnings, s) }, + }); err != nil { + t.Fatalf("Restore: %v", err) + } + for _, w := range warnings { + if strings.Contains(w, "you are restoring onto") { + t.Fatalf("same-version restore warned about a version gap: %s", w) + } + } +} +``` + +Run: `cd shared && MONGO_TEST_URI=mongodb://localhost:27017 go test ./backup/... -run Restore` +Expected: PASS. + +- [ ] **Step 6: Note the compound-index ordering limitation in the plan's own terms** + +`indexModelFrom` sorts key fields alphabetically because JSON objects do not +preserve order. For a compound index whose field order differs from alphabetical +this produces a functionally different index. Add a test that documents the +current behaviour so a later change is deliberate: + +```go +func TestCompoundIndexKeyOrderIsAlphabetical(t *testing.T) { + model, name, unique, ok := indexModelFrom(map[string]any{ + "name": "b_1_a_1", + "key": map[string]any{"b": float64(1), "a": float64(1)}, + }) + if !ok { + t.Fatal("spec rejected") + } + if unique { + t.Fatal("index reported as unique") + } + if name != "b_1_a_1" { + t.Fatalf("name %q", name) + } + keys, isD := model.Keys.(bson.D) + if !isD { + t.Fatalf("keys are %T, want bson.D", model.Keys) + } + // Documents current behaviour: field order is alphabetical, not the order + // the server reported. Storing the key document as raw BSON in the archive + // instead of JSON would fix this and is the change to make if compound + // index order ever matters here. + if keys[0].Key != "a" || keys[1].Key != "b" { + t.Fatalf("got %v", keys) + } +} +``` + +Run: `cd shared && go test ./backup/... -run CompoundIndex` +Expected: PASS. + +- [ ] **Step 7: Run the whole package** + +Run: `cd shared && MONGO_TEST_URI=mongodb://localhost:27017 go test ./backup/...` +Expected: PASS. + +- [ ] **Step 8: Commit** + +```bash +git add shared/backup/restore.go shared/backup/restore_test.go +git commit -m "feat: Add the backup restore + +Every refusal happens before the first write: format, checksums, key +policy, then target inspection. A unique index that will not build aborts, +because the unique indexes here are tenant-isolation properties rather +than optimisations." +``` + +--- + +### Task 7: Verify + +**Files:** +- Create: `shared/backup/verify.go` +- Create: `shared/backup/verify_test.go` + +**Interfaces:** +- Consumes: `Reader`, `Manifest`, `FingerprintHex`, `cryptobox.Open`, `CiphertextCollections`. +- Produces: + - `backup.VerifyOptions{Archive *Reader; KeyHex string; Client *mongo.Client; Database string}` + - `backup.VerifyReport{ArchiveFingerprint *string; KeyFingerprint *string; KeyMatchesArchive bool; ProbeAttempted bool; ProbeCollection string; ProbeDecrypted bool; Problems []string}` + - `backup.Verify(ctx context.Context, opt VerifyOptions) (VerifyReport, error)` + - `(VerifyReport).OK() bool` + +- [ ] **Step 1: Write the failing test** + +Create `shared/backup/verify_test.go`: + +```go +package backup + +import ( + "context" + "testing" + + "gitea.hostxtra.co.uk/mrhid6/vantage/shared/cryptobox" + "go.mongodb.org/mongo-driver/v2/bson" +) + +func TestVerifyMatchingKey(t *testing.T) { + client, _ := testDB(t) + archive := archiveOf(t, client, validKeyHex, false) + + rep, err := Verify(context.Background(), VerifyOptions{Archive: archive, KeyHex: validKeyHex}) + if err != nil { + t.Fatalf("Verify: %v", err) + } + if !rep.KeyMatchesArchive { + t.Fatal("matching key reported as a mismatch") + } + if rep.ProbeAttempted { + t.Fatal("probe ran with no client supplied") + } + if !rep.OK() { + t.Fatalf("report not OK: %v", rep.Problems) + } +} + +func TestVerifyMismatchedKeyIsNotOK(t *testing.T) { + client, _ := testDB(t) + archive := archiveOf(t, client, validKeyHex, false) + other := "0000000000000000000000000000000000000000000000000000000000000002" + + rep, err := Verify(context.Background(), VerifyOptions{Archive: archive, KeyHex: other}) + if err != nil { + t.Fatalf("Verify: %v", err) + } + if rep.KeyMatchesArchive { + t.Fatal("mismatched key reported as matching") + } + if rep.OK() { + t.Fatal("a mismatch must not report OK") + } +} + +func TestVerifyProbeDecryptsLiveCiphertext(t *testing.T) { + client, dbName := testDB(t) + ctx := context.Background() + + key, err := ParseKey(validKeyHex) + if err != nil { + t.Fatalf("ParseKey: %v", err) + } + sealed, err := cryptobox.Seal(key, "s3cret") + if err != nil { + t.Fatalf("Seal: %v", err) + } + if _, err := client.Database(dbName).Collection("secrets").InsertOne(ctx, bson.M{ + "instance_id": "i1", + "values": bson.M{"TOKEN": sealed}, + }); err != nil { + t.Fatalf("insert: %v", err) + } + + path, _ := dumpToFile(t, DumpOptions{Client: client, Database: dbName, KeyHex: validKeyHex}) + archive, err := Open(path) + if err != nil { + t.Fatalf("Open: %v", err) + } + defer archive.Close() + + rep, err := Verify(ctx, VerifyOptions{ + Archive: archive, KeyHex: validKeyHex, Client: client, Database: dbName, + }) + if err != nil { + t.Fatalf("Verify: %v", err) + } + if !rep.ProbeAttempted { + t.Fatal("probe did not run with a client supplied") + } + if !rep.ProbeDecrypted { + t.Fatalf("probe failed to decrypt live ciphertext: %v", rep.Problems) + } + if rep.ProbeCollection != "secrets" { + t.Fatalf("probe collection %q, want secrets", rep.ProbeCollection) + } + if !rep.OK() { + t.Fatalf("report not OK: %v", rep.Problems) + } +} + +func TestVerifyProbeFailsWithWrongKey(t *testing.T) { + client, dbName := testDB(t) + ctx := context.Background() + + key, err := ParseKey(validKeyHex) + if err != nil { + t.Fatalf("ParseKey: %v", err) + } + sealed, err := cryptobox.Seal(key, "s3cret") + if err != nil { + t.Fatalf("Seal: %v", err) + } + if _, err := client.Database(dbName).Collection("secrets").InsertOne(ctx, bson.M{ + "values": bson.M{"TOKEN": sealed}, + }); err != nil { + t.Fatalf("insert: %v", err) + } + + other := "0000000000000000000000000000000000000000000000000000000000000002" + path, _ := dumpToFile(t, DumpOptions{Client: client, Database: dbName, KeyHex: other}) + archive, err := Open(path) + if err != nil { + t.Fatalf("Open: %v", err) + } + defer archive.Close() + + rep, err := Verify(ctx, VerifyOptions{ + Archive: archive, KeyHex: other, Client: client, Database: dbName, + }) + if err != nil { + t.Fatalf("Verify: %v", err) + } + if rep.ProbeDecrypted { + t.Fatal("probe decrypted with the wrong key") + } + if rep.OK() { + t.Fatal("a failed probe must not report OK") + } +} + +func TestVerifyProbeAbsentCiphertextIsNotAFailure(t *testing.T) { + client, dbName := testDB(t) + seed(t, client, dbName) + + path, _ := dumpToFile(t, DumpOptions{Client: client, Database: dbName, KeyHex: validKeyHex}) + archive, err := Open(path) + if err != nil { + t.Fatalf("Open: %v", err) + } + defer archive.Close() + + rep, err := Verify(context.Background(), VerifyOptions{ + Archive: archive, KeyHex: validKeyHex, Client: client, Database: dbName, + }) + if err != nil { + t.Fatalf("Verify: %v", err) + } + if rep.ProbeAttempted { + t.Fatal("probe claims to have run against a database with no ciphertext") + } + if !rep.OK() { + t.Fatalf("a database storing no secrets must still verify: %v", rep.Problems) + } +} +``` + +- [ ] **Step 2: Run the test and verify it fails** + +Run: `cd shared && MONGO_TEST_URI=mongodb://localhost:27017 go test ./backup/... -run Verify` +Expected: FAIL — `undefined: Verify`. + +- [ ] **Step 3: Write the implementation** + +Create `shared/backup/verify.go`: + +```go +package backup + +import ( + "context" + "fmt" + + "gitea.hostxtra.co.uk/mrhid6/vantage/shared/cryptobox" + "go.mongodb.org/mongo-driver/v2/bson" + "go.mongodb.org/mongo-driver/v2/mongo" +) + +// VerifyOptions configures a verification. Client and Database are optional; +// supplying them turns on the live probe. +type VerifyOptions struct { + Archive *Reader + KeyHex string + Client *mongo.Client + Database string +} + +// VerifyReport is what verify found. +type VerifyReport struct { + ArchiveFingerprint *string + KeyFingerprint *string + KeyMatchesArchive bool + + // ProbeAttempted is false when no client was supplied, and also when the + // database holds no ciphertext to probe. + ProbeAttempted bool + ProbeCollection string + ProbeDecrypted bool + + Problems []string +} + +// OK reports whether this archive is usable with the key in hand. +func (r VerifyReport) OK() bool { return len(r.Problems) == 0 } + +func (r *VerifyReport) problem(format string, args ...any) { + r.Problems = append(r.Problems, fmt.Sprintf(format, args...)) +} + +// Verify checks an already-opened archive against the key in hand and, when a +// client is supplied, against a live database. +// +// Open has already verified every member's checksum, so integrity is not +// rechecked here. What this adds is the question an operator actually has: +// will the key I hold open the data this archive carries. A fingerprint +// comparison proves two archives agree; only the probe proves the key opens +// real ciphertext. +func Verify(ctx context.Context, opt VerifyOptions) (VerifyReport, error) { + m := opt.Archive.Manifest() + rep := VerifyReport{ArchiveFingerprint: m.KeyFingerprint} + + if opt.KeyHex != "" { + fp, err := FingerprintHex(opt.KeyHex) + if err != nil { + return rep, err + } + rep.KeyFingerprint = &fp + } + + switch { + case m.KeyFingerprint == nil && rep.KeyFingerprint == nil: + rep.problem("neither the archive nor this environment names a key; nothing here " + + "proves the archive's ciphertext can ever be read") + case m.KeyFingerprint == nil: + rep.problem("the archive carries no key fingerprint, so it cannot be matched " + + "against the key you hold") + case rep.KeyFingerprint == nil: + rep.problem("KEY_ENCRYPTION_KEY is not set, so the archive's fingerprint %s " + + "cannot be checked against anything") + case *m.KeyFingerprint == *rep.KeyFingerprint: + rep.KeyMatchesArchive = true + default: + rep.problem("key mismatch: archive fingerprint %s, your key fingerprints as %s", + *m.KeyFingerprint, *rep.KeyFingerprint) + } + + if opt.Client == nil || opt.Database == "" || opt.KeyHex == "" { + return rep, nil + } + if err := probe(ctx, opt, &rep); err != nil { + return rep, err + } + return rep, nil +} + +// probe reads one ciphertext field from the live database and tries to open it. +func probe(ctx context.Context, opt VerifyOptions, rep *VerifyReport) error { + key, err := ParseKey(opt.KeyHex) + if err != nil { + return err + } + for _, coll := range CiphertextCollections() { + ciphertext, ok, err := findCiphertext(ctx, opt.Client.Database(opt.Database), coll) + if err != nil { + return err + } + if !ok { + continue + } + rep.ProbeAttempted = true + rep.ProbeCollection = coll + if _, err := cryptobox.Open(key, ciphertext); err != nil { + rep.problem("the key in hand does not decrypt live ciphertext in %s", coll) + return nil + } + rep.ProbeDecrypted = true + return nil + } + // No ciphertext anywhere is an ordinary state — a deployment that has + // stored no secrets, keys or SSO configuration yet — and is not a failure. + return nil +} + +// ciphertextFields names, per collection, the fields that hold hex ciphertext. +// A value is a candidate only if it is a hex string long enough to carry a GCM +// nonce and tag, which is what keeps this from probing a plaintext field. +var ciphertextFields = map[string][]string{ + "keys": {"private_key_enc", "passphrase_enc"}, + "secrets": {"values"}, + "auth_providers": {"client_secret_enc"}, + "console_sessions": {"rdp_password_enc", "vnc_password_enc"}, + "settings": {"secrets_token_hash_enc"}, +} + +func findCiphertext(ctx context.Context, db *mongo.Database, coll string) (string, bool, error) { + fields, ok := ciphertextFields[coll] + if !ok { + return "", false, nil + } + cur, err := db.Collection(coll).Find(ctx, bson.M{}) + if err != nil { + return "", false, fmt.Errorf("probe %s: %w", coll, err) + } + defer cur.Close(ctx) + + for cur.Next(ctx) { + var doc bson.M + if err := cur.Decode(&doc); err != nil { + return "", false, fmt.Errorf("probe %s: %w", coll, err) + } + for _, f := range fields { + if v, ok := looksLikeCiphertext(doc[f]); ok { + return v, true, nil + } + } + } + return "", false, cur.Err() +} + +// looksLikeCiphertext accepts a hex string long enough to be a sealed value, and +// descends one level into a map so secrets' values sub-document is reachable. +func looksLikeCiphertext(v any) (string, bool) { + switch t := v.(type) { + case string: + // 12-byte nonce plus a 16-byte tag is 56 hex characters before any + // plaintext at all, so anything shorter is not a sealed value. + if len(t) < 56 || !isHex(t) { + return "", false + } + return t, true + case bson.M: + for _, inner := range t { + if s, ok := looksLikeCiphertext(inner); ok { + return s, true + } + } + } + return "", false +} + +func isHex(s string) bool { + for _, c := range s { + switch { + case c >= '0' && c <= '9', c >= 'a' && c <= 'f', c >= 'A' && c <= 'F': + default: + return false + } + } + return true +} +``` + +- [ ] **Step 4: Fix the format-string bug the compiler will flag** + +`go vet` will report the `rep.problem("... fingerprint %s ...")` case that takes +no argument. Change that branch to: + +```go + case rep.KeyFingerprint == nil: + rep.problem("KEY_ENCRYPTION_KEY is not set, so the archive's fingerprint %s "+ + "cannot be checked against anything", *m.KeyFingerprint) +``` + +- [ ] **Step 5: Run the test and verify it passes** + +Run: `cd shared && MONGO_TEST_URI=mongodb://localhost:27017 go test ./backup/... -run Verify -v` +Expected: PASS, five tests. + +- [ ] **Step 6: Vet the package** + +Run: `cd shared && go vet ./backup/... ./cryptobox/...` +Expected: no output. + +- [ ] **Step 7: Commit** + +```bash +git add shared/backup/verify.go shared/backup/verify_test.go +git commit -m "feat: Add backup verify with a live decrypt probe + +A fingerprint comparison proves two archives agree about a key. Only +opening real ciphertext from the target proves the key in hand reads the +data, which is the question an operator actually has." +``` + +--- +### Task 8: The `vantagectl` module and cobra root + +**Files:** +- Create: `vantagectl/go.mod` +- Create: `vantagectl/main.go` +- Create: `vantagectl/internal/cmd/root.go` +- Create: `vantagectl/internal/cmd/root_test.go` +- Modify: `go.work` + +**Interfaces:** +- Consumes: nothing from `shared/backup` yet. +- Produces: + - `cmd.Execute(version string) error` + - `cmd.NewRoot(version string) *cobra.Command` + - `cmd.globalOpts{MongoURI, Database, KeyHex string}` populated by `cmd.resolveGlobals(c *cobra.Command) (*globalOpts, error)` + - `cmd.connect(ctx context.Context, g *globalOpts) (*mongo.Client, error)` + +- [ ] **Step 1: Create the module and wire it into the workspace** + +```bash +mkdir -p vantagectl/internal/cmd +cd vantagectl +cat > go.mod <<'MOD' +module gitea.hostxtra.co.uk/mrhid6/vantage/vantagectl + +go 1.26 + +replace gitea.hostxtra.co.uk/mrhid6/vantage/shared => ../shared +MOD +cd .. +``` + +Edit `go.work` so the `use` block reads: + +``` +use ( + ./admin + ./agent + ./server + ./shared + ./sitesvc + ./vantagectl +) +``` + +- [ ] **Step 2: Add the dependencies** + +```bash +cd vantagectl +go get github.com/spf13/cobra@latest +go get go.mongodb.org/mongo-driver/v2@v2.8.0 +go get golang.org/x/term@latest +go mod tidy +cd .. +``` + +Confirm `shared/go.mod` did **not** gain cobra: + +```bash +grep -c cobra shared/go.mod +``` +Expected: `0`. + +- [ ] **Step 3: Write the failing test** + +Create `vantagectl/internal/cmd/root_test.go`: + +```go +package cmd + +import ( + "bytes" + "strings" + "testing" +) + +func TestRootListsEverySubcommand(t *testing.T) { + root := NewRoot("test") + var buf bytes.Buffer + root.SetOut(&buf) + root.SetArgs([]string{"--help"}) + if err := root.Execute(); err != nil { + t.Fatalf("Execute: %v", err) + } + for _, want := range []string{"backup", "restore", "inspect", "verify"} { + if !strings.Contains(buf.String(), want) { + t.Fatalf("help does not mention %q:\n%s", want, buf.String()) + } + } +} + +func TestGlobalFlagsFallBackToEnvironment(t *testing.T) { + t.Setenv("MONGO_URI", "mongodb://env:27017") + t.Setenv("MONGO_DB", "envdb") + t.Setenv("KEY_ENCRYPTION_KEY", "envkey") + + root := NewRoot("test") + g, err := resolveGlobals(root) + if err != nil { + t.Fatalf("resolveGlobals: %v", err) + } + if g.MongoURI != "mongodb://env:27017" { + t.Fatalf("MongoURI %q", g.MongoURI) + } + if g.Database != "envdb" { + t.Fatalf("Database %q", g.Database) + } + if g.KeyHex != "envkey" { + t.Fatalf("KeyHex %q", g.KeyHex) + } +} + +func TestExplicitFlagsBeatEnvironment(t *testing.T) { + t.Setenv("MONGO_URI", "mongodb://env:27017") + t.Setenv("MONGO_DB", "envdb") + + root := NewRoot("test") + if err := root.PersistentFlags().Set("mongo-uri", "mongodb://flag:27017"); err != nil { + t.Fatalf("set flag: %v", err) + } + if err := root.PersistentFlags().Set("db", "flagdb"); err != nil { + t.Fatalf("set flag: %v", err) + } + g, err := resolveGlobals(root) + if err != nil { + t.Fatalf("resolveGlobals: %v", err) + } + if g.MongoURI != "mongodb://flag:27017" { + t.Fatalf("MongoURI %q; the flag must win over the environment", g.MongoURI) + } + if g.Database != "flagdb" { + t.Fatalf("Database %q", g.Database) + } +} + +func TestDatabaseFallsBackToURIPath(t *testing.T) { + t.Setenv("MONGO_URI", "mongodb://host:27017/fromuri") + root := NewRoot("test") + g, err := resolveGlobals(root) + if err != nil { + t.Fatalf("resolveGlobals: %v", err) + } + if g.Database != "fromuri" { + t.Fatalf("Database %q, want fromuri", g.Database) + } +} + +func TestMissingURIIsAnError(t *testing.T) { + t.Setenv("MONGO_URI", "") + root := NewRoot("test") + if _, err := resolveGlobals(root); err == nil { + t.Fatal("resolveGlobals accepted an empty MONGO_URI") + } +} + +func TestVersionIsReported(t *testing.T) { + root := NewRoot("1.2.3") + if root.Version != "1.2.3" { + t.Fatalf("Version %q", root.Version) + } +} +``` + +- [ ] **Step 4: Run the test and verify it fails** + +Run: `cd vantagectl && go test ./internal/cmd/...` +Expected: FAIL — `undefined: NewRoot`. + +- [ ] **Step 5: Write the root command** + +Create `vantagectl/internal/cmd/root.go`: + +```go +// Package cmd is vantagectl's command tree. +// +// It holds argument parsing and operator-facing output only. Everything it does +// to a database goes through shared/backup, which the server can also import. +package cmd + +import ( + "context" + "fmt" + "net/url" + "os" + "strings" + "time" + + "github.com/spf13/cobra" + "go.mongodb.org/mongo-driver/v2/mongo" + "go.mongodb.org/mongo-driver/v2/mongo/options" +) + +const connectTimeout = 30 * time.Second + +// globalOpts is what every subcommand needs. +type globalOpts struct { + MongoURI string + Database string + KeyHex string +} + +// NewRoot builds the command tree. +func NewRoot(version string) *cobra.Command { + root := &cobra.Command{ + Use: "vantagectl", + Short: "Back up and restore a Vantage control plane", + Version: version, + Long: "vantagectl backs up and restores the MongoDB database behind a Vantage\n" + + "control plane.\n\n" + + "It talks to MongoDB directly and never to the Vantage API, so it works\n" + + "against a control plane that is down, half-migrated, or gone.\n\n" + + "KEY_ENCRYPTION_KEY is never written into an archive. What an archive\n" + + "records is a fingerprint of it, so a restore can tell you that the key\n" + + "you hold is the wrong one before it writes a database nobody can read.", + SilenceUsage: true, + } + + f := root.PersistentFlags() + f.String("mongo-uri", "", "MongoDB connection string (env MONGO_URI)") + f.String("db", "", "database name (env MONGO_DB, or the URI path)") + + root.AddCommand(newBackupCmd(), newRestoreCmd(), newInspectCmd(), newVerifyCmd()) + return root +} + +// Execute runs the tree. +func Execute(version string) error { + return NewRoot(version).Execute() +} + +// resolveGlobals applies the environment fallback. +// +// Explicit flags win. The check is on Changed rather than on emptiness, so +// `--db ""` is an explicit empty value rather than an invitation to read the +// environment behind the operator's back. +func resolveGlobals(c *cobra.Command) (*globalOpts, error) { + root := c.Root() + f := root.PersistentFlags() + + uri, err := f.GetString("mongo-uri") + if err != nil { + return nil, err + } + if !f.Changed("mongo-uri") { + uri = os.Getenv("MONGO_URI") + } + if uri == "" { + return nil, fmt.Errorf("no MongoDB URI: pass --mongo-uri or set MONGO_URI") + } + + db, err := f.GetString("db") + if err != nil { + return nil, err + } + if !f.Changed("db") { + db = os.Getenv("MONGO_DB") + } + if db == "" { + db = databaseFromURI(uri) + } + if db == "" { + return nil, fmt.Errorf("no database name: pass --db, set MONGO_DB, or put one in the URI path") + } + + return &globalOpts{ + MongoURI: uri, + Database: db, + KeyHex: strings.TrimSpace(os.Getenv("KEY_ENCRYPTION_KEY")), + }, nil +} + +// databaseFromURI reads the database out of the URI path. sitesvc takes its +// database name this way too, so an operator who has configured one has +// configured both. +func databaseFromURI(uri string) string { + u, err := url.Parse(uri) + if err != nil { + return "" + } + return strings.Trim(u.Path, "/") +} + +// connect dials MongoDB and proves the connection before a caller commits to +// anything. +func connect(ctx context.Context, g *globalOpts) (*mongo.Client, error) { + client, err := mongo.Connect(options.Client().ApplyURI(g.MongoURI)) + if err != nil { + return nil, fmt.Errorf("connect to MongoDB: %w", err) + } + pingCtx, cancel := context.WithTimeout(ctx, connectTimeout) + defer cancel() + if err := client.Ping(pingCtx, nil); err != nil { + _ = client.Disconnect(context.Background()) + return nil, fmt.Errorf("MongoDB did not answer: %w", err) + } + return client, nil +} +``` + +Create `vantagectl/main.go`: + +```go +// Command vantagectl backs up and restores a Vantage control plane. +package main + +import ( + "fmt" + "os" + + "gitea.hostxtra.co.uk/mrhid6/vantage/vantagectl/internal/cmd" +) + +// Version is stamped at build time with -ldflags "-X main.Version=...". +var Version = "dev" + +func main() { + if err := cmd.Execute(Version); err != nil { + fmt.Fprintln(os.Stderr, "error:", err) + os.Exit(1) + } +} +``` + +- [ ] **Step 6: Add empty subcommand constructors so the package compiles** + +Create `vantagectl/internal/cmd/backup.go`, `restore.go`, `inspect.go` and +`verify.go`, each holding only its constructor for now. These are replaced in +full by Tasks 9 and 10. + +```go +package cmd + +import "github.com/spf13/cobra" + +func newBackupCmd() *cobra.Command { + return &cobra.Command{Use: "backup", Short: "Write an archive of the database"} +} +``` + +```go +package cmd + +import "github.com/spf13/cobra" + +func newRestoreCmd() *cobra.Command { + return &cobra.Command{Use: "restore ARCHIVE", Short: "Load an archive into a database"} +} +``` + +```go +package cmd + +import "github.com/spf13/cobra" + +func newInspectCmd() *cobra.Command { + return &cobra.Command{Use: "inspect ARCHIVE", Short: "Print an archive's manifest"} +} +``` + +```go +package cmd + +import "github.com/spf13/cobra" + +func newVerifyCmd() *cobra.Command { + return &cobra.Command{Use: "verify ARCHIVE", Short: "Check an archive against the key in hand"} +} +``` + +- [ ] **Step 7: Run the test and verify it passes** + +Run: `cd vantagectl && go test ./internal/cmd/...` +Expected: PASS, six tests. + +- [ ] **Step 8: Confirm the other modules are untouched** + +```bash +cd server && go build ./... && cd ../admin && go build ./... && cd ../sitesvc && go build ./... +git diff --stat server/go.sum admin/go.sum sitesvc/go.sum +``` +Expected: builds succeed, `git diff --stat` prints nothing — cobra stayed out of their module graphs. + +- [ ] **Step 9: Commit** + +```bash +git add go.work vantagectl +git commit -m "feat: Add the vantagectl module and its cobra root + +Its own module rather than a package under shared, so cobra and pflag stay +out of the module graphs of server, admin and sitesvc, which never use +them." +``` + +--- + +### Task 9: `backup` and `inspect` subcommands + +**Files:** +- Modify: `vantagectl/internal/cmd/backup.go` +- Modify: `vantagectl/internal/cmd/inspect.go` +- Create: `vantagectl/internal/cmd/inspect_test.go` + +**Interfaces:** +- Consumes: `resolveGlobals`, `connect`, `backup.Dump`, `backup.DumpOptions`, `backup.Open`, `backup.Manifest`. +- Produces: `cmd.archiveName(database string, at time.Time) string`, `cmd.renderManifest(w io.Writer, m backup.Manifest)`. + +- [ ] **Step 1: Write the failing test** + +Create `vantagectl/internal/cmd/inspect_test.go`: + +```go +package cmd + +import ( + "bytes" + "strings" + "testing" + "time" + + "gitea.hostxtra.co.uk/mrhid6/vantage/shared/backup" +) + +func TestArchiveNameIsSortableAndNamesTheDatabase(t *testing.T) { + at := time.Date(2026, 9, 7, 14, 30, 5, 0, time.UTC) + got := archiveName("vantage", at) + if !strings.HasPrefix(got, "vantage-backup-vantage-") { + t.Fatalf("name %q does not name the database", got) + } + if !strings.HasSuffix(got, ".tar.gz") { + t.Fatalf("name %q has the wrong extension", got) + } + if strings.ContainsAny(got, ":") { + t.Fatalf("name %q contains a colon, which Windows will not accept", got) + } + if !strings.Contains(got, "20260907") { + t.Fatalf("name %q does not carry a sortable date", got) + } +} + +func TestRenderManifestShowsWhatMatters(t *testing.T) { + fp := "ab12" + m := backup.Manifest{ + FormatVersion: backup.FormatVersion, + CreatedAt: time.Date(2026, 9, 7, 14, 0, 0, 0, time.UTC), + VantageVersion: "1.4.0", + Hostname: "ops-box", + MongoDB: "vantage", + MongoServerVersion: "7.0.5", + KeyFingerprint: &fp, + Collections: []backup.CollectionEntry{ + {Name: "servers", Documents: 12, Bytes: 4096}, + {Name: "keys", Documents: 3, Bytes: 900}, + }, + Excluded: []string{"audit_logs"}, + } + + var buf bytes.Buffer + renderManifest(&buf, m) + out := buf.String() + + for _, want := range []string{ + "vantage", "1.4.0", "ops-box", "7.0.5", "ab12", + "servers", "12", "keys", "audit_logs", "2026-09-07", + } { + if !strings.Contains(out, want) { + t.Fatalf("inspect output missing %q:\n%s", want, out) + } + } +} + +func TestRenderManifestFlagsAMissingFingerprint(t *testing.T) { + var buf bytes.Buffer + renderManifest(&buf, backup.Manifest{FormatVersion: backup.FormatVersion}) + out := buf.String() + if !strings.Contains(out, "none recorded") { + t.Fatalf("a null fingerprint must be called out, got:\n%s", out) + } + if !strings.Contains(out, "cannot be checked") { + t.Fatalf("a null fingerprint must explain the consequence, got:\n%s", out) + } +} +``` + +- [ ] **Step 2: Run the test and verify it fails** + +Run: `cd vantagectl && go test ./internal/cmd/... -run 'ArchiveName|RenderManifest'` +Expected: FAIL — `undefined: archiveName`. + +- [ ] **Step 3: Write `inspect`** + +Replace `vantagectl/internal/cmd/inspect.go`: + +```go +package cmd + +import ( + "fmt" + "io" + "strings" + "text/tabwriter" + + "gitea.hostxtra.co.uk/mrhid6/vantage/shared/backup" + "github.com/spf13/cobra" +) + +func newInspectCmd() *cobra.Command { + return &cobra.Command{ + Use: "inspect ARCHIVE", + Short: "Print an archive's manifest", + Long: "inspect reads an archive and prints what it holds. It contacts no\n" + + "database, so it is safe to run against an archive of unknown origin\n" + + "and is the fastest way to find out whether one is worth anything.", + Args: cobra.ExactArgs(1), + RunE: func(c *cobra.Command, args []string) error { + archive, err := backup.Open(args[0]) + if err != nil { + return err + } + defer archive.Close() + renderManifest(c.OutOrStdout(), archive.Manifest()) + return nil + }, + } +} + +// renderManifest prints a manifest for a human. +func renderManifest(w io.Writer, m backup.Manifest) { + fmt.Fprintf(w, "Created %s\n", m.CreatedAt.UTC().Format("2006-01-02 15:04:05 MST")) + fmt.Fprintf(w, "Database %s\n", m.MongoDB) + fmt.Fprintf(w, "MongoDB %s\n", m.MongoServerVersion) + fmt.Fprintf(w, "Written by vantagectl %s on %s\n", m.VantageVersion, m.Hostname) + fmt.Fprintf(w, "Format version %d\n", m.FormatVersion) + + if m.KeyFingerprint == nil { + fmt.Fprintf(w, "Key none recorded — this archive cannot be checked "+ + "against any KEY_ENCRYPTION_KEY\n") + } else { + fmt.Fprintf(w, "Key %s\n", *m.KeyFingerprint) + } + if len(m.Excluded) > 0 { + fmt.Fprintf(w, "Excluded %s\n", strings.Join(m.Excluded, ", ")) + } + + var docs, bytes int64 + for _, c := range m.Collections { + docs += c.Documents + bytes += c.Bytes + } + fmt.Fprintf(w, "\n%d collections, %d documents, %s\n\n", + len(m.Collections), docs, humanBytes(bytes)) + + tw := tabwriter.NewWriter(w, 0, 0, 2, ' ', 0) + fmt.Fprintln(tw, "COLLECTION\tDOCUMENTS\tSIZE") + for _, c := range m.Collections { + fmt.Fprintf(tw, "%s\t%d\t%s\n", c.Name, c.Documents, humanBytes(c.Bytes)) + } + tw.Flush() +} + +func humanBytes(n int64) string { + const unit = 1024 + if n < unit { + return fmt.Sprintf("%d B", n) + } + div, exp := int64(unit), 0 + for v := n / unit; v >= unit; v /= unit { + div *= unit + exp++ + } + return fmt.Sprintf("%.1f %cB", float64(n)/float64(div), "KMGTP"[exp]) +} +``` + +- [ ] **Step 4: Write `backup`** + +Replace `vantagectl/internal/cmd/backup.go`: + +```go +package cmd + +import ( + "context" + "fmt" + "io" + "os" + "path/filepath" + "time" + + "gitea.hostxtra.co.uk/mrhid6/vantage/shared/backup" + "github.com/spf13/cobra" +) + +func newBackupCmd() *cobra.Command { + var ( + out string + exclude []string + allowNoKey bool + ) + + c := &cobra.Command{ + Use: "backup", + Short: "Write an archive of the database", + Long: "backup writes every collection in the database to a gzipped tar\n" + + "archive, along with a fingerprint of KEY_ENCRYPTION_KEY.\n\n" + + "The key itself is never written. The fingerprint is what lets a later\n" + + "restore refuse rather than produce a database whose secrets nobody\n" + + "can read.\n\n" + + "Pass --out - to stream to stdout, which is how this composes with\n" + + "restic, age, or aws s3 cp -.", + Args: cobra.NoArgs, + RunE: func(c *cobra.Command, _ []string) error { + ctx := c.Context() + g, err := resolveGlobals(c) + if err != nil { + return err + } + client, err := connect(ctx, g) + if err != nil { + return err + } + defer client.Disconnect(context.Background()) + + w, closeOut, name, err := backupDestination(out, g.Database) + if err != nil { + return err + } + defer closeOut() + + m, err := backup.Dump(ctx, backup.DumpOptions{ + Client: client, + Database: g.Database, + Exclude: exclude, + KeyHex: g.KeyHex, + AllowNoKey: allowNoKey, + VantageVersion: c.Root().Version, + Out: w, + }) + if err != nil { + return err + } + + // Progress goes to stderr so --out - stays a clean pipe. + var docs int64 + for _, coll := range m.Collections { + docs += coll.Documents + } + fmt.Fprintf(c.ErrOrStderr(), "wrote %s: %d collections, %d documents\n", + name, len(m.Collections), docs) + if m.KeyFingerprint == nil { + fmt.Fprintln(c.ErrOrStderr(), + "warning: no key recorded; nothing in this archive proves its "+ + "ciphertext can ever be read") + } + return nil + }, + } + + c.Flags().StringVar(&out, "out", ".", "directory to write the archive into, or - for stdout") + c.Flags().StringSliceVar(&exclude, "exclude", nil, + "collections to leave out, comma separated (recorded in the manifest)") + c.Flags().BoolVar(&allowNoKey, "allow-no-key", false, + "back up without KEY_ENCRYPTION_KEY set; only for a deployment storing no encrypted data") + return c +} + +// backupDestination resolves --out to a writer, a closer and a name to print. +func backupDestination(out, database string) (io.Writer, func(), string, error) { + if out == "-" { + return os.Stdout, func() {}, "stdout", nil + } + name := archiveName(database, time.Now().UTC()) + path := filepath.Join(out, name) + f, err := os.OpenFile(path, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600) + if err != nil { + return nil, nil, "", fmt.Errorf("create %s: %w", path, err) + } + return f, func() { f.Close() }, path, nil +} + +// archiveName is sortable and carries no colon, because an operator will copy +// these onto a Windows share sooner or later and a colon is not a legal +// filename character there. +func archiveName(database string, at time.Time) string { + return fmt.Sprintf("vantage-backup-%s-%s.tar.gz", database, at.Format("20060102T150405Z")) +} +``` + +- [ ] **Step 5: Run the tests** + +Run: `cd vantagectl && go test ./internal/cmd/...` +Expected: PASS, nine tests. + +- [ ] **Step 6: Exercise both commands end to end against a real database** + +```bash +docker run -d --rm -p 27017:27017 --name vantage-test-mongo mongo:7 +cd vantagectl +export MONGO_URI=mongodb://localhost:27017 +export MONGO_DB=vantage_smoke +export KEY_ENCRYPTION_KEY=0000000000000000000000000000000000000000000000000000000000000001 +go run . backup --out /tmp +go run . inspect /tmp/vantage-backup-vantage_smoke-*.tar.gz +``` + +Expected: `backup` reports what it wrote; `inspect` prints the manifest with a +key fingerprint and a collection table. An empty database is fine — the point +here is that both commands run. + +Then confirm the refusal: + +```bash +KEY_ENCRYPTION_KEY= go run . backup --out /tmp +``` +Expected: exits non-zero with `KEY_ENCRYPTION_KEY is not set`. + +- [ ] **Step 7: Commit** + +```bash +git add vantagectl/internal/cmd/backup.go vantagectl/internal/cmd/inspect.go vantagectl/internal/cmd/inspect_test.go +git commit -m "feat: Add the vantagectl backup and inspect subcommands + +Progress output goes to stderr so --out - stays a clean pipe into restic, +age or aws s3 cp. Archive names carry no colon, because these get copied +onto Windows shares." +``` + +--- + +### Task 10: `restore` and `verify` subcommands + +**Files:** +- Modify: `vantagectl/internal/cmd/restore.go` +- Modify: `vantagectl/internal/cmd/verify.go` +- Create: `vantagectl/internal/cmd/restore_test.go` + +**Interfaces:** +- Consumes: `resolveGlobals`, `connect`, `backup.Open`, `backup.Restore`, `backup.RestoreOptions`, `backup.Verify`, `backup.VerifyOptions`. +- Produces: `cmd.confirmDestruction(in io.Reader, out io.Writer, isTTY bool, confirmDB, database string) error`, `cmd.ErrNotConfirmed`. + +- [ ] **Step 1: Write the failing test** + +Create `vantagectl/internal/cmd/restore_test.go`: + +```go +package cmd + +import ( + "bytes" + "errors" + "strings" + "testing" +) + +func TestConfirmDestructionNonTTYRequiresMatchingFlag(t *testing.T) { + var out bytes.Buffer + err := confirmDestruction(strings.NewReader(""), &out, false, "", "vantage") + if !errors.Is(err, ErrNotConfirmed) { + t.Fatalf("got %v, want ErrNotConfirmed", err) + } + if !strings.Contains(err.Error(), "--confirm-db vantage") { + t.Fatalf("the error must tell the operator exactly what to pass, got: %v", err) + } +} + +func TestConfirmDestructionNonTTYRejectsWrongDatabase(t *testing.T) { + var out bytes.Buffer + err := confirmDestruction(strings.NewReader(""), &out, false, "staging", "production") + if !errors.Is(err, ErrNotConfirmed) { + t.Fatalf("got %v, want ErrNotConfirmed", err) + } + if !strings.Contains(err.Error(), "production") { + t.Fatalf("the error must name the real target, got: %v", err) + } +} + +func TestConfirmDestructionNonTTYAcceptsMatchingFlag(t *testing.T) { + var out bytes.Buffer + if err := confirmDestruction(strings.NewReader(""), &out, false, "vantage", "vantage"); err != nil { + t.Fatalf("matching --confirm-db rejected: %v", err) + } +} + +func TestConfirmDestructionTTYRequiresTypedName(t *testing.T) { + var out bytes.Buffer + if err := confirmDestruction(strings.NewReader("vantage\n"), &out, true, "", "vantage"); err != nil { + t.Fatalf("typed name rejected: %v", err) + } + if !strings.Contains(out.String(), "vantage") { + t.Fatalf("the prompt must name the database, got: %s", out.String()) + } +} + +func TestConfirmDestructionTTYRejectsWrongTypedName(t *testing.T) { + var out bytes.Buffer + err := confirmDestruction(strings.NewReader("something else\n"), &out, true, "", "vantage") + if !errors.Is(err, ErrNotConfirmed) { + t.Fatalf("got %v, want ErrNotConfirmed", err) + } +} + +func TestConfirmDestructionTTYFlagSkipsThePrompt(t *testing.T) { + var out bytes.Buffer + if err := confirmDestruction(strings.NewReader(""), &out, true, "vantage", "vantage"); err != nil { + t.Fatalf("matching --confirm-db rejected on a TTY: %v", err) + } + if out.Len() != 0 { + t.Fatalf("--confirm-db must skip the prompt, got: %s", out.String()) + } +} +``` + +- [ ] **Step 2: Run the test and verify it fails** + +Run: `cd vantagectl && go test ./internal/cmd/... -run Confirm` +Expected: FAIL — `undefined: confirmDestruction`. + +- [ ] **Step 3: Write `restore`** + +Replace `vantagectl/internal/cmd/restore.go`: + +```go +package cmd + +import ( + "bufio" + "context" + "errors" + "fmt" + "io" + "os" + "strings" + "text/tabwriter" + + "gitea.hostxtra.co.uk/mrhid6/vantage/shared/backup" + "github.com/spf13/cobra" + "golang.org/x/term" +) + +// ErrNotConfirmed is returned when a destructive restore was not confirmed. +var ErrNotConfirmed = errors.New("restore not confirmed") + +func newRestoreCmd() *cobra.Command { + var ( + force bool + confirmDB string + ignoreKeyErr bool + ) + + c := &cobra.Command{ + Use: "restore ARCHIVE", + Short: "Load an archive into a database", + Long: "restore loads an archive into a MongoDB database.\n\n" + + "The target is expected to be empty. A database that already holds data\n" + + "is refused unless --force is given, which drops each collection in the\n" + + "archive before loading it. There are no merge semantics: merging two\n" + + "control planes reconciles nothing, and upserting would resurrect\n" + + "revoked keys and deleted users.\n\n" + + "Restore does not touch Redis. Sessions are all it holds, so everyone\n" + + "signs in again.", + Args: cobra.ExactArgs(1), + RunE: func(c *cobra.Command, args []string) error { + ctx := c.Context() + g, err := resolveGlobals(c) + if err != nil { + return err + } + + archive, err := backup.Open(args[0]) + if err != nil { + return err + } + defer archive.Close() + + if force { + isTTY := term.IsTerminal(int(os.Stdin.Fd())) + if err := confirmDestruction(c.InOrStdin(), c.OutOrStdout(), isTTY, + confirmDB, g.Database); err != nil { + return err + } + } + + client, err := connect(ctx, g) + if err != nil { + return err + } + defer client.Disconnect(context.Background()) + + res, err := backup.Restore(ctx, backup.RestoreOptions{ + Client: client, + Database: g.Database, + Archive: archive, + Force: force, + KeyHex: g.KeyHex, + IgnoreKeyMismatch: ignoreKeyErr, + Warn: func(s string) { + fmt.Fprintln(c.ErrOrStderr(), "warning:", s) + }, + }) + if err != nil { + return err + } + + tw := tabwriter.NewWriter(c.OutOrStdout(), 0, 0, 2, ' ', 0) + fmt.Fprintln(tw, "COLLECTION\tDOCUMENTS\tINDEXES") + for _, coll := range res.Collections { + fmt.Fprintf(tw, "%s\t%d\t%d\n", coll.Name, coll.Documents, coll.Indexes) + } + tw.Flush() + fmt.Fprintf(c.OutOrStdout(), "\nrestored %d collections into %s\n", + len(res.Collections), g.Database) + return nil + }, + } + + c.Flags().BoolVar(&force, "force", false, + "drop each collection in the archive before loading it") + c.Flags().StringVar(&confirmDB, "confirm-db", "", + "name of the database being overwritten; required with --force when there is no terminal") + c.Flags().BoolVar(&ignoreKeyErr, "ignore-key-mismatch", false, + "restore even though KEY_ENCRYPTION_KEY does not match the archive") + return c +} + +// confirmDestruction gates a --force restore. +// +// On a terminal the operator types the database name. Without one — a +// Kubernetes Job, a CI step, a cron entry — the same assurance comes from +// --confirm-db, whose value must equal the target. Naming the database in the +// argument means a copy-pasted command carries its intended target with it and +// cannot destroy a different one. +func confirmDestruction(in io.Reader, out io.Writer, isTTY bool, confirmDB, database string) error { + if confirmDB != "" { + if confirmDB != database { + return fmt.Errorf("%w: --confirm-db says %q but the target is %q", + ErrNotConfirmed, confirmDB, database) + } + return nil + } + if !isTTY { + return fmt.Errorf("%w: --force with no terminal needs --confirm-db %s", + ErrNotConfirmed, database) + } + + fmt.Fprintf(out, "This drops every collection in the archive from %q and reloads it.\n", database) + fmt.Fprintf(out, "Type the database name to continue: ") + + line, err := bufio.NewReader(in).ReadString('\n') + if err != nil && err != io.EOF { + return fmt.Errorf("%w: %v", ErrNotConfirmed, err) + } + if strings.TrimSpace(line) != database { + return fmt.Errorf("%w: that is not %q", ErrNotConfirmed, database) + } + return nil +} +``` + +- [ ] **Step 4: Write `verify`** + +Replace `vantagectl/internal/cmd/verify.go`: + +```go +package cmd + +import ( + "context" + "fmt" + + "gitea.hostxtra.co.uk/mrhid6/vantage/shared/backup" + "github.com/spf13/cobra" + "go.mongodb.org/mongo-driver/v2/mongo" +) + +func newVerifyCmd() *cobra.Command { + return &cobra.Command{ + Use: "verify ARCHIVE", + Short: "Check an archive against the key in hand", + Long: "verify checks that an archive is intact and that the\n" + + "KEY_ENCRYPTION_KEY in this environment matches the one it was made\n" + + "with.\n\n" + + "Given --mongo-uri it goes further and opens a real ciphertext value\n" + + "from that database. A fingerprint proves two archives agree about a\n" + + "key; only the probe proves the key you hold reads the data.\n\n" + + "Exit status is non-zero when anything is wrong, so this is the command\n" + + "to put on a schedule.", + Args: cobra.ExactArgs(1), + RunE: func(c *cobra.Command, args []string) error { + ctx := c.Context() + + archive, err := backup.Open(args[0]) + if err != nil { + return err + } + defer archive.Close() + + opt := backup.VerifyOptions{Archive: archive} + + // A database is optional here. resolveGlobals fails without a URI, + // so its error is a signal to verify the archive alone rather than + // a reason to stop. + var client *mongo.Client + if g, gerr := resolveGlobals(c); gerr == nil { + client, err = connect(ctx, g) + if err != nil { + return err + } + defer client.Disconnect(context.Background()) + opt.Client = client + opt.Database = g.Database + opt.KeyHex = g.KeyHex + } else { + fmt.Fprintln(c.ErrOrStderr(), + "note: no MongoDB URI, so this checks the archive and the key only") + } + + rep, err := backup.Verify(ctx, opt) + if err != nil { + return err + } + + out := c.OutOrStdout() + fmt.Fprintln(out, "Archive intact, every member matches its checksum") + if rep.ArchiveFingerprint != nil { + fmt.Fprintf(out, "Archive key %s\n", *rep.ArchiveFingerprint) + } + if rep.KeyFingerprint != nil { + fmt.Fprintf(out, "Your key %s\n", *rep.KeyFingerprint) + } + if rep.KeyMatchesArchive { + fmt.Fprintln(out, "Key match yes") + } + switch { + case rep.ProbeDecrypted: + fmt.Fprintf(out, "Live probe decrypted a value from %s\n", rep.ProbeCollection) + case rep.ProbeAttempted: + fmt.Fprintf(out, "Live probe FAILED against %s\n", rep.ProbeCollection) + case opt.Client != nil: + fmt.Fprintln(out, "Live probe skipped; this database stores no ciphertext yet") + } + + if rep.OK() { + fmt.Fprintln(out, "\nThis archive will restore.") + return nil + } + fmt.Fprintln(out) + for _, p := range rep.Problems { + fmt.Fprintln(out, "problem:", p) + } + return fmt.Errorf("verification failed") + }, + } +} +``` + +- [ ] **Step 5: Run the tests** + +Run: `cd vantagectl && go test ./internal/cmd/...` +Expected: PASS, fifteen tests. + +- [ ] **Step 6: Exercise the full cycle against a real database** + +```bash +cd vantagectl +export MONGO_URI=mongodb://localhost:27017 +export KEY_ENCRYPTION_KEY=0000000000000000000000000000000000000000000000000000000000000001 + +MONGO_DB=vantage_smoke go run . backup --out /tmp +ARCHIVE=$(ls -t /tmp/vantage-backup-vantage_smoke-*.tar.gz | head -1) + +MONGO_DB=vantage_restored go run . restore "$ARCHIVE" +MONGO_DB=vantage_restored go run . verify "$ARCHIVE" + +# The refusal path: a second restore over a populated database. +MONGO_DB=vantage_restored go run . restore "$ARCHIVE" +``` + +Expected: the first restore succeeds, `verify` reports the archive will +restore, and the second restore exits non-zero with `target database is not +empty`. + +Then the non-TTY confirmation: + +```bash +MONGO_DB=vantage_restored go run . restore "$ARCHIVE" --force < /dev/null +``` +Expected: exits non-zero, telling you to pass `--confirm-db vantage_restored`. + +```bash +MONGO_DB=vantage_restored go run . restore "$ARCHIVE" --force --confirm-db vantage_restored < /dev/null +``` +Expected: succeeds. + +- [ ] **Step 7: Commit** + +```bash +git add vantagectl/internal/cmd/restore.go vantagectl/internal/cmd/verify.go vantagectl/internal/cmd/restore_test.go +git commit -m "feat: Add the vantagectl restore and verify subcommands + +--force requires a typed database name on a terminal and --confirm-db +without one, so a copy-pasted restore command carries its intended target +and cannot destroy a different database." +``` + +--- +### Task 11: Container image and CI + +**Files:** +- Create: `vantagectl/Dockerfile` +- Create: `.gitea/workflows/vantagectl-release.yml` +- Modify: `.gitea/workflows/server-deploy.yml` + +**Interfaces:** +- Consumes: the built `vantagectl` module from Tasks 8 to 10. +- Produces: image `${DOCKER_HOST}//vantage/vantagectl:latest`, and release assets `vantagectl-{linux-amd64,linux-arm64,darwin-arm64,windows-amd64.exe}` plus `checksums.txt`. + +- [ ] **Step 1: Write the Dockerfile** + +Create `vantagectl/Dockerfile`: + +```dockerfile +# Build stage +# +# Context is the repository root, not vantagectl/, because vantagectl depends on +# the shared module through a replace directive. +FROM golang:1.26 AS builder + +WORKDIR /src + +# Manifests first so the dependency layer caches independently of source edits. +COPY shared/go.mod shared/go.sum ./shared/ +COPY vantagectl/go.mod vantagectl/go.sum ./vantagectl/ +RUN cd vantagectl && go mod download + +COPY shared/ ./shared/ +COPY vantagectl/ ./vantagectl/ + +ARG VERSION=dev +RUN cd vantagectl && CGO_ENABLED=0 GOOS=linux go build \ + -ldflags="-s -w -X main.Version=${VERSION}" -o /vantagectl . + +# Staged so the scratch image below can have a /tmp. It cannot mkdir one +# itself — scratch has no shell. +RUN mkdir -p /staging/tmp && chmod 1777 /staging/tmp + +# Runtime stage +FROM scratch + +COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ + +# restore extracts an archive here before verifying its checksums, and backup +# stages nothing but still inherits os.MkdirTemp's requirements. Without this +# every restore stops at "temp dir: stat /tmp: no such file or directory". +COPY --from=builder /staging/tmp /tmp +COPY --from=builder /vantagectl /vantagectl + +ENTRYPOINT ["/vantagectl"] +``` + +- [ ] **Step 2: Build the image and run it** + +```bash +docker build -t vantagectl:test -f vantagectl/Dockerfile . +docker run --rm vantagectl:test --help +``` +Expected: the help text lists `backup`, `restore`, `inspect` and `verify`. + +- [ ] **Step 3: Prove the `/tmp` copy is load-bearing** + +Temporarily comment out the `COPY --from=builder /staging/tmp /tmp` line, +rebuild as `vantagectl:notmp`, and run a restore against any archive. It must +fail with a `/tmp` error. Restore the line and rebuild. This is a manual check, +not a committed test — the point is that the next person to trim the Dockerfile +learns why the line is there. + +- [ ] **Step 4: Add the release workflow** + +Create `.gitea/workflows/vantagectl-release.yml`: + +```yaml +name: vantagectl Release + +on: + push: + tags: + - "vantagectl/v*" + +jobs: + build: + runs-on: ubuntu-docker + container: node:26 + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Set up Go + uses: actions/setup-go@v5 + with: + go-version: "1.26" + cache: true + cache-dependency-path: vantagectl/go.sum + + - name: Extract version + id: version + run: echo "VERSION=${GITHUB_REF_NAME#vantagectl/}" >> $GITHUB_OUTPUT + + - name: Test + working-directory: vantagectl + run: go test ./... + + - name: Build + working-directory: vantagectl + env: + VERSION: ${{ steps.version.outputs.VERSION }} + run: | + mkdir -p dist + for target in linux/amd64 linux/arm64 darwin/arm64 windows/amd64; do + goos="${target%/*}" + goarch="${target#*/}" + out="dist/vantagectl-${goos}-${goarch}" + if [ "$goos" = "windows" ]; then out="${out}.exe"; fi + CGO_ENABLED=0 GOOS="$goos" GOARCH="$goarch" go build \ + -ldflags="-s -w -X main.Version=${VERSION}" \ + -o "$out" . + done + + - name: Checksums + working-directory: vantagectl/dist + run: sha256sum vantagectl-* > checksums.txt + + - name: Create release + uses: https://gitea.com/actions/gitea-release-action@v1 + with: + token: ${{ secrets.RELEASE_TOKEN }} + files: | + vantagectl/dist/vantagectl-linux-amd64 + vantagectl/dist/vantagectl-linux-arm64 + vantagectl/dist/vantagectl-darwin-arm64 + vantagectl/dist/vantagectl-windows-amd64.exe + vantagectl/dist/checksums.txt +``` + +- [ ] **Step 5: Add the image to `server-deploy.yml`** + +In the change-detection block, add a line after the three existing Go flags: + +```bash + flag vantagectl '^(vantagectl/|shared/|go\.work)' +``` + +Update the comment above those flags so it stays true: + +```bash + # The four Go images build from the repo root and COPY + # shared/ plus their own directory, so shared/ rebuilds all + # four. proto/ is in server's list as insurance: the +``` + +Add a build step alongside the others: + +```yaml + - name: Build and push vantagectl image + if: steps.changed.outputs.vantagectl == 'true' + run: | + IMAGE="${{ vars.DOCKER_HOST }}/${{ github.repository_owner }}/vantage/vantagectl:latest" + # Root context: vantagectl depends on the shared module. + docker build -t "$IMAGE" -f vantagectl/Dockerfile . + docker push "$IMAGE" +``` + +- [ ] **Step 6: Check the workflow parses** + +Run: `python3 -c "import yaml,sys; [yaml.safe_load(open(p)) for p in ['.gitea/workflows/server-deploy.yml','.gitea/workflows/vantagectl-release.yml']]; print('ok')"` +Expected: `ok`. + +- [ ] **Step 7: Commit** + +```bash +git add vantagectl/Dockerfile .gitea/workflows/vantagectl-release.yml .gitea/workflows/server-deploy.yml +git commit -m "feat: Build and publish vantagectl + +The scratch runtime stage copies an explicit /tmp: restore extracts an +archive there before verifying it, and a scratch image has none. + +shared/ now fans out to four Go images rather than three." +``` + +--- + +### Task 12: Helm CronJob + +**Files:** +- Create: `deploy/chart/vantage/templates/backup-cronjob.yaml` +- Modify: `deploy/chart/vantage/values.yaml` +- Modify: `deploy/chart/vantage/templates/NOTES.txt` + +**Interfaces:** +- Consumes: the image from Task 11. +- Produces: values `backup.enabled`, `backup.schedule`, `backup.image`, `backup.pvcName`, `backup.exclude`, `backup.successfulJobsHistoryLimit`, `backup.failedJobsHistoryLimit`, `backup.resources`. + +- [ ] **Step 1: Read how the existing templates reference secrets** + +Run: `sed -n '1,80p' deploy/chart/vantage/templates/server.yaml` + +Match whatever that file does for `MONGO_URI` and `KEY_ENCRYPTION_KEY` exactly. +The CronJob must reference the same secret keys rather than declaring its own — +a backup job with its own copy of the encryption key is a second place for it to +be wrong. + +- [ ] **Step 2: Add the values** + +Append to `deploy/chart/vantage/values.yaml`: + +```yaml +# Scheduled backups. +# +# Off by default, deliberately. A backup with nowhere durable to land is a +# false sense of safety, and the chart cannot know where that is — pvcName +# must name a volume you have decided will outlive the cluster. +# +# There is no restore manifest here on purpose: a restore is an operator +# decision with a confirmation attached, and must never be something a +# `helm upgrade` can trigger. Run one as a `kubectl run` Job with +# --confirm-db. +backup: + enabled: false + schedule: "0 2 * * *" + image: "" + pvcName: "" + # Collections to leave out. Recorded in each archive's manifest, so an + # archive can never claim to be complete when it is not. + exclude: [] + successfulJobsHistoryLimit: 3 + failedJobsHistoryLimit: 3 + resources: {} +``` + +- [ ] **Step 3: Write the template** + +Create `deploy/chart/vantage/templates/backup-cronjob.yaml`: + +```yaml +{{- if .Values.backup.enabled }} +{{- if not .Values.backup.pvcName }} +{{- fail "backup.enabled requires backup.pvcName: a backup needs somewhere durable to land, and the chart cannot guess where that is" }} +{{- end }} +{{- if not .Values.backup.image }} +{{- fail "backup.enabled requires backup.image: the vantagectl image to run" }} +{{- end }} +apiVersion: batch/v1 +kind: CronJob +metadata: + name: {{ include "vantage.fullname" . }}-backup + labels: + {{- include "vantage.labels" . | nindent 4 }} + app.kubernetes.io/component: backup +spec: + schedule: {{ .Values.backup.schedule | quote }} + concurrencyPolicy: Forbid + successfulJobsHistoryLimit: {{ .Values.backup.successfulJobsHistoryLimit }} + failedJobsHistoryLimit: {{ .Values.backup.failedJobsHistoryLimit }} + jobTemplate: + spec: + backoffLimit: 2 + template: + metadata: + labels: + {{- include "vantage.selectorLabels" . | nindent 12 }} + app.kubernetes.io/component: backup + spec: + restartPolicy: Never + containers: + - name: vantagectl + image: {{ .Values.backup.image | quote }} + args: + - backup + - --out + - /backups + {{- with .Values.backup.exclude }} + - --exclude + - {{ join "," . | quote }} + {{- end }} + env: + # Referenced, never redeclared. A backup job holding its own + # copy of KEY_ENCRYPTION_KEY is a second place for it to be + # wrong, and the fingerprint it stamps would then be a + # fingerprint of the wrong key. + {{- include "vantage.mongoEnv" . | nindent 16 }} + {{- include "vantage.encryptionKeyEnv" . | nindent 16 }} + volumeMounts: + - name: backups + mountPath: /backups + resources: + {{- toYaml .Values.backup.resources | nindent 16 }} + volumes: + - name: backups + persistentVolumeClaim: + claimName: {{ .Values.backup.pvcName | quote }} +{{- end }} +``` + +If `_helpers.tpl` has no `vantage.mongoEnv` or `vantage.encryptionKeyEnv`, add +them there by lifting the exact env blocks `server.yaml` already uses, and change +`server.yaml` to call the helpers too. Two copies of an env block that must agree +is the drift this chart already avoids elsewhere. + +- [ ] **Step 4: Render the chart four ways** + +```bash +cd deploy/chart +helm lint vantage +helm template vantage vantage > /dev/null +helm template vantage vantage --set backup.enabled=true --set backup.image=img --set backup.pvcName=pvc > /dev/null +``` + +Then the refusals, which must fail: + +```bash +helm template vantage vantage --set backup.enabled=true --set backup.image=img 2>&1 | grep -q "backup.pvcName" && echo "refused without a PVC" +helm template vantage vantage --set backup.enabled=true --set backup.pvcName=pvc 2>&1 | grep -q "backup.image" && echo "refused without an image" +``` +Expected: both echo their message. `helm lint` accepts a chart whose templates +never execute, so rendering is what proves the `fail` calls still fire. + +- [ ] **Step 5: Add the CI render cases** + +In `.gitea/workflows/chart-release.yml`, add the backup-enabled render to the +existing set of four, and add the two refusals to the must-be-refused set. +Follow whatever shape that file already uses; do not restructure it. + +- [ ] **Step 6: Add the NOTES.txt warning** + +Append to `deploy/chart/vantage/templates/NOTES.txt`: + +``` +{{- if not .Values.backup.enabled }} + +No backups are scheduled. Vantage encrypts SSH private keys, vault secrets and +SSO client secrets with KEY_ENCRYPTION_KEY, and that key is not stored anywhere +but your own configuration — a database restored without it is permanently +unreadable. + +Set backup.enabled, backup.image and backup.pvcName, and store +KEY_ENCRYPTION_KEY somewhere that survives this cluster. +{{- end }} +``` + +- [ ] **Step 7: Commit** + +```bash +git add deploy/chart/vantage +git commit -m "feat: Add an optional scheduled backup CronJob to the chart + +Off by default: a backup with nowhere durable to land is a false sense of +safety and the chart cannot know where that is. NOTES.txt says so when it +is off. + +No restore manifest ships: a restore must never be something a helm +upgrade can trigger." +``` + +--- + +### Task 13: Documentation + +**Files:** +- Create: `docsite/docs/operations/backup-and-restore.md` +- Modify: `docsite/sidebars.ts` +- Modify: `CLAUDE.md` + +**Interfaces:** +- Consumes: everything above. +- Produces: no code. + +- [ ] **Step 1: Read the existing operations docs for house style** + +Run: `ls docsite/docs/operations && head -40 docsite/docs/operations/*.md | head -60` + +Match the front matter, heading level and tone of what is already there. + +- [ ] **Step 2: Write the page** + +Create `docsite/docs/operations/backup-and-restore.md`. Front matter must match +the sibling pages' shape. Content, in this order: + +1. **The key comes first.** Vantage encrypts SSH private keys, key passphrases, + vault secrets, SSO client secrets and console credentials with + `KEY_ENCRYPTION_KEY`. It is not in your backup and it is not recoverable. A + database restored without it is permanently unreadable. Store it wherever you + store the credentials you could not rebuild. +2. **What a backup holds:** every collection in the database, the index + definitions, and a SHA-256 fingerprint of the key — never the key. +3. **What it does not hold:** Redis sessions (everyone signs in again, which is + already true whenever Redis restarts), the vulnerability database (re-pulled + automatically), and any agent state on managed servers. Agents reconnect on + their own because `servers.agent_token_hash` is in the backup, so no server + needs re-enrolling. +4. **Taking a backup**, three copyable forms: the loose binary, the container, + and the Kubernetes CronJob values. Use the exact commands from the spec's + Distribution section. +5. **Where to put the archive.** `--out -` streams to stdout; show one `restic` + and one `aws s3 cp -` example. Note that an archive is as sensitive as a + database dump and should be encrypted at rest by whatever you pipe it into. +6. **Checking a backup is real:** `vantagectl verify ARCHIVE` with + `--mongo-uri`, what each line of its output means, and that it exits non-zero + so it can go on a schedule. +7. **Restoring**, in order: restore into an empty database, what the refusal on + a non-empty one means, `--force` and its confirmation, and `--confirm-db` for + a Job or CI step with no terminal. +8. **The restore drill**, with a heading of its own: restore last night's + archive into a scratch database, run `verify` against it, drop it. An + untested backup is a hypothesis. Recommend monthly. +9. **When the key is wrong**, describing what `--ignore-key-mismatch` does and + which collections it names, and stating plainly that there is no way to + recover the ciphertext afterwards. + +- [ ] **Step 3: Add the page to the sidebar** + +`docsite/sidebars.ts` is authored by hand. Add the new page to the Operations +section in the position that reads correctly, not necessarily last. + +- [ ] **Step 4: Build the docs site** + +```bash +cd docsite && npm ci && npm run build +``` +Expected: build succeeds with no broken-link warnings for the new page. + +- [ ] **Step 5: Add the `CLAUDE.md` section** + +Add a `### Backup and restore` subsection under Subsystems, covering, briefly: + +- `vantagectl` is a separate module and a separate image, and why (cobra out of + three module graphs; a tool that must run when the control plane does not). +- `shared/backup` holds the logic so `server` can import it later; + `shared/cryptobox` is now the single AES-GCM implementation and + `services/crypto.go` delegates to it. +- The archive carries a fingerprint of `KEY_ENCRYPTION_KEY`, never the key, and + backup refuses without one. +- Collections are enumerated live, the opposite choice to `ScopedCollections`, + and why that is right here. +- Restore refuses a non-empty target and has no merge semantics. +- The `vantagectl/Dockerfile` scratch stage needs its explicit `/tmp`, same as + `server`. +- `shared/` now fans out to **four** Go images in `server-deploy.yml`. + +Also update the existing sentence in the CI section that says "seven images" and +the one that says `shared/` "fans out to all three Go images", both of which are +now wrong. + +- [ ] **Step 6: Verify the counts you just changed** + +Run: `grep -n "seven images\|three Go images\|all three" CLAUDE.md` +Expected: no stale hits remain. + +- [ ] **Step 7: Commit** + +```bash +git add docsite CLAUDE.md +git commit -m "docs: Document backup and restore + +The page leads with KEY_ENCRYPTION_KEY rather than mentioning it in a +note, because holding a good database dump and no key is the way this goes +wrong." +``` + +--- + +## Final verification + +- [ ] **Run every module's tests** + +```bash +cd shared && go test ./... && cd ../vantagectl && go test ./... && cd ../server && go test ./... +``` +With `MONGO_TEST_URI` set for the first two, so the database-backed tests +actually run rather than skipping. + +- [ ] **Vet everything new** + +```bash +cd shared && go vet ./backup/... ./cryptobox/... && cd ../vantagectl && go vet ./... +``` + +- [ ] **Confirm the module boundaries held** + +```bash +grep -r "spf13/cobra" shared/ server/ admin/ sitesvc/ --include="*.go" --include="go.mod" +grep -r "vantage/server" vantagectl/ --include="*.go" +``` +Expected: both print nothing. + +- [ ] **Confirm the full round trip once more, from the built image** + +```bash +docker build -t vantagectl:final -f vantagectl/Dockerfile . +docker run --rm --network host \ + -e MONGO_URI=mongodb://localhost:27017 -e MONGO_DB=vantage_smoke \ + -e KEY_ENCRYPTION_KEY=0000000000000000000000000000000000000000000000000000000000000001 \ + -v /tmp:/out vantagectl:final backup --out /out +``` +Expected: an archive appears in `/tmp`, written by the scratch image, which +proves the staged `/tmp` and the CA bundle are both in place. diff --git a/docs/superpowers/specs/2026-09-07-control-plane-backup-restore-design.md b/docs/superpowers/specs/2026-09-07-control-plane-backup-restore-design.md index ac680d7..695946a 100644 --- a/docs/superpowers/specs/2026-09-07-control-plane-backup-restore-design.md +++ b/docs/superpowers/specs/2026-09-07-control-plane-backup-restore-design.md @@ -208,8 +208,10 @@ vantagectl root; prints help ``` Persistent flags on the root command, so every subcommand accepts them and they -are documented once: `--mongo-uri` (env `MONGO_URI`), `--db` (env `MONGO_DB`, -falling back to the URI path), `--log-level`. +are documented once: `--mongo-uri` (env `MONGO_URI`) and `--db` (env `MONGO_DB`, +falling back to the URI path). There is no `--log-level`: the tool's entire +output is what it is telling the operator, and a level that could hide a key +warning is worth not having. Environment fallback is wired with an explicit `Changed` check on each flag rather than through viper. Viper is a configuration-file and remote-config