mrhid6/vantage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 29 Wiki Activity

refactor: consume vantage-shared as an external private module
Chart Release / chart (push) Successful in 19s
Details
Server Deploy / deploy (push) Failing after 1m11s
Details

Browse Source 
shared/ is extracted to gitea.hostxtra.co.uk/vantage/vantage-shared and
pinned at v0.1.0 by server, agent, admin, sitesvc and vantagectl. The
replace directives and the ./shared entry in go.work are gone.

Every Go build now needs a credential for the private module: CI writes a
netrc per job from REGISTRY_USER + RELEASE_TOKEN and sets GOPRIVATE, and
the four Go Dockerfiles take it as a BuildKit secret rather than a build
arg, which would survive in the builder layer's history. RELEASE_TOKEN
needs read access to the vantage org.

admin, sitesvc and vantagectl now build from their own directory; only
server still needs the repository root, for default_steps/. The rebuild
triggers in server-deploy.yml lose their shared/ patterns, since a
service now moves when its own go.mod pin does.
This commit is contained in:
mrhid6
2026-09-08 07:42:58 +00:00
parent 5326639918
commit ee1f9f3b32
91 changed files with 251 additions and 5751 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+74 -36
View File
@@ -99,17 +99,47 @@ vantage/
│ ├── src/css/custom.css # site/'s tokens, copied, mapped onto --ifm-*
│ ├── sidebars.ts # authored by hand, not autogenerated
│ └── nginx.conf # serves the build under /docs
├── shared/ # imported by server, sitesvc and admin
│ ├── mail/ # the one email system: transport + tmpl templates
│ ├── license/ # payload, sign, verify, trusted keys, plans
│ ├── models/ # Instance, User, Settings
│ └── cmd/lkctl/ # issue and inspect licences by hand
├── proto/vantage/v1/vantage.proto
├── installer/ # Windows: setup.ps1, nssm.exe, WiX .wxs
├── deploy/ # docker-compose.yml, agent.service
└── .gitea/workflows/ # agent-release.yml, server-deploy.yml
```
**`shared/` is not in this repository.** It is the private module
`gitea.hostxtra.co.uk/vantage/vantage-shared`, and it holds `mail/` (the one
email system: transport plus templates), `license/` (payload, sign, verify,
trusted keys, plans), `models/` (Instance, User, Settings), `provision/`,
`backup/`, `cryptobox/`, `indexes/`, `grpc/pb` + `grpc/codec`, and
`cmd/lkctl/`. Five modules here depend on it — `server`, `agent`, `admin`,
`sitesvc`, `vantagectl` — each pinning a version in its own `go.mod`. It was a
directory in this repository until it was extracted with its history; the
`replace ../shared` directives and the `./shared` entry in `go.work` are gone
with it.
**A version pin is now the coupling, and that is the point.** While it was a
directory, every service in a given commit built against exactly one `shared/`,
and a change there rebuilt three images at once whether or not they were ready
for it. Now a service moves when somebody bumps its pin, which is a commit under
that service's own directory — so the existing per-directory rebuild triggers
already cover it, and there is no longer any way to ship a service against a
`shared/` it was never built against. The cost is the obvious one: a fix in
`vantage-shared` is live nowhere until each consumer's pin is bumped, and
nothing in this repository will remind you.
Every Go build now needs a credential for it — `GOPRIVATE=gitea.hostxtra.co.uk/*`
plus a netrc. CI writes one per job from `REGISTRY_USER` + `RELEASE_TOKEN`
(**that token needs read access to the `vantage` org, not only `mrhid6`**), and
the four Go Dockerfiles take it as a **BuildKit secret** rather than a build
arg, because an arg survives in the builder layer's history and this one is a
Gitea token. Locally, either a netrc or
`git config --global url."git@gitea.hostxtra.co.uk:".insteadOf https://gitea.hostxtra.co.uk/`.
**Three build contexts shrank as a result.** `admin`, `sitesvc` and `vantagectl`
build from their own directory now; only `server` still builds from the
repository root, and only because its runtime stage copies `default_steps/`.
One side effect worth knowing: `admin/.dockerignore` was inert while the context
was the root, and is live now.
---
## Subsystems
@@ -460,12 +490,15 @@ drifted: the agent's `UnimplementedVantageServer` was three methods stale and
carried no `ReportWorkloads` at all. The agent links the server half as dead
code, which the linker drops.
This makes `agent` the **fifth** consumer of `shared/`, and the second one CI
does not rebuild on a push to main: like `vantagectl`, the agent image is cut by
`agent-release.yml` on an `agent/v*` tag, so a wire change reaches the server at
the next push and the fleet at the next agent release. That gap existed before
too — it is just now a compile error in the same tree rather than a silent
mismatch between two copies that both compiled.
This makes `agent` the **fifth** consumer of `shared/`, and the wire contract now
lives outside this repository entirely: a message added to `vantage-shared` is
not a message either side has until its pin is bumped. What that buys is that
the mismatch is a compile error rather than two copies that both compiled and
disagreed on the wire. What it costs is ordering — a wire change needs a
`vantage-shared` release, then a pin bump in `server/` (live at the next push to
main) and a pin bump in `agent/` (live at the next `agent/v*` tag). The
server-ahead-of-fleet gap existed before too; it is now explicit in two `go.mod`
files instead of implicit in a shared directory.
### Status pages
@@ -645,8 +678,8 @@ mid-write. Reconstructing the key document from JSON also lost compound key
order, which is significant.
**`backup.ciphertextFields` mirrors `server/internal/models` by hand.**
`shared/` is a separate module and `models` is under `server/internal`, so
`shared/backup` cannot import it; the map naming each collection's `*_enc`
`shared/` is a separate module — a separate *repository* now — and `models` is
under `server/internal`, so `shared/backup` cannot import it; the map naming each collection's `*_enc`
fields (`keys`, `secrets`, `auth_providers`, `console_sessions`) must change in
the same commit as any of those bson tags, the same hazard as
`web/lib/targets.ts` and `services.MaxWorkloadLogLines`. Wrong field names are
@@ -668,11 +701,10 @@ temporary directory before verifying its checksums, and a scratch image has no
`server`, but here it would break every restore rather than only vulnerability
scanning.
**`shared/` reaches four Go images, but only three of them from
`server-deploy.yml`** (`server`, `sitesvc`, `admin`). The `vantagectl` image is
built by `vantagectl-release.yml` on a `vantagectl/v*` tag instead, so a
`shared/` change reaches it at the next release rather than the next push to
main — see the CI section below.
**Nothing in this repository rebuilds when `vantage-shared` changes.** It is an
external module, so a `shared/backup` fix reaches `vantagectl` when somebody
bumps `vantagectl/go.mod` and cuts a `vantagectl/v*` tag, and reaches nothing
else until its pin moves too — see the CI section below.
### API tokens and OpenAPI
@@ -1343,24 +1375,30 @@ cd /opt/vantage && docker compose -f docker-compose.yml -f docker-compose.site.y
**Each image only rebuilds when its own inputs changed.** A `git diff` against `github.event.before` decides, which is why the checkout uses `fetch-depth: 0` — the default shallow clone has one commit and nothing to diff — and why `git` is installed in the `docker:dind` container. The mapping follows the build contexts exactly:
| Image | Rebuilds when |
| ---------------------------- | ----------------------------------------- |
| `server` | `server/`, `shared/`, `proto/`, `go.work` |
| `admin` | `admin/`, `shared/`, `go.work` |
| `sitesvc` | `sitesvc/`, `shared/`, `go.work` |
| `web` · `site` · `adminsite` · `docsite` | their own directory only |
| Image | Rebuilds when |
| ---------------------------- | -------------------------------- |
| `server` | `server/`, `proto/`, `go.work` |
| `admin` | `admin/`, `go.work` |
| `sitesvc` | `sitesvc/`, `go.work` |
| `web` · `site` · `adminsite` · `docsite` | their own directory only |
`shared/` fans out to **three** images here (`server`, `sitesvc`, `admin`)
because each of their Dockerfiles copies `shared/` from a root context — **if a
fourth service ever imports `shared/`, add it to that list or it will ship
stale**. `vantagectl` also imports `shared/` and is the exception: it is built
by `vantagectl-release.yml`, so a `shared/` fix reaches it only when someone
cuts a `vantagectl/v*` tag. That is deliberate — an operator restoring a
database should be running a version they can name — but it does mean a
`shared/backup` fix is not live until it is released. `agent` is the same shape
of exception since `shared/grpc/pb` moved there: it is built by
`agent-release.yml` on an `agent/v*` tag, so a wire change lands on the server
at the next push to main and on the fleet only at the next agent release. A change to the workflow file rebuilds everything, since
**No path in this table names `shared/` any more**, and no fan-out rule replaces
it: `vantage-shared` is an external module pinned per service, so a service
rebuilds when its own `go.mod` moves, which its own directory pattern already
matches. What that removes is the failure where a `shared/` edit rebuilt three
images and one of them was not ready; what it adds is that nothing here reminds
you a pin is stale.
Every Go build in these workflows writes a netrc from `REGISTRY_USER` +
`RELEASE_TOKEN` before it runs, and sets `GOPRIVATE=gitea.hostxtra.co.uk/*`.
There are **five** such places, and each needs its own because jobs do not share
a filesystem: `server-deploy.yml`'s single job, both jobs of
`agent-release.yml` (the `msi` job is Windows, where Go reads `%USERPROFILE%\_netrc`,
not `.netrc`) and both jobs of `vantagectl-release.yml`. The docker builds pass
it on as `--secret id=netrc`, never a build arg. **`RELEASE_TOKEN` needs read
access to the `vantage` org** on top of its existing scopes; without it every Go
build fails at `go mod download` with a 404 on the module, which reads like a
missing tag rather than a missing permission. A change to the workflow file rebuilds everything, since
a build arg is baked into the image. So does anything that leaves no
trustworthy base commit: a manual `workflow_dispatch`, a new branch, or a
force-push whose old head is gone.
@@ -1392,7 +1430,7 @@ git push origin main # server + web deploy
| Name | Type | Value |
| ----------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `RELEASE_TOKEN` | Secret | Gitea API token. Needs `write:release` (agent releases), `write:package` (container images and the Helm chart). **This is the only token any workflow authenticates with** — `docker login` and the chart publish both pair it with `REGISTRY_USER` |
| `RELEASE_TOKEN` | Secret | Gitea API token. Needs `write:release` (agent releases), `write:package` (container images and the Helm chart) and **read access to the `vantage` org**, which is where the private `vantage-shared` module lives — without that last one every Go build fails at `go mod download` with what looks like a missing tag. **This is the only token any workflow authenticates with** — `docker login`, the chart publish and the module netrc all pair it with `REGISTRY_USER` |
| `REGISTRY_USER` | Secret | Gitea username. Must own `RELEASE_TOKEN`, or basic auth is rejected |
| ~~`REGISTRY_PASSWORD`~~ | — | **Not used.** Named here historically; no workflow reads it. Referencing an unset secret yields an empty password and a `401 Failed to authenticate user` that looks like a token scope problem. Use `RELEASE_TOKEN` |
| `DOCKER_HOST` | Variable | registry host used for image tags |