@@ -14,7 +14,7 @@ Each server's detail page shows the version it reported at its last sync.
|
||||
|
||||
## Updating from the UI
|
||||
|
||||
**Servers → *a server* → Update agent** pushes `UpdateAgentCmd` with a target
|
||||
**Servers → _a server_ → Update agent** pushes `UpdateAgentCmd` with a target
|
||||
version. The agent then:
|
||||
|
||||
1. Downloads the binary for its platform from the release.
|
||||
@@ -38,7 +38,7 @@ irm https://vantage.example.com/update.ps1 | iex
|
||||
```
|
||||
|
||||
It does the same download, checksum and replace, then restarts the service. Use
|
||||
this when the control plane cannot push — for example, when the machine is
|
||||
this when the control plane cannot push for example, when the machine is
|
||||
reachable but its command stream is not.
|
||||
|
||||
## Rolling out across a fleet
|
||||
@@ -57,20 +57,20 @@ Do one, confirm it returns to `active`, then do the rest.
|
||||
|
||||
## Version compatibility
|
||||
|
||||
The gRPC API is versioned to tolerate an agent older than the control plane. The
|
||||
reverse — an agent newer than the control plane — is not a case anyone tests.
|
||||
The agent API is versioned to tolerate an agent older than the control plane. The
|
||||
reverse an agent newer than the control plane is not a case anyone tests.
|
||||
Upgrade the control plane first.
|
||||
|
||||
Agents report their version on every `SyncKeys`, so a fleet running mixed
|
||||
Agents report their version on every poll, so a fleet running mixed
|
||||
versions is visible in the server list rather than something you have to go
|
||||
looking for.
|
||||
|
||||
## If an update fails
|
||||
|
||||
| Symptom | Cause |
|
||||
| --- | --- |
|
||||
| "Checksum mismatch" | Interrupted download, or a proxy rewriting the body. Retry |
|
||||
| Downloads nothing | The machine cannot reach `gitea.hostxtra.co.uk` |
|
||||
| Symptom | Cause |
|
||||
| --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| "Checksum mismatch" | Interrupted download, or a proxy rewriting the body. Retry |
|
||||
| Downloads nothing | The machine cannot reach `gitea.hostxtra.co.uk` |
|
||||
| Service will not start afterwards | Wrong architecture binary, or the file was replaced while a different service manager held it. Reinstall with the install one-liner |
|
||||
|
||||
Reinstalling is always safe: the config file is left alone, so the agent comes
|
||||
|
||||
@@ -9,12 +9,12 @@ either one restores to something unusable.
|
||||
|
||||
## What holds what
|
||||
|
||||
| Store | Contents | Back up |
|
||||
| --- | --- | --- |
|
||||
| MongoDB | Everything durable — servers, keys, assignments, workflows, runs, monitors, incidents, secrets, settings, audit | **Yes** |
|
||||
| Redis | Sessions only | No. Losing it signs everyone out and nothing else |
|
||||
| `./data` bind mount | Workflow run logs | Optional |
|
||||
| `KEY_ENCRYPTION_KEY` | Not stored anywhere by the app | **Yes, separately** |
|
||||
| Store | Contents | Back up |
|
||||
| -------------------- | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------- |
|
||||
| MongoDB | Everything durable servers, keys, assignments, workflows, runs, monitors, incidents, secrets, settings, audit | **Yes** |
|
||||
| Redis | Sessions only | No. Losing it signs everyone out and nothing else |
|
||||
| `./data` bind mount | Workflow run logs | Optional |
|
||||
| `KEY_ENCRYPTION_KEY` | Not stored anywhere by the app | **Yes, separately** |
|
||||
|
||||
:::danger The database alone is not a backup
|
||||
Private keys, vault secrets, OIDC client secrets and console credentials are
|
||||
@@ -50,7 +50,7 @@ nothing writes during the restore.
|
||||
cp /opt/vantage/.env /secure-location/vantage.env
|
||||
```
|
||||
|
||||
Treat it as a credential in its own right — it holds the encryption key.
|
||||
Treat it as a credential in its own right it holds the encryption key.
|
||||
|
||||
## Run logs
|
||||
|
||||
@@ -69,17 +69,17 @@ What it does **not** do is reconcile the world. After a restore:
|
||||
- Agents reconnect with their existing tokens, since the token hashes are in the
|
||||
database.
|
||||
- If the restore is older than an enrolment, that server's token hash is missing
|
||||
and the agent will fail to authenticate — re-enrol it.
|
||||
and the agent will fail to authenticate re-enrol it.
|
||||
- The next agent poll rewrites `authorized_keys` to match the restored desired
|
||||
state, which may remove keys added since the backup.
|
||||
|
||||
## A workable schedule
|
||||
|
||||
| What | When |
|
||||
| --- | --- |
|
||||
| MongoDB dump | Nightly, retained per your policy |
|
||||
| Environment file | On change, held in a password manager or secret store |
|
||||
| Restore rehearsal | Occasionally, into a throwaway host |
|
||||
| What | When |
|
||||
| ----------------- | ----------------------------------------------------- |
|
||||
| MongoDB dump | Nightly, retained per your policy |
|
||||
| Environment file | On change, held in a password manager or secret store |
|
||||
| Restore rehearsal | Occasionally, into a throwaway host |
|
||||
|
||||
The rehearsal is the part that gets skipped and the part that finds the
|
||||
problems.
|
||||
|
||||
@@ -1,102 +0,0 @@
|
||||
---
|
||||
id: ci-cd
|
||||
title: CI/CD
|
||||
sidebar_label: CI/CD
|
||||
---
|
||||
|
||||
Two Gitea Actions workflows: one releases agents, one builds images.
|
||||
|
||||
## Agent releases
|
||||
|
||||
Triggered by an `agent/v*` tag.
|
||||
|
||||
```bash
|
||||
git tag agent/v1.0.0 && git push origin agent/v1.0.0
|
||||
```
|
||||
|
||||
Builds `linux/amd64`, `linux/arm64` and `windows/amd64`, writes `checksums.txt`
|
||||
and creates a Gitea release. A second job on Windows packages the WiX MSI.
|
||||
|
||||
The install and update scripts read the newest `agent/v*` release from the Gitea
|
||||
API, so tagging is what makes a new agent available to every install.
|
||||
|
||||
## Image builds
|
||||
|
||||
Triggered on every push to `main`. Builds and pushes seven images: `server`,
|
||||
`web`, `site`, `sitesvc`, `admin`, `adminsite` and `docsite`.
|
||||
|
||||
:::warning Despite the name, this workflow does not deploy
|
||||
There is no SSH step. Rolling images out is a manual step on the host:
|
||||
|
||||
```bash
|
||||
cd /opt/vantage && \
|
||||
docker compose -f docker-compose.yml -f docker-compose.site.yml pull && \
|
||||
docker compose -f docker-compose.yml -f docker-compose.site.yml up -d --remove-orphans
|
||||
```
|
||||
:::
|
||||
|
||||
### Each image rebuilds only when its own inputs change
|
||||
|
||||
A `git diff` against the previous head decides. That is why the checkout uses
|
||||
`fetch-depth: 0` — a shallow clone has one commit and nothing to diff against.
|
||||
|
||||
| Image | Rebuilds when |
|
||||
| --- | --- |
|
||||
| `server` | `server/`, `shared/`, `proto/`, `go.work` |
|
||||
| `admin` | `admin/`, `shared/`, `go.work` |
|
||||
| `sitesvc` | `sitesvc/`, `shared/`, `go.work` |
|
||||
| `web` · `site` · `adminsite` · `docsite` | their own directory only |
|
||||
|
||||
`shared/` fans out to all three Go images because each of their Dockerfiles
|
||||
copies it from a root context. **If a fourth service ever imports `shared/`, it
|
||||
must be added to that list or it will ship stale.**
|
||||
|
||||
Everything rebuilds when there is no trustworthy base commit to diff against: a
|
||||
manual `workflow_dispatch`, a new branch, or a force-push whose old head is
|
||||
gone. Changing the workflow file itself also rebuilds everything, since a build
|
||||
argument is baked into each image.
|
||||
|
||||
### The gap: repository variables
|
||||
|
||||
:::danger Editing a repository variable pushes no commit, so nothing rebuilds
|
||||
Values like `ADMIN_API_URL`, `HQ_URL`, `ADMIN_ENV`, `PADDLE_ENV`,
|
||||
`PADDLE_CLIENT_TOKEN`, `DOCS_URL` and `DOCS_BASE_URL` are baked into images at
|
||||
build time. After editing one, run the workflow manually — that is what
|
||||
`workflow_dispatch` is for.
|
||||
|
||||
The symptom is a frontend that keeps using the old value with no error anywhere,
|
||||
which is a long afternoon if you do not know about this.
|
||||
:::
|
||||
|
||||
The same applies to base images: a service nobody touches stops being rebuilt on
|
||||
newer base layers. A periodic manual run covers it.
|
||||
|
||||
## Secrets and variables
|
||||
|
||||
| Name | Type | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `RELEASE_TOKEN` | Secret | Gitea API token, `write:release` |
|
||||
| `REGISTRY_USER` / `REGISTRY_PASSWORD` | Secret | Registry push credentials |
|
||||
| `PADDLE_API_KEY` | Secret | Read by admin at runtime |
|
||||
| `PADDLE_WEBHOOK_SECRET` | Secret | Webhook signature verification |
|
||||
| `GITEA_HOST` / `DOCKER_HOST` | Variable | Hosts used in tags and URLs |
|
||||
| `HQ_URL` | Variable | Baked into `web`; empty on self-hosted |
|
||||
| `SITE_API_URL` / `SITE_CONTACT_EMAIL` | Variable | Baked into `site` |
|
||||
| `ADMIN_API_URL` | Variable | Baked into `adminsite` **and** `site` |
|
||||
| `ADMIN_ENV` | Variable | Environment badge in the portal |
|
||||
| `PADDLE_ENV` | Variable | Baked into `adminsite`, read by `admin`. Must match on both sides |
|
||||
| `PADDLE_CLIENT_TOKEN` | Variable | Browser Paddle token for checkout |
|
||||
| `DOCS_URL` / `DOCS_BASE_URL` | Variable | Baked into `docsite` |
|
||||
|
||||
Anything marked "browser-reachable" must be an origin a browser can actually
|
||||
resolve — not an internal service name. Get it wrong and every request fails at
|
||||
runtime with a not-connected panel rather than at build time.
|
||||
|
||||
## The documentation site
|
||||
|
||||
`docsite/` builds to static files and is served by nginx under `/docs` on the
|
||||
marketing host, routed by its own proxy location.
|
||||
|
||||
`DOCS_BASE_URL` has to agree with three things at once: that proxy location, the
|
||||
directory the runtime image serves from, and the value baked into the build. When
|
||||
they disagree the page loads and every stylesheet and script 404s.
|
||||
@@ -5,7 +5,12 @@ sidebar_label: Upgrading
|
||||
---
|
||||
|
||||
Upgrading the control plane is a pull and a recreate. Agents are versioned and
|
||||
upgraded separately — see [Agent updates](./agent-updates.md).
|
||||
upgraded separately see [Agent updates](./agent-updates.md).
|
||||
|
||||
:::info Cloud instances upgrade themselves
|
||||
This page is for self-hosted installs. If your instance is hosted by us, there
|
||||
is nothing here for you to do.
|
||||
:::
|
||||
|
||||
## Upgrade
|
||||
|
||||
@@ -25,7 +30,7 @@ renamed or removed.
|
||||
2. **Indexes** are ensured. Auth and settings index builders are fatal on
|
||||
failure; secret and workflow ones only warn.
|
||||
3. **Default steps** are reseeded from the image, overwriting the `default`
|
||||
library — which is why those steps are read-only.
|
||||
library which is why those steps are read-only.
|
||||
|
||||
Watch it:
|
||||
|
||||
@@ -52,28 +57,15 @@ is the reason the backup is not optional.
|
||||
The stack is not designed for it. `docker compose up -d` recreates the server
|
||||
container, which is a short interruption:
|
||||
|
||||
- Agents reconnect on their own — they retry, and the poll loop is idempotent.
|
||||
- Agents reconnect on their own they retry, and the poll loop is idempotent.
|
||||
- Workflow runs in progress lose their command stream. Steps already dispatched
|
||||
finish on the agent, but their results have nowhere to go. **Do not upgrade
|
||||
during a run.**
|
||||
- Sessions survive, because they live in Redis rather than in the server.
|
||||
|
||||
## Upgrading the hosted deployment
|
||||
|
||||
Both Compose files, together:
|
||||
|
||||
```bash
|
||||
cd /opt/vantage
|
||||
docker compose -f docker-compose.yml -f docker-compose.site.yml pull
|
||||
docker compose -f docker-compose.yml -f docker-compose.site.yml up -d --remove-orphans
|
||||
```
|
||||
|
||||
CI builds and pushes images but does **not** deploy them; rolling out is this
|
||||
manual step. See [CI/CD](./ci-cd.md).
|
||||
|
||||
## After upgrading
|
||||
|
||||
- Confirm every service is `running`.
|
||||
- Confirm servers return to `active` within a couple of poll intervals.
|
||||
- Open a page that touches encryption — a secret group — to confirm
|
||||
- Open a page that touches encryption a secret group to confirm
|
||||
`KEY_ENCRYPTION_KEY` came through.
|
||||
|
||||
Reference in New Issue
Block a user