mrhid6
b5b9775d2b
The window deadline no longer kills a running package manager: it only gates the start of each phase, and a started upgrade runs under a 2 hour backstop that sends SIGTERM on Linux. A PatchResult whose send fails is queued and flushed on the next command stream, retaking the reboot decision. deb822 folded Suites continuation lines are filtered with the field. The startup static inventory report is retried until it succeeds.
175 lines
9.4 KiB
Markdown
175 lines
9.4 KiB
Markdown
# Vantage agent (`vantage-agent`)
|
|
|
|
The lightweight Go agent installed on every managed server, and the Windows
|
|
installer that packages it. Extracted from the `vantage` monorepo with its
|
|
history; the agent is the repository root, so the module is
|
|
`gitea.hostxtra.co.uk/vantage/vantage-agent`.
|
|
|
|
```
|
|
vantage-agent/
|
|
├── cmd/main.go # flags: -generate-key
|
|
├── internal/
|
|
│ ├── checker/ # monitor check execution
|
|
│ ├── config/ # config.yaml load/save
|
|
│ ├── exec/ # workflow step execution
|
|
│ ├── grpc/ # client + the codec registration
|
|
│ ├── inventory/ # CPU/mem/disk collection (linux/other/windows)
|
|
│ ├── keys/ # authorized_keys read/diff/write
|
|
│ ├── monitors/ # agent-run monitor loop
|
|
│ ├── proxy/ # console relay, always from 127.0.0.1
|
|
│ ├── sync/ # poll loop + command stream + self-update
|
|
│ ├── updates/ # OS package update check/apply
|
|
│ ├── winexec/ # PowerShell invocation on Windows
|
|
│ └── workloads/ # containers and units/services
|
|
├── installer/ # Windows: setup.ps1, nssm.exe, WiX .wxs
|
|
└── .gitea/workflows/agent-release.yml
|
|
```
|
|
|
|
## Relationship to the other repositories
|
|
|
|
| Repository | Relationship |
|
|
| ---------------- | ------------------------------------------------------------------------------------------------------ |
|
|
| `vantage-shared` | a private Go module. `grpc/pb` and `grpc/codec` are the wire contract; `proto/` there documents them |
|
|
| `vantage` | the control plane. **No import in either direction** - the coupling is the gRPC wire, and it is entirely mediated by `vantage-shared` |
|
|
|
|
**`vantage-shared` is private**, so every Go build needs
|
|
`GOPRIVATE=gitea.hostxtra.co.uk/*` plus a credential. CI writes a netrc from
|
|
`REGISTRY_USER` + `RELEASE_TOKEN` (**that token needs read access to the
|
|
`vantage` org**) - twice, because the `msi` job is Windows and Go looks for
|
|
`_netrc` in the profile directory there, not `.netrc`. Locally, either a netrc
|
|
or `git config --global url."git@gitea.hostxtra.co.uk:".insteadOf https://gitea.hostxtra.co.uk/`.
|
|
|
|
### A wire change is three steps, in order
|
|
|
|
`shared/grpc/pb` is hand-written and shared by both sides, so a new message is a
|
|
compile error rather than a silent disagreement - but only once each side moves:
|
|
|
|
1. release `vantage-shared` (and change `proto/vantage/v1/vantage.proto` in the
|
|
same commit as the Go types)
|
|
2. bump the pin in `vantage`'s `server/go.mod` - live at the next push to main
|
|
3. bump the pin here - live only at the next `agent/v*` tag
|
|
|
|
The control plane runs ahead of the fleet in between. That was true before the
|
|
split too; it is now explicit in two `go.mod` files rather than implicit in a
|
|
shared directory.
|
|
|
|
## Releases
|
|
|
|
Tags are `agent/v*`, **not** bare `v*`, even though this repository is only the
|
|
agent. The tag is not a private detail: it is what the fleet downloads by, what
|
|
`UpdateAgentCmd` carries, and what the control plane greps release tag names for
|
|
when answering `GET /api/agent/latest-version`. Renaming it would be a second
|
|
breaking change stacked on the repository move.
|
|
|
|
```bash
|
|
git tag agent/v1.2.0 && git push origin agent/v1.2.0
|
|
```
|
|
|
|
Builds `linux/amd64`, `linux/arm64` and `windows/amd64`, writes `checksums.txt`,
|
|
creates the Gitea release. A second `msi` job on `windows-2022` builds the exe
|
|
again, packages it with WiX and appends the MSI to the same release **through
|
|
the API** - `gitea-release-action` cannot find a tag with a slash in it.
|
|
|
|
### The self-update path, and what the move broke
|
|
|
|
`internal/sync` downloads its own replacement from
|
|
`<gitea>/vantage/vantage-agent/releases/download/<tag>/…`, verifying the
|
|
SHA-256 from `checksums.txt` before swapping itself. That path is **compiled
|
|
into the binary**.
|
|
|
|
**Every agent built before this move has the old path - `mrhid6/vantage` -
|
|
baked in, and releases are no longer published there.** For those agents the
|
|
push-button update in the UI will fail: the download 404s. They are not
|
|
stranded, because `/update` and `/update.ps1` are generated by the control plane
|
|
at request time and point wherever the current server says, so re-running the
|
|
update one-liner on a host moves it onto a build that knows the new address.
|
|
After that, self-update works again permanently.
|
|
|
|
This was a deliberate choice - the alternative was publishing releases to a
|
|
repository that no longer holds the source - but it means **the fleet needs one
|
|
pass of the update one-liner**, and the control plane must be redeployed with
|
|
the new release paths *first*, or the one-liner points at the old repository
|
|
too.
|
|
|
|
## What the agent will not do
|
|
|
|
- **It reboots a host only when told to and only when owed.** An
|
|
`ApplyUpdatesCmd` with `reboot_if_required` set, on a host whose OS reports a
|
|
reboot is owed, with at least 5 minutes left before `deadline_unix`, reboots
|
|
after a one-minute grace period (`shutdown -r +1`, `shutdown /r /t 60`), and
|
|
only once the `PatchResult` announcing it has been sent. Anything else
|
|
installs and stops there, and `inventory.reboot_required` reports what is
|
|
owed.
|
|
- **It decides what it will not touch.** The protected workload set is computed
|
|
and enforced agent-side - `vantage-agent.service`, `VantageAgent` on Windows,
|
|
and its own container ID from `/proc/self/cgroup`. The control plane may name
|
|
a target; the agent decides what it will do to itself. A server-side denylist
|
|
alone would be bypassed by the next dispatch path someone adds, and the
|
|
failure is unrecoverable from the UI.
|
|
- **The console relay dials `127.0.0.1` only.** The host is hardcoded here, so
|
|
the control plane can name a port and nothing else.
|
|
- **No `authorized_keys` management on Windows**, and no package inventory: a
|
|
Windows agent never calls `ReportPackages`, so no `server_packages` document
|
|
exists for it at all - a different, earlier state than the `unsupported` a
|
|
Linux distribution reaches when its family has no security feed.
|
|
|
|
## Platform split
|
|
|
|
Windows support is build tags, not runtime branches - `systemd_linux.go` /
|
|
`services_windows.go` and the matching `control_` and `logs_` pairs. Windows
|
|
collection runs PowerShell through `internal/winexec`, and **every script that
|
|
reports data emits JSON that a build-tag-free parser reads**, so those parsers
|
|
are tested on Linux. This module has no Windows CI: the control verbs and
|
|
`serviceDisplayName` emit no JSON, have no parser, and are exercised only by
|
|
running the agent on Windows.
|
|
|
|
Windows updates go through the Windows Update COM API
|
|
(`Microsoft.Update.Session`) rather than the PSWindowsUpdate module, which would
|
|
need a PowerShell Gallery install on every host and fails on an air-gapped
|
|
fleet. `CurrentVersion` is empty on Windows and `NewVersion` carries the KB
|
|
article ID: a Windows update is not a version bump of a named package.
|
|
|
|
## Patching
|
|
|
|
`updates.Apply` takes a scope and a deadline and answers with the tail of the
|
|
package manager's output. Security-only uses `--security` on dnf/yum,
|
|
`zypper patch --category security`, the Security and Critical classifications
|
|
on Windows, and for apt a temporary `SourceParts` directory holding only the
|
|
`-security` suites (with `APT::Get::List-Cleanup=0`, or the reduced update
|
|
deletes every other list file). apk and pacman have no security metadata and
|
|
report `unsupported`; security-only never falls back to installing
|
|
everything. One run at a time: a second command answers `busy`.
|
|
|
|
The deadline (`deadline_unix`, the window end) only gates the **start** of each
|
|
phase: the apt index refresh, the upgrade command, the Windows install script.
|
|
A phase that has not started by then is refused with "the maintenance window
|
|
ended before <phase> could start" (`canStart` in `internal/updates/phase.go`).
|
|
A started upgrade is never killed by the window: it runs under a 2 hour
|
|
backstop from its own start (`defaultApplyCap`), which on Linux sends SIGTERM
|
|
and waits 5 minutes before a kill. Interrupting a package manager mid-transaction
|
|
is worse than letting it finish late.
|
|
|
|
A `PatchResult` whose send fails (the command stream reconnected while the
|
|
patch ran) is kept in a bounded queue (32, oldest dropped) and flushed on the
|
|
next stream right after `AgentReady`. A queued result that announced a reboot
|
|
has the reboot decision taken again at flush time, and the host still reboots
|
|
only once the result is delivered. The startup static inventory report is
|
|
retried every 30 seconds, up to 10 attempts: it carries the boot time the
|
|
control plane uses to prove a patch reboot.
|
|
|
|
## Two constants that mirror the control plane
|
|
|
|
Neither can be shared - this is a separate module and the control plane's are
|
|
under `internal/` - so both must change in step, by hand:
|
|
|
|
- the workload log cap, **500 lines and 256KB whichever binds first**, mirrored
|
|
in the control plane's `services.MaxWorkloadLogLines`
|
|
- `streamHealthyAfter` and the 70s command-stream watchdog, which pair with the
|
|
control plane's 20s `PingCmd`. The watchdog arms only **after** a first ping
|
|
has been seen, so an older server that sends none is treated as working rather
|
|
than put into a reconnect loop.
|
|
|
|
## Writing style
|
|
|
|
Never use em dashes (the long dash character) anywhere: code, comments, UI copy, docs, commit messages. Use a plain hyphen ` - `, a comma, a colon, or split the sentence instead.
|