23 Commits
Author SHA1 Message Date
mrhid6 02a6502f12 docs: add CLAUDE.md covering the module and its mirrored contracts 2026-09-08 08:43:44 +00:00
mrhid6 66eed23f79 chore: bring proto/ in with its history 2026-09-08 08:43:17 +00:00
mrhid6 abd28f2e17 feat: Report whether a managed host is waiting on a reboot 2026-08-13 10:39:27 +00:00
mrhid6 26f00c2f7f feat: store workload reports and route log results 2026-08-07 08:56:26 +01:00
mrhid6 b004143ea7 feat: workload registry proto messages 2026-08-07 08:53:33 +01:00
mrhid6 f29b75e325 feat: ReportPackages wire types with hash short-circuit
The pb packages are hand-written, not protoc-generated, and the wire
codec is JSON (encoding.RegisterCodec(JSONCodec{})). Field numbers in
the .proto are documentation; JSON field names are the contract. Both pb
packages edited by hand to match.

SyncResponse.collect_packages is omitempty and absent decodes as false,
so an older server leaves agents collecting nothing rather than
collecting without a licence.
2026-08-06 13:17:44 +01:00
mrhid6 58bd26030c feat: Added ping command 2026-07-31 17:10:59 +01:00
mrhid6 131570da01 feat: add ProxyStream wire types for agent-relayed console 2026-07-29 12:37:37 +01:00
mrhid6 ece5384739 chore: updated deps 2026-07-28 15:07:19 +01:00
mrhid6 3c54ac92e9 feat: Updated package path to match repo 2026-07-28 10:01:40 +01:00
mrhid6 c6894e2a24 feat: edit monitors + notification channels; HTTP monitor insecure-TLS option 2026-07-21 14:55:41 +01:00
mrhid6 cdcb8754ad feat(proto): SyncMonitors + ReportChecks RPCs 2026-07-21 14:18:49 +01:00
mrhid6 4440e14320 feat(proto): add ReportInventory RPC and inventory model 2026-07-21 14:04:28 +01:00
mrhid6 1afa250203 feat: More verbose logging on workflow logs 2026-07-20 17:35:58 +01:00
mrhid6 d0219def80 feat(proto): add StepOutputChunk streaming message 2026-07-20 12:34:32 +01:00
mrhid6 dc49f2d5eb feat(proto): add RunStepCmd and StepResult messages 2026-07-20 11:24:08 +01:00
domrichardson 2c23a794b9 feat: Added package management 2026-06-24 16:31:51 +01:00
domrichardson 943b4c32a5 feat: Updated brand to be vantage 2026-06-24 15:48:13 +01:00
domrichardson c8d81944e7 feat: update agent button on server page 2026-06-24 14:33:17 +01:00
domrichardson 5446286562 updates 2026-06-24 13:57:48 +01:00
domrichardson d6f4a807d0 updates 2026-06-16 10:28:46 +01:00
domrichardson baf99e79b6 updates 2026-06-16 09:37:32 +01:00
domrichardson 81182ee8cf first commit 2026-06-15 13:58:45 +01:00
2 changed files with 480 additions and 0 deletions
+88
View File
@@ -0,0 +1,88 @@
# vantage-shared
The private Go module every Vantage service imports. Extracted from the
`vantage` monorepo with its history.
```
vantage-shared/
├── grpc/pb/ # the agent↔control-plane wire types, hand-written
├── grpc/codec/ # the JSON codec they travel over
├── proto/vantage/v1/ # vantage.proto — documentation for the above
├── mail/ # the one email system: transport + templates
├── license/ # payload, sign, verify, trusted keys, plans
├── models/ # Instance, User, Settings
├── provision/ # slug rules, reserved names, instance/user creation
├── backup/ # dump, restore, verify, manifest, fingerprint
├── cryptobox/ # the single AES-256-GCM implementation
├── indexes/ # users.email and instances.slug
└── cmd/lkctl/ # issue and inspect licences by hand
```
Module path is `gitea.hostxtra.co.uk/vantage/vantage-shared` — **lowercase
`vantage`**, though the Gitea org is canonically `Vantage`. Gitea serves both
spellings; Go module paths are case-sensitive strings, and two spellings would
cache as two modules. Keep the lowercase one.
## Who depends on this, and when they find out
| Repository | Modules | A change here reaches them |
| ---------------- | ---------------------------------------- | ---------------------------------------------------- |
| `vantage` | `server`, `agent`, `vantagectl` | `server` at the next push to main after a pin bump; `agent` and `vantagectl` only at their next release tag |
| `vantage-admin` | `server` | at the next push to main after a pin bump |
| `vantage-site` | `server` | at the next push to main after a pin bump |
**Nothing bumps a pin automatically.** A fix here is live nowhere until each
consumer's `go.mod` moves, and nothing in any of those repositories will remind
you. That is the trade this split made: a service can no longer be shipped
against a version of this module it was never built against, and in exchange the
staleness is now silent rather than impossible.
Tag a release when you change anything: `git tag v0.2.0 && git push origin
v0.2.0`. Consumers pin exact versions.
## `proto/` is documentation, not a generator input
`grpc/pb` is **hand-written** JSON-tagged structs over the codec in
`grpc/codec`. Nothing generates them, and `vantage.proto` is not compiled by any
build. It is the readable statement of the wire contract, and it lives in the
same repository as the Go types precisely so that a message added to one can be
added to the other **in the same commit** — that co-location is the only thing
enforcing the match, so do not split them again.
There used to be two copies of `pb`, in the agent and the server, and they had
already drifted: the agent's `UnimplementedVantageServer` was three methods
stale and carried no `ReportWorkloads` at all. One package now serves both
sides. The agent links the server half as dead code, which the linker drops.
**A wire change lands in three steps, in this order**: release this module, bump
the pin in `vantage/server` (live at the next push to main), bump the pin in
`vantage/agent` (live only at the next `agent/v*` tag). The control plane will
be ahead of the fleet in between, which was true before too — it is just written
down in two `go.mod` files now instead of implied by a shared directory.
## Things that mirror something outside this repository
These cannot be enforced by the compiler and must be changed by hand, in step
with a file in another repository:
- **`backup.ciphertextFields`** names each collection's `*_enc` fields and
mirrors `server/internal/models` in the `vantage` repository, which this
module cannot import. Wrong field names fail **silently**: `verify`'s live
probe finds no ciphertext and reports "this database stores no ciphertext
yet", so the one gate that catches what a key fingerprint cannot becomes a
no-op. `settings` is deliberately in neither that map nor
`CiphertextCollections()` — its ESO read token is a SHA-256 hash, not
ciphertext.
- **`mail/templates/layout.html.tmpl`** carries the control plane's dark theme
values as **literal hex**. Email clients support neither `var()` nor a
reliable `prefers-color-scheme`, so the token indirection the four front ends
use is not available here. Every colour in the email system is in that one
file. The tokens it mirrors live in `vantage-site`.
- **`provision`** is the single implementation of the slug rules that both the
control plane and Vantage HQ depend on. It is the one place those two
repositories must agree on behaviour, which is why it is here rather than
copied — but it now also means a change to it is a release and two pin bumps.
`mail/render_test.go` renders every template and fails if one exists that no
case covers. The templates are parsed in `init()`, so without that test a
mistyped field is a boot-time panic in three services.
+392
View File
@@ -0,0 +1,392 @@
syntax = "proto3";
package vantage.v1;
option go_package="gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/grpc/pb";
service Vantage {
rpc Register(RegisterRequest) returns (RegisterResponse);
rpc SyncKeys(SyncRequest) returns (SyncResponse);
rpc UploadGeneratedKey(UploadKeyRequest) returns (UploadKeyResponse);
rpc ReportUpdates(ReportUpdatesRequest) returns (ReportUpdatesResponse);
rpc ReportPackages(ReportPackagesRequest) returns (ReportPackagesResponse);
rpc ReportWorkloads(ReportWorkloadsRequest) returns (ReportWorkloadsResponse);
rpc ReportInventory(InventoryReport) returns (InventoryReportResponse);
rpc SyncMonitors(SyncMonitorsRequest) returns (SyncMonitorsResponse);
rpc ReportChecks(ReportChecksRequest) returns (ReportChecksResponse);
// Bidirectional stream: agent sends auth once, server pushes commands.
rpc CommandStream(stream AgentMessage) returns (stream ServerCommand);
rpc ProxyStream(stream ProxyClientMsg) returns (stream ProxyServerMsg);
}
message RegisterRequest {
string server_id = 1;
string pre_reg_token = 2;
string hostname = 3;
string ip_address = 4;
string os_info = 5;
}
message RegisterResponse {
string agent_token = 1;
}
message SyncRequest {
string server_id = 1;
string agent_token = 2;
string agent_version = 3;
}
message SyncResponse {
repeated string public_keys = 1;
// collect_packages tells the agent whether this instance's licence grants
// vulnerability scanning. False means do not collect at all: no gRPC body,
// no document, no storage. The server re-checks on ReportPackages — this
// flag is the optimisation, the server check is the boundary.
//
// Absent reads as false, which is the safe direction: an old server that
// does not send it leaves agents collecting nothing.
bool collect_packages = 2;
}
// ReportPackages carries a server's installed package set.
//
// The agent calls twice at most. The first call sends only the hash; if the
// server already holds that hash it answers need_full = false and the ~150KB
// body is never sent. A machine's package set changes rarely, so almost every
// hour costs one small message.
message ReportPackagesRequest {
string server_id = 1;
string agent_token = 2;
string hash = 3;
OSRelease os = 4;
repeated InstalledPackage packages = 5; // empty on the offer call
}
message ReportPackagesResponse {
bool need_full = 1;
}
message OSRelease {
string family = 1;
// version_id is not optional: Ubuntu 22.04 and 24.04 publish different fixed
// versions for the same CVE, so a scan without it is guesswork.
string version_id = 2;
string arch = 3;
}
message InstalledPackage {
string name = 1;
string version = 2;
int32 epoch = 3;
string arch = 4;
// source_name is what the Debian and Ubuntu feeds are keyed on: one advisory
// against "openssl" covers libssl3, openssl and libssl-dev.
string source_name = 5;
}
message UploadKeyRequest {
string server_id = 1;
string agent_token = 2;
string public_key = 3;
string label = 4;
string private_key = 5;
}
message UploadKeyResponse {
string key_id = 1;
}
// CommandStream messages
message AgentMessage {
string server_id = 1;
string agent_token = 2;
oneof payload {
AgentReady ready = 3;
CommandResult result = 4;
StepResult step_result = 5;
StepOutputChunk step_output = 6;
WorkloadLogsResult workload_logs_result = 7;
}
}
message AgentReady {
}
message CommandResult {
string command_id = 1;
bool success = 2;
string message = 3;
}
message PackageUpdate {
string name = 1;
string current_version = 2;
string new_version = 3;
}
message ReportUpdatesRequest {
string server_id = 1;
string agent_token = 2;
repeated PackageUpdate updates = 3;
}
message ReportUpdatesResponse {
}
message CPUReport {
string model = 1;
int32 cores = 2;
double usage_pct = 3;
double load1 = 4;
}
message MemReport {
uint64 total_bytes = 1;
uint64 used_bytes = 2;
}
message PartitionReport {
string device = 1;
string mountpoint = 2;
string fstype = 3;
uint64 total_bytes = 4;
uint64 used_bytes = 5;
}
message InventoryReport {
string server_id = 1;
string agent_token = 2;
bool include_static = 3;
CPUReport cpu = 4;
MemReport memory = 5;
uint64 swap_total = 6;
uint64 swap_used = 7;
repeated PartitionReport partitions = 8;
string kernel = 9;
// Set on static snapshots only. The agent never reboots; it reports that one
// is owed and leaves the decision to a person or a workflow.
bool reboot_required = 10;
}
message InventoryReportResponse {
}
message MonitorSpec {
string monitor_id = 1;
string type = 2;
string url = 3;
string host = 4;
int32 port = 5;
string method = 6;
int32 expected_status = 7;
string keyword = 8;
int32 tls_warn_days = 9;
int32 interval_sec = 10;
int32 retries = 11;
bool insecure = 12;
}
message SyncMonitorsRequest {
string server_id = 1;
string agent_token = 2;
}
message SyncMonitorsResponse {
repeated MonitorSpec monitors = 1;
}
message CheckResult {
string monitor_id = 1;
bool up = 2;
int32 latency_ms = 3;
string message = 4;
int64 cert_expiry_unix = 5;
}
message ReportChecksRequest {
string server_id = 1;
string agent_token = 2;
repeated CheckResult results = 3;
}
message ReportChecksResponse {
}
message ApplyUpdatesCmd {
}
message ServerCommand {
string command_id = 1;
oneof command {
GenerateKeyCmd generate_key = 2;
DeleteKeyCmd delete_key = 3;
UpdateAgentCmd update_agent = 4;
ApplyUpdatesCmd apply_updates = 5;
RunStepCmd run_step = 6;
CleanupWorkspaceCmd cleanup_workspace = 7;
OpenProxyCmd open_proxy = 8;
PingCmd ping = 9;
RefreshWorkloadsCmd refresh_workloads = 10;
ControlWorkloadCmd control_workload = 11;
WorkloadLogsCmd workload_logs = 12;
}
}
// PingCmd is a liveness beat, carrying nothing and requiring no reply.
//
// It exists because gRPC keepalive cannot prove what the agent needs to know.
// Behind an L7 proxy the agent's HTTP/2 connection terminates at the proxy, so
// keepalive pings are answered by the proxy whether or not the server behind it
// is still there. A pod that dies leaves the agent blocked in Recv on a stream
// that will never produce another message and never error — commands are
// dispatched into it and silently lost. Only traffic that originates at the
// server itself distinguishes a live stream from an orphaned one.
message PingCmd {
}
// CleanupWorkspaceCmd tells the agent to recursively remove the run's working
// directory once all steps on that server have finished.
message CleanupWorkspaceCmd {
string workspace_id = 1;
}
message DeleteKeyCmd {
string label = 1;
}
message UpdateAgentCmd {
string version = 1; // e.g. "1.2.3"
string gitea_base_url = 2; // e.g. "https://gitea.example.com"
}
message GenerateKeyCmd {
string label = 1;
string key_type = 2; // ed25519 | rsa | ecdsa (default: ed25519)
int32 key_size = 3; // bits; used for rsa and ecdsa
string passphrase = 4; // empty = no passphrase
string comment = 5; // embedded in public key
}
message RunStepCmd {
string interpreter = 1; // "bash" | "powershell"
string script = 2;
map<string, string> env = 3;
int32 timeout_seconds = 4;
string workspace_id = 5;
}
message StepResult {
string command_id = 1;
int32 exit_code = 2;
string stdout = 3;
string stderr = 4;
map<string, string> output_env = 5;
}
message StepOutputChunk {
string command_id = 1;
uint64 seq = 2;
bytes data = 3;
bool eof = 4;
}
// OpenProxyCmd tells the agent to dial 127.0.0.1:port locally and relay that
// connection back over a fresh ProxyStream identified by proxy_id.
message OpenProxyCmd {
string proxy_id = 1;
uint32 port = 2;
}
message ProxyOpen {
string server_id = 1;
string agent_token = 2;
string proxy_id = 3;
}
message ProxyClose { string reason = 1; }
message ProxyClientMsg {
oneof payload {
ProxyOpen open = 1; // first message only
bytes data = 2;
ProxyClose close = 3;
}
}
message ProxyServerMsg {
oneof payload {
bytes data = 1;
ProxyClose close = 2;
}
}
// ---------------------------------------------------------------------------
// Workload registry
// ReportWorkloads carries what a server is running.
//
// Offer-then-send, the same handshake as ReportPackages: the agent calls once
// with workloads empty, and resends with the body only if need_full is set.
message ReportWorkloadsRequest {
string server_id = 1;
string agent_token = 2;
string hash = 3;
bool docker_ok = 4;
string docker_error = 5;
bool systemd_ok = 6;
string systemd_error = 7;
repeated Workload workloads = 8; // empty on the offer call
// full marks the second call. It is not inferred from an empty workloads
// list: a host running nothing sends an empty list as its full report.
bool full = 9;
}
message ReportWorkloadsResponse {
bool need_full = 1;
}
message Workload {
string kind = 1; // "container" | "unit"
string id = 2;
string name = 3;
string state = 4;
string health = 5;
string image = 6;
string stack = 7;
repeated string ports = 8;
int32 restarts = 9;
string started_at = 10; // RFC3339, empty when not running
bool protected = 11;
}
// RefreshWorkloadsCmd carries no payload back. It makes the agent report
// immediately through ReportWorkloads, so there is exactly one writer for the
// server_workloads collection rather than two arriving by different routes.
message RefreshWorkloadsCmd {}
message ControlWorkloadCmd {
string kind = 1;
string id = 2;
string action = 3; // "start" | "stop" | "restart"
}
message WorkloadLogsCmd {
string kind = 1;
string id = 2;
int32 tail = 3;
}
message WorkloadLogsResult {
string command_id = 1;
string text = 2;
bool truncated = 3;
string error = 4;
}