Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
02a6502f12 | ||
|
|
66eed23f79 | ||
|
|
abd28f2e17 | ||
|
|
26f00c2f7f | ||
|
|
b004143ea7 | ||
|
|
f29b75e325 | ||
|
|
58bd26030c | ||
|
|
131570da01 | ||
|
|
ece5384739 | ||
|
|
3c54ac92e9 | ||
|
|
c6894e2a24 | ||
|
|
cdcb8754ad | ||
|
|
4440e14320 | ||
|
|
1afa250203 | ||
|
|
d0219def80 | ||
|
|
dc49f2d5eb | ||
|
|
2c23a794b9 | ||
|
|
943b4c32a5 | ||
|
|
c8d81944e7 | ||
|
|
5446286562 | ||
|
|
d6f4a807d0 | ||
|
|
baf99e79b6 | ||
|
|
81182ee8cf |
@@ -0,0 +1,88 @@
|
||||
# vantage-shared
|
||||
|
||||
The private Go module every Vantage service imports. Extracted from the
|
||||
`vantage` monorepo with its history.
|
||||
|
||||
```
|
||||
vantage-shared/
|
||||
├── grpc/pb/ # the agent↔control-plane wire types, hand-written
|
||||
├── grpc/codec/ # the JSON codec they travel over
|
||||
├── proto/vantage/v1/ # vantage.proto — documentation for the above
|
||||
├── mail/ # the one email system: transport + templates
|
||||
├── license/ # payload, sign, verify, trusted keys, plans
|
||||
├── models/ # Instance, User, Settings
|
||||
├── provision/ # slug rules, reserved names, instance/user creation
|
||||
├── backup/ # dump, restore, verify, manifest, fingerprint
|
||||
├── cryptobox/ # the single AES-256-GCM implementation
|
||||
├── indexes/ # users.email and instances.slug
|
||||
└── cmd/lkctl/ # issue and inspect licences by hand
|
||||
```
|
||||
|
||||
Module path is `gitea.hostxtra.co.uk/vantage/vantage-shared` — **lowercase
|
||||
`vantage`**, though the Gitea org is canonically `Vantage`. Gitea serves both
|
||||
spellings; Go module paths are case-sensitive strings, and two spellings would
|
||||
cache as two modules. Keep the lowercase one.
|
||||
|
||||
## Who depends on this, and when they find out
|
||||
|
||||
| Repository | Modules | A change here reaches them |
|
||||
| ---------------- | ---------------------------------------- | ---------------------------------------------------- |
|
||||
| `vantage` | `server`, `agent`, `vantagectl` | `server` at the next push to main after a pin bump; `agent` and `vantagectl` only at their next release tag |
|
||||
| `vantage-admin` | `server` | at the next push to main after a pin bump |
|
||||
| `vantage-site` | `server` | at the next push to main after a pin bump |
|
||||
|
||||
**Nothing bumps a pin automatically.** A fix here is live nowhere until each
|
||||
consumer's `go.mod` moves, and nothing in any of those repositories will remind
|
||||
you. That is the trade this split made: a service can no longer be shipped
|
||||
against a version of this module it was never built against, and in exchange the
|
||||
staleness is now silent rather than impossible.
|
||||
|
||||
Tag a release when you change anything: `git tag v0.2.0 && git push origin
|
||||
v0.2.0`. Consumers pin exact versions.
|
||||
|
||||
## `proto/` is documentation, not a generator input
|
||||
|
||||
`grpc/pb` is **hand-written** JSON-tagged structs over the codec in
|
||||
`grpc/codec`. Nothing generates them, and `vantage.proto` is not compiled by any
|
||||
build. It is the readable statement of the wire contract, and it lives in the
|
||||
same repository as the Go types precisely so that a message added to one can be
|
||||
added to the other **in the same commit** — that co-location is the only thing
|
||||
enforcing the match, so do not split them again.
|
||||
|
||||
There used to be two copies of `pb`, in the agent and the server, and they had
|
||||
already drifted: the agent's `UnimplementedVantageServer` was three methods
|
||||
stale and carried no `ReportWorkloads` at all. One package now serves both
|
||||
sides. The agent links the server half as dead code, which the linker drops.
|
||||
|
||||
**A wire change lands in three steps, in this order**: release this module, bump
|
||||
the pin in `vantage/server` (live at the next push to main), bump the pin in
|
||||
`vantage/agent` (live only at the next `agent/v*` tag). The control plane will
|
||||
be ahead of the fleet in between, which was true before too — it is just written
|
||||
down in two `go.mod` files now instead of implied by a shared directory.
|
||||
|
||||
## Things that mirror something outside this repository
|
||||
|
||||
These cannot be enforced by the compiler and must be changed by hand, in step
|
||||
with a file in another repository:
|
||||
|
||||
- **`backup.ciphertextFields`** names each collection's `*_enc` fields and
|
||||
mirrors `server/internal/models` in the `vantage` repository, which this
|
||||
module cannot import. Wrong field names fail **silently**: `verify`'s live
|
||||
probe finds no ciphertext and reports "this database stores no ciphertext
|
||||
yet", so the one gate that catches what a key fingerprint cannot becomes a
|
||||
no-op. `settings` is deliberately in neither that map nor
|
||||
`CiphertextCollections()` — its ESO read token is a SHA-256 hash, not
|
||||
ciphertext.
|
||||
- **`mail/templates/layout.html.tmpl`** carries the control plane's dark theme
|
||||
values as **literal hex**. Email clients support neither `var()` nor a
|
||||
reliable `prefers-color-scheme`, so the token indirection the four front ends
|
||||
use is not available here. Every colour in the email system is in that one
|
||||
file. The tokens it mirrors live in `vantage-site`.
|
||||
- **`provision`** is the single implementation of the slug rules that both the
|
||||
control plane and Vantage HQ depend on. It is the one place those two
|
||||
repositories must agree on behaviour, which is why it is here rather than
|
||||
copied — but it now also means a change to it is a release and two pin bumps.
|
||||
|
||||
`mail/render_test.go` renders every template and fails if one exists that no
|
||||
case covers. The templates are parsed in `init()`, so without that test a
|
||||
mistyped field is a boot-time panic in three services.
|
||||
@@ -0,0 +1,392 @@
|
||||
syntax = "proto3";
|
||||
|
||||
package vantage.v1;
|
||||
|
||||
option go_package="gitea.hostxtra.co.uk/mrhid6/vantage/server/internal/grpc/pb";
|
||||
|
||||
service Vantage {
|
||||
rpc Register(RegisterRequest) returns (RegisterResponse);
|
||||
rpc SyncKeys(SyncRequest) returns (SyncResponse);
|
||||
rpc UploadGeneratedKey(UploadKeyRequest) returns (UploadKeyResponse);
|
||||
rpc ReportUpdates(ReportUpdatesRequest) returns (ReportUpdatesResponse);
|
||||
rpc ReportPackages(ReportPackagesRequest) returns (ReportPackagesResponse);
|
||||
rpc ReportWorkloads(ReportWorkloadsRequest) returns (ReportWorkloadsResponse);
|
||||
rpc ReportInventory(InventoryReport) returns (InventoryReportResponse);
|
||||
rpc SyncMonitors(SyncMonitorsRequest) returns (SyncMonitorsResponse);
|
||||
rpc ReportChecks(ReportChecksRequest) returns (ReportChecksResponse);
|
||||
// Bidirectional stream: agent sends auth once, server pushes commands.
|
||||
rpc CommandStream(stream AgentMessage) returns (stream ServerCommand);
|
||||
rpc ProxyStream(stream ProxyClientMsg) returns (stream ProxyServerMsg);
|
||||
}
|
||||
|
||||
message RegisterRequest {
|
||||
string server_id = 1;
|
||||
string pre_reg_token = 2;
|
||||
string hostname = 3;
|
||||
string ip_address = 4;
|
||||
string os_info = 5;
|
||||
}
|
||||
|
||||
message RegisterResponse {
|
||||
string agent_token = 1;
|
||||
}
|
||||
|
||||
message SyncRequest {
|
||||
string server_id = 1;
|
||||
string agent_token = 2;
|
||||
string agent_version = 3;
|
||||
}
|
||||
|
||||
message SyncResponse {
|
||||
repeated string public_keys = 1;
|
||||
|
||||
// collect_packages tells the agent whether this instance's licence grants
|
||||
// vulnerability scanning. False means do not collect at all: no gRPC body,
|
||||
// no document, no storage. The server re-checks on ReportPackages — this
|
||||
// flag is the optimisation, the server check is the boundary.
|
||||
//
|
||||
// Absent reads as false, which is the safe direction: an old server that
|
||||
// does not send it leaves agents collecting nothing.
|
||||
bool collect_packages = 2;
|
||||
}
|
||||
|
||||
// ReportPackages carries a server's installed package set.
|
||||
//
|
||||
// The agent calls twice at most. The first call sends only the hash; if the
|
||||
// server already holds that hash it answers need_full = false and the ~150KB
|
||||
// body is never sent. A machine's package set changes rarely, so almost every
|
||||
// hour costs one small message.
|
||||
message ReportPackagesRequest {
|
||||
string server_id = 1;
|
||||
string agent_token = 2;
|
||||
string hash = 3;
|
||||
OSRelease os = 4;
|
||||
repeated InstalledPackage packages = 5; // empty on the offer call
|
||||
}
|
||||
|
||||
message ReportPackagesResponse {
|
||||
bool need_full = 1;
|
||||
}
|
||||
|
||||
message OSRelease {
|
||||
string family = 1;
|
||||
// version_id is not optional: Ubuntu 22.04 and 24.04 publish different fixed
|
||||
// versions for the same CVE, so a scan without it is guesswork.
|
||||
string version_id = 2;
|
||||
string arch = 3;
|
||||
}
|
||||
|
||||
message InstalledPackage {
|
||||
string name = 1;
|
||||
string version = 2;
|
||||
int32 epoch = 3;
|
||||
string arch = 4;
|
||||
// source_name is what the Debian and Ubuntu feeds are keyed on: one advisory
|
||||
// against "openssl" covers libssl3, openssl and libssl-dev.
|
||||
string source_name = 5;
|
||||
}
|
||||
|
||||
message UploadKeyRequest {
|
||||
string server_id = 1;
|
||||
string agent_token = 2;
|
||||
string public_key = 3;
|
||||
string label = 4;
|
||||
string private_key = 5;
|
||||
}
|
||||
|
||||
message UploadKeyResponse {
|
||||
string key_id = 1;
|
||||
}
|
||||
|
||||
// CommandStream messages
|
||||
|
||||
message AgentMessage {
|
||||
string server_id = 1;
|
||||
string agent_token = 2;
|
||||
oneof payload {
|
||||
AgentReady ready = 3;
|
||||
CommandResult result = 4;
|
||||
StepResult step_result = 5;
|
||||
StepOutputChunk step_output = 6;
|
||||
WorkloadLogsResult workload_logs_result = 7;
|
||||
}
|
||||
}
|
||||
|
||||
message AgentReady {
|
||||
|
||||
}
|
||||
|
||||
message CommandResult {
|
||||
string command_id = 1;
|
||||
bool success = 2;
|
||||
string message = 3;
|
||||
}
|
||||
|
||||
message PackageUpdate {
|
||||
string name = 1;
|
||||
string current_version = 2;
|
||||
string new_version = 3;
|
||||
}
|
||||
|
||||
message ReportUpdatesRequest {
|
||||
string server_id = 1;
|
||||
string agent_token = 2;
|
||||
repeated PackageUpdate updates = 3;
|
||||
}
|
||||
|
||||
message ReportUpdatesResponse {
|
||||
|
||||
}
|
||||
|
||||
message CPUReport {
|
||||
string model = 1;
|
||||
int32 cores = 2;
|
||||
double usage_pct = 3;
|
||||
double load1 = 4;
|
||||
}
|
||||
|
||||
message MemReport {
|
||||
uint64 total_bytes = 1;
|
||||
uint64 used_bytes = 2;
|
||||
}
|
||||
|
||||
message PartitionReport {
|
||||
string device = 1;
|
||||
string mountpoint = 2;
|
||||
string fstype = 3;
|
||||
uint64 total_bytes = 4;
|
||||
uint64 used_bytes = 5;
|
||||
}
|
||||
|
||||
message InventoryReport {
|
||||
string server_id = 1;
|
||||
string agent_token = 2;
|
||||
bool include_static = 3;
|
||||
CPUReport cpu = 4;
|
||||
MemReport memory = 5;
|
||||
uint64 swap_total = 6;
|
||||
uint64 swap_used = 7;
|
||||
repeated PartitionReport partitions = 8;
|
||||
string kernel = 9;
|
||||
// Set on static snapshots only. The agent never reboots; it reports that one
|
||||
// is owed and leaves the decision to a person or a workflow.
|
||||
bool reboot_required = 10;
|
||||
}
|
||||
|
||||
message InventoryReportResponse {
|
||||
|
||||
}
|
||||
|
||||
message MonitorSpec {
|
||||
string monitor_id = 1;
|
||||
string type = 2;
|
||||
string url = 3;
|
||||
string host = 4;
|
||||
int32 port = 5;
|
||||
string method = 6;
|
||||
int32 expected_status = 7;
|
||||
string keyword = 8;
|
||||
int32 tls_warn_days = 9;
|
||||
int32 interval_sec = 10;
|
||||
int32 retries = 11;
|
||||
bool insecure = 12;
|
||||
}
|
||||
|
||||
message SyncMonitorsRequest {
|
||||
string server_id = 1;
|
||||
string agent_token = 2;
|
||||
}
|
||||
|
||||
message SyncMonitorsResponse {
|
||||
repeated MonitorSpec monitors = 1;
|
||||
}
|
||||
|
||||
message CheckResult {
|
||||
string monitor_id = 1;
|
||||
bool up = 2;
|
||||
int32 latency_ms = 3;
|
||||
string message = 4;
|
||||
int64 cert_expiry_unix = 5;
|
||||
}
|
||||
|
||||
message ReportChecksRequest {
|
||||
string server_id = 1;
|
||||
string agent_token = 2;
|
||||
repeated CheckResult results = 3;
|
||||
}
|
||||
|
||||
message ReportChecksResponse {
|
||||
|
||||
}
|
||||
|
||||
message ApplyUpdatesCmd {
|
||||
|
||||
}
|
||||
|
||||
message ServerCommand {
|
||||
string command_id = 1;
|
||||
oneof command {
|
||||
GenerateKeyCmd generate_key = 2;
|
||||
DeleteKeyCmd delete_key = 3;
|
||||
UpdateAgentCmd update_agent = 4;
|
||||
ApplyUpdatesCmd apply_updates = 5;
|
||||
RunStepCmd run_step = 6;
|
||||
CleanupWorkspaceCmd cleanup_workspace = 7;
|
||||
OpenProxyCmd open_proxy = 8;
|
||||
PingCmd ping = 9;
|
||||
RefreshWorkloadsCmd refresh_workloads = 10;
|
||||
ControlWorkloadCmd control_workload = 11;
|
||||
WorkloadLogsCmd workload_logs = 12;
|
||||
}
|
||||
}
|
||||
|
||||
// PingCmd is a liveness beat, carrying nothing and requiring no reply.
|
||||
//
|
||||
// It exists because gRPC keepalive cannot prove what the agent needs to know.
|
||||
// Behind an L7 proxy the agent's HTTP/2 connection terminates at the proxy, so
|
||||
// keepalive pings are answered by the proxy whether or not the server behind it
|
||||
// is still there. A pod that dies leaves the agent blocked in Recv on a stream
|
||||
// that will never produce another message and never error — commands are
|
||||
// dispatched into it and silently lost. Only traffic that originates at the
|
||||
// server itself distinguishes a live stream from an orphaned one.
|
||||
message PingCmd {
|
||||
}
|
||||
|
||||
// CleanupWorkspaceCmd tells the agent to recursively remove the run's working
|
||||
// directory once all steps on that server have finished.
|
||||
message CleanupWorkspaceCmd {
|
||||
string workspace_id = 1;
|
||||
}
|
||||
|
||||
message DeleteKeyCmd {
|
||||
string label = 1;
|
||||
}
|
||||
|
||||
message UpdateAgentCmd {
|
||||
string version = 1; // e.g. "1.2.3"
|
||||
string gitea_base_url = 2; // e.g. "https://gitea.example.com"
|
||||
}
|
||||
|
||||
message GenerateKeyCmd {
|
||||
string label = 1;
|
||||
string key_type = 2; // ed25519 | rsa | ecdsa (default: ed25519)
|
||||
int32 key_size = 3; // bits; used for rsa and ecdsa
|
||||
string passphrase = 4; // empty = no passphrase
|
||||
string comment = 5; // embedded in public key
|
||||
}
|
||||
|
||||
message RunStepCmd {
|
||||
string interpreter = 1; // "bash" | "powershell"
|
||||
string script = 2;
|
||||
map<string, string> env = 3;
|
||||
int32 timeout_seconds = 4;
|
||||
string workspace_id = 5;
|
||||
}
|
||||
|
||||
message StepResult {
|
||||
string command_id = 1;
|
||||
int32 exit_code = 2;
|
||||
string stdout = 3;
|
||||
string stderr = 4;
|
||||
map<string, string> output_env = 5;
|
||||
}
|
||||
|
||||
message StepOutputChunk {
|
||||
string command_id = 1;
|
||||
uint64 seq = 2;
|
||||
bytes data = 3;
|
||||
bool eof = 4;
|
||||
}
|
||||
|
||||
// OpenProxyCmd tells the agent to dial 127.0.0.1:port locally and relay that
|
||||
// connection back over a fresh ProxyStream identified by proxy_id.
|
||||
message OpenProxyCmd {
|
||||
string proxy_id = 1;
|
||||
uint32 port = 2;
|
||||
}
|
||||
|
||||
message ProxyOpen {
|
||||
string server_id = 1;
|
||||
string agent_token = 2;
|
||||
string proxy_id = 3;
|
||||
}
|
||||
|
||||
message ProxyClose { string reason = 1; }
|
||||
|
||||
message ProxyClientMsg {
|
||||
oneof payload {
|
||||
ProxyOpen open = 1; // first message only
|
||||
bytes data = 2;
|
||||
ProxyClose close = 3;
|
||||
}
|
||||
}
|
||||
|
||||
message ProxyServerMsg {
|
||||
oneof payload {
|
||||
bytes data = 1;
|
||||
ProxyClose close = 2;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Workload registry
|
||||
|
||||
// ReportWorkloads carries what a server is running.
|
||||
//
|
||||
// Offer-then-send, the same handshake as ReportPackages: the agent calls once
|
||||
// with workloads empty, and resends with the body only if need_full is set.
|
||||
message ReportWorkloadsRequest {
|
||||
string server_id = 1;
|
||||
string agent_token = 2;
|
||||
string hash = 3;
|
||||
bool docker_ok = 4;
|
||||
string docker_error = 5;
|
||||
bool systemd_ok = 6;
|
||||
string systemd_error = 7;
|
||||
repeated Workload workloads = 8; // empty on the offer call
|
||||
// full marks the second call. It is not inferred from an empty workloads
|
||||
// list: a host running nothing sends an empty list as its full report.
|
||||
bool full = 9;
|
||||
}
|
||||
|
||||
message ReportWorkloadsResponse {
|
||||
bool need_full = 1;
|
||||
}
|
||||
|
||||
message Workload {
|
||||
string kind = 1; // "container" | "unit"
|
||||
string id = 2;
|
||||
string name = 3;
|
||||
string state = 4;
|
||||
string health = 5;
|
||||
string image = 6;
|
||||
string stack = 7;
|
||||
repeated string ports = 8;
|
||||
int32 restarts = 9;
|
||||
string started_at = 10; // RFC3339, empty when not running
|
||||
bool protected = 11;
|
||||
}
|
||||
|
||||
// RefreshWorkloadsCmd carries no payload back. It makes the agent report
|
||||
// immediately through ReportWorkloads, so there is exactly one writer for the
|
||||
// server_workloads collection rather than two arriving by different routes.
|
||||
message RefreshWorkloadsCmd {}
|
||||
|
||||
message ControlWorkloadCmd {
|
||||
string kind = 1;
|
||||
string id = 2;
|
||||
string action = 3; // "start" | "stop" | "restart"
|
||||
}
|
||||
|
||||
message WorkloadLogsCmd {
|
||||
string kind = 1;
|
||||
string id = 2;
|
||||
int32 tail = 3;
|
||||
}
|
||||
|
||||
message WorkloadLogsResult {
|
||||
string command_id = 1;
|
||||
string text = 2;
|
||||
bool truncated = 3;
|
||||
string error = 4;
|
||||
}
|
||||
Reference in New Issue
Block a user