docs: add Fleet Inventory and SaaS auth/orgs specs + plans
This commit is contained in:
@@ -0,0 +1,142 @@
|
||||
# Fleet Inventory — Design
|
||||
|
||||
**Date:** 2026-07-20
|
||||
**Status:** Approved (design) — ready for implementation planning
|
||||
**Scope:** Fleet Inventory only. Server Workflows and SaaS/auth are separate sub-projects.
|
||||
|
||||
---
|
||||
|
||||
## 1. Summary
|
||||
|
||||
Each agent collects hardware/OS inventory about its host and reports it to the server, which stores the latest snapshot per server and surfaces it in the UI. Two cadences:
|
||||
|
||||
- **Metrics (near-real-time):** CPU load/usage, RAM used/total, swap used/total — every **30s** (aligned with existing poll rhythm).
|
||||
- **Static inventory (slow):** disks, partitions and their usage, CPU model/cores, total RAM, OS details — every **15 min**.
|
||||
|
||||
Transport: a **new unary gRPC `ReportInventory` RPC** (mirrors the existing `ReportUpdates` pattern). No streaming.
|
||||
|
||||
---
|
||||
|
||||
## 2. Locked decisions
|
||||
|
||||
| Topic | Decision |
|
||||
|-------|----------|
|
||||
| Transport | New `ReportInventory` unary RPC. |
|
||||
| Cadence | Metrics every 30s; static inventory every 15 min. One RPC carries both, but static fields are only populated on the 15-min tick (empty/omitted otherwise → server keeps prior static snapshot). |
|
||||
| Storage | Latest snapshot embedded on the `servers` document (`inventory` sub-doc). No history/time-series in v1. |
|
||||
| Collection | Pure-Go where practical (`/proc`, `gopsutil`-style). Agent already runs as root. |
|
||||
| Platform | Linux primary; Windows agent populates what it can, leaves the rest empty. |
|
||||
|
||||
---
|
||||
|
||||
## 3. Data model
|
||||
|
||||
Add an `Inventory` sub-document to the existing `Server` model (`server/internal/models/server.go`):
|
||||
|
||||
```go
|
||||
type CPUInfo struct {
|
||||
Model string `bson:"model,omitempty" json:"model,omitempty"`
|
||||
Cores int `bson:"cores,omitempty" json:"cores,omitempty"`
|
||||
UsagePct float64 `bson:"usage_pct" json:"usage_pct"` // metrics tick
|
||||
Load1 float64 `bson:"load1,omitempty" json:"load1,omitempty"`
|
||||
}
|
||||
|
||||
type MemInfo struct {
|
||||
TotalBytes uint64 `bson:"total_bytes" json:"total_bytes"`
|
||||
UsedBytes uint64 `bson:"used_bytes" json:"used_bytes"` // metrics tick
|
||||
}
|
||||
|
||||
type Partition struct {
|
||||
Device string `bson:"device" json:"device"`
|
||||
Mountpoint string `bson:"mountpoint" json:"mountpoint"`
|
||||
Fstype string `bson:"fstype,omitempty" json:"fstype,omitempty"`
|
||||
TotalBytes uint64 `bson:"total_bytes" json:"total_bytes"`
|
||||
UsedBytes uint64 `bson:"used_bytes" json:"used_bytes"`
|
||||
}
|
||||
|
||||
type Inventory struct {
|
||||
CPU CPUInfo `bson:"cpu" json:"cpu"`
|
||||
Memory MemInfo `bson:"memory" json:"memory"`
|
||||
SwapTotalBytes uint64 `bson:"swap_total_bytes" json:"swap_total_bytes"`
|
||||
SwapUsedBytes uint64 `bson:"swap_used_bytes" json:"swap_used_bytes"`
|
||||
Partitions []Partition `bson:"partitions,omitempty" json:"partitions,omitempty"`
|
||||
Kernel string `bson:"kernel,omitempty" json:"kernel,omitempty"`
|
||||
MetricsAt *time.Time `bson:"metrics_at,omitempty" json:"metrics_at,omitempty"`
|
||||
StaticAt *time.Time `bson:"static_at,omitempty" json:"static_at,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
Add `Inventory *Inventory` field to `Server`.
|
||||
|
||||
Server-side update rules:
|
||||
- Metrics fields (`cpu.usage_pct`, `cpu.load1`, `memory.used_bytes`, swap used) always updated + `metrics_at`.
|
||||
- Static fields (`cpu.model/cores`, `memory.total_bytes`, `partitions`, `kernel`, swap total) updated only when the report includes them (non-zero/non-empty) + `static_at`.
|
||||
|
||||
---
|
||||
|
||||
## 4. gRPC protocol (`proto/vantage/v1/vantage.proto` + both `pb.go` files)
|
||||
|
||||
```protobuf
|
||||
rpc ReportInventory(InventoryReport) returns (InventoryReportResponse);
|
||||
|
||||
message InventoryReport {
|
||||
string server_id = 1;
|
||||
string agent_token = 2;
|
||||
bool include_static = 3; // true on the 15-min tick
|
||||
CPUReport cpu = 4;
|
||||
MemReport memory = 5;
|
||||
uint64 swap_total = 6;
|
||||
uint64 swap_used = 7;
|
||||
repeated PartitionReport partitions = 8; // only when include_static
|
||||
string kernel = 9; // only when include_static
|
||||
}
|
||||
message CPUReport { string model = 1; int32 cores = 2; double usage_pct = 3; double load1 = 4; }
|
||||
message MemReport { uint64 total_bytes = 1; uint64 used_bytes = 2; }
|
||||
message PartitionReport { string device = 1; string mountpoint = 2; string fstype = 3; uint64 total_bytes = 4; uint64 used_bytes = 5; }
|
||||
message InventoryReportResponse {}
|
||||
```
|
||||
|
||||
Hand-written JSON-codec structs added to `server/internal/grpc/pb/vantage.pb.go` and `agent/internal/grpc/pb/vantage.pb.go`, plus the RPC method wiring (service interface, client method, handler registration) mirroring `ReportUpdates`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Agent collection (`agent/internal/inventory/`)
|
||||
|
||||
- `Collect(includeStatic bool) *pb.InventoryReport` — reads:
|
||||
- CPU usage: sample `/proc/stat` delta; load from `/proc/loadavg`; model/cores from `/proc/cpuinfo` (static).
|
||||
- Memory/swap: `/proc/meminfo`.
|
||||
- Partitions: `/proc/mounts` filtered to real filesystems + `statfs` for total/used (static).
|
||||
- Kernel: `uname` / `/proc/version` (static).
|
||||
- Windows: best-effort via `wmic`/PS or leave empty.
|
||||
- Scheduler in the agent main loop: a 30s ticker calls `Collect(false)` and `ReportInventory`; every 30th tick (15 min) calls `Collect(true)`.
|
||||
- Reuse existing gRPC client; add `Client.ReportInventory(...)` like `ReportUpdates`.
|
||||
|
||||
Prefer implementing the `/proc` readers directly (no new heavy deps) unless a `gopsutil` dependency is already vendored.
|
||||
|
||||
---
|
||||
|
||||
## 6. Server handler + service
|
||||
|
||||
- gRPC handler `ReportInventory` in `server/internal/grpc/server.go`: validate agent token (`ValidateAgentToken`), then call `services.StoreInventory(serverID, report)`.
|
||||
- `services.StoreInventory` (in `server/internal/services/inventory.go`): builds the `$set` per the update rules in §3 and `UpdateOne` on `servers`.
|
||||
|
||||
---
|
||||
|
||||
## 7. Frontend
|
||||
|
||||
Surface inventory on the existing server detail page (`web/app/servers/[id]/page.tsx`) — add an "Inventory" panel:
|
||||
- CPU usage gauge + model/cores, load.
|
||||
- RAM used/total bar, swap bar.
|
||||
- Partitions table: device, mount, fstype, used/total with a usage bar.
|
||||
- "Updated Xs ago" from `metrics_at`/`static_at`.
|
||||
|
||||
Optionally add compact CPU/RAM badges to the servers list (`web/app/servers/page.tsx`). Reuse `@/components/ui` + Tailwind tokens. Poll the server detail query while the page is open (react-query `refetchInterval` ~30s) so metrics stay fresh.
|
||||
|
||||
---
|
||||
|
||||
## 8. Out of scope
|
||||
|
||||
- Time-series history / graphs (only latest snapshot stored).
|
||||
- Alerting thresholds on usage (settings/alerts is a separate concern).
|
||||
- Per-process / network / GPU inventory.
|
||||
- Tests (skipped, consistent with the Workflows iteration).
|
||||
@@ -0,0 +1,142 @@
|
||||
# SaaS: Auth + Organizations — Design
|
||||
|
||||
**Date:** 2026-07-20
|
||||
**Status:** Approved (design) — ready for implementation planning
|
||||
**Scope:** Local auth + organizations + per-org OIDC, and org-scoping of existing data. Billing/plan-limits explicitly deferred. Fleet Inventory and Server Workflows are separate sub-projects.
|
||||
|
||||
---
|
||||
|
||||
## 1. Summary
|
||||
|
||||
Turn Vantage from a single-admin, single global-OIDC tool into a multi-tenant app:
|
||||
|
||||
1. **Replace** the global Authentik/env-based OIDC with **local email/password accounts** as the primary login.
|
||||
2. **Organizations** — every user belongs to an org; every domain object (servers, keys, secrets, assignments, workflows, steps, runs, audit) carries an `org_id` and all queries are scoped to the caller's org.
|
||||
3. **Per-org OpenID** — an org admin can configure their own OIDC provider (issuer/client id/secret); users in that org can then sign in through it.
|
||||
|
||||
No billing, no seat/server limits this iteration (schema leaves room).
|
||||
|
||||
---
|
||||
|
||||
## 2. Locked decisions
|
||||
|
||||
| Topic | Decision |
|
||||
|-------|----------|
|
||||
| Primary auth | Local email + password (bcrypt). Replaces global Authentik. |
|
||||
| Org SSO | Per-org OIDC provider, configured by org admin, resolved dynamically at login. |
|
||||
| Isolation | `org_id` on every collection; every service query filtered by org. Enforced in the request layer via session→org. |
|
||||
| Roles | `owner`, `admin`, `member` (v1: owner/admin can manage users + org OIDC + all resources; member can use resources). Keep minimal. |
|
||||
| Bootstrapping | First-run creates the initial org + owner account (setup flow) when no users exist. |
|
||||
| Sessions | Keep existing Redis session store; session now carries `user_id`, `org_id`, `role`, `email`. |
|
||||
| Agent auth | Unchanged (per-server agent tokens). Servers gain `org_id`; agent RPCs resolve org from the server record. |
|
||||
|
||||
---
|
||||
|
||||
## 3. Data model
|
||||
|
||||
### `orgs`
|
||||
```json
|
||||
{ "_id":"ObjectId", "org_id":"uuid", "name":"Acme", "created_at":"ISODate" }
|
||||
```
|
||||
|
||||
### `users`
|
||||
```json
|
||||
{
|
||||
"_id":"ObjectId", "user_id":"uuid", "org_id":"uuid",
|
||||
"email":"a@b.com", "password_hash":"bcrypt...", "role":"owner|admin|member",
|
||||
"auth_source":"local|oidc", "created_at":"ISODate", "last_login":"ISODate|null"
|
||||
}
|
||||
```
|
||||
Unique index on `email` (global — email identifies the account and its org).
|
||||
|
||||
### `org_oidc` (per-org provider config)
|
||||
```json
|
||||
{
|
||||
"_id":"ObjectId", "org_id":"uuid",
|
||||
"issuer":"https://id.acme.com", "client_id":"...",
|
||||
"client_secret_enc":"AES...", // encrypted with existing crypto.go
|
||||
"redirect_url":"https://vantage.../auth/oidc/callback",
|
||||
"enabled": true, "updated_at":"ISODate"
|
||||
}
|
||||
```
|
||||
|
||||
### Existing collections — add `org_id`
|
||||
`servers`, `keys`, `assignments`, `secrets`, `workflows`, `workflow_steps`, `workflow_runs`, `audit` each gain `org_id string`. A **migration** backfills all existing documents into a default org (see §7).
|
||||
|
||||
---
|
||||
|
||||
## 4. Auth flows
|
||||
|
||||
### Local
|
||||
- `POST /auth/register` — only allowed during first-run bootstrap (creates org + owner) OR by an org admin inviting a user (see below). Not open self-serve.
|
||||
- `POST /auth/login` — email + password → verify bcrypt → create session with `{user_id, org_id, role, email}`.
|
||||
- `POST /auth/logout` — destroy session.
|
||||
- `GET /auth/me` — returns current user + org.
|
||||
|
||||
### Org-admin user management
|
||||
- `GET /api/org/users` / `POST /api/org/users` (create local user in caller's org) / `PUT /api/org/users/:id/role` / `DELETE /api/org/users/:id`.
|
||||
|
||||
### Per-org OIDC
|
||||
- `GET/PUT /api/org/oidc` — read/save the caller org's provider config (admin only). Secret stored encrypted.
|
||||
- `GET /auth/oidc/start?org=<org_id or slug>` — look up org's `org_oidc`, build the OIDC provider on demand (cache per org), redirect to authorize.
|
||||
- `GET /auth/oidc/callback` — exchange code, match/provision the user by email **within that org**, create session.
|
||||
- If the email exists in the org → log in. If not → provision a `member` with `auth_source=oidc` (org admin can promote). Reject if email belongs to a different org.
|
||||
|
||||
### First-run bootstrap
|
||||
- `GET /auth/bootstrap-status` → `{ needs_setup: bool }` (true when `users` is empty).
|
||||
- Setup page collects org name + owner email/password → creates org + owner → session.
|
||||
|
||||
---
|
||||
|
||||
## 5. Request scoping
|
||||
|
||||
- `auth.Middleware` already loads the session; extend `Session` to include `OrgID`, `UserID`, `Role`. Add helper `auth.OrgID(c) string`.
|
||||
- **Every service function that reads/writes a scoped collection takes an `orgID` argument** and adds `"org_id": orgID` to its filter and on insert. Handlers pass `auth.OrgID(c)`.
|
||||
- Add a `requireRole(role)` gin middleware for admin-only routes (org user mgmt, org OIDC).
|
||||
- Agent-facing gRPC: resolve `org_id` from the `servers` record (already tied to `server_id`); inventory/keys/sync operate on that org implicitly.
|
||||
|
||||
---
|
||||
|
||||
## 6. Removing global Authentik
|
||||
|
||||
- Delete/retire env-driven `InitOIDC` global provider (`OIDC_ISSUER` etc.). Keep the `go-oidc`/`oauth2` machinery but move it behind the per-org resolver.
|
||||
- `authEnabled` global replaced by "auth always on" (there is always local auth). Update `middleware.go` accordingly (no more `if !authEnabled { next }` bypass — except the bootstrap endpoints and login/register which are unauthenticated).
|
||||
- Login page (`web/app/login` or existing) offers: email/password form + "Sign in with your organization's SSO" (enter org, redirect to `/auth/oidc/start`).
|
||||
|
||||
---
|
||||
|
||||
## 7. Migration
|
||||
|
||||
One-shot migration run at startup (idempotent):
|
||||
1. If `orgs` is empty AND `servers`/`keys`/etc. contain documents without `org_id`: create a **default org** ("Default").
|
||||
2. Set `org_id = <default>` on all existing `servers`, `keys`, `assignments`, `secrets`, `workflows`, `workflow_steps`, `workflow_runs`, `audit` documents missing it.
|
||||
3. If `OIDC_ISSUER` env was set previously and an admin email is known, optionally seed an owner user (documented manual step) — otherwise first-run bootstrap handles owner creation.
|
||||
Guard with a marker (e.g. a `migrations` collection entry) so it runs once.
|
||||
|
||||
---
|
||||
|
||||
## 8. Frontend
|
||||
|
||||
- **Login/Setup:** `web/app/login/page.tsx` (email/password + org SSO entry) and `web/app/setup/page.tsx` (first-run). Redirect logic based on `bootstrap-status` and `auth/me`.
|
||||
- **Org settings:** `web/app/settings/org/` — members list + invite/create user + role management; OIDC provider form (issuer/client id/secret/enabled).
|
||||
- Existing pages unchanged functionally but now implicitly org-scoped by the backend. Show current org + user in the sidebar/header.
|
||||
|
||||
---
|
||||
|
||||
## 9. Security
|
||||
|
||||
- Passwords: bcrypt (cost ≥ 12). Never returned.
|
||||
- Org OIDC client secret encrypted at rest (reuse `services/crypto.go` AES).
|
||||
- Cross-org access prevented at the service layer (org_id in every filter) — the primary isolation boundary. Handlers must never accept an `org_id` from the client; always derive from session.
|
||||
- OIDC callback must bind the returned identity to the org that initiated the flow (state carries org_id) to prevent org-mixing.
|
||||
- Role checks on all org-admin mutations.
|
||||
|
||||
---
|
||||
|
||||
## 10. Out of scope
|
||||
|
||||
- Billing, plans, seat/server limits.
|
||||
- Cross-org resource sharing, org switching for a single user (one user = one org in v1).
|
||||
- SCIM / directory sync, SAML.
|
||||
- Email delivery for invites (create-user sets a password or invite token; email sending deferred — document as manual/console output).
|
||||
- Tests (skipped, consistent with prior iterations).
|
||||
Reference in New Issue
Block a user