docs: design spec for the MCP server feature
This commit is contained in:
@@ -0,0 +1,403 @@
|
||||
# MCP server
|
||||
|
||||
Date: 2026-09-08
|
||||
|
||||
## Goal
|
||||
|
||||
Expose Vantage to LLM agents as a first-class tool surface, so that an agent
|
||||
acting for a user can answer questions about the fleet and — when explicitly
|
||||
permitted — act on it, under the same identity, scopes, licence and audit trail
|
||||
as every other API caller.
|
||||
|
||||
Concretely: a user mints a Vantage API token, points Claude (or any MCP client)
|
||||
at `https://<instance>/api/mcp`, and asks "which hosts are still on OpenSSL
|
||||
1.1?" or "run the patch workflow on staging". Nothing an agent can do is
|
||||
something the token's owner could not already do through the UI.
|
||||
|
||||
Out of scope, deliberately:
|
||||
|
||||
- **OAuth 2.1 authorization server.** A first-class remote connector on
|
||||
claude.ai would need one. Bearer tokens work today in every client that
|
||||
matters, and the token model already exists. Its own sub-project.
|
||||
- **MCP resources and prompts.** Client support is uneven, and tools alone are
|
||||
the whole value. Both can be added later without a protocol break.
|
||||
- **Secret reveal.** Secret names and metadata are exposed; plaintext never is,
|
||||
at any scope. An LLM context window is the wrong place for a credential, and
|
||||
the human path through the UI still exists.
|
||||
- **Console and exec.** Interactive terminal access is a streaming, stateful
|
||||
problem that does not fit a tool call, and an agent with a shell is a
|
||||
different security conversation.
|
||||
- **An approval queue.** Writes are gated by an explicit scope, not by a
|
||||
human-in-the-loop workflow. A pending-action subsystem is a real feature and
|
||||
would roughly double this one.
|
||||
- **Agent-authored workflows.** Generating and saving a workflow from natural
|
||||
language is compelling and separable. Not here.
|
||||
|
||||
## Current state
|
||||
|
||||
Most of the hard parts already exist, which is the reason this is worth doing
|
||||
now rather than as a large project later.
|
||||
|
||||
| Capability | Where |
|
||||
| --- | --- |
|
||||
| Bearer token auth, role recomputation, expiry auditing | `auth.sessionFromToken` |
|
||||
| Token model, hash-only storage, immutable role and scopes | `models.APIToken` |
|
||||
| Coarse scope vocabulary, write implies read | `services.ScopeResources` |
|
||||
| Route-to-scope map, boot-time completeness assertion | `api.routeScopes`, `AssertScopeMapComplete` |
|
||||
| Licence feature gate as middleware | `api.RequireFeature`, `license.HasFeature` |
|
||||
| Feature catalogue and Paddle pricing | `models.CatalogueRow`, `catalogue.LineItems` |
|
||||
| Tag selectors over servers | `services.MatchesTags`, `ListServersFiltered`, `ResolveTargets` |
|
||||
| Audit event write | `services.LogEvent` |
|
||||
|
||||
Three things do not exist: any MCP protocol handling, any notion of a
|
||||
credential restricted to part of the fleet, and any licence feature for either.
|
||||
|
||||
## Approach
|
||||
|
||||
A new package `server/internal/mcp` registers a tool set against the existing
|
||||
service layer and serves it over Streamable HTTP at `/api/mcp`, mounted inside
|
||||
the existing `/api` group so that every middleware already on that group applies
|
||||
unchanged.
|
||||
|
||||
The design principle throughout: **MCP is a presentation layer over the service
|
||||
layer, and introduces no new authority.** It calls the same service functions
|
||||
the REST handlers call, and every decision about who may do what is made by
|
||||
machinery that already exists. Where MCP needs something new — tag-scoped
|
||||
tokens — that thing is built as a general capability of the API, not as an MCP
|
||||
feature.
|
||||
|
||||
Three independent gates gate every tool call, and all three must pass:
|
||||
|
||||
1. The licence grants `license.FeatureMCP`.
|
||||
2. The token holds `mcp:read` (any tool) or `mcp:write` (write tools).
|
||||
3. The token holds the per-tool resource scope, e.g. `workflows:write`.
|
||||
|
||||
## Scope vocabulary
|
||||
|
||||
`services.ScopeResources` gains one entry, `"mcp"`. That is the whole change:
|
||||
`AllScopes()` derives `mcp:read` and `mcp:write` from it, `validScope` accepts
|
||||
them, `ScopeSatisfied` already implements write-implies-read, and the token
|
||||
creation UI advertises them without modification.
|
||||
|
||||
A bespoke `mcp:use` scope was rejected. The vocabulary is deliberately uniform —
|
||||
every resource has exactly `:read` and `:write` — and one special-cased action
|
||||
verb would be the first exception in a table whose value is having none.
|
||||
|
||||
The meanings:
|
||||
|
||||
- **`mcp:read`** — the token may reach `/api/mcp` at all. A token without it is
|
||||
not an agent token, whatever else it holds. Read tools are listed and callable
|
||||
subject to their own resource scopes.
|
||||
- **`mcp:write`** — write tools are listed and callable, again subject to their
|
||||
own resource scopes. Implied by the existing rule when a token holds
|
||||
`mcp:write`, so `mcp:read` need not be requested separately.
|
||||
|
||||
Write tools are **omitted from `tools/list`** for a token without `mcp:write`,
|
||||
not merely refused on call. An agent cannot be talked into using a tool it has
|
||||
never been told exists, and a read-only agent that cannot see destructive tools
|
||||
produces better behaviour than one that keeps trying them and reading errors.
|
||||
|
||||
## Tag-scoped API tokens
|
||||
|
||||
This is a general API token capability, not an MCP one, and it ships ungated by
|
||||
licence. Restricting what a credential can touch is a security control, and
|
||||
putting a security control behind a paywall is the wrong instinct.
|
||||
|
||||
`models.APIToken` gains:
|
||||
|
||||
```go
|
||||
// TagSelector restricts this token to servers carrying every tag in the map.
|
||||
// Empty or nil means the whole fleet. Immutable after creation, like Role and
|
||||
// Scopes: narrowing or widening what a deployed credential reaches, with no
|
||||
// record of what it reached before, is worse than requiring a rotation.
|
||||
TagSelector map[string]string `bson:"tag_selector,omitempty" json:"tag_selector,omitempty"`
|
||||
```
|
||||
|
||||
Validated on creation by the existing `services.ValidateTags`, so a token
|
||||
selector cannot express a tag a server could never carry. A caller may only
|
||||
create a token whose selector is at least as narrow as their own — the same
|
||||
rule `ScopeSatisfied` already enforces for scopes, applied to tags.
|
||||
|
||||
`auth.Session` carries `TagSelector`, populated in `sessionFromToken` and always
|
||||
empty for a cookie session. `auth.ServerScope(c)` returns it.
|
||||
|
||||
### Enforcement
|
||||
|
||||
The selector is intersected at the points where servers are resolved, not at
|
||||
each handler:
|
||||
|
||||
| Path | Change |
|
||||
| --- | --- |
|
||||
| `services.ListServers` | Handlers call `ListServersFiltered` with the session selector merged into any request selector |
|
||||
| `services.GetServer` | Returns not-found when `!MatchesTags(srv, sel)` |
|
||||
| `services.ResolveTargets` | Intersects the caller's selector with the requested one; a request naming an out-of-scope ID resolves to nothing |
|
||||
|
||||
`ResolveTargets` is the chokepoint that matters most: workflow runs, console
|
||||
connections and update application all pass through it, so a correct
|
||||
intersection there covers the mutating surface.
|
||||
|
||||
**Out-of-scope hosts read as 404, never 403.** A scoped token must not be able
|
||||
to enumerate the fleet it cannot see by observing which IDs answer differently.
|
||||
|
||||
### Completeness assertion
|
||||
|
||||
Mirroring `AssertScopeMapComplete`, a boot-time assertion in `api` lists every
|
||||
route that returns or acts on server-derived data and asserts each is declared
|
||||
either tag-filtered or explicitly fleet-wide. A route added tomorrow that reads
|
||||
server data without honouring the selector fails at deploy rather than leaking
|
||||
silently. The precedent is deliberate: this codebase already prefers a
|
||||
maintained map that fails boot over a decorator someone can forget.
|
||||
|
||||
## Licence feature
|
||||
|
||||
`vantage-shared/license` gains:
|
||||
|
||||
```go
|
||||
FeatureMCP = "mcp" // agent access over the Model Context Protocol
|
||||
```
|
||||
|
||||
No plan bundles it. Every tier's `Features` stays `[]string{}`, consistent with
|
||||
console and OIDC being opt-in per customer.
|
||||
|
||||
Enforced in three places:
|
||||
|
||||
1. **Route** — `RequireFeature(license.FeatureMCP)` on the `/api/mcp` group,
|
||||
answering the standard `feature_unavailable` 403.
|
||||
2. **Token minting** — creating a token with `mcp:read` or `mcp:write` is
|
||||
refused without the feature. A licence downgrade should not leave live agent
|
||||
credentials that fail confusingly mid-conversation, and the same
|
||||
guard-at-source thinking is already in `services/packages.go`.
|
||||
3. **UI** — the token form's MCP scopes and the MCP connection panel are hidden
|
||||
when the licence does not grant it, as console is today.
|
||||
|
||||
Existing tokens are unaffected: absent the new scopes, no token can reach the
|
||||
endpoint, so enabling the feature grants nothing by itself.
|
||||
|
||||
## Transport and protocol
|
||||
|
||||
Streamable HTTP, stateless. `POST /api/mcp` carries the JSON-RPC request and
|
||||
returns either a JSON response or an SSE stream; `GET /api/mcp` opens the
|
||||
server-to-client stream where a client asks for one. No session resumption in
|
||||
v1 — each request stands alone, which is what lets the endpoint sit behind
|
||||
ordinary request middleware with no special-casing.
|
||||
|
||||
Protocol framing comes from `github.com/modelcontextprotocol/go-sdk`. Everything
|
||||
below the framing is the existing service layer, called directly in-process.
|
||||
The MCP layer never issues HTTP requests to Vantage's own API: doing so would
|
||||
duplicate auth and double every request's cost for no benefit.
|
||||
|
||||
`routeScopes` gains `POST /api/mcp` and `GET /api/mcp`, both mapped to
|
||||
`mcp:read`, satisfying `AssertScopeMapComplete`. Per-tool scope enforcement
|
||||
happens inside the handler, because one route serves many operations — this is
|
||||
the first route where the route-level scope is a floor rather than the whole
|
||||
answer, and the map entry's comment says so.
|
||||
|
||||
Server metadata advertises the instance name and Vantage version, so a user with
|
||||
several instances connected can tell them apart in a client.
|
||||
|
||||
## Tool set
|
||||
|
||||
Roughly twenty tools, written to how an agent asks questions rather than to how
|
||||
the REST API is shaped. Each declares its resource scope and whether it is a
|
||||
write.
|
||||
|
||||
| Tool | Scope | Write |
|
||||
| --- | --- | --- |
|
||||
| `list_servers` | `servers:read` | |
|
||||
| `get_server` | `servers:read` | |
|
||||
| `search_fleet` | `vulns:read` | |
|
||||
| `list_monitors` | `monitors:read` | |
|
||||
| `get_monitor_status` | `monitors:read` | |
|
||||
| `list_incidents` | `monitors:read` | |
|
||||
| `get_monitor_samples` | `monitors:read` | |
|
||||
| `list_pending_updates` | `servers:read` | |
|
||||
| `list_vulnerabilities` | `vulns:read` | |
|
||||
| `get_server_packages` | `vulns:read` | |
|
||||
| `list_workflows` | `workflows:read` | |
|
||||
| `get_workflow` | `workflows:read` | |
|
||||
| `get_run` | `workflows:read` | |
|
||||
| `get_run_logs` | `workflows:read` | |
|
||||
| `list_audit_events` | `settings:read` | |
|
||||
| `list_secret_names` | `secrets:read` | |
|
||||
| `run_workflow` | `workflows:write` | yes |
|
||||
| `cancel_run` | `workflows:write` | yes |
|
||||
| `apply_updates` | `servers:write` | yes |
|
||||
| `update_agent` | `servers:write` | yes |
|
||||
| `assign_key` | `keys:write` | yes |
|
||||
|
||||
Rules every tool follows:
|
||||
|
||||
- **Trimmed projections, not API JSON.** `list_servers` over thirty hosts must
|
||||
cost a few hundred tokens, not several thousand. Each tool defines its own
|
||||
response struct containing what an agent needs to decide what to do next, and
|
||||
a `get_*` tool exists for the detail.
|
||||
- **Pagination with a hard cap.** Every list takes `limit` and `cursor`, caps
|
||||
`limit`, and states the total so an agent knows it is seeing a page.
|
||||
- **Descriptions state blast radius in plain words.** A tool description is
|
||||
prompt text; `run_workflow` says that it executes commands on real servers.
|
||||
- **No blocking.** `run_workflow` returns a run ID immediately. The agent polls
|
||||
`get_run`. A tool call must never hold a connection open for a long job.
|
||||
- **Fan-out guard.** Any write tool resolving more than a configurable number of
|
||||
servers (default 25) refuses unless called with `confirm: true`, and says how
|
||||
many it would have touched. Cheap insurance against a mis-parsed selector
|
||||
reaching the whole fleet.
|
||||
|
||||
## Audit
|
||||
|
||||
Every tool call writes an audit event through `services.LogEvent`, reads
|
||||
included. The point of an agent-facing surface is being able to reconstruct
|
||||
afterwards what the agent looked at, not only what it changed.
|
||||
|
||||
Event type `mcp.tool_call`; actor is the token name, as REST token actions
|
||||
already record; detail is the tool name, a compact argument summary, and the
|
||||
number of servers affected. Failures record `mcp.tool_denied` with the gate that
|
||||
refused — licence, MCP scope, resource scope, or tag selector — which is what
|
||||
turns "the agent said it couldn't" into a diagnosable event.
|
||||
|
||||
Arguments are summarised, never dumped verbatim: an argument could carry
|
||||
arbitrary text from a model, and the audit log is read by humans in a UI.
|
||||
|
||||
A chatty agent can produce many events. If that becomes a problem the throttle
|
||||
pattern already used for `token.expired_use` applies, but v1 records everything —
|
||||
under-recording a new and sensitive surface is the worse failure.
|
||||
|
||||
## Errors
|
||||
|
||||
Scope, licence and selector failures return **MCP tool errors**, not transport
|
||||
errors, carrying a plain-language remedy: "this token does not hold
|
||||
workflows:write". The agent must be able to read the refusal and adapt or tell
|
||||
its user, and a transport-level failure is invisible to the model.
|
||||
|
||||
Out-of-scope hosts are not-found, matching the REST rule. Upstream service
|
||||
errors are summarised — a raw Mongo error is neither useful to a model nor safe
|
||||
to expose.
|
||||
|
||||
## HQ, catalogue and Paddle
|
||||
|
||||
### Catalogue
|
||||
|
||||
`models.seedRows()` gains `license.FeatureMCP` to its shared feature list, one
|
||||
more `KindFeature` row at `ScopeShared`, sold by every paid plan at one price.
|
||||
`SeedCatalogue` is `$setOnInsert` only, so the row appears empty on deploy and
|
||||
staff-entered price IDs are never blanked. The comment naming the row count
|
||||
("nine rows") is updated — the file explicitly asks the next person to keep that
|
||||
number deliberate.
|
||||
|
||||
`catalogue.LineItems` needs no change: a `KindFeature` row the customer selected
|
||||
becomes a line item, and one with no price ID in the running environment is
|
||||
granted free. That is what makes the pre-pricing window safe.
|
||||
|
||||
### HQ UI
|
||||
|
||||
`vantage-admin/web/lib/features.ts` gains the label "Agent access (MCP)" and the
|
||||
description "Let AI agents query and act on your fleet through the Model Context
|
||||
Protocol, under a scoped token you control." It then appears automatically in
|
||||
the purchase form, the staff pricing page and the account detail view, all of
|
||||
which render from that map.
|
||||
|
||||
### App licence page
|
||||
|
||||
`vantage-app/web/app/(app)/settings/license/page.tsx` gains
|
||||
`<Feature label="Agent Access (MCP)" included={Boolean(license.features.mcp)} />`
|
||||
alongside the existing four. `licenceResponse.Features` is already a
|
||||
`map[string]bool` built from the licence, so no server change is needed.
|
||||
|
||||
### Paddle
|
||||
|
||||
One product, two prices, created in the sandbox environment first:
|
||||
|
||||
| Field | Value |
|
||||
| --- | --- |
|
||||
| Product name | Vantage — Agent Access (MCP) |
|
||||
| Description | AI agent access to a Vantage instance over the Model Context Protocol |
|
||||
| Tax category | `standard` |
|
||||
| Currency | GBP |
|
||||
| Monthly price | £9.00, billing interval `month` × 1 |
|
||||
| Annual price | £90.00, billing interval `year` × 1 |
|
||||
|
||||
Annual is ten months' money for twelve, matching the convention the other add-on
|
||||
rows use.
|
||||
|
||||
Creation runs through the connected `paddle-sandbox` MCP server during
|
||||
implementation, with the exact payload confirmed before each call. The resulting
|
||||
price IDs are recorded in the catalogue row's `price_ids.sandbox` map through
|
||||
the existing staff pricing page — not by a migration, because that page is the
|
||||
only place price IDs are meant to be entered and a migration writing them would
|
||||
be a second source of truth.
|
||||
|
||||
Production prices are created by hand in the Paddle dashboard when the feature
|
||||
ships, and pasted into `price_ids.production` the same way. Nothing in this spec
|
||||
writes to a production billing account.
|
||||
|
||||
## Frontend
|
||||
|
||||
A new **Agent access** panel on the API tokens settings page, visible only when
|
||||
the licence grants the feature:
|
||||
|
||||
- The endpoint URL for this instance, with a copy button.
|
||||
- A short client configuration snippet, again copyable.
|
||||
- A link to the docs page.
|
||||
|
||||
The token creation form gains the two MCP scopes in its scope list — no special
|
||||
UI, they are ordinary scopes — and a **tag restriction** field, which is shown
|
||||
for every token regardless of licence because tag scoping is not gated. The
|
||||
field offers the tag keys and values already in use on servers, as the workflow
|
||||
target selector does.
|
||||
|
||||
The token list shows a token's tag restriction as a chip beside its scopes, so
|
||||
that "what can this credential reach" is answerable at a glance.
|
||||
|
||||
## Documentation
|
||||
|
||||
`vantage-docs` gains `docs/vantage/mcp.md`: what MCP is in two sentences, how to
|
||||
mint a suitable token, how to connect Claude and other clients, the full tool
|
||||
list with what each one does, and an explicit section on what an agent cannot do
|
||||
(reveal secrets, open a console, exceed its tags, act without `mcp:write`).
|
||||
|
||||
`docs/reference/api-tokens.md` gains the tag restriction field.
|
||||
|
||||
## Testing
|
||||
|
||||
Table-driven, over the tool registry rather than per tool, because the registry
|
||||
is the thing that must stay correct as tools are added:
|
||||
|
||||
- **Gate matrix.** Each tool × token shape (no MCP scope, `mcp:read`,
|
||||
`mcp:write`, missing resource scope, missing licence feature): assert listed
|
||||
or not listed, and allowed or refused. This is the security test of the
|
||||
feature.
|
||||
- **Registry completeness.** Every registered tool declares a resource scope
|
||||
from `ScopeResources` and a write flag. Same spirit as
|
||||
`AssertScopeMapComplete`; a tool added without a scope fails the build.
|
||||
- **Tag scoping at the chokepoints.** `GetServer` on an out-of-scope host is
|
||||
not-found; `ResolveTargets` intersects rather than unions; a run naming
|
||||
out-of-scope IDs targets nothing. Service-level tests, since the property is a
|
||||
service-level one.
|
||||
- **Token creation.** A caller cannot mint a token with scopes or a tag
|
||||
selector broader than their own; MCP scopes are refused without the licence.
|
||||
- **Audit.** A successful call and a refused call each write exactly one event
|
||||
of the expected type.
|
||||
- **Response size.** `list_servers` over a seeded fleet stays under a stated
|
||||
byte budget — a regression here degrades every agent interaction and is
|
||||
otherwise invisible.
|
||||
|
||||
`services/statuspages_test.go` is the style model.
|
||||
|
||||
## Migration and rollout
|
||||
|
||||
No data migration. `TagSelector` absent on existing tokens means fleet-wide,
|
||||
which is what those tokens do today. `SeedCatalogue` adds the row on the next
|
||||
admin deploy. No licence gains the feature until staff grant it.
|
||||
|
||||
Order of work:
|
||||
|
||||
1. `vantage-shared`: `FeatureMCP` constant.
|
||||
2. `vantage-app` server: `mcp` scope resource, tag selector on tokens plus
|
||||
enforcement and the completeness assertion, then the MCP package and tools.
|
||||
3. `vantage-app` web: token form fields, agent access panel, licence page row.
|
||||
4. `vantage-admin`: catalogue seed row, feature label.
|
||||
5. Paddle sandbox product and prices; price IDs entered through the staff page.
|
||||
6. `vantage-docs`: the MCP page.
|
||||
|
||||
Steps 1–3 are independently useful: tag-scoped tokens are a security improvement
|
||||
whether or not MCP ever ships, which is the argument for building them as a
|
||||
general capability rather than folding them into the MCP package.
|
||||
Reference in New Issue
Block a user