Two failures found by tracing the migration path against a real upgrade.
Bootstrap orphaned the entire dataset. On upgrade, 0001 creates the
Default org and stamps every legacy document with it, but the instance
has no users, so the operator must run /auth/bootstrap to get in — and
that unconditionally created a SECOND org and put the owner in it. Every
org-scoped read then filtered on the new org, so the operator would log
into an empty Vantage while all their data sat under "default". Nothing
errored and agents kept syncing, so it presented as total data loss.
Bootstrap now adopts the sole existing org, renaming and re-slugging it,
and only creates one when no org exists. More than one org with no users
is refused rather than guessed.
Fresh installs crash-looped. Nothing creates the settings collection
before EnsureSettingsIndexes, so DropOne returned NamespaceNotFound (26),
isIndexNotFound matched only IndexNotFound (27), and that check is fatal.
The same early return also skipped index creation in the secrets and
workflow ensures.
Also: only insert the backfill org on ErrNoDocuments, so a transient read
error can't race the fatal unique slug index; run 0002 before 0003 so the
settings migration can't be pushed into its ambiguous branch; fail 0002's
ambiguous case with a remedy instead of continuing into a fatal index
build; and skip non-string ids in the owner backfill rather than aborting.
Findings from the final whole-branch review.
- scopedCollections named "audit" and "channels", but the code writes to
audit_logs and notification_channels. On upgrade from single-tenant,
legacy audit events and channels would never get org_id, becoming
invisible to org-filtered reads while channels silently stopped firing
— and the detection loop counted the wrong names, so the 0001 marker
could be written having migrated nothing. Names fixed, plus migration
0003 so an incorrectly-migrated instance converges with a fresh one.
- EnsureAuthIndexes failure is now fatal. GetUserByEmail is unscoped and
the OIDC cross-org guard compares against whichever duplicate Mongo
returns first, so users.email uniqueness is a security invariant, and a
legacy collection with duplicate emails is the realistic upgrade case.
- Evict the per-org OIDC provider cache on save; rotating away from a
compromised IdP previously had no effect until restart.
- Build the oauth2 config per request instead of mutating a shared cached
pointer outside the mutex, which raced on RedirectURL between
concurrent logins for the same org.
- Stamp org_id on console_sessions, incidents and monitor_rollups, the
last collections with no tenant column. 0003 derives their org from the
owning server/monitor rather than defaulting, so one org's console
history and incident timeline cannot merge into another's.
- Seed default steps when an org is created, not only at boot.
- Reject an empty session OrgID at the middleware.
- Derive the app root label from APP_ROOT_LABEL instead of hardcoding
"vantage", which silently disabled the host guard off that domain.
- Stop caching negative slug lookups, so a new org's subdomain resolves
immediately.
Review follow-ups on e5363a6:
- MigrateSettingsOrg no longer guesses via the "default" slug. One org
means stamp that org; zero orgs means synthesise Default; more than
one means leave it alone and log, since guessing would hand one org
another's SMTP config and ESO token.
- EnsureSettingsIndexes failure is now fatal. Without the unique index
on org_id, GetSettings returns an arbitrary duplicate; without the one
on the token hash, ResolveSecretsReadToken picks an arbitrary org.
- Name the token-hash index explicitly so it stops colliding with the
legacy name DropOne targets, and exclude the empty string from the
partial filter.
- Log retention: distinguish a missing run doc from a Mongo error, so a
transient failure skips the directory rather than purging it at the
30-day default.
- Offline sweep: fresh context per org, log-and-continue on a per-org
error, plus a final pass for servers whose org no longer exists.
- ESO handler 401s on an empty token-derived org rather than querying
org_id "".
The settings collection was a single global document, so every org
shared one SMTP config, alert config, retention policy and ESO read
token. GetSecretGroupDecryptedAny then flattened every org's secrets
for a group into one map, meaning any tenant's token read every
tenant's secrets.
- settings gains org_id; GetSettings/SaveSettings/RotateSecretsReadToken/
GetWorkflowLogRetentionDays all take orgID
- VerifySecretsReadToken replaced by ResolveSecretsReadToken, which
resolves the org from the presented token's hash; the ESO endpoint
derives its org from the token rather than a session, since it is
called machine-to-machine
- GetSecretGroupDecryptedAny deleted in favour of the org-scoped variant
- settings and token-rotation routes now require owner/admin
- offline sweep and log retention resolve org per server / per run
- migration 0002 stamps the legacy settings doc with the default org
Note: /api/settings now 403s for members; the web settings page needs a
matching role check.
Threads org_id through every admin-facing service function (servers, keys,
assignments, secrets, workflows/steps/runs, monitors, channels, audit),
adds RequireRole middleware, and wires /api/org user + OIDC management
routes. Agent/scheduler paths keep unique-key signatures and resolve org
from the loaded record; internal-only helpers (getServerByID,
getRunByID, getMonitorByID) preserve those call sites.
mrhid6
domrichardson