diff --git a/docs/superpowers/specs/2026-07-21-adhoc-steps-import-export-defaults-design.md b/docs/superpowers/specs/2026-07-21-adhoc-steps-import-export-defaults-design.md new file mode 100644 index 0000000..60bf34d --- /dev/null +++ b/docs/superpowers/specs/2026-07-21-adhoc-steps-import-export-defaults-design.md @@ -0,0 +1,216 @@ +# Ad-hoc Steps, Step Import/Export, and Default Steps — Design + +Date: 2026-07-21 + +## Summary + +Three related additions to the workflow step system: + +1. **Ad-hoc steps** — steps defined inline in a single workflow, not written to the + shared step library. +2. **Import/Export** — single-step portable JSON (`vantage.step/v1`). Import can + target the shared library or a workflow as an inline ad-hoc step. +3. **Default steps** — JSON files in a bind-mounted directory, seeded into the + library on boot and re-syncable on demand. Org-ready for a future SaaS plan. + +Existing model: shared steps live in the `workflow_steps` collection; a +`Workflow.Steps[]` is a list of `WorkflowStepRef` that reference a library step by +`step_id` and may carry `Overrides` + `Inputs`. `resolveSteps` freezes each ref +into a `ResolvedStep` snapshot at run time. + +## 1. Data model + +`server/internal/models/workflow.go`. + +### WorkflowStep + +Add a provenance field: + +```go +Source string `bson:"source" json:"source"` // "user" | "default" +Slug string `bson:"slug" json:"slug"` // kebab of name; stable key for default seeding +``` + +`Slug` is set for `source="default"` steps (used as the upsert key by the seeder). +For `source="user"` steps it may be empty. Existing steps default to +`source="user"` (absent field decodes to ""). + +### WorkflowStepRef + +Add an inline definition. A ref is **either** a library ref (`StepID` set) **or** +ad-hoc (`Inline` set). Never both. + +```go +type WorkflowStepRef struct { + StepID string `bson:"step_id,omitempty" json:"step_id"` + Inline *WorkflowStep `bson:"inline,omitempty" json:"inline,omitempty"` + Order int `bson:"order" json:"order"` + OnFailure string `bson:"on_failure" json:"on_failure"` + MaxRetries int `bson:"max_retries" json:"max_retries"` + Overrides *StepOverride `bson:"overrides,omitempty" json:"overrides,omitempty"` // library-ref only + Inputs map[string]string `bson:"inputs,omitempty" json:"inputs,omitempty"` +} +``` + +`Inline` reuses `WorkflowStep` (name, description, interpreter, script, +declared_inputs, declared_outputs, secret_refs). Its `ID`, `StepID`, `Slug`, +`Source`, and timestamps stay empty and are never persisted to `workflow_steps`. + +Validation on workflow create/update: for each ref, exactly one of `StepID` / +`Inline` must be set. `Overrides` is ignored when `Inline` is set. + +## 2. Resolve at run time + +`server/internal/services/workflow_runner.go`, `resolveSteps`. + +For each ref: + +- If `ref.Inline != nil`: build `ResolvedStep` from `ref.Inline` directly + (name/interpreter/script/secret_refs), apply `ref.Inputs` against + `Inline.DeclaredInputs` defaults. Skip `getStep`, skip `Overrides`. +- Else: current library path unchanged (load step, apply overrides). + +`ResolvedStep` output shape and the run snapshot are unchanged, so the runner and +the run-history UI need no changes. + +`DeleteStep` cascade is unaffected — ad-hoc refs carry no `step_id`, so they never +match the cascade query. + +## 3. Import / Export + +Portable single-step JSON, `kind: "vantage.step/v1"`: + +```json +{ + "kind": "vantage.step/v1", + "name": "...", + "description": "...", + "interpreter": "bash", + "script": "...", + "declared_inputs": [ { "name": "...", "default": "...", "description": "..." } ], + "declared_outputs": ["..."], + "secret_refs": ["NAME"] +} +``` + +Export strips `_id`, `step_id`, `slug`, `source`, and timestamps. +Secret refs are exported as names only. On import, dangling secret refs are kept +verbatim (not auto-created). + +### Service functions (`services/workflows.go`) + +- `ExportStep(stepID string) ([]byte, error)` — load library step, marshal to the + v1 shape. +- `ParseStepDoc(b []byte) (models.WorkflowStep, error)` — validate `kind`, decode + into a `WorkflowStep` (no id/source). Shared by both import targets. +- `ImportStepToLibrary(b []byte) (*models.WorkflowStep, error)` — `ParseStepDoc` + then `CreateStep` (fresh `step_id`, `source="user"`). + +Import-to-inline needs no new service fn: the web editor calls `ParseStepDoc`'s +API equivalent (see routes) and drops the returned step object into a new +`WorkflowStepRef.Inline` in the workflow it's editing, then saves the workflow +normally. + +### Routes (`api/workflows.go`) + +- `GET /api/steps/:id/export` — returns JSON as a downloadable attachment + (`Content-Disposition`). +- `POST /api/steps/import` — body is the v1 JSON; imports to library; returns the + created step. (Used by the "import to library" flow.) +- `POST /api/steps/parse` — body is the v1 JSON; validates and returns the + normalized step object **without** persisting. Used by "import to inline" so the + editor can insert it as an ad-hoc ref. (Keeps parsing/validation server-side.) + +Audit: `workflow.step_imported` logged on library import. + +## 4. Default steps (seed + admin re-sync) + +Mirrors the existing `WorkflowLogDir` pattern — a bind-mounted directory, no Go +`embed`. + +### Directory + +```go +// DefaultStepsDir returns the directory holding default step JSON files, creating it. +func DefaultStepsDir() string { + dir := os.Getenv("VANTAGE_DEFAULT_STEPS_DIR") + if dir == "" { + dir = filepath.Join("data", "default-steps") + } + _ = os.MkdirAll(dir, 0700) + return dir +} +``` + +Compose already bind-mounts `./data:/data`. Set +`VANTAGE_DEFAULT_STEPS_DIR=/data/default-steps` for explicitness (optional). +Operator drops `*.json` (`vantage.step/v1`) files into that folder. + +### Seeder + +`SeedDefaultSteps() (created, updated int, err error)`: + +1. Glob `DefaultStepsDir()/*.json`. +2. For each file: `ParseStepDoc`, compute `slug = kebab(name)`. +3. Upsert into `workflow_steps` keyed on `{ slug, source: "default" }`: + - absent → insert with fresh `step_id`, `source="default"`, `slug`. (`created++`) + - present → `$set` name/description/interpreter/script/declared_*/secret_refs + + `updated_at`. (`updated++`) + +**Override rule (confirmed):** re-sync is authoritative for `source="default"` +steps and overwrites their content, reverting any user edits to those steps. +`source="user"` steps are never touched by the seeder, even on a slug collision +(the seeder query is scoped to `source: "default"`). + +Add a partial unique index on `slug` where `source == "default"` (or enforce +uniqueness in the seeder loop) to keep default slugs unambiguous. + +### Boot + +Call `SeedDefaultSteps()` from server startup after `EnsureWorkflowIndexes()` +(alongside index setup in `server/cmd/main.go`). Log the created/updated counts; +a seed error is logged but non-fatal (server still boots). + +### Route + +- `POST /api/steps/seed-defaults` (admin) — runs `SeedDefaultSteps()`, returns + `{ "created": n, "updated": m }`. Audit `workflow.defaults_synced`. + +### Org readiness + +Signature stays global today. When Orgs land, `SeedDefaultSteps(orgID)` seeds +per-org and the upsert key becomes `{ org_id, slug, source }`. No schema churn +blocks that later change. + +## 5. Web + +`web/app/workflows/[id]/page.tsx` and the steps list page. + +- **Steps list:** per-row **Export** (downloads JSON) and top-level **Import** + (file picker → `POST /api/steps/import` → library). **Sync defaults** admin + button → `POST /api/steps/seed-defaults`, toast the counts. `source="default"` + rows get a "default" badge. +- **Workflow editor — Add step:** existing "add from library" plus **Add ad-hoc + step** (inline mini-form: name, interpreter, script, optional inputs) stored as + a `WorkflowStepRef.Inline`. Also **Import ad-hoc from file** → `POST + /api/steps/parse` → inserts the returned step as a new inline ref. +- Ad-hoc rows in the editor are visually distinguished from library refs (badge) + and are editable in place; library refs keep the existing override UI. + +## Testing + +- `resolveSteps`: inline ref resolves without touching the library; inputs apply + from `Inline.DeclaredInputs` defaults and ref overrides; library path unchanged. +- Workflow validation: rejects a ref with both `StepID` and `Inline`, and one with + neither. +- Export → import round-trips to an equivalent library step with a new `step_id`. +- `ParseStepDoc` rejects a wrong/missing `kind`. +- `SeedDefaultSteps`: insert-then-update idempotency; user steps untouched; + user-edited default step reverted on re-sync; counts correct. +- `DeleteStep` cascade ignores ad-hoc refs. + +## Out of scope + +- Whole-workflow export/import. +- Auto-creating secret refs on import. +- Multi-tenant Org model (design is forward-compatible only).