feat(flows-yaml): 9 defaults + JSON Schema pipeline + REST CRUD + main wiring (closes #1076) #1081

Merged

reviewer merged 3 commits from flows-yaml/api-defaults into main 2026-05-10 18:55:04 +00:00

Conversation 1 Commits 3 Files changed 52 +7206 -181

claude-desktop commented 2026-05-10 18:31:16 +00:00

Collaborator

Copy link

Closes #1076. Builds on #1075 (engine + ops + dispatcher merged in #1079).

Lands the operator-facing surface so the YAML engine is ready to ship behind a feature flag. flows_yaml_settings.mode defaults to off — the engine boots, the routes mount, no real dispatches fire until #1078 cutover.

Commits

1. feat(flows-yaml): 9 default *.yml flows + Zod→JSON Schema pipeline — defaults, fixture-replay tests, schema generator wired into CI, plus an engine fix for the multi-template interpolate fast-path collapse.
2. feat(flows-yaml): REST CRUD + SSE + main.ts dispatcher boot — operator-facing routes, validation pipeline, SSE broadcast, live-capabilities adapter, bootstrapFlowsYamlEngine() in main.ts.

What landed

Defaults (flows/defaults/*.yml)

9 files mirroring the legacy graphs: pr-opened, review-requested, issue-opened, issue-labeled, issue-assigned, issue-unassigned, issue-closed, pr-merged (internal trigger), breakdown-comment. Each has a fixture-replay test under apps/server/src/domain/flows-yaml/defaults/__tests__/.

JSON Schema pipeline

• .describe() on every Zod field across schema.ts, trigger-events.ts, all 15 ops/*.ts. Expression-string fields tagged format: "flow-expression" for Monaco hover (#1077).
• apps/server/scripts/generate-flow-schema.ts — Zod v4 native z.toJSONSchema(), no extra dep. Combines per-op argsSchema into a discriminated oneOf keyed on uses: plus the parallel-step branch.
• Output committed at apps/server/src/domain/flows-yaml/flows.schema.json + apps/web/public/schemas/flows.schema.json (byte-identical, snapshot-tested).
• just flow-schema-check regenerates into a tmp file and diffs; wired into qa. Web prebuild runs the generator before vite build.

REST CRUD (/api/flows-yaml/*)

Verb	Path	Behaviour
GET	/api/flows-yaml	list {name, source, trigger_summary, enabled, mtime}
GET	/api/flows-yaml/runs	paginated runs (flow, limit, cursor)
GET	/api/flows-yaml/runs/:id	run + step trace
GET	/api/flows-yaml/schema	JSON Schema
GET	/api/flows-yaml/:name	body_yaml + parsed + last_run
POST	/api/flows-yaml	create custom (409 conflict, 422 invalid)
PUT	/api/flows-yaml/:name	mtime check, default → fork
DELETE	/api/flows-yaml/:name	409 on pure default; custom-shadowed default → restore
POST	/api/flows-yaml/:name/disable	toggle .disabled.json overlay
POST	/api/flows-yaml/:name/dry-run	simulate (no DB writes)
POST	/api/flows-yaml/reload	full re-scan

Validation pipeline: YAML parse (422 yaml-parse) → Zod FlowFileSchema (422 schema) → semantic checks (duplicate step.id, unknown uses:, expression parse — 422 semantic). All routes gated by existing dashboard auth.

Path prefix /api/flows-yaml/* keeps the legacy /api/flows/* JSON node engine intact through the #1078 cutover.

Engine fix

interpolate() whole-string fast path now skips when the template contains more than one ${{ }} placeholder. The original ^\${{\s*([\s\S]*?)\s*}}$-anchored non-greedy regex collapsed multi-template chains like ${{ a }}#${{ b }} into one bad capture (a }}#${{ b). Adds a dedicated expr/index.test.ts covering each interpolation mode. The three default YAMLs that originally carried repo: workaround prefixes are reverted now that the engine is correct.

Boot wiring (main.ts)

bootstrapFlowsYamlEngine() runs after seedDefaultFlowAtBoot():

• Reads service_settings.flows_yaml_settings.mode (default off)
• When non-off: createTriggerBus → setTriggerBus (worker + event-handlers), loadFlowsFromDisk, createFlowWatcher, createPersistentHooks, createDispatcher
• registerFlowsYamlRoutes always runs so operators can author offline even when mode is off

Live capabilities adapter (live-capabilities.ts)

buildLiveCapabilities(deps) maps every OpDepKey to a thin adapter over the existing service surface (workers, dedup, agent-resolver, forge, breakdown, review-guard, pr-flow). #1078 wires real deps; until then bootstrapFlowsYamlEngine() passes empty deps so any actual dispatch faults loudly with MissingCapabilityError. assertCapabilitiesSatisfyOps() does a boot-time conformance check.

#1075 ACs closed via this PR

• POST /api/flows-yaml/reload — implemented (inherited via comment on #1076)

Test plan

• bun test apps/server/src/domain/flows-yaml/ — 233 / 0 (engine + ops + 9 defaults + dispatcher + hooks + interpolate)
• Full server suite (cd apps/server && bun test) — 3692 / 0
• bun x tsc --noEmit — clean (server + shared)
• just flow-schema-check — passes (committed schema in sync)
• No edits to legacy domain/flows/, webhook.ts, or flows-routes.ts

Out of scope (ships in follow-ups)

• #1077 — Monaco editor consuming GET /api/flows-yaml/* + POST /api/flows-yaml/:name/dry-run + SSE flow.changed
• #1078 — wire real deps into buildLiveCapabilities, flip mode off → shadow → live, port webhook imperative handlers to YAML ops, drop the legacy node engine, rename flow_id → flow_name / node_id → step_id

🤖 Generated with Claude Code

Closes #1076. Builds on #1075 (engine + ops + dispatcher merged in #1079). Lands the operator-facing surface so the YAML engine is ready to ship behind a feature flag. `flows_yaml_settings.mode` defaults to `off` — the engine boots, the routes mount, no real dispatches fire until #1078 cutover. ## Commits 1. `feat(flows-yaml): 9 default *.yml flows + Zod→JSON Schema pipeline` — defaults, fixture-replay tests, schema generator wired into CI, plus an engine fix for the multi-template `interpolate` fast-path collapse. 2. `feat(flows-yaml): REST CRUD + SSE + main.ts dispatcher boot` — operator-facing routes, validation pipeline, SSE broadcast, live-capabilities adapter, `bootstrapFlowsYamlEngine()` in main.ts. ## What landed ### Defaults (`flows/defaults/*.yml`) 9 files mirroring the legacy graphs: `pr-opened`, `review-requested`, `issue-opened`, `issue-labeled`, `issue-assigned`, `issue-unassigned`, `issue-closed`, `pr-merged` (internal trigger), `breakdown-comment`. Each has a fixture-replay test under `apps/server/src/domain/flows-yaml/defaults/__tests__/`. ### JSON Schema pipeline - `.describe()` on every Zod field across `schema.ts`, `trigger-events.ts`, all 15 `ops/*.ts`. Expression-string fields tagged `format: "flow-expression"` for Monaco hover (#1077). - `apps/server/scripts/generate-flow-schema.ts` — Zod v4 native `z.toJSONSchema()`, no extra dep. Combines per-op `argsSchema` into a discriminated `oneOf` keyed on `uses:` plus the parallel-step branch. - Output committed at `apps/server/src/domain/flows-yaml/flows.schema.json` + `apps/web/public/schemas/flows.schema.json` (byte-identical, snapshot-tested). - `just flow-schema-check` regenerates into a tmp file and diffs; wired into `qa`. Web `prebuild` runs the generator before `vite build`. ### REST CRUD (`/api/flows-yaml/*`) | Verb | Path | Behaviour | |---|---|---| | GET | `/api/flows-yaml` | list `{name, source, trigger_summary, enabled, mtime}` | | GET | `/api/flows-yaml/runs` | paginated runs (`flow`, `limit`, `cursor`) | | GET | `/api/flows-yaml/runs/:id` | run + step trace | | GET | `/api/flows-yaml/schema` | JSON Schema | | GET | `/api/flows-yaml/:name` | body_yaml + parsed + last_run | | POST | `/api/flows-yaml` | create custom (409 conflict, 422 invalid) | | PUT | `/api/flows-yaml/:name` | mtime check, default → fork | | DELETE | `/api/flows-yaml/:name` | 409 on pure default; custom-shadowed default → restore | | POST | `/api/flows-yaml/:name/disable` | toggle `.disabled.json` overlay | | POST | `/api/flows-yaml/:name/dry-run` | simulate (no DB writes) | | POST | `/api/flows-yaml/reload` | full re-scan | Validation pipeline: YAML parse (422 yaml-parse) → Zod `FlowFileSchema` (422 schema) → semantic checks (duplicate `step.id`, unknown `uses:`, expression parse — 422 semantic). All routes gated by existing dashboard auth. Path prefix `/api/flows-yaml/*` keeps the legacy `/api/flows/*` JSON node engine intact through the #1078 cutover. ### Engine fix `interpolate()` whole-string fast path now skips when the template contains more than one `${{ }}` placeholder. The original `^\${{\s*([\s\S]*?)\s*}}$`-anchored non-greedy regex collapsed multi-template chains like `${{ a }}#${{ b }}` into one bad capture (`a }}#${{ b`). Adds a dedicated `expr/index.test.ts` covering each interpolation mode. The three default YAMLs that originally carried `repo:` workaround prefixes are reverted now that the engine is correct. ### Boot wiring (`main.ts`) `bootstrapFlowsYamlEngine()` runs after `seedDefaultFlowAtBoot()`: - Reads `service_settings.flows_yaml_settings.mode` (default `off`) - When non-off: `createTriggerBus` → `setTriggerBus` (worker + event-handlers), `loadFlowsFromDisk`, `createFlowWatcher`, `createPersistentHooks`, `createDispatcher` - `registerFlowsYamlRoutes` always runs so operators can author offline even when mode is `off` ### Live capabilities adapter (`live-capabilities.ts`) `buildLiveCapabilities(deps)` maps every `OpDepKey` to a thin adapter over the existing service surface (workers, dedup, agent-resolver, forge, breakdown, review-guard, pr-flow). #1078 wires real deps; until then `bootstrapFlowsYamlEngine()` passes empty deps so any actual dispatch faults loudly with `MissingCapabilityError`. `assertCapabilitiesSatisfyOps()` does a boot-time conformance check. ## #1075 ACs closed via this PR - [x] `POST /api/flows-yaml/reload` — implemented (inherited via comment on #1076) ## Test plan - [x] `bun test apps/server/src/domain/flows-yaml/` — 233 / 0 (engine + ops + 9 defaults + dispatcher + hooks + interpolate) - [x] Full server suite (`cd apps/server && bun test`) — 3692 / 0 - [x] `bun x tsc --noEmit` — clean (server + shared) - [x] `just flow-schema-check` — passes (committed schema in sync) - [x] No edits to legacy `domain/flows/`, `webhook.ts`, or `flows-routes.ts` ## Out of scope (ships in follow-ups) - **#1077** — Monaco editor consuming `GET /api/flows-yaml/*` + `POST /api/flows-yaml/:name/dry-run` + SSE `flow.changed` - **#1078** — wire real deps into `buildLiveCapabilities`, flip mode `off` → `shadow` → `live`, port webhook imperative handlers to YAML ops, drop the legacy node engine, rename `flow_id` → `flow_name` / `node_id` → `step_id` 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-desktop added 2 commits 2026-05-10 18:31:16 +00:00

feat(flows-yaml): 9 default *.yml flows + Zod→JSON Schema pipeline ... 80af13b40c

#1076 Wave 1 — defaults shipped on disk, schema generator wired into CI.

Defaults (flows/defaults/):
- pr-opened.yml — pull_request: [opened, synchronize] → guard_reviewer →
  resolve_agent → dispatch
- review-requested.yml — pull_request_review: [requested]
- issue-opened.yml — placeholder dedup_check (no real op until skill ops land)
- issue-labeled.yml — area:design / area:design-review routing branches
- issue-assigned.yml — dedup → cancel_stale → record → resolve → dispatch
- issue-unassigned.yml — cancel_tasks
- issue-closed.yml — propagate_dependencies(issue-closed)
- pr-merged.yml — propagate_dependencies(rebase-cascade) on internal trigger
- breakdown-comment.yml — comment_matches('^\\s*/breakdown')
  → dispatch_breakdown

Per-flow fixture-replay tests under
apps/server/src/domain/flows-yaml/defaults/__tests__/ load each YAML from
disk via loadFlowsFromDisk, build a synthetic TriggerEvent, and assert
the recorded op call sequence + args against legacy graph behaviour.

JSON Schema pipeline:
- Every Zod field in flows-yaml/{schema,trigger-events,ops/*}.ts gets a
  `.describe()` for Monaco hover text. Expression-string fields tagged
  with `.meta({ format: "flow-expression" })`.
- `apps/server/scripts/generate-flow-schema.ts` walks the registry +
  combines per-op argsSchemas into a discriminated `oneOf` keyed on
  `uses:`, plus the parallel-step branch.
- Output committed at apps/server/src/domain/flows-yaml/flows.schema.json
  + apps/web/public/schemas/flows.schema.json (byte-identical;
  snapshot-tested).
- Uses Zod v4's native z.toJSONSchema() — no zod-to-json-schema dep
  needed.
- `just flow-schema-check` regenerates into a tmp file + diffs; wired
  into the `qa` recipe alongside paraglide-check / sql-layer-check.
- apps/web prebuild script runs the generator before vite build so dev
  + prod ship identical schemas.

Engine fix: interpolate() whole-string fast path now skips when more
than one ${{ }} placeholder is present. The original
`^\\${{ ... }}$`-anchored non-greedy regex collapsed multi-template
chains like `${{ a }}-${{ b }}` into one bad capture (`a }}-${{ b`).
Adds a dedicated index.test.ts covering: no-template, whole-string
native type preservation, embedded interpolation, multi-template
without literal separator, env reference, undefined member access.
Workaround `repo:` prefixes in three default YAMLs reverted now that
the engine is correct.

3668 / 3668 server tests pass. tsc --noEmit clean.

Refs: #1076

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

feat(flows-yaml): REST CRUD + SSE + main.ts dispatcher boot ... 88b33b7067

#1076 Wave 2/3 — operator-facing surface for the YAML engine.

apps/server/src/http/flows-yaml-routes.ts (new):
- GET /api/flows-yaml — list `{name, source, trigger_summary, enabled, mtime}`
- GET /api/flows-yaml/runs[?flow=&limit=&cursor=] — paginated runs
- GET /api/flows-yaml/runs/:id — flow_runs row + step trace
- GET /api/flows-yaml/schema — JSON Schema (defence-in-depth copy)
- GET /api/flows-yaml/:name — body_yaml + parsed + last_run
- POST /api/flows-yaml — create under flows/custom (409 on conflict)
- PUT /api/flows-yaml/:name — mtime-checked update (409 on stale);
  default-fork writes a copy to flows/custom on first edit
- DELETE /api/flows-yaml/:name — 409 on pure default; custom-shadowed
  default → restore default
- POST /api/flows-yaml/:name/disable — toggle .disabled.json overlay
- POST /api/flows-yaml/:name/dry-run — no-op hooks, empty caps,
  deterministic per-step trace, no DB writes
- POST /api/flows-yaml/reload — full re-scan, returns
  {loaded, errors[]}

Validation pipeline: YAML parse (422 yaml-parse) → Zod
FlowFileSchema (422 schema) → semantic checks (duplicate step.id,
unknown uses, expression parse, 422 semantic). All routes gated by
the existing dashboard auth middleware.

Path prefix `/api/flows-yaml/*` keeps the legacy `/api/flows/*` JSON
node engine routes intact through the #1078 cutover.

apps/server/src/domain/flows-yaml/live-capabilities.ts (new):
- buildLiveCapabilities(deps) maps every OpDepKey to a thin adapter
  over the existing service surface (workers, dedup, agent-resolver,
  forge, breakdown, review-guard, pr-flow). #1078 wires real deps;
  for now buildLiveCapabilities returns empty caps so shadow mode
  faults loudly with MissingCapabilityError if invoked.
- assertCapabilitiesSatisfyOps(opRegistry, caps) — boot-time check
  that every declared dep on every op is satisfied; raised on first
  fault, not on first dispatch.

apps/server/src/main.ts:
- bootstrapFlowsYamlEngine() runs after seedDefaultFlowAtBoot.
  Reads service_settings.flows_yaml_settings (mode: off|shadow|live);
  defaults off. When non-off:
    - createTriggerBus + setTriggerBus (worker.ts + event-handlers.ts)
    - loadFlowsFromDisk(flows/defaults + flows/custom) → FlowRegistry
    - createFlowWatcher → flow.changed SSE on add/change/unlink
    - createPersistentHooks → flow_runs / flow_node_runs writes
    - createDispatcher
    - registerFlowsYamlRoutes always runs (CRUD lives even when
      mode=off so operators can author offline)

apps/server/src/infrastructure/database/service-config-store.ts:
- getFlowsYamlSettings / setFlowsYamlSettings — JSON value under
  `flows_yaml_settings` key, default `{ mode: "off" }`. No schema
  migration; matches the existing billing_mode / design_pipeline_enabled
  pattern.

Engine fix carried over from Wave 1: interpolate() whole-string fast
path skips when more than one ${{ }} placeholder is present (multi-
template strings like `${{ a }}#${{ b }}` would otherwise collapse
into one bad capture). Workaround `repo:` prefixes in three default
YAMLs reverted now that the engine is correct. Adds expr/index.test.ts
covering: no-template, native-type preservation, embedded
interpolation, multi-template without literal separator, env, undefined.

JSON Schema regenerated with the current `.describe()` set
(63695 bytes; 15 ops + parallel branch).

3692 / 3692 server tests pass. tsc --noEmit clean.

Refs: #1076

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

charles added 1 commit 2026-05-10 18:38:09 +00:00

fix(flows-yaml): exclude flows.schema.json from biome to keep generator output stable ... c862ccbf2d

The schema generator emits 2-space JSON; biome's `format --write`
reformats it to tab-indent on every commit, which then makes the
generator's next run diff against the committed file and fails CI on
the snapshot test (`generate-flow-schema > produces a stable,
committed-on-disk artifact`).

Add `flows.schema.json` to biome's `files.includes` ignore list and
re-stage the generator-shaped output. Generated artefacts shouldn't
be reformatted by the linter — same precedent as `routeTree.gen.ts`
and `paraglide/`.

Refs: #1076

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

charles requested review from reviewer 2026-05-10 18:50:03 +00:00

reviewer approved these changes 2026-05-10 18:54:58 +00:00

reviewer left a comment

Copy link

All ACs from #1076 met. CI green. Code is correct.

Nit (not blocking): dedup_record in issue-assigned.yml fires unconditionally even when no agent resolves (no dispatch), leaving a claimed dedup slot with no backing task. Behavior parity with legacy is deferred to #1078, so this is in-scope for that ticket, not here.

All ACs from #1076 met. CI green. Code is correct. Nit (not blocking): `dedup_record` in `issue-assigned.yml` fires unconditionally even when no agent resolves (no dispatch), leaving a claimed dedup slot with no backing task. Behavior parity with legacy is deferred to #1078, so this is in-scope for that ticket, not here.

reviewer merged commit 7b38eaff77 into main 2026-05-10 18:55:04 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

reviewer

Labels

Clear labels   

area:agents

Agent types, pool scheduling, per-instance config

  

area:dashboard

Dashboard UI and observability surfaces

  

area:database

DB layer — schema, migrations, ORM, raw SQL

  

area:design

UI/UX mockup work — routes to designer agent

  

area:design-review

Design review dispatch — routes to design-reviewer agent

  

area:flows

Flow runner — YAML loader, executor, op registry, expression eval

  

area:infra

Deployment, isolation, containers, systemd units

  

area:meta

Tracking, scaffolding, project setup

  

area:security

Security — routes to reviewer-security (opus)

  

area:sessions

Session-id store, Claude SDK resume logic

  

area:webhook

Forgejo webhook routing and handlers

  

area:workdir

Clone cache, worktrees, git identity

  

security

Security-sensitive issue

  

type:bug

Bug

  

type:chore

Chore

  

type:meta

Tracking or decisions, not implementation work

  

type:user-story

User story

No labels

area:agents

area:dashboard

area:database

area:design

area:design-review

area:flows

area:infra

area:meta

area:security

area:sessions

area:webhook

area:workdir

security

type:bug

type:chore

type:meta

type:user-story

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

3 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

charles/claude-hooks!1081

No description provided.