Flows YAML — shadow mode, per-flow cutover, legacy + canvas deletion #1078

Closed
opened 2026-05-10 15:42:52 +00:00 by claude-desktop · 1 comment
Collaborator

User story

As the platform operator, I want to run the YAML executor in shadow mode alongside the legacy JSON node engine, validate behaviour parity per flow, cut flows over one at a time, and finally delete the legacy server code + React Flow canvas, so that the migration is incremental and reversible at every step.

Acceptance criteria

Shadow mode

  • New service setting: service_settings.flows_yaml.mode"off" | "shadow" | "live", with per-flow override.
  • In shadow, both engines see the event; YAML engine writes to flow_runs_yaml (separate table) and runs no real dispatches (op dispatch becomes a no-op recorder; same for label/comment ops).
  • Daily diff job compares flow_runs_yaml against flow_runs for the same triggers; mismatch report surfaced in dashboard.

Per-flow cutover

  • CLI just flows-yaml-cutover <name> flips one flow to live after operator review.
  • 24h shadow-clean window required before cutover succeeds (--force override for emergencies).
  • Rollback: same CLI, --off reverts that flow to legacy engine.

Legacy server deletion (after all 9 cut over)

  • Remove apps/server/src/domain/flows/{executor,registry,validation,flow-dispatch}.ts.
  • Remove inline imperative handlers from apps/server/src/http/webhook.ts:296–341.
  • Remove defaultArgInjections from flow-dispatch.ts.
  • Drop flows table; rename flow_node_runs.node_*step_* (Drizzle migration with backfill).
  • Drop flow_runs_yaml shadow table.
  • Retire specs/node-flows.md + specs/node-flows-ui.md (note in spec).

Canvas UI cutover

  • New Monaco editor (Editor UI ticket) lands at /flows.
  • Old canvas stays available behind ?legacy=1 for one release.
  • After release N+1: apps/web/src/features/flows/ (canvas, palette, inspector) deleted; React Flow dep removed; ?legacy=1 redirects to /flows.

Tests / safety

  • Shadow diff job has fixtures covering each default flow.
  • Cutover CLI refuses on mismatch; refuses without 24h clean window unless --force.
  • Legacy delete PR runs migration on a copy of production DB and asserts read paths still load.

Out of scope

  • Unifying flow_runs + task_history into a single events table (separate spec).

References

  • Spec: docs/specs/flows-yaml.md §11 (all phases).
## User story As the platform operator, I want to run the YAML executor in shadow mode alongside the legacy JSON node engine, validate behaviour parity per flow, cut flows over one at a time, and finally delete the legacy server code + React Flow canvas, so that the migration is incremental and reversible at every step. ## Acceptance criteria ### Shadow mode - [ ] New service setting: `service_settings.flows_yaml.mode` ∈ `"off" | "shadow" | "live"`, with per-flow override. - [ ] In `shadow`, both engines see the event; YAML engine writes to `flow_runs_yaml` (separate table) and runs no real dispatches (op `dispatch` becomes a no-op recorder; same for label/comment ops). - [ ] Daily diff job compares `flow_runs_yaml` against `flow_runs` for the same triggers; mismatch report surfaced in dashboard. ### Per-flow cutover - [ ] CLI `just flows-yaml-cutover <name>` flips one flow to `live` after operator review. - [ ] 24h shadow-clean window required before cutover succeeds (`--force` override for emergencies). - [ ] Rollback: same CLI, `--off` reverts that flow to legacy engine. ### Legacy server deletion (after all 9 cut over) - [ ] Remove `apps/server/src/domain/flows/{executor,registry,validation,flow-dispatch}.ts`. - [ ] Remove inline imperative handlers from `apps/server/src/http/webhook.ts:296–341`. - [ ] Remove `defaultArgInjections` from `flow-dispatch.ts`. - [ ] Drop `flows` table; rename `flow_node_runs.node_*` → `step_*` (Drizzle migration with backfill). - [ ] Drop `flow_runs_yaml` shadow table. - [ ] Retire `specs/node-flows.md` + `specs/node-flows-ui.md` (note in spec). ### Canvas UI cutover - [ ] New Monaco editor (Editor UI ticket) lands at `/flows`. - [ ] Old canvas stays available behind `?legacy=1` for one release. - [ ] After release N+1: `apps/web/src/features/flows/` (canvas, palette, inspector) deleted; React Flow dep removed; `?legacy=1` redirects to `/flows`. ### Tests / safety - [ ] Shadow diff job has fixtures covering each default flow. - [ ] Cutover CLI refuses on mismatch; refuses without 24h clean window unless `--force`. - [ ] Legacy delete PR runs migration on a copy of production DB and asserts read paths still load. ## Out of scope - Unifying `flow_runs` + `task_history` into a single events table (separate spec). ## References - Spec: `docs/specs/flows-yaml.md` §11 (all phases).
Author
Collaborator

Inherited acceptance criteria from #1075

The engine PR (#1079) defers every column rename to this cutover so the legacy NF-2 dispatch path keeps reading + writing its existing schema unchanged. Track explicitly:

  • Drizzle migration: flow_runs.flow_idflow_name
  • Drizzle migration: flow_node_runs.node_idstep_id and any other node_*step_* columns the wave executor reads
  • Update flows-yaml/hooks-impl.ts adapters tagged // #1078 to drop the legacy column-name aliases
  • Backfill / migration script preserves history rows during the rename

Already covered in this ticket's body, restated here so #1075's "Schema rename" AC bullet has a concrete forward link.

## Inherited acceptance criteria from #1075 The engine PR (#1079) defers every column rename to this cutover so the legacy NF-2 dispatch path keeps reading + writing its existing schema unchanged. Track explicitly: - [ ] Drizzle migration: `flow_runs.flow_id` → `flow_name` - [ ] Drizzle migration: `flow_node_runs.node_id` → `step_id` and any other `node_*` → `step_*` columns the wave executor reads - [ ] Update `flows-yaml/hooks-impl.ts` adapters tagged `// #1078` to drop the legacy column-name aliases - [ ] Backfill / migration script preserves history rows during the rename Already covered in this ticket's body, restated here so #1075's "Schema rename" AC bullet has a concrete forward link.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
charles/claude-hooks#1078
No description provided.