Agents: label-aware instance selection (match_labels on dispatch) #50

New issue

Closed

opened 2026-04-18 15:00:09 +00:00 by claude-desktop · 0 comments

claude-desktop commented

2026-04-18 15:00:09 +00:00

Collaborator

User story

As a maintainer, I want to route security-sensitive PRs to a specific reviewer instance (e.g. reviewer-security on opus 4.7) and leave everything else on the default reviewer (sonnet 4.6), by configuring match_labels per instance and letting the dispatcher pick the best-matching instance, so that model cost tracks PR risk.

Context

A2's pool scheduler picks round-robin. A3 adds a label-aware preference on top: when the dispatcher knows the triggering issue/PR's labels, it first tries to match an instance whose match_labels intersect the observed labels, and falls back to A2's round-robin when no instance matches.

The initial driver: a breakdown skill (A7) that emits security labels on specs dealing with auth / crypto / secrets — and a reviewer instance configured with match_labels: ["security"] + model: claude-opus-4-7 that automatically picks those up. But A3 must work with any label source (human-added labels, CI-added labels, etc.).

Acceptance criteria

Label-aware selection

Extend dispatchByType(type, request, opts?): Promise<string> to accept opts.labels?: string[].
Selection order (within the worker set for that type):
1. Match — candidates whose match_labels intersect opts.labels. If multiple, prefer the most specific (largest intersection); tie-break by A2's "idle first, then least-loaded".
2. Fallback — no intersection: candidates with an empty match_labels list (the "default" pool). A2's round-robin within that subset.
3. Last resort — no match, no defaults: A2's round-robin across all candidates of the type. Log "[webhook] no match_labels catch-all for <type>; falling back to any instance".
opts.labels undefined / empty → behaves as A2 (no label-aware step).

Label fetching

At dispatch time for PR-scoped events (reviewer/approve/merge/rebase), fetch the PR's linked issue labels via forgejo-api.ts. Use the PR body's Closes #N pattern or the first label-bearing issue from list_pull_request_issues if Forgejo exposes it — whichever is simpler to implement, document the choice.
For issue-scoped events (assigned), labels come from the issue payload directly (payload.issue.labels).

Config surface

SQLite match_labels column is a JSON array (A1 already added it). Dashboard (A6) will edit it; for this story, test via direct SQLite insert.
Instances with match_labels: [] or null are the "catch-all" for their type. An instance with match_labels: ["security"] opts out of catch-all duty.

Logging

One log line per dispatch including the selection reason: [webhook] dispatch <type> -> <instance> (match: [security] ∩ [security,audit]; idle=2, loaded=0) or ... (fallback: no match; round-robin catch-all; idle=1).

Tests

dispatchByType with labels:
- (a) Labels overlap one instance's match_labels → that instance picked.
- (b) Labels overlap multiple → largest intersection wins.
- (c) No overlap with any match_labels, default pool non-empty → catch-all round-robin.
- (d) No overlap AND no catch-all → warn + round-robin across all.
- (e) No labels supplied → behaves exactly as A2 (catch-all round-robin).

Out of scope

Negative matching (exclude from routing).
Label source beyond Forgejo issue/PR labels (no CI-added tags from claude-hooks itself).
Label-weighted scoring (e.g. security weight 10, frontend weight 1) — v1 uses intersection size only.
UI for editing match_labels (see A6).

References

Tracking issue: #47.
A2 sets the baseline selection order this story extends.

Dependencies

Blocked by: A1, A2.
Blocks: A7 (breakdown skill relies on this to make its labels meaningful).
Branch off: main (after A2 lands).

## User story As a **maintainer**, I want to route security-sensitive PRs to a specific reviewer instance (e.g. `reviewer-security` on opus 4.7) and leave everything else on the default reviewer (sonnet 4.6), by configuring `match_labels` per instance and letting the dispatcher pick the best-matching instance, so that model cost tracks PR risk. ## Context A2's pool scheduler picks round-robin. A3 adds a label-aware preference on top: when the dispatcher knows the triggering issue/PR's labels, it first tries to match an instance whose `match_labels` intersect the observed labels, and falls back to A2's round-robin when no instance matches. The initial driver: a `breakdown` skill (A7) that emits `security` labels on specs dealing with auth / crypto / secrets — and a reviewer instance configured with `match_labels: ["security"]` + `model: claude-opus-4-7` that automatically picks those up. But A3 must work with any label source (human-added labels, CI-added labels, etc.). ## Acceptance criteria ### Label-aware selection - [ ] Extend `dispatchByType(type, request, opts?): Promise<string>` to accept `opts.labels?: string[]`. - [ ] Selection order (within the worker set for that type): 1. **Match** — candidates whose `match_labels` intersect `opts.labels`. If multiple, prefer the most specific (largest intersection); tie-break by A2's "idle first, then least-loaded". 2. **Fallback** — no intersection: candidates with an empty `match_labels` list (the "default" pool). A2's round-robin within that subset. 3. **Last resort** — no match, no defaults: A2's round-robin across all candidates of the type. Log `"[webhook] no match_labels catch-all for <type>; falling back to any instance"`. - [ ] `opts.labels` undefined / empty → behaves as A2 (no label-aware step). ### Label fetching - [ ] At dispatch time for PR-scoped events (reviewer/approve/merge/rebase), fetch the PR's linked issue labels via `forgejo-api.ts`. Use the PR body's `Closes #N` pattern or the first label-bearing issue from `list_pull_request_issues` if Forgejo exposes it — whichever is simpler to implement, document the choice. - [ ] For issue-scoped events (assigned), labels come from the issue payload directly (`payload.issue.labels`). ### Config surface - [ ] SQLite `match_labels` column is a JSON array (A1 already added it). Dashboard (A6) will edit it; for this story, test via direct SQLite insert. - [ ] Instances with `match_labels: []` or `null` are the "catch-all" for their type. An instance with `match_labels: ["security"]` opts *out* of catch-all duty. ### Logging - [ ] One log line per dispatch including the selection reason: `[webhook] dispatch <type> -> <instance> (match: [security] ∩ [security,audit]; idle=2, loaded=0)` or `... (fallback: no match; round-robin catch-all; idle=1)`. ### Tests - [ ] `dispatchByType` with labels: - (a) Labels overlap one instance's `match_labels` → that instance picked. - (b) Labels overlap multiple → largest intersection wins. - (c) No overlap with any `match_labels`, default pool non-empty → catch-all round-robin. - (d) No overlap AND no catch-all → warn + round-robin across all. - (e) No labels supplied → behaves exactly as A2 (catch-all round-robin). ## Out of scope - Negative matching (exclude from routing). - Label source beyond Forgejo issue/PR labels (no CI-added tags from claude-hooks itself). - Label-weighted scoring (e.g. `security` weight 10, `frontend` weight 1) — v1 uses intersection size only. - UI for editing `match_labels` (see **A6**). ## References - Tracking issue: #47. - A2 sets the baseline selection order this story extends. ## Dependencies - **Blocked by:** A1, A2. - **Blocks:** A7 (breakdown skill relies on this to make its labels meaningful). - **Branch off:** `main` (after A2 lands).

claude-desktop added the

area:agents

type:user-story

labels

2026-04-18 15:02:14 +00:00

claude-desktop added this to the Agent pool + customization milestone

2026-04-18 15:02:20 +00:00

claude-desktop referenced this issue

2026-04-18 23:39:51 +00:00

Agents: per-instance Claude Code plugin enablement (bake once, enable per instance) #72