Designing dynamic fan-out

Fan out over multiple services deliberately hardcodes identify-upstreams’s list — ["github", "npm"], fixed at authoring time — to keep that tutorial’s focus on the fan-out mechanics themselves, not list-generation. This guide is the other half: inferring a genuinely runtime-determined list, the way a real pipeline actually would. ~/.vectorstep/pipelines/fan-out-multi-service-triage.yaml, seeded by the installer, is a complete, working example of everything below — this guide is the walkthrough of why it’s built that way, including the one real limitation it doesn’t fully solve.

Replace the hardcoded list with an inferring step

An LLM step reads the alert and returns a list it actually determined, not one the pipeline author already knew — the real version of identify-upstreams from the tutorial. The sample’s identify-services step does this with an ordinary structured-output prompt: describe the task, ask for a JSON array under a named field, and bound it explicitly (“Keep the list short (≤10 services). If the impact is unclear, include all plausible candidates rather than guessing.”) so the model doesn’t return an unbounded list on a vague alert.

Name that field something like services or affected_hosts, never items — the naming gotcha already documented on the reference page. Jinja2 attribute access tries getattr before getitem, so a field named items resolves to Python’s dict method object instead of your data, and the failure doesn’t look like a naming mistake — it looks like over failed to resolve at all. The sample’s own prompt spells this out to the model directly (“Use services not items”), which is worth copying literally: it’s cheap insurance against the one field name that silently breaks everything downstream.

Consolidating without knowing the branch count in advance

The tutorial’s consolidate step references check-upstreams/0 and check-upstreams/1 directly — a shortcut that only works because the list is fixed at two. A genuinely dynamic list can’t be enumerated by index at authoring time, and it’s worth being precise about what VectorStep actually gives you here, because it’s less than it first appears: a fan-out group’s branches are registered individually as {fan_out_name}/{index} — triage-services/0, triage-services/1, and so on — but the group’s own name is never registered at all. There is no {{steps.triage_services.summary}} to fall back on for an aggregate view; a fan-out group produces no step-level output of its own that a later step can reference, only its branches. Reference it and you get the same silent empty-string Jinja2 lookup any other missing field produces — not an error, just nothing.

The sample’s own remediation-decision step works around this two ways at once, and it’s worth reading both for what they actually buy you rather than copying the shape uncritically:

• It gives the consolidator {{steps.identify_services.next_step_context}} as framing — the list-producing step’s own output, which is a real, always-populated field, unlike anything from the fan-out group itself. This is reliable regardless of branch count, because it doesn’t depend on how many branches ran.
• For actual per-branch detail, it hardcodes triage-services/0 through /2 with a comment owning the limitation directly: “first 3 branches shown — check all steps["triage-services/*"]”. This is a real gap, not a clean pattern — branches beyond whatever index you hardcode are invisible to the consolidator’s prompt specifically, even though they still ran, still contributed to the joined effective_confidence, and still show up in full in the run’s trace and the UI.

If the consolidator genuinely needs every branch’s structured verdict in its own prompt, the honest options are hardcoding indices up to max_items (verbose, but complete) or leaning on the coarser signals that are reliably available — the joined confidence, the identify step’s scope, and the fact that a human reviewing an escalation can always open the run and see every branch directly. Don’t reach for a loop construct that doesn’t exist in the template language; there isn’t one.

Sizing max_items for a real distribution, not a round number

max_items (default 20) isn’t a soft cap — if the inferring step’s list comes back longer, the fan-out step fails outright with a message naming the count and the limit, rather than silently truncating to the first N. That’s the right failure mode (a silently truncated triage that looks complete is worse than a loud one that isn’t), but it means the number you pick is a real operational decision: too low, and a genuinely large incident fails the pipeline exactly when it matters most; too high, and you’ve sized for a distribution that never happens. Think about it as a cost/blast-radius control as much as a safety one — every item is a real LLM call, run concurrently, and a fan-out step with a verifier: and a grounding: block multiplies that per branch, not just per step; see Cost control for the arithmetic. Set it from what a real alert for this pipeline could plausibly produce, not from the sample’s own max_items: 10 or the schema’s default of 20.

Choosing on_empty deliberately

The three options read like they cover meaningfully different outcomes, but two of them are closer than the names suggest. complete (the default) and skip both leave the step status: completed with effective_confidence: 1.0 when the inferring step’s list comes back empty — functionally, downstream when: conditions and confidence gating see identical results either way. What actually differs is the run log: skip logs a plain info-level fan_out_skipped event (“nothing to do, not an error”), while complete logs a warning-level fan_out_empty event — same mechanics, a different claim about whether an empty list was expected. Pick between them based on what you want an incident reviewer to see in the log, not because you expect different pipeline behavior; they don’t differ there.

abort is the one that’s genuinely different — it fails the step outright (status: failed), which does change what happens next. Reach for it when an empty result is itself alarming for this specific pipeline (an inferring step that should always find something for this alert type returning nothing is a sign something upstream broke, not that there was nothing to do). The tutorial’s fixed two-item list never exercises this choice at all, since it can’t be empty by construction — a genuinely dynamic list is the first place on_empty actually matters.

Testing a variable-count fan-out

You can’t manually eyeball every possible branch count the way the tutorial’s fixed two branches allow. Before trusting a dynamic fan-out in production, run it against enough varied input (in stage: testing, with allow_testing=true, the same pattern every tutorial in this series uses) to exercise:

• The join arithmetic — trigger a run where one branch is deliberately weak and confirm all_must_pass actually drags the effective confidence down to that branch’s own score, not an average of it.
• A genuinely empty list — confirm on_empty does what you configured, not just what you assumed reading the YAML.
• A single-item list — the smallest non-empty case, and often the one authors forget to check because it’s “basically sequential.”
• A list at (and one over) max_items — confirm the step fails the way you expect at the boundary, not just comfortably under it.

If the pipeline has any downstream step referencing fan-out branches by hardcoded index (see the consolidation section above), re-check it every time the realistic branch count changes — a step that only ever ran with two or three services identified can hide a hardcoded-index assumption for a long time before an alert that identifies eight services finally exposes it.

Where next

• Parallel groups & fan-out — the full reference this guide is the deeper companion to.
• Fan out over multiple services — the hands-on tutorial with the hardcoded stand-in this guide replaces.
• Cost control — max_items as a spend control, not just a safety one.
• Writing good prompts — the general rule behind the silent-empty-lookup behavior this guide’s consolidation section relies on.
• ~/.vectorstep/pipelines/fan-out-multi-service-triage.yaml — the complete, real worked example referenced throughout.