Tutorial: build your first agent

The quick start got a pipeline running end to end, but it borrowed a ready-made agent to do it. This tutorial builds one from nothing: two real MCP servers, an agent.yaml and a soul.md you write yourself, and a prompt that puts both tools to work.

It deliberately turns none of the trust knobs on. That’s the next tutorial — this one is just about getting an agent to do something real and watching it happen.

What you’ll build

A first-responder agent that triages the same critical alert from the quick start, but actually gathers evidence before handing off:

1. Checks GitHub’s public status API — if the alerting service depends on GitHub (CI, package registry, container registry), an upstream incident changes the whole story.
2. Checks a local known-issues.md file — has this exact alert already been triaged and understood?

Two tools, two MCP servers, zero accounts to sign up for.

Models used for testing

This series was validated internally against openrouter/anthropic/claude-sonnet-4.5 for first-responder and openrouter/anthropic/claude-haiku-4.5 for every other agent introduced later in the series (verifiers, grounding-judge, upstream-checker, generic-pipeline-step). Other models and providers should work too, but exact wording and confidence numbers can vary by model — worth knowing if what you see doesn’t match a “what you should see” section exactly.

Prerequisites

• The quick start completed — Gateway and VectorStep both running.

Both MCP servers below run as subprocesses inside the Gateway container, which already ships Node.js/npx and uv/uvx — there’s nothing to install on your own machine for this tutorial.

1. Give the filesystem server something to read

The Gateway container only has one directory bind-mounted from your host: ~/.vectorstep/agents/, at /data/agents inside the container. The MCP filesystem server’s path argument is resolved inside that container, so scope it to a directory under agents/ rather than somewhere else on your host — an arbitrary host path like ~/vectorstep-tutorial wouldn’t be visible to the subprocess at all.

Create the agent’s directory now and drop a known-issues log in it; you’ll add agent.yaml and soul.md alongside it in step 3.

mkdir -p ~/.vectorstep/agents/first-responder
cat > ~/.vectorstep/agents/first-responder/known-issues.md <<'EOF'
# Known issues

- payments-api: intermittent 5xx during the nightly batch export job
  (02:00-02:15 UTC). Not a page — self-resolves within minutes.
EOF

2. Add both MCP servers to the Gateway

In the Gateway’s ~/.vectorstep/config/gateway.yaml, add:

mcp_servers:
  fetch:
    command: uvx
    args: ["mcp-server-fetch", "--ignore-robots-txt"]
  filesystem:
    command: npx
    args: ["-y", "@modelcontextprotocol/server-filesystem", "/data/agents/first-responder"]

/data/agents/first-responder is the container’s path to the directory you just created — the compose file bind-mounts your host’s ~/.vectorstep/agents/ there. The filesystem server scopes every file operation to that directory and everything under it (which will also include agent.yaml and soul.md once you add them in step 3 — harmless for a tutorial).

--ignore-robots-txt is needed because githubstatus.com/robots.txt disallows /api/ for automated fetchers — a courtesy convention aimed at crawlers, not an access control, but mcp-server-fetch honours it by default and refuses the request otherwise. This deliberately overrides that site’s stated preference for this one tool; it’s not something to apply automatically to every fetch server you configure later.

3. Write the agent

Agents live under ~/.vectorstep/agents/ on the host (/data/agents inside the container) — you already created this agent’s directory in step 1. The agent below is written narrow on purpose — one job, two tools, an explicit output contract — following the principles in Writing good agents, worth a read once this tutorial is done.

~/.vectorstep/agents/first-responder/agent.yaml:

name: first-responder
model: anthropic/claude-sonnet-4-6
max_tokens: 4096
tools:
  - fetch
  - filesystem

Nothing above is Anthropic-specific except that model: line. Use whatever provider and model you’d like here — the Gateway supports OpenRouter, Google, Azure OpenAI, and Ollama too, and this agent works the same regardless; see Providers for every supported prefix and model string format. If you’re on OpenRouter, openrouter/anthropic/claude-sonnet-4.5 is the exact model this tutorial was validated against, per the note near the top of this page. Either way, if you haven’t already added your provider to ~/.vectorstep/config/gateway.yaml (for example, because you used Anthropic in the quick start), see the quick start’s OpenRouter setup steps for the .env key and providers: block — the Gateway restart later in this step picks up both changes together.

~/.vectorstep/agents/first-responder/soul.md:

# First Responder

You are a first-response triage agent for infrastructure alerts. Your job is
narrow: gather two pieces of evidence and hand off a clear brief. You do not
remediate anything, and you do not guess at a root cause you haven't checked
for.

## What you do

1. Use the `fetch` tool to check whether GitHub itself is having an
   incident — relevant if the alerting service depends on it (CI, package
   registry, container registry).
2. Use the `filesystem` tool to read `known-issues.md` and check whether
   this exact alert has already been triaged before.
3. Summarise what you found and hand off.

## Confidence

Confidence measures how completely you gathered the two pieces of evidence
above — not how serious the alert is. Both tool calls succeeded and gave you
a clear answer → confidence should be high. A tool failed, timed out, or gave
you nothing useful → say so honestly and score low, rather than filling the
gap with a plausible-sounding guess.

## Output format

Respond with ONLY the JSON object your prompt asks for. No preamble, no
markdown fences, no commentary outside the JSON.

Restart the Gateway so it picks up the new mcp_servers entries and the new agent — hot reload covers agent config changes, not new MCP server subprocesses, so a restart is the safe move here. This has to be docker compose restart, not docker compose up -d: up -d only recreates a container when the compose file, image, or environment changed, and none of those did here — agent.yaml and gateway.yaml are bind-mounted files the running container reads at startup, so up -d sees nothing to do and silently leaves the old process running with none of your changes, no error either way. restart actually stops and starts the process, which re-reads both files fresh:

cd ~/.vectorstep
docker compose restart gateway
docker compose logs -f --tail=50 gateway

--tail=50 matters here: with no limit, docker compose logs -f replays the container’s entire log history before following new lines — if your Gateway has been running a while, that can be a long scroll of old output before you reach anything from this restart.

Expect a wall of ERROR/WARNING lines that have nothing to do with first-responder. The installer’s seeded sample agents (order-intake, scoped-tools-example, sre-triage, and others) deliberately reference MCP servers (grafana, tavily, atlassian) and providers that this tutorial never configures — quick start’s own text calls them out as “not a fit for this walkthrough.” Seeing them error on every reload is normal and unrelated to anything you’ve done; it’d happen on a totally untouched install too. What actually matters is the summary line at the end:

INFO gateway.main Gateway ready — 8 agent(s) loaded: [..., 'first-responder', ...]

first-responder in that list, with no ERROR/WARNING line naming it specifically, means your agent and both MCP servers loaded cleanly.

4. Check the tools actually loaded

Press Ctrl-C first to exit the logs -f tail from the previous step — it follows indefinitely, so your terminal is stuck there until you do.

The Gateway publishes no host port by default, so this runs from inside its own container rather than against localhost:18780 directly — curl with the Gateway’s own token, already set in its environment. The raw response is the full tool schema for every registered tool, which is a lot of text to eyeball for just “did both servers load” — pipe it through python3 (already in the container) to get a one-line tool count per server instead:

docker compose exec gateway sh -c 'curl -s localhost:18780/mcp/tools -H "Authorization: Bearer $VECTORSTEP_GATEWAY_TOKEN" | python3 -c "import json, sys; d = json.load(sys.stdin); print({k: len(v) for k, v in d.items()})"'

You should see something like {'fetch': 1, 'filesystem': 14} — both servers present with at least one tool each. If a server is missing from that output entirely, check the Gateway’s startup logs — a bad command/args fails loudly there. (Want the full detail behind a specific tool? Drop the | python3 ... part to see the raw JSON.)

5. Wire the pipeline to your new agent

Switch back to the service side, in ~/.vectorstep/ — the quick start’s ~/.vectorstep/pipelines/alert-triage.yaml already triggers on severity: critical and pulls its step from the step library (use: quickstart-triage). Rather than add a second pipeline that would collide with the same trigger match — pipeline resolution is first-match-wins, so the two would race — replace that file’s contents to point at your new agent with an inline step instead of the library one:

name: alert-triage
description: First-responder agent gathers evidence before anyone escalates
trigger:
  match: { source: alertmanager, severity: critical }
  dedup:
    enabled: false   # quick-start convenience — see the quick start's note on this

context_template:
  include:
    - severity
    - summary

steps:
  - name: triage
    executor: gateway
    executor_config:
      agent: first-responder
      session_key: "agent:first-responder:{{pipeline_run_id}}:triage"
    prompt_template: |
      A {{severity}} alert fired for {{labels.service}} in {{labels.environment}}.
      Summary: {{summary}}

      1. Use the fetch tool to check https://www.githubstatus.com/api/v2/summary.json
         for any active GitHub incident.
      2. Use the filesystem tool to read known-issues.md and check whether an
         entry already matches this alert.

      Return ONLY this JSON, no other text:
      {
        "confidence": 0.0,
        "summary": "One sentence: what's happening and what you found",
        "next_step_context": "",
        "upstream_incident": true,
        "known_issue": true,
        "reasoning": {
          "supports": "Evidence that makes this alert credible",
          "contradicts": "Evidence that suggests noise or a known cause",
          "assumptions": "What you're assuming in the absence of data"
        }
      }

confidence, summary and next_step_context are the three mandatory fields every agent response must include — see the LLMOutput contract — next_step_context can be an empty string for a terminal step like this one, but it has to be present or the response fails validation. Everything else in the JSON above (upstream_incident, known_issue) is a free-form extra field, stored and available to any later step as {{steps.triage.upstream_incident}}.

Notice what’s not here: no confidence_threshold, no on_low_confidence, no verifier. This is rung 0 on the trust ladder — a fully working pipeline with no gating at all, which is a legitimate place to stop for a step that only informs rather than acts.

Reload the service — run this, and the trigger command below, from ~/.vectorstep (the directory quick-start’s step 1 left you in):

curl -X POST http://localhost:8000/reload \
  -H "Authorization: Bearer $(grep VECTORSTEP_ADMIN_TOKEN ~/.vectorstep/.env | cut -d= -f2)"

6. Trigger it

Reuse the same fixture from the quick start — the point of this tutorial is the agent, not a new trigger. This pipeline is still stage: testing by default, same as in the quick start, so allow_testing=true is required:

curl -X POST "http://localhost:8000/webhook?source=alertmanager&allow_testing=true" \
  -H "Authorization: Bearer $(grep VECTORSTEP_WEBHOOK_TOKEN ~/.vectorstep/.env | cut -d= -f2)" \
  -H "Content-Type: application/json" \
  -d @webhooks/alertmanager_critical.json

7. Watch it run

Open the run in http://localhost:8000/ui/ and look at the step’s trace. You should see two real tool calls — fetch hitting GitHub’s status API and filesystem reading known-issues.md — plus the agent’s JSON response built from what they returned, not from what the model assumed.

What you should see

Expect the triage step to show completed at (or near) 100% confidence — a real contrast with the quick start’s escalated result at ~10%. That’s the whole point of this tutorial: same alert, same shape of task, but this agent actually has evidence to reason from instead of bare labels.

The summary should reference both tools concretely — something like “GitHub reports all systems operational and known-issues.md documents intermittent 5xx during the nightly batch export job” — and next-step-context should reflect the known-issue match, e.g. checking whether the alert falls inside that documented window. In REASONING → CONTRADICTS, you should see the model explicitly weighing the known-issue match against the alert firing at all — evidence of it actually reasoning over what the tools returned, not pattern-matching a generic “alert fired, escalate” response. The two extra fields from the prompt’s JSON schema, upstream_incident and known_issue, show up under OTHER FIELDS.

Nothing here is being checked yet — there’s no confidence threshold, no verifier, no grounding enforcement on this step (that’s deliberate, per “what’s not here” above). A confident-sounding response and a confident-and-grounded one look identical until something actually verifies the trace behind it. That’s exactly what grounding does next: not whether the output sounds right, but whether it’s backed by a real tool call in the agent’s own trace.

Where next

Go to Turn on the trust knobs next: it takes this exact pipeline and adds a confidence floor and a verifier — the natural continuation of the “nothing here is being checked yet” point above.

Once you’re comfortable with the mechanics:

• Writing good agents — the principles behind the agent you just wrote: narrow scope, minimal tools, honest uncertainty, and why each of those matters more than it looks like it should.
• Writing good prompts — the same treatment for the prompt_template you just wrote, including the soul.md-vs-prompt split and the {{steps.x.y}} hyphen gotcha.
• Adding trust, one signal at a time — the full ladder this tutorial and the next one are climbing.
• Creating agents — the full agent.yaml reference, including scoping tools: to specific tool names and model fallbacks.