Day 11: Tool Use as an API, Not a Prompt Trick

Coding agents become dangerous the moment they can call tools. Day 11 reframes tool use as API design: OpenAPI (or equivalent) contracts, idempotency keys for mutating calls, deadlines, and typed errors the model can recover from — not a paragraph that says “you may call functions.”

⚡ TL;DR: Publish tools as versioned schemas shared with handlers. Require idempotency keys on writes. Bound every call with timeouts and size limits. Return structured errors (retryable, code, message). Never invent ad-hoc JSON in prose.

Tools are HTTP APIs with a weird client

The model is an untrusted client. Design like you would for a public API:

# ✅ Fragment of an action-group / tool schema
paths:
  /apply_patch:
    post:
      operationId: apply_patch
      parameters:
        - in: header
          name: Idempotency-Key
          required: true
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ApplyPatch'

❌ “Helper tools” described only in natural language with optional args the handler half-implements.

Idempotency and retries

Agents retry. Networks retry. Humans click twice. Mutating tools without idempotency will double-apply patches or double-comment on PRs.

# ✅ Server-side idempotency
def apply_patch(key: str, path: str, patch: str):
    existing = redis.get(f"idem:{key}")
    if existing:
        return json.loads(existing)
    result = do_apply(path, patch)
    redis.setex(f"idem:{key}", 86400, json.dumps(result))
    return result

Generate keys in the orchestrator (uuid4) and force the model to echo them, or inject them server-side and ignore model-supplied keys for privilege.

Timeouts, size limits, and backpressure

Every tool needs: connection timeout, overall deadline, max payload bytes, and concurrency limits. Long tools should be async (Day 16/17 patterns): return a job ID, expose get_status, do not block the agent session until Lambda’s 15 minutes expire.

const TOOL_LIMITS = {
  apply_patch: { timeoutMs: 10_000, maxPatchBytes: 20_000 },
  run_tests: { timeoutMs: 120_000, mode: "async" },
};

Typed errors beat “something went wrong”

{
  "ok": false,
  "code": "PATH_NOT_ALLOWED",
  "retryable": false,
  "message": "path must be under src/"
}

The agent can branch: retryable → backoff; validation → fix args; authz → stop and ask human. Stringly errors cause infinite ReAct loops (Day 12).

Closing checklist

• [ ] Schemas checked into repo beside handlers
• [ ] Idempotency on all mutating tools
• [ ] Timeouts and payload caps enforced server-side
• [ ] Typed error envelope with retryable
• [ ] Authz allowlist per caller role
• [ ] CI fails when OpenAPI and handler types drift

Worked example: Bedrock action groups

Keep the OpenAPI file as the source of truth. Generate handler stubs and run contract tests that POST sample tool payloads through the same validators the Lambda uses. When the model invents file_path, the action group should reject before your filesystem sees it.

Instrument tool_name, latency_ms, error_code, and idempotency_replay metrics. Spikes in replays mean clients are retrying aggressively — often a timeout misconfig, not model creativity.

Failure modes to watch

• Prompt-only tools with no server schema.
• Missing idempotency on comment/patch tools.
• Infinite retries on non-retryable validation errors.
• Sync wrappers around 10-minute test suites inside one agent turn.
• Over-broad shell tools (run_command) instead of narrow argv APIs.

Field notes from production

Idempotency stores need retention and privacy reviews — they contain paths and patches. Encrypt at rest, TTL ≥ longest client retry, and scrub on user deletion requests. Expose a ‘dry_run’ flag on mutating tools for plan-then-act previews.

Implementation sketch

# Implementation sketch: tool registry
REGISTRY = {
  "apply_patch": ToolSpec(schema=..., timeout=10, mutating=True, idempotent=True),
  "read_file": ToolSpec(schema=..., timeout=5, mutating=False),
}

Operator addendum

Prefer argv arrays over shell strings even in ‘safe’ tools. If you must expose a formatter, run it in a gVisor/Firecracker sandbox with no credentials (Day 14/17 territory for elevation).

Designing tools as product surfaces

Name tools after user intents (quarantine_test, open_draft_pr) not implementation leftovers (invoke_lambda_17). Write one-paragraph docs per tool for humans and for the model description field — they often diverge. Include examples of valid args in the schema examples if the provider supports it, but never only in prose. Load-test the tool layer without the model: fuzz args, hammer idempotency, confirm timeouts. Agents amplify tool bugs; they do not create them from nothing.

Extended discussion

Return to the core angle for Day 11: OpenAPI tools, idempotency keys, and timeouts for coding agents. That sentence is the acceptance lens for every design review this week. If a proposed change does not make this angle easier to measure or enforce, it is a distraction.

Write down three metrics you will look at after shipping Day 11 ideas, schedule a 45-minute readout, and archive the notes next to the eval artifacts. Architecture without a readout becomes slideshow archaeology.

Pair this day with the adjacent lessons in the series navigation below. Forward links exist so you can keep momentum; backward links exist so you can repair foundations when a later lab fails for boring earlier reasons.

Practically, allocate half a day to implement the smallest vertical slice, half a day to wire measurement, and refuse to polish UI until both are done. This ordering is how bootcamp projects stay honest under time pressure.

Series navigation

← Day 10 · Day 12 →

Last updated September 11, 2026