Posted inAI AWS Machine Learning

Bedrock Converse toolConfig: Idempotent Tool Results Under Retries

No Comments
1 views

This guide focuses on Bedrock Converse toolConfig for production systems, with practical trade-offs for reliability, security, and cost.
Converse / ConverseStream with toolConfig is the right production API for coding agents — until a retry, a mid-stream disconnect, or a Step Functions task token replay re-invokes apply_patch with the same toolUseId. At-least-once is the default. The unfair advantage is treating every tool as an idempotent command keyed in the schema, with durable toolResult caching keyed by toolUseId + args hash.

⚡ TL;DR: Put idempotencyKey (or derive from toolUseId) in every mutating tool schema. Before side effects, claim a DynamoDB lock / conditional write. On retry, return the cached toolResult content. Align with AI Coding Agent Tool Schemas, Exactly-Once Illusions, and Day-35 patterns like Idempotent Tool Calls Against DynamoDB without rehashing the bootcamp article.

Bedrock Converse toolConfig: production guidance

typescript
import {
  BedrockRuntimeClient,
  ConverseCommand,
  ToolConfiguration,
} from "@aws-sdk/client-bedrock-runtime";

const toolConfig: ToolConfiguration = {
  tools: [
    {
      toolSpec: {
        name: "apply_patch",
        description:
          "Apply a unified diff to a repo workspace. Safe under retries when idempotencyKey is stable.",
        inputSchema: {
          json: {
            type: "object",
            properties: {
              workspaceId: { type: "string" },
              unifiedDiff: { type: "string" },
              // ✅ Client/model must pass a stable key per logical mutation
              idempotencyKey: {
                type: "string",
                minLength: 8,
                maxLength: 128,
                description: "Stable key for this patch intent (UUID or hash)",
              },
            },
            required: ["workspaceId", "unifiedDiff", "idempotencyKey"],
            additionalProperties: false,
          },
        },
      },
    },
  ],
  toolChoice: { auto: {} },
};

❌ Omitting idempotencyKey and hoping toolUseId alone is enough can work if you always key on Bedrock’s toolUseId — but multi-agent replays and your own “regenerate” buttons often mint a new toolUseId for the same logical patch. Prefer an explicit business key plus toolUseId for result caching.

Claim → execute → cache toolResult

typescript
import { createHash } from "crypto";
import {
  DynamoDBClient,
  PutItemCommand,
  GetItemCommand,
} from "@aws-sdk/client-dynamodb";

const ddb = new DynamoDBClient({});
const TABLE = process.env.TOOL_IDEMPOTENCY_TABLE!;

function argsHash(input: Record<string, unknown>): string {
  const { idempotencyKey: _k, ...rest } = input;
  return createHash("sha256").update(JSON.stringify(rest)).digest("hex").slice(0, 32);
}

export async function runApplyPatch(toolUseId: string, input: {
  workspaceId: string;
  unifiedDiff: string;
  idempotencyKey: string;
}) {
  const pk = `tool#apply_patch#${input.idempotencyKey}`;
  const hash = argsHash(input);

  // 1) Fast path: prior success
  const existing = await ddb.send(
    new GetItemCommand({
      TableName: TABLE,
      Key: { pk: { S: pk } },
    })
  );
  if (existing.Item?.status?.S === "SUCCEEDED") {
    if (existing.Item.argsHash?.S !== hash) {
      // ✅ Same key, different args = conflict, do not silently apply
      return {
        toolUseId,
        content: [
          {
            text: JSON.stringify({
              ok: false,
              error: "IdempotencyKeyConflict",
              message: "idempotencyKey reused with different arguments",
            }),
          },
        ],
        status: "error" as const,
      };
    }
    return {
      toolUseId,
      content: [{ json: JSON.parse(existing.Item.resultJson!.S!) }],
      status: "success" as const,
    };
  }

  // 2) Claim in-flight (conditional)
  try {
    await ddb.send(
      new PutItemCommand({
        TableName: TABLE,
        Item: {
          pk: { S: pk },
          status: { S: "IN_FLIGHT" },
          toolUseId: { S: toolUseId },
          argsHash: { S: hash },
          ttl: { N: String(Math.floor(Date.now() / 1000) + 86400) },
        },
        ConditionExpression:
          "attribute_not_exists(pk) OR #s = :failed OR (attribute_exists(ttl) AND ttl < :now)",
        ExpressionAttributeNames: { "#s": "status" },
        ExpressionAttributeValues: {
          ":failed": { S: "FAILED" },
          ":now": { N: String(Math.floor(Date.now() / 1000)) },
        },
      })
    );
  } catch (e: any) {
    if (e?.name === "ConditionalCheckFailedException") {
      // Another worker owns it — wait/poll or return in-progress error for model retry
      return {
        toolUseId,
        content: [{ text: JSON.stringify({ ok: false, error: "InFlight" }) }],
        status: "error" as const,
      };
    }
    throw e;
  }

  // 3) Side effect once
  try {
    const result = await applyDiffOnce(input.workspaceId, input.unifiedDiff);
    await ddb.send(
      new PutItemCommand({
        TableName: TABLE,
        Item: {
          pk: { S: pk },
          status: { S: "SUCCEEDED" },
          toolUseId: { S: toolUseId },
          argsHash: { S: hash },
          resultJson: { S: JSON.stringify(result) },
          ttl: { N: String(Math.floor(Date.now() / 1000) + 7 * 86400) },
        },
      })
    );
    return {
      toolUseId,
      content: [{ json: result }],
      status: "success" as const,
    };
  } catch (err: any) {
    await ddb.send(
      new PutItemCommand({
        TableName: TABLE,
        Item: {
          pk: { S: pk },
          status: { S: "FAILED" },
          toolUseId: { S: toolUseId },
          argsHash: { S: hash },
          error: { S: String(err?.message ?? err) },
          ttl: { N: String(Math.floor(Date.now() / 1000) + 3600) },
        },
      })
    );
    return {
      toolUseId,
      content: [{ text: JSON.stringify({ ok: false, error: "ApplyFailed" }) }],
      status: "error" as const,
    };
  }
}

async function applyDiffOnce(workspaceId: string, unifiedDiff: string) {
  // workspace FS / git apply — must itself be safe if partially applied
  return { workspaceId, filesTouched: 3, ok: true };
}

Wire results back into Converse correctly

typescript
const bedrock = new BedrockRuntimeClient({});

async function turn(messages: any[]) {
  const res = await bedrock.send(
    new ConverseCommand({
      modelId: "anthropic.claude-3-5-sonnet-20241022-v2:0",
      messages,
      toolConfig,
    })
  );

  const out = res.output?.message;
  if (!out) return res;

  const toolUses = (out.content ?? []).filter((c: any) => c.toolUse);
  if (!toolUses.length) return res;

  const toolResults = [];
  for (const block of toolUses) {
    const tu = block.toolUse!;
    const result = await runApplyPatch(tu.toolUseId!, tu.input as any);
    toolResults.push({
      toolResult: {
        toolUseId: result.toolUseId,
        content: result.content,
        status: result.status,
      },
    });
  }

  // Continue conversation with tool results — same messages array, no duplicate applies
  return turn([
    ...messages,
    out,
    { role: "user", content: toolResults },
  ]);
}

✅ Always echo the same toolUseId in toolResult. ❌ Inventing a new id breaks the model’s tool loop.

For schema evolution and strict JSON, reuse lessons from AI Coding Agent Tool Schemas. For async long tools, combine with Bedrock Agents Plus EventBridge — store the idempotency record before enqueue.

Read-only tools still need contracts

Idempotency is mandatory for mutators, but read tools (grep_repo, read_file) still need strict schemas and size limits so retries do not blow token budgets. Cap maxBytes, require workspaceId, and return truncated markers explicitly so the model does not invent the rest of the file.

typescript
{
  name: "read_file",
  inputSchema: {
    json: {
      type: "object",
      properties: {
        workspaceId: { type: "string" },
        path: { type: "string", maxLength: 512 },
        maxBytes: { type: "integer", minimum: 1, maximum: 100_000 },
      },
      required: ["workspaceId", "path"],
      additionalProperties: false,
    },
  },
}

Testing retries without burning Bedrock

Unit-test the claim table with three concurrent runApplyPatch calls sharing one idempotencyKey — assert a single applyDiffOnce. Integration-test by recording a Converse transcript fixture and replaying the tool phase twice. Only then run a live Bedrock soak. This keeps CI cheap while still proving the unfair advantage: retries become free no-ops instead of corrupted workspaces.

Checklist

  • [ ] Every mutating tool schema requires idempotencyKey (or documented toolUseId-only policy)
  • [ ] DynamoDB (or similar) claim with conditional write before side effects
  • [ ] Cached success returns identical toolResult JSON on retry
  • [ ] Key reuse with different args → explicit conflict error (not second apply)
  • [ ] toolResult.toolUseId matches Bedrock’s toolUseId
  • [ ] TTL on idempotency rows; FAILED reusable after backoff
  • [ ] Load test: duplicate Converse retries do not double-apply patches


Related: Bedrock Agents: Idempotent Tool Calls Against DynamoDB Writes; Day 32: Bedrock Converse API Tool Choice in Production; Bedrock Converse API Tool Choice Modes for Production Coding Agents.

Most viewed

  1. Python Decorators Explained: From Simple Wrappers to Production Patterns10 views
  2. Distributed Locks Reality Check: When Redis Redlock Is the Wrong Tool7 views
  3. REST API Design Best Practices: The Patterns That Make APIs a Joy to Use7 views
  4. Python asyncio vs Threading: The Benchmark That Changes How You Think About Concurrency7 views
  5. LLM context windows: Tokens, Context Windows, and Why Models Forget Mid-Task6 views · 1 like

Newly added

  1. AWS Budgets + Cost Anomaly Detection: Cap Runaway Coding-Agent Spend
  2. OpenSearch Serverless: Semantic Scratch Memory for Multi-Turn Coding Agents
  3. ECR + Lambda Container Images: Heavyweight Coding Tools Without Zip Limits
  4. CloudTrail Lake: Query Agent IAM Abuses Without Spreadsheets
  5. DynamoDB Transactions: Atomic Tool-Ledger Writes for Coding Agents

Deep-dive PDF

Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.

Already subscribed? or open the subscribe page.

Discover more from CheatCoders

Subscribe to get the latest posts sent to your email.

Comments

No comments yet. Why don’t you start the discussion?

Leave a comment

No account needed. Name and email are optional.