S3 Conditional Writes: Idempotent Agent Artifact Uploads With If-None-Match

This guide focuses on S3 Conditional Writes for production systems, with practical trade-offs for reliability, security, and cost.
Your coding agent finishes a patch, uploads s3://bucket/sessions/abc/patch.diff, and a retried tool call from the same turn uploads again. Or turn 7 and turn 8 race two different patches onto the same key. Last writer wins — until your evaluator applies the wrong diff. S3 conditional writes (If-None-Match: *, If-Match with ETag) give you compare-and-swap semantics so agent artifact uploads become idempotent instead of silently clobbering.

⚡ TL;DR: For agent patches, logs, and build outputs, PutObject with IfNoneMatch: "*" creates-if-absent; use IfMatch + ETag for safe updates. Combine with idempotency keys like SQS FIFO agent queues and tool contracts in AI Coding Agent Tool Schemas. See also CodeBuild Spec Sandboxes for build artifact keys. Never overwrite session artifacts on blind PutObject.

S3 Conditional Writes: production guidance

Multi-turn agents retry. Bedrock may re-send a toolUse after a stream blip. Your Lambda might process the same SQS message twice. Two writers + one key = lost update:

// ❌ Blind put — retries and parallel turns clobber
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({});

export async function uploadPatch(sessionId: string, body: string) {
  await s3.send(
    new PutObjectCommand({
      Bucket: process.env.ARTIFACT_BUCKET!,
      Key: `sessions/${sessionId}/patch.diff`,
      Body: body,
      ContentType: "text/plain",
    })
  );
  return { ok: true, key: `sessions/${sessionId}/patch.diff` };
}

If-None-Match: * — create only once

S3 supports conditional puts: when If-None-Match is *, the put succeeds only if the object does not already exist. Retries of the same create become PreconditionFailed (412) — which you treat as success-if-same or conflict-if-different.

// ✅ Idempotent create for a unique artifact key
import {
  S3Client,
  PutObjectCommand,
  GetObjectCommand,
} from "@aws-sdk/client-s3";
import { createHash } from "node:crypto";

const s3 = new S3Client({});
const BUCKET = process.env.ARTIFACT_BUCKET!;

export async function putArtifactOnce(input: {
  key: string;
  body: Buffer | string;
  contentType: string;
  idempotencyKey: string;
}) {
  const bodyBuf = Buffer.isBuffer(input.body)
    ? input.body
    : Buffer.from(input.body, "utf8");
  const checksum = createHash("sha256").update(bodyBuf).digest("hex");

  try {
    const out = await s3.send(
      new PutObjectCommand({
        Bucket: BUCKET,
        Key: input.key,
        Body: bodyBuf,
        ContentType: input.contentType,
        IfNoneMatch: "*", // ✅ create-only
        Metadata: {
          idempotencykey: input.idempotencyKey,
          checksumsha256: checksum,
        },
      })
    );
    return { status: "created" as const, etag: out.ETag, checksum };
  } catch (e: any) {
    if (e?.name === "PreconditionFailed" || e?.$metadata?.httpStatusCode === 412) {
      // Object exists — verify same payload vs true conflict
      const existing = await s3.send(
        new GetObjectCommand({ Bucket: BUCKET, Key: input.key })
      );
      const existingMeta = existing.Metadata ?? {};
      if (
        existingMeta.idempotencykey === input.idempotencyKey ||
        existingMeta.checksumsha256 === checksum
      ) {
        return {
          status: "already_exists_same" as const,
          etag: existing.ETag,
          checksum,
        };
      }
      return {
        status: "conflict" as const,
        etag: existing.ETag,
        checksum,
      };
    }
    throw e;
  }
}

Key design: include the idempotency key in the object key when you want automatic uniqueness:

sessions/{sessionId}/patches/{idempotencyKey}.diff

Then If-None-Match: * means “this tool result uploads at most once.” Parallel turns with different keys do not fight.

If-Match + ETag — safe updates

When you must update a mutable pointer (e.g. sessions/abc/latest.json), read ETag, then put with IfMatch:

// ✅ Compare-and-swap update
import {
  GetObjectCommand,
  PutObjectCommand,
} from "@aws-sdk/client-s3";

export async function casLatestManifest(
  sessionId: string,
  next: object
): Promise<"ok" | "conflict"> {
  const key = `sessions/${sessionId}/latest.json`;
  const cur = await s3.send(
    new GetObjectCommand({ Bucket: BUCKET, Key: key })
  );
  const etag = cur.ETag;
  if (!etag) return "conflict";

  try {
    await s3.send(
      new PutObjectCommand({
        Bucket: BUCKET,
        Key: key,
        Body: Buffer.from(JSON.stringify(next), "utf8"),
        ContentType: "application/json",
        IfMatch: etag, // ✅ only if nobody else wrote
      })
    );
    return "ok";
  } catch (e: any) {
    if (e?.name === "PreconditionFailed" || e?.$metadata?.httpStatusCode === 412) {
      return "conflict";
    }
    throw e;
  }
}

On conflict, the tool returns an error the model can understand: “manifest changed; re-read latest.” That beats applying a stale patch.

Combining with idempotency keys

S3 conditionals handle object identity. Idempotency keys handle tool identity across queues and Converse retries:

1. Tool schema requires idempotencyKey (UUID per logical upload)
2. Object key includes that key or metadata stores it
3. SQS FIFO MessageDeduplicationId = same key when the upload is driven from a queue
4. DynamoDB conditional put records “upload completed” for session-level bookkeeping

export const uploadArtifactToolSchema = {
  name: "upload_artifact",
  description: "Upload a session artifact exactly once for this idempotencyKey",
  inputSchema: {
    type: "object",
    additionalProperties: false,
    required: ["sessionId", "kind", "content", "idempotencyKey"],
    properties: {
      sessionId: { type: "string", minLength: 8 },
      kind: { type: "string", enum: ["patch", "log", "build"] },
      content: { type: "string" },
      idempotencyKey: { type: "string", minLength: 8, maxLength: 128 },
    },
  },
} as const;

Pitfalls

Checklist

• [ ] Agent artifact keys include sessionId + idempotencyKey where possible
• [ ] Creates use IfNoneMatch: "*"; updates use IfMatch: etag
• [ ] 412 + same checksum/idempotency → success; else conflict tool error
• [ ] Tool schema requires idempotencyKey
• [ ] Align SQS dedup IDs with the same key for queue-driven uploads
• [ ] Alarm / metric on conflict rates (possible agent loops)
• [ ] Bucket policy: agent role can Put only under sessions/${sessionId}/*
• [ ] Never blind-overwrite patch.diff on a shared key

Idempotent uploads are not optional once two turns can touch S3. Conditional writes turn “last writer wins” into create-once and compare-and-swap — the same discipline you already use for DynamoDB and FIFO queues.

Related: Bedrock Agents: Idempotent Tool Calls Against DynamoDB Writes; AWS S3 Presigned URLs: The Security Mistakes 90% of Developers Make; Multi-Agent Systems: How to Build Agent Teams That Do Not Hallucinate Each Other Into Failure.