EventBridge Scheduler: Overnight Agent Batches Without Cron Drift

You scheduled a “nightly” RAG reindex with an EventBridge rule cron: cron(30 23 * * ? *) in UTC. Someone assumed that meant 11:30 PM in India. Six months later the job runs at the wrong wall-clock time, overlaps the morning traffic spike, and a missed invoke from a transient Lambda throttle gets replayed into a double reindex that corrupts the vector store. The unfair advantage for overnight agent batches is EventBridge Scheduler — first-class timezones, flexible windows, one-time and rate schedules, and dead-lettering — plus idempotency tokens so a late delivery never double-applies.

⚡ TL;DR: Prefer EventBridge Scheduler over classic EventBridge rule cron for overnight agent work (RAG reindex, eval harnesses, batch Converse). Set ScheduleExpressionTimezone to Asia/Kolkata, use flexible time windows, wire a DLQ, and stamp every run with an idempotency key derived from the schedule window. Pair with Bedrock Agents + EventBridge Async Tools, EventBridge Pipes to Lambda, and SQS FIFO Ordered Agent Job Queues.

Why classic EventBridge cron drifts for agents

EventBridge rules evaluate cron in UTC unless you carefully document otherwise. Humans think in local time. Overnight agent batches usually need:

• Wall-clock alignment with team SLAs (“after 1 AM IST, before 5 AM IST”)
• Catch-up without stampeding when the account was throttled
• Exactly-once business effect even if the target is invoked twice

Rules also lack first-class flexible time windows and one-time schedules with automatic completion. Scheduler was built for that.

// ❌ Classic rule: UTC cron, easy to misread as local
{
  ScheduleExpression: "cron(30 19 * * ? *)", // "7:30 PM UTC" — is that IST?
  // No timezone field, no flexible window, no schedule-level DLQ of the same shape
}

✅ Scheduler: cron(30 1 * * ? *) with ScheduleExpressionTimezone: Asia/Kolkata.

One-time vs rate vs cron schedules

For overnight RAG and eval, prefer cron + timezone. Use one-time schedules for operator-triggered backfills so you do not leave a permanent rule that someone forgets.

import {
  SchedulerClient,
  CreateScheduleCommand,
} from "@aws-sdk/client-scheduler";

const scheduler = new SchedulerClient({});

export async function createNightlyRagReindex(input: {
  scheduleName: string;
  lambdaArn: string;
  roleArn: string;
  dlqArn: string;
}) {
  await scheduler.send(
    new CreateScheduleCommand({
      Name: input.scheduleName,
      ScheduleExpression: "cron(0 2 * * ? *)", // 02:00
      ScheduleExpressionTimezone: "Asia/Kolkata", // ✅ wall clock IST
      FlexibleTimeWindow: {
        Mode: "FLEXIBLE",
        MaximumWindowInMinutes: 30, // ✅ avoid thundering herd at :00
      },
      Target: {
        Arn: input.lambdaArn,
        RoleArn: input.roleArn,
        Input: JSON.stringify({
          jobType: "rag_reindex",
          // static template; runtime adds window id below in handler
        }),
        DeadLetterConfig: { Arn: input.dlqArn },
        RetryPolicy: {
          MaximumEventAgeInSeconds: 3600,
          MaximumRetryAttempts: 2,
        },
      },
      State: "ENABLED",
    })
  );
}

Flexible time windows and Asia/Kolkata

A flexible window of 15–30 minutes lets Scheduler spread invocations so you do not synchronize every account’s nightly job to the same second. For agent batches that hit Bedrock or shared OpenSearch, that alone prevents self-inflicted throttle storms.

Timezone Asia/Kolkata has no DST, which removes one class of “cron drift” — but document the timezone on the schedule name and in runbooks. Future you will assume UTC.

// ✅ Name encodes timezone intent
const name = "agent-rag-reindex-0200-ist";

Idempotency so missed runs do not double-apply

Scheduler retries and DLQ redrives can deliver the same logical night twice. Your Lambda must treat “Tuesday IST reindex” as a single business operation.

Derive a stable key from the schedule name + local date of the intended window, not from Date.now().

import { createHash } from "crypto";
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
import {
  DynamoDBDocumentClient,
  PutCommand,
} from "@aws-sdk/lib-dynamodb";

const ddb = DynamoDBDocumentClient.from(new DynamoDBClient({}));

function istDateString(d = new Date()): string {
  // en-CA gives YYYY-MM-DD; timeZone forces Asia/Kolkata calendar date
  return new Intl.DateTimeFormat("en-CA", {
    timeZone: "Asia/Kolkata",
    year: "numeric",
    month: "2-digit",
    day: "2-digit",
  }).format(d);
}

export async function claimNightlyRun(scheduleName: string): Promise<{
  claimed: boolean;
  idempotencyKey: string;
}> {
  const day = istDateString();
  const idempotencyKey = createHash("sha256")
    .update(`${scheduleName}#${day}`)
    .digest("hex")
    .slice(0, 32);

  try {
    await ddb.send(
      new PutCommand({
        TableName: process.env.IDEMPOTENCY_TABLE!,
        Item: {
          pk: `job#${idempotencyKey}`,
          sk: "claim",
          scheduleName,
          day,
          status: "running",
          ttl: Math.floor(Date.now() / 1000) + 86400 * 7,
        },
        ConditionExpression: "attribute_not_exists(pk)",
      })
    );
    return { claimed: true, idempotencyKey };
  } catch (e: any) {
    if (e.name === "ConditionalCheckFailedException") {
      return { claimed: false, idempotencyKey };
    }
    throw e;
  }
}

export async function handler(event: { jobType: string }) {
  const scheduleName = process.env.SCHEDULE_NAME ?? "agent-rag-reindex-0200-ist";
  const { claimed, idempotencyKey } = await claimNightlyRun(scheduleName);
  if (!claimed) {
    // ✅ Late retry / double delivery — no-op
    return { ok: true, skipped: true, idempotencyKey };
  }

  // run RAG reindex / eval / batch Converse with idempotencyKey in logs + EMF
  await runAgentBatch({ jobType: event.jobType, idempotencyKey });
  return { ok: true, skipped: false, idempotencyKey };
}

Same pattern pairs with Bedrock Converse toolConfig idempotent tool results when the overnight job itself drives tool-using agents.

Scheduler vs EventBridge rules — when to keep rules

Keep rules when you need event-pattern matching (S3 Object Created → agent pipeline) or bus fan-out. Use Scheduler when time is the primary trigger.

For async agent tools that should not block a Bedrock session, combine Scheduler (time) with EventBridge buses (facts) — do not force everything through one cron.

Dead-letter and observability

Always set DeadLetterConfig on the schedule target. Alarm on DLQ depth and on “skipped due to idempotency” vs “failed before claim” so you can tell catch-up from poison messages.

Emit EMF with dimensions jobType, scheduleName, day (IST). If a night is skipped because the claim already exists, that is a success metric — not an error — unless the first run never finished (store status=running with a watchdog).

// ✅ Watchdog: if status=running older than 2h, allow reclaim or page

Checklist

• [ ] Overnight jobs use EventBridge Scheduler, not UTC-only rule cron
• [ ] ScheduleExpressionTimezone = Asia/Kolkata (or your real ops TZ)
• [ ] Flexible window 15–30 minutes for Bedrock / shared index workloads
• [ ] Target DLQ + low retry count; alarm on DLQ visible messages
• [ ] Idempotency key = scheduleName + IST calendar date claimed in DynamoDB
• [ ] Double delivery returns skipped success, not a second reindex
• [ ] One-time at() for backfills; cron for recurring nights
• [ ] Runbooks state timezone explicitly next to the cron expression

Cron drift is a people problem disguised as infrastructure. Scheduler makes the timezone explicit; idempotency makes retries safe. Overnight agent batches need both.