Chaos for IAM Boundaries: Prove Apps Fail Closed on Denied Actions

Chaos for IAM Boundaries: Prove Apps Fail Closed on Denied Actions
8 views
This guide focuses on Chaos for IAM Boundaries for production systems, with practical trade-offs for reliability, security, and cost. IAM policies look correct in reviews and still fail open in code: a denied kms:Decrypt becomes a swallowed exception, a missing s3:GetObject falls back to a public URL, or a half-written Dynamo item looks like success. Chaos for IAM injects deliberate denials in staging so you prove fail-closed behavior before production.
⚡ TL;DR: Maintain a chaos suite that temporarily attaches Deny statements (or SCPs in a sandbox account) for critical actions, runs synthetic traffic, and asserts apps return typed authz errors + metrics—not partial writes. Pair with IAM Access Analyzer, IAM Credential Canaries, and Lambda Authorizers.

Chaos for IAM Boundaries: production guidance

text
Inject: Deny s3:GetObject on private bucket
Expect: 403/typed AuthZError, no plaintext fallback, metric+trace
Fail:   empty body treated as "not found", retry storms, or public CDN fetch

Inject: Deny dynamodb:PutItem
Expect: transaction abort, no dual-write drift
Fail:   "success" after only the outbox write

Injecting denials safely

json
{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "ChaosDenyKmsDecrypt",
    "Effect": "Deny",
    "Action": ["kms:Decrypt"],
    "Resource": ["arn:aws:kms:us-east-1:123:key/chaos-target"],
    "Condition": {
      "StringEquals": { "aws:PrincipalTag/chaos": "iam-boundary" }
    }
  }]
}
bash
# Staging-only: tag the task role principal, attach chaos policy, run suite, detach
aws iam put-role-policy --role-name checkout-task-staging \
  --policy-name ChaosDenyKms --policy-document file://chaos-deny-kms.json

pnpm chaos:iam --scenario kms-decrypt --expect fail-closed
aws iam delete-role-policy --role-name checkout-task-staging --policy-name ChaosDenyKms
Never run IAM chaos against production customer data. Prefer a dedicated staging account with SCPs that already block dangerous actions.

Assertions in the app and tests

typescript
import { KMSServiceException } from "@aws-sdk/client-kms";

export async function decryptField(blob: Buffer) {
  try {
    return await kms.decrypt({ CiphertextBlob: blob });
  } catch (e) {
    if (e instanceof KMSServiceException && e.name === "AccessDeniedException") {
      metrics.increment("authz.denied", { action: "kms:Decrypt" });
      throw new AuthZError("kms_decrypt_denied"); // typed, no fallback
    }
    throw e;
  }
}

// BAD
export async function decryptFieldBad(blob: Buffer) {
  try {
    return await kms.decrypt({ CiphertextBlob: blob });
  } catch {
    return blob; // fail OPEN — chaos must catch this
  }
}
yaml
# Synthetic expectations
scenarios:
  - name: deny-s3-get
    inject: Deny s3:GetObject
    expect_http: 403
    expect_metric: authz.denied{action=s3:GetObject}
    forbid_logs: ["fallback_public_url"]

Closing checklist

  • [ ] Chaos scenarios cover KMS, S3, DynamoDB, SQS publish, and STS assume
  • [ ] Apps map AccessDenied to typed errors; no silent fallbacks
  • [ ] Metrics and traces emit on authz denial
  • [ ] Suite runs in CI/staging on IAM policy changes
  • [ ] Policies cleaned up after every run (no leftover Denies)
  • [ ] Production remains out of scope for injection

Related reading

Last updated on September 11, 2026 Related: EventBridge Pipes to Lambda: Enrich, Filter, and Fail Closed; CI/CD Pipeline Best Practices: From Code Push to Production in Under 10 Minutes; Cursor Rules for AWS CDK: Stop AI From Inventing IAM Wildcards.

Watch: 100 Days of AI on YouTube

Short videos from CheatCoders, one AI engineering topic per day.

Open the playlist on YouTube · CheatCoders on YouTube

Deep-dive PDF

Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.

Already subscribed? or open the subscribe page.


Discover more from CheatCoders

Subscribe to get the latest posts sent to your email.

Comments

No comments yet. Why don’t you start the discussion?

Leave a comment

No account needed. Name and email are optional.