AWS Systems Manager Session Manager: Audited Break-Glass Shell into Coding-Agent Sandboxes

Coding agents fail in ways dashboards cannot ssh into: a half-applied patch, a stuck npm install, a mystery process holding the workspace lock. The wrong fix is a standing SSH key on every Fargate/EC2 sandbox. AWS Systems Manager Session Manager gives operators an IAM-gated shell through the SSM agent — no inbound 22, optional session encryption, and logs to S3/CloudWatch that actually survive the “who typed rm -rf?” review. Distinct from Network Firewall egress filtering (what the sandbox can call) and IAM Roles Anywhere (how on-prem runners get AWS creds): Session Manager is human break-glass with an audit trail.

⚡ TL;DR: Install/enable SSM agent on sandbox hosts (or ECS exec where appropriate), deny SSH security-group ingress, grant ssm:StartSession to an on-call role with MFA, enable session logging to a locked S3 bucket, and alert on every break-glass via CloudTrail. Related: CloudTrail Lake IAM abuses, Fargate Spot sandboxes, CodeBuild sandboxes, GuardDuty compromised creds.

Why SSH keys rot on agent fleets

❌ Baking a forever SSH public key into the sandbox AMI “just for debugging.”

Prerequisites on the sandbox

# ✅ EC2 / Bottlerocket / Amazon Linux — SSM agent + instance profile
# Instance profile needs AmazonSSMManagedInstanceCore (or tighter custom)
aws ec2 associate-iam-instance-profile \
  --instance-id i-0abc \
  --iam-instance-profile Name=coding-agent-sandbox-ssm

# Security group: egress for SSM endpoints (or VPC interface endpoints);
# ✅ no inbound 22 from 0.0.0.0/0
aws ec2 revoke-security-group-ingress \
  --group-id sg-sandbox \
  --protocol tcp --port 22 --cidr 0.0.0.0/0 || true

Prefer VPC interface endpoints for ssm, ssmmessages, ec2messages so sandboxes with locked egress (Network Firewall) can still accept Sessions without public SSM API access.

For ECS/Fargate, use ECS Exec (Session Manager under the hood) with enableExecuteCommand and task IAM — document which path your fleet uses so on-call runbooks do not invent SSH.

Start an audited session

# ✅ MFA-protected on-call role assumed first
aws sts assume-role \
  --role-arn arn:aws:iam::123456789012:role/coding-agent-breakglass \
  --role-session-name oncall-$(date +%s) \
  --serial-number arn:aws:iam::123456789012:mfa/you \
  --token-code 123456

aws ssm start-session --target i-0abc

# ECS Exec equivalent
aws ecs execute-command \
  --cluster agent-sandboxes \
  --task TASK_ARN \
  --container sandbox \
  --interactive \
  --command "/bin/bash"

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "BreakGlassSession",
      "Effect": "Allow",
      "Action": ["ssm:StartSession"],
      "Resource": [
        "arn:aws:ec2:*:*:instance/*",
        "arn:aws:ssm:*:*:document/AWS-StartSSHSession",
        "arn:aws:ssm:*:*:document/SSM-SessionManagerRunShell"
      ],
      "Condition": {
        "StringEquals": {
          "ssm:resourceTag/workload": "coding-agent",
          "aws:MultiFactorAuthPresent": "true"
        },
        "Bool": { "aws:MultiFactorAuthPresent": "true" }
      }
    },
    {
      "Sid": "TerminateOwn",
      "Effect": "Allow",
      "Action": ["ssm:TerminateSession", "ssm:ResumeSession"],
      "Resource": "arn:aws:ssm:*:*:session/${aws:userid}-*"
    }
  ]
}

❌ ssm:StartSession on * without tag conditions or MFA — that is a lateral-movement gift.

Session logging that holds up in IR

# ✅ Session Manager preferences → S3 bucket + CloudWatch Logs
# Bucket: Block Public Access, Object Lock optional for IR retention,
# deny delete for breakglass role
aws ssm update-document \
  --name SSM-SessionManagerRunShell \
  --document-version '$LATEST' \
  --content file://session-prefs.json

Example prefs fragment:

{
  "schemaVersion": "1.0",
  "description": "coding-agent break-glass logging",
  "sessionType": "Standard_Stream",
  "inputs": {
    "s3BucketName": "coding-agent-session-logs",
    "s3KeyPrefix": "ssm/",
    "s3EncryptionEnabled": true,
    "cloudWatchLogGroupName": "/ssm/coding-agent-sessions",
    "cloudWatchEncryptionEnabled": true,
    "idleSessionTimeout": "15",
    "maxSessionDuration": "60",
    "runAsEnabled": true,
    "runAsDefaultUser": "ssm-user"
  }
}

Send CloudTrail StartSession events into CloudTrail Lake and page on-call when sessions open outside change windows. Pair with Detective if the same principal also spikes STS anomalies.

Runbook: break-glass without making it normal

1. Confirm agent run ID / task ARN and customer impact
2. Assume break-glass role with MFA; note ticket ID in session reason (tags / chat)
3. start-session / ECS Exec — read-only first (ls, logs); write only with peer ack
4. Capture evidence to IR bucket; do not “fix prod” inside the sandbox if promote-path exists
5. Terminate session; rotate any secrets touched; open postmortem stub

# ✅ detect sessions via CloudTrail Lake (sketch)
# SELECT userIdentity.arn, eventTime, requestParameters
# FROM cloudtrail_lake
# WHERE eventName = 'StartSession'
# AND eventTime > date_add('hour', -24, current_timestamp)

Failure modes

Production checklist

• [ ] No inbound SSH to sandbox security groups
• [ ] SSM agent + instance/task role with least privilege
• [ ] VPC endpoints for SSM in locked-egress accounts
• [ ] Break-glass role: MFA, tags, short session max, idle timeout
• [ ] Session logs to encrypted S3 + CloudWatch; retain per IR policy
• [ ] CloudTrail alert on StartSession / ExecuteCommand
• [ ] Runbook distinguishes read-only vs mutate; ticket ID required
• [ ] Quarterly access review; remove standing humans from sandbox roles
• [ ] Document ECS Exec vs EC2 Session Manager per environment

FAQ

Q: Does Session Manager work on Fargate?
A: Use ECS Exec for Fargate tasks (Session Manager plumbing). Classic start-session to instance IDs is the EC2 path. Do not assume one CLI covers both.

Q: Is this a replacement for Network Firewall?
A: No. Firewall constrains egress. Session Manager constrains and audits human ingress. You want both.

Q: What about CodeBuild sandboxes?
A: Prefer ephemeral CodeBuild with logs in CloudWatch; break-glass is rarer. If you need a shell into a long-lived builder host, treat it like EC2 SSM — same IAM + logging bar.

Related reading

• AWS Network Firewall: Egress Filtering for Coding-Agent Sandboxes
• AWS IAM Roles Anywhere: Cert-Based Creds for On-Prem Runners
• CloudTrail Lake: Query Agent IAM Abuses
• AWS Fargate Spot: Cheap Ephemeral Sandboxes

Break glass with IAM and tape. Not with a master key in the AMI.

Last updated on October 3, 2026