23 views
Your factory still runs coding-agent runners on bare metal next to the GPU box — no EC2 instance profile, so someone baked AKIA… into the golden image in 2024. Those keys outlive employees and AMI rebuilds. AWS IAM Roles Anywhere is the unfair advantage for on-prem / hybrid agent runners: exchange an X.509 client certificate for temporary AWS credentials via a trust anchor and profile — same least-privilege roles you already use in-cloud, without static access keys. Pair with Secrets Manager for forge tokens and SCPs on the target account — this post is the cert-based credential layer.
⚡ TL;DR: Create a Roles Anywhere trust anchor (ACMPCA or your CA), map profiles to agent task roles, run aws_signing_helper / credential process on each on-prem runner. Short sessions, cert rotation, no long-lived AKIA in images. Related: IAM condition keys, CodeBuild sandboxes, PrivateLink Bedrock, GuardDuty.
When on-prem runners still make sense
Not every coding-agent fleet is pure Fargate:- Air-gapped or regulated build halls that must keep source on-prem
- Specialized accelerators not in your AWS region yet
- Customer-VPC agents that reach AWS APIs over Direct Connect
- Gradual migrations off Jenkins/Buildkite workers that already have a PKI
| Credential style | Lifetime | Rotation | Agent runner fit |
|---|---|---|---|
| Static AKIA in image | ❌ Years | Manual pain | ❌ Never |
| Long-lived IAM user key on disk | ❌ Months | Manual | ❌ |
| Roles Anywhere + X.509 | ✅ Hours | Cert + role session | ✅ On-prem / hybrid |
| OIDC (GitHub Actions, etc.) | ✅ Minutes | Provider | ✅ Hosted CI |
| EC2/ECS task role | ✅ Hours | AWS-managed | ✅ In-cloud |
Trust anchors, profiles, and roles
Concepts:- Trust anchor — your CA (AWS Private CA or external) that Roles Anywhere trusts
- Profile — which IAM roles can be assumed + session policies
- Role — the actual permissions (S3 artifacts, CodeArtifact, Bedrock invoke)
- Certificate — issued per runner (or per runner fleet with careful CN/SAN mapping)
bash
# ✅ create trust anchor from ACM Private CA ARN (illustrative)
aws rolesanywhere create-trust-anchor \
--name coding-agent-onprem-ca \
--source "sourceData={acmPcaArn=arn:aws:acm-pca:REGION:ACCOUNT:certificate-authority/CAID},sourceType=AWS_ACM_PCA" \
--enabled
# ✅ profile binding agent runner role
aws rolesanywhere create-profile \
--name coding-agent-runners \
--role-arns arn:aws:iam::ACCOUNT:role/coding-agent-onprem-runner \
--duration-seconds 3600 \
--enabled
json
// ✅ role trust policy for Roles Anywhere
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": { "Service": "rolesanywhere.amazonaws.com" },
"Action": ["sts:AssumeRole", "sts:TagSession", "sts:SetSourceIdentity"],
"Condition": {
"ArnEquals": {
"aws:SourceArn": "arn:aws:rolesanywhere:REGION:ACCOUNT:trust-anchor/TA_ID"
},
"StringEquals": {
"aws:PrincipalTag/x509Subject/CN": "coding-agent-runner"
}
}
}]
}
Credential process on the runner
Use the IAM Roles Anywhere credential helper as an AWS CLI / SDK credential_process so every tool turn gets fresh temporary keys.ini
# ✅ ~/.aws/config on on-prem runner
[profile agent-runner]
credential_process = aws_signing_helper credential-process
--certificate /etc/agent/certs/runner.pem
--private-key /etc/agent/certs/runner.key
--trust-anchor-arn arn:aws:rolesanywhere:REGION:ACCOUNT:trust-anchor/TA_ID
--profile-arn arn:aws:rolesanywhere:REGION:ACCOUNT:profile/PROFILE_ID
--role-arn arn:aws:iam::ACCOUNT:role/coding-agent-onprem-runner
region = us-east-1
python
Pin file permissions on # ✅ boto3 picks up credential_process via profile
import boto3
session = boto3.Session(profile_name="agent-runner")
s3 = session.client("s3")
s3.put_object(Bucket=ARTIFACTS, Key=f"tenants/{tenant}/turns/{turn}/out.tgz", Body=data)
# ❌ still hardcoding keys somewhere "as fallback"
# os.environ["AWS_ACCESS_KEY_ID"] = "AKIA..."
runner.key (0400, root:agent), prefer TPM/HSM-backed keys when the hardware allows, and never check certs into the agent monorepo.
Session tags, SCPs, and least privilege
Map certificate attributes into session tags so IAM condition keys work the same as in-cloud:tenant_id/runner_idfrom SAN URI or custom OID- Deny
iam:*,rolesanywhere:DeleteTrustAnchorvia permission boundary + SCPs - Authorize tool actions with Verified Permissions using
runner_id
json
Prefer PrivateLink / Direct Connect paths to S3, STS, Bedrock (PrivateLink Bedrock) so on-prem runners do not need broad internet for AWS APIs. Package installs still go through CodeArtifact.
{
"Effect": "Allow",
"Action": ["s3:PutObject", "s3:GetObject"],
"Resource": "arn:aws:s3:::agent-artifacts-prod/tenants/${aws:PrincipalTag/tenant_id}/*"
}
Rotation and compromise response
| Event | Action |
|---|---|
| Scheduled cert expiry | Issue new cert; leave old valid overlap window |
| Runner stolen | Revoke cert at CA; Roles Anywhere sessions die at expiry — keep duration ≤ 1h |
| Suspicious API use | GuardDuty + disable profile / role; kill tool via AppConfig |
| CA compromise | Rotate trust anchor; treat all issued certs as burned |
bash
❌ Multi-day session durations recreate the static-key problem with extra steps.
# ✅ keep sessions short — prefer 15–60 minutes for agent runners
aws rolesanywhere update-profile \
--profile-id PROFILE_ID \
--duration-seconds 900
Production checklist
- [ ] Trust anchor owned by security; agent teams only request certs via ticketed CA flow
- [ ] Profiles map to least-privilege runner roles — no AdminAccess
- [ ] credential_process only; no AKIA leftovers in images (scan with CI)
- [ ] Session tags → tenant-scoped IAM; SCP deny privilege escalation
- [ ] Cert private keys HSM/TPM or locked disk; 0400 perms
- [ ] Duration ≤ 1h; CRL / CA revocation tested quarterly
- [ ] GuardDuty + CloudTrail Lake monitoring on the runner role
- [ ] Prefer migrating workloads to CodeBuild / Fargate Spot when hardware allows — Roles Anywhere is bridge, not forever for everything
Migration path off static keys
- Inventory every
AKIAin runner images / Ansible vaults / Jenkins creds - Stand up trust anchor + one pilot profile for a non-prod runner
- Switch that runner to
credential_process; confirm S3/CodeArtifact/Bedrock calls - Revoke the old access key; watch GuardDuty for residual use
- Roll fleet-wide; SCP-deny
iam:CreateAccessKeyfor agent human users in that account
AKIA use as an incident — Roles Anywhere only wins if static keys are actually deleted, not left “for break-glass” on the same hosts.
FAQ
Q: Roles Anywhere vs OIDC for GitHub-hosted agents?A: Use OIDC for GitHub Actions / cloud CI. Roles Anywhere when the runner is your machine with a real cert PKI. Q: Can Lambda use Roles Anywhere?
A: Do not — Lambda should use its execution role. Roles Anywhere is for outside AWS identity. Q: What if we already use HashiCorp Vault for AWS creds?
A: Vault can remain the broker; Roles Anywhere is the AWS-native cert→STS path. Pick one source of truth for temporary creds to avoid dual-expiry bugs. IAM Roles Anywhere lets on-prem coding-agent runners speak AWS with short-lived, cert-backed sessions instead of immortal access keys. Stand up the trust anchor, wire credential_process, keep sessions short — and migrate to native AWS compute roles wherever the hardware allows.
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- Python String Methods: Every str Method With Real Production Examples
- API Debugging in 30 Seconds: The Fastest Ways to Fix API Issues
Newly added
- n8n vs AI Agent: I Gave Both the Same Job (Local, Ollama)
- Local AI Video Generator: Build Your Own App (Free, Python, No GPU)
- AI Wrote Raw Binary Instead of Code. Here’s What Happened (WebAssembly)
- I Built a Browser That Only Runs Encrypted Binary (Ep 1)
- RAG From Scratch: Chunking, Embeddings, Reranking and Evaluation (Real Numbers)
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.