A coding agent’s tools need credentials: a token to open pull requests, a registry token to publish packages, a database password for a migration check. The usual shortcut is an environment variable set months ago. From there the value tends to spread. It’s printed by a verbose curl into a tool result, which the model reads and may repeat; it’s captured in a trace; it’s copied into a support ticket. Rotation doesn’t stop a leak, but it limits how long a leaked value works, and fetching at call time keeps the value out of places it doesn’t need to be.
⚡ TL;DR
- Store tool credentials in Secrets Manager and fetch them when the tool runs. The model only ever sees the tool’s name and result, never the value.
- Pass the secret to the tool’s child process only, and redact it from the output before the output reaches the model.
- Rotate with a Lambda function that implements Secrets Manager’s four steps: createSecret, setSecret, testSecret, finishSecret. Secrets Manager can rotate as often as every four hours.
- For GitHub, prefer a GitHub App: store the App’s private key and mint installation tokens that expire after one hour, scoped to one repository and the permissions the tool needs.
- For RDS, let RDS manage the master password in Secrets Manager; it rotates every seven days by default.
Who this is for: engineers running agent tools on AWS (Lambda, ECS or EC2) that call third-party or internal APIs. The rotation function was tested against moto’s Secrets Manager mock, the GitHub token code against a locally generated RSA key, and the tool runner with a real subprocess.
Where tool secrets leak, and what limits each leak
| Pattern | How long a leaked value works | Can the model see it? |
|---|---|---|
| Token in the system prompt | Until someone notices and replaces it | Yes, always |
| Plaintext Lambda environment variable | Until redeploy with a new value | If any tool prints its environment |
| Secrets Manager, fetched at call time, no rotation | Until manual rotation | Only if a tool echoes it (redact output) |
| Secrets Manager with scheduled rotation | Up to the rotation interval (plus overlap) | Only if echoed (redact output) |
| GitHub App installation token | At most one hour | Only if echoed (redact output) |

Step 1: Fetch at call time, and keep the value out of model context
Store each credential as a secret, named by environment, tool and (if needed) tenant, for example agents/prod/tools/registry/publish-token. The tool runner fetches it when the tool is called and passes it only to the process that needs it:
"""Tool runner: the secret goes into the child process only, and never back into model context."""
import subprocess
def run_with_secret(cmd: list[str], secret: str, env_name: str, cwd: str = ".") -> dict:
proc = subprocess.run(cmd, cwd=cwd, env={"PATH": "/usr/bin:/bin", env_name: secret},
capture_output=True, text=True, timeout=120)
out = (proc.stdout + proc.stderr).replace(secret, f"[{env_name} redacted]") # e.g. verbose curl output
return {"exit_code": proc.returncode, "output": out[-4000:]} # only this reaches the model
if __name__ == "__main__":
leaky = ["sh", "-c", 'echo "Authorization: Bearer $NPM_TOKEN"; exit 3']
print(run_with_secret(leaky, "npm_EXAMPLEexampleEXAMPLE", "NPM_TOKEN")){'exit_code': 3, 'output': 'Authorization: Bearer [NPM_TOKEN redacted]\n'}The child process gets a minimal environment containing just the one secret, not the runner’s whole environment. Whatever the command prints, the exact secret value is replaced before the output becomes a tool result. That covers the most common leak: a verbose HTTP client printing its Authorization header. It won’t catch a transformed value (for example base64), so keep commands quiet as well. For broader detection of secrets in prompts and outputs, see Secret-Aware Context Filters.
On Lambda, the AWS Parameters and Secrets Lambda Extension can retrieve and cache secrets without SDK calls in your code. If you cache (with the extension or in your own code), keep the cache lifetime shorter than the period during which a rotated-out token still works, which Step 2 sets up.
Limit which secrets each role can read
For multi-tenant agents, tag secrets with the tenant and use attribute-based access control. The Secrets Manager documentation shows a condition that compares aws:ResourceTag/<key> on the secret with aws:PrincipalTag/<key> on the caller, so a role tagged for tenant A can’t read tenant B’s secret even if a tool asks for the wrong name.
Step 2: Rotate with the four-step Lambda function
Secrets Manager rotation calls your Lambda function four times per rotation, with a Step of createSecret, setSecret, testSecret and finishSecret, and a ClientRequestToken that is the new version’s ID. The new value is stored with the staging label AWSPENDING, tested, and then AWSCURRENT is moved to it. Each step has to be safe to run twice, because Secrets Manager can retry.
"""Secrets Manager rotation function for an API token that the issuing service mints and revokes."""
import json
import boto3
sm = boto3.client("secretsmanager")
class Issuer:
"""Adapter for the service that issues tokens (a package registry, an internal API...)."""
def mint(self) -> dict: raise NotImplementedError
def works(self, token: str) -> bool: raise NotImplementedError
def revoke(self, token_id: str) -> None: raise NotImplementedError
def handler(event, context, issuer: Issuer):
arn, token, step = event["SecretId"], event["ClientRequestToken"], event["Step"]
meta = sm.describe_secret(SecretId=arn)
if not meta.get("RotationEnabled"):
raise ValueError(f"rotation is not enabled for {arn}")
stages = meta["VersionIdsToStages"].get(token)
if not stages:
raise ValueError(f"version {token} has no stage")
if "AWSCURRENT" in stages:
return # already finished: retries are safe
if "AWSPENDING" not in stages:
raise ValueError(f"version {token} is not AWSPENDING")
{"createSecret": create, "setSecret": lambda *a: None, # the issuer already activated it
"testSecret": test, "finishSecret": finish}[step](arn, token, issuer, meta)
def create(arn, token, issuer, meta):
try:
sm.get_secret_value(SecretId=arn, VersionId=token, VersionStage="AWSPENDING")
return # created on an earlier attempt
except sm.exceptions.ResourceNotFoundException:
pass
new = issuer.mint() # {"token_id": ..., "token": ...}
sm.put_secret_value(SecretId=arn, ClientRequestToken=token,
SecretString=json.dumps(new), VersionStages=["AWSPENDING"])
def test(arn, token, issuer, meta):
pending = json.loads(sm.get_secret_value(SecretId=arn, VersionId=token,
VersionStage="AWSPENDING")["SecretString"])
if not issuer.works(pending["token"]):
raise RuntimeError("new token failed its test call; AWSCURRENT is unchanged")
def finish(arn, token, issuer, meta):
current = next(v for v, s in meta["VersionIdsToStages"].items() if "AWSCURRENT" in s)
previous = next((v for v, s in meta["VersionIdsToStages"].items() if "AWSPREVIOUS" in s), None)
sm.update_secret_version_stage(SecretId=arn, VersionStage="AWSCURRENT",
MoveToVersionId=token, RemoveFromVersionId=current)
# Revoke the token from two rotations ago. The one just demoted stays valid for one more
# cycle, so callers holding a cached copy keep working until their cache expires.
if previous and previous != current:
old = json.loads(sm.get_secret_value(SecretId=arn, VersionId=previous)["SecretString"])
issuer.revoke(old["token_id"])Two design choices matter for agent tools:
- Test before switching. If the new token fails its test call, the step raises, and
AWSCURRENTstill points to the working token. A broken rotation then shows up as a failed rotation, not as tools failing at 3 a.m. - Overlap by one cycle. When
AWSCURRENTmoves, the old version getsAWSPREVIOUS. The function revokes the token from two rotations ago, not the one it just replaced, so a tool that fetched and cached the previous value a minute ago keeps working.
The test runs three rotations against moto’s Secrets Manager mock, with a fake registry that tracks which tokens are live. Each rotation also calls finishSecret a second time, to check retries are harmless:
import json, uuid
import boto3
from moto import mock_aws
with mock_aws():
import rotate_token
rotate_token.sm = boto3.client("secretsmanager", region_name="us-east-1")
sm = rotate_token.sm
class FakeRegistry(rotate_token.Issuer):
def __init__(self): self.live, self.n = set(), 0
def mint(self):
self.n += 1; tid = f"tok-{self.n}"; self.live.add(tid)
return {"token_id": tid, "token": f"secret-value-{self.n}"}
def works(self, token): return f"tok-{token.rsplit('-', 1)[1]}" in self.live
def revoke(self, token_id): self.live.discard(token_id)
reg = FakeRegistry()
first = reg.mint()
arn = sm.create_secret(Name="agents/prod/tools/registry/publish-token", SecretString=json.dumps(first))["ARN"]
real_describe = sm.describe_secret # test shim: moto needs a real Lambda to enable rotation
sm.describe_secret = lambda **kw: {**real_describe(**kw), "RotationEnabled": True}
def rotate():
token = str(uuid.uuid4())
# Stand-in for the service: RotateSecret registers an AWSPENDING version, then calls the steps.
rotate_token.create(arn, token, reg, None)
for step in ("createSecret", "setSecret", "testSecret", "finishSecret"):
rotate_token.handler({"SecretId": arn, "ClientRequestToken": token, "Step": step}, None, reg)
rotate_token.handler({"SecretId": arn, "ClientRequestToken": token, "Step": "finishSecret"}, None, reg) # retry
cur = json.loads(sm.get_secret_value(SecretId=arn, VersionStage="AWSCURRENT")["SecretString"])
print(f"current={cur['token_id']} live={sorted(reg.live)}")
rotate(); rotate(); rotate()current=tok-2 live=['tok-1', 'tok-2']
current=tok-3 live=['tok-2', 'tok-3']
current=tok-4 live=['tok-3', 'tok-4']After each rotation exactly two tokens are live, the current one and the previous one, and the token from two rotations ago has been revoked. The test uses a small shim because moto requires a real Lambda function before it marks rotation as enabled; in AWS you attach the function with RotateSecret and a schedule. Secrets Manager can rotate as often as every four hours, within a rotation window of at least one hour.
Step 3: Use GitHub App tokens instead of personal access tokens
For GitHub there’s a better option than rotating a long-lived token. A GitHub App’s installation access tokens expire after one hour, and when you request one you can limit it to specific repositories (the repositories body parameter) and to a subset of the App’s permissions (the permissions parameter). Secrets Manager then stores only the App’s private key and client ID.
To request a token, sign a JSON Web Token with the App’s private key. GitHub requires RS256, recommends setting iat 60 seconds in the past to allow for clock drift, limits exp to 10 minutes ahead, and recommends the client ID for iss. Then POST to /app/installations/{installation_id}/access_tokens:
"""Mint a short-lived GitHub App installation token per tool call. Only the App key lives in Secrets Manager."""
import calendar
import json
import time
import urllib.request
import boto3
import jwt # PyJWT, with the cryptography package for RS256
sm = boto3.client("secretsmanager")
SECRET_ID = "agents/prod/tools/github/app" # {"client_id": "...", "private_key": "-----BEGIN ..."}
_cache: dict[tuple, tuple[str, float]] = {}
def app_jwt(client_id: str, private_key: str, now: float | None = None) -> str:
now = int(now or time.time())
# GitHub: RS256, iat 60 s in the past for clock drift, exp no more than 10 minutes ahead.
return jwt.encode({"iat": now - 60, "exp": now + 540, "iss": client_id}, private_key, algorithm="RS256")
def installation_token(installation_id: int, repo: str, permissions: dict, post=None) -> str:
key = (installation_id, repo, json.dumps(permissions, sort_keys=True))
if key in _cache and _cache[key][1] - time.time() > 300: # reuse until 5 minutes before expiry
return _cache[key][0]
app = json.loads(sm.get_secret_value(SecretId=SECRET_ID)["SecretString"])
body = {"repositories": [repo], "permissions": permissions} # narrower than the App's own grants
resp = (post or _post)(f"https://api.github.com/app/installations/{installation_id}/access_tokens",
app_jwt(app["client_id"], app["private_key"]), body)
expires = calendar.timegm(time.strptime(resp["expires_at"], "%Y-%m-%dT%H:%M:%SZ")) # UTC
_cache[key] = (resp["token"], expires)
return resp["token"]
def _post(url: str, bearer: str, body: dict) -> dict:
req = urllib.request.Request(url, data=json.dumps(body).encode(), method="POST", headers={
"Accept": "application/vnd.github+json", "Authorization": f"Bearer {bearer}"})
with urllib.request.urlopen(req, timeout=10) as r:
return json.load(r)import json, time
import boto3, jwt
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from moto import mock_aws
key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
pem = key.private_bytes(serialization.Encoding.PEM, serialization.PrivateFormat.PKCS8,
serialization.NoEncryption()).decode()
with mock_aws():
import github_token as gt
gt.sm = boto3.client("secretsmanager", region_name="us-east-1")
gt.sm.create_secret(Name=gt.SECRET_ID, SecretString=json.dumps({"client_id": "Iv23liEXAMPLE", "private_key": pem}))
calls = []
def fake_post(url, bearer, body):
claims = jwt.decode(bearer, key.public_key(), algorithms=["RS256"])
calls.append((url, body, claims["exp"] - claims["iat"]))
expires = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(time.time() + 3600))
return {"token": f"ghs_example{len(calls)}", "expires_at": expires}
perms = {"contents": "write", "pull_requests": "write"}
t1 = gt.installation_token(42, "web-app", perms, post=fake_post)
t2 = gt.installation_token(42, "web-app", perms, post=fake_post) # cached
t3 = gt.installation_token(42, "web-app", {"contents": "read"}, post=fake_post)
print(t1, t2, t3, "| API calls:", len(calls))
print("JWT lifetime (s):", calls[0][2], "| body:", calls[0][1])ghs_example1 ghs_example1 ghs_example2 | API calls: 2
JWT lifetime (s): 600 | body: {'repositories': ['web-app'], 'permissions': {'contents': 'write', 'pull_requests': 'write'}}The test signs with a freshly generated RSA key and checks the JWT verifies with the public key. Repeated calls with the same repository and permissions reuse the cached token; a call that needs different permissions (read-only here) mints a separate, narrower token. A tool that only opens pull requests on one repository never holds a token that can push to every repository the App is installed on.
Earlier versions of this post suggested automating the creation of fine-grained personal access tokens. GitHub doesn’t document an API for creating personal access tokens, so that approach isn’t something you can build on; use a GitHub App.
Step 4: Databases
For Amazon RDS, let RDS manage the master user password in Secrets Manager. RDS creates the secret, rotates it every seven days by default (you can change the schedule), and applications read the current value from the secret. Agent tools that need database access should use a separate, least-privilege database user rather than the master user; rotate that user’s password with a rotation function in the same four-step pattern.
Worked example: a token printed into a tool result
- A publish tool fails, and the agent retries it with verbose output. The registry client prints its request headers, including the token.
- The runner’s redaction replaces the token in the output before the tool result goes to the model, so the transcript and the trace contain
[NPM_TOKEN redacted]. - Suppose redaction had been missing. The token would now be in the conversation history and traces. Because of the one-cycle overlap, a single rotation leaves the leaked token valid as
AWSPREVIOUS; it’s only revoked when it becomes two versions old. With weekly rotation that could be up to 14 days, so after a known leak revoke the token directly with the issuer (or rotate twice) instead of waiting. - The follow-up fix: run the publish command without verbose flags, and add a test that a tool echoing its secret gets it redacted.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
Rotation fails at testSecret | New token not active yet, or wrong scope | Check the issuer; AWSCURRENT is unchanged, so tools keep working |
| Tools fail right after rotation | Old token revoked while callers still cache it | Revoke two versions back (as above); shorten caches |
| Rotation retries create extra tokens | createSecret not idempotent | Check for an existing AWSPENDING value for the token before minting |
| GitHub returns 401 for the JWT | Clock drift or exp more than 10 minutes ahead | Set iat 60 seconds back; sync the clock |
| Wrong tenant’s secret read | Role can read every secret by name | Tag-based conditions on GetSecretValue |
Checklist
- ☐ No credential values in prompts, tool schemas or plaintext environment variables
- ☐ Secrets fetched at call time; passed to the child process only
- ☐ Tool output redacted before it reaches the model
- ☐ Rotation function implements all four steps, idempotently, and tests before switching
- ☐ Previous token stays valid for one cycle; caches are shorter than that
- ☐ GitHub access through an App with per-call, per-repository tokens
- ☐ Tag-based access so roles only read their tenant’s secrets
FAQ
Isn’t Parameter Store enough?
Parameter Store SecureString parameters are fine for configuration and values you rotate yourself. Secrets Manager adds managed rotation with staging labels, which is what this pattern relies on.
How often should I rotate?
As often as the issuing service and your callers can handle. Secrets Manager supports every four hours at most; weekly is a common start for tokens. GitHub App tokens make the question moot for GitHub.
Can the agent ever be given a secret value?
There’s rarely a reason. If a task needs a credential, it belongs in a tool that uses it, not in the conversation.
Does rotation help after a leak?
It limits how long the leaked value works. After a known leak, revoke the leaked token with the issuer right away; with the overlap described above, one rotation alone leaves it valid for another cycle.
Official documentation
- Secrets Manager rotation function steps
- Rotation schedules
- Attribute-based access control for secrets
- Parameters and Secrets Lambda Extension
- RDS password management with Secrets Manager
- GitHub: generating a JWT for a GitHub App and installation access tokens
Last updated on October 10, 2026
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Python String Methods: Every str Method With Real Production Examples
- REST API Design Best Practices: The Patterns That Make APIs a Joy to Use
Newly added
- n8n vs AI Agent: I Gave Both the Same Job (Local, Ollama)
- Local AI Video Generator: Build Your Own App (Free, Python, No GPU)
- AI Wrote Raw Binary Instead of Code. Here’s What Happened (WebAssembly)
- I Built a Browser That Only Runs Encrypted Binary (Ep 1)
- RAG From Scratch: Chunking, Embeddings, Reranking and Evaluation (Real Numbers)
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.
From what I’ve seen, an SEO consultant is kind of like a doctor for your website. They diagnose problems like slow pages, missing meta tags, or weak content, then prescribe fixes based on data instead of guesswork. The good ones also keep an eye on algorithm updates so your rankings don’t randomly tank overnight, which happened to me once before I hired someone.
I guess the short version is they make sure Google and actual users can find and trust your site, through a mix of technical tweaks, content advice, and link building. It’s not an overnight fix though, mine warned me it’d take months before seeing real movement, and he was right. Has anyone had a consultant who delivered faster results, or is that just unrealistic?
Been doing my own SEO for years but finally brought in outside help when traffic plateaued. What surprised me was how much time he spent asking about my business goals before even touching keywords. Turns out half my problem wasn’t technical, it was just targeting the wrong audience the whole time.
If I could go back I’d ask way more questions before signing anything, like how they measure success and what tools they actually use. A lot of this industry runs on jargon that sounds impressive but means nothing in practice. What questions do you guys usually ask before trusting someone with your site?
hire an SEO consultant [url=https://yourhomeyourwayltd.co.uk/what-an-seo-consultant-actually-does-advice/#hire-an-SEO-consultant]https://yourhomeyourwayltd.co.uk/what-an-seo-consultant-actually-does-advice/[/url] .