AWS Systems Manager Parameter Store: Hierarchical Runtime Config for Coding Agents

19 views
Every coding agent has knobs: which model to call, how many tool hops it may take, which repositories it may touch, how long a test run may take. When those values live as constants in source or as Lambda environment variables, every tweak is a redeploy, every environment drifts a little, and nobody can say what production was running last Tuesday. AWS Systems Manager Parameter Store gives you a hierarchical, versioned key-value store with IAM access control and optional KMS encryption. This guide shows how to design a path hierarchy for agents, scope access per role and tenant (including one IAM trap that catches many teams), read configuration efficiently without hitting throughput limits, and promote or roll back changes with labels. It merges our two earlier Parameter Store posts into one.
TL;DR
  • Use paths like /<env>/coding-agent/<role>/<key>. Hierarchies can be up to 15 levels deep.
  • Store knobs as String, lightly sensitive values as SecureString, and rotating credentials in Secrets Manager.
  • Granting GetParametersByPath on a parent path exposes everything under it, even sub-paths you explicitly deny. Grant only the leaf paths a role needs.
  • Default throughput is 40 TPS shared by GetParameter, GetParameters and GetParametersByPath. Cache reads.
  • Standard tier is free with 4 KB values; the advanced tier adds 8 KB values and parameter policies, for a charge. Throughput is a separate setting.

What belongs where

Parameter Store sits next to several similar services. Picking the right one for each value avoids most configuration problems.
Kind of valueStoreWhy
Model ID, max tool hops, timeouts, repo allowlistsParameter Store StringVersioned, hierarchical, cheap to read
Webhook signing salt, internal endpoint namesParameter Store SecureStringEncrypted with KMS, IAM-scoped
GitHub tokens, database passwords, API keysSecrets ManagerBuilt-in rotation
Kill switches, percentage rolloutsAWS AppConfigValidators, deployment strategies, rollback
Prompt textBedrock Prompt ManagementStore only the version pin in Parameter Store
You don’t have to copy secrets into Parameter Store to read them through one API. Parameter Store can reference Secrets Manager secrets through the reserved path /aws/reference/secretsmanager/<secret-name> with WithDecryption. The caller still needs permission on the secret itself, and these references don’t use Parameter Store versioning.

Design the hierarchy first

Paths are the unit of access control, so design them around who reads what. A layout that works for most agent fleets:
  • /prod/coding-agent/shared/: values every role uses, such as the Bedrock Region and audit bucket.
  • /prod/coding-agent/patcher/ and /prod/coding-agent/reviewer/: per-role knobs.
  • /prod/coding-agent/tenants/acme/: per-tenant overrides, only if you need them.
  • The same tree under /staging/, so nothing in staging can ever overwrite production.
The agent loads shared first, then its role path, then its tenant path, and later values win. Keep the structure flat below the role level; a key like tools/run_tests/timeout_ms is as deep as you’ll need.
Figure 1. Configuration changes are reviewed, versioned and announced; agents read only their own paths and cache the merged result. Diagram by CheatCoders.
Figure 1. Configuration changes are reviewed, versioned and announced; agents read only their own paths and cache the merged result. Diagram by CheatCoders.

Step 1: Create parameters

bash
# Plain knobs (standard tier, no extra charge)
aws ssm put-parameter \
  --name /prod/coding-agent/patcher/model_id \
  --type String \
  --value "MODEL_OR_INFERENCE_PROFILE_ID" \
  --tags Key=workload,Value=coding-agent Key=env,Value=prod

aws ssm put-parameter \
  --name /prod/coding-agent/patcher/max_tool_hops \
  --type String \
  --value "8"

# Lightly sensitive value, encrypted with a customer managed key
aws ssm put-parameter \
  --name /prod/coding-agent/patcher/webhook_salt \
  --type SecureString \
  --key-id alias/coding-agent-config \
  --value "REPLACE_ME"

# Pointer to a secret that Secrets Manager rotates
aws ssm put-parameter \
  --name /prod/coding-agent/patcher/github_secret_id \
  --type String \
  --value "agents/github"
To change a value later, run put-parameter again with --overwrite. Each change creates a new version, and Parameter Store keeps up to 100 versions per parameter. In production, apply these through CloudFormation, CDK or Terraform so every change goes through a pull request.

Step 2: Scope access per role (and avoid the parent-path trap)

The GetParametersByPath API reference contains an important warning: if a principal can access a path, it can access all levels below it. Even if IAM explicitly denies /a/b, a principal allowed on /a can call GetParametersByPath recursively on /a and see /a/b. So a “deny the other tenants” statement doesn’t protect anything if the role is also allowed on a parent path. Grant only the exact paths a role needs:
json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadPatcherConfigOnly",
      "Effect": "Allow",
      "Action": ["ssm:GetParameter", "ssm:GetParameters", "ssm:GetParametersByPath"],
      "Resource": [
        "arn:aws:ssm:ap-south-1:111122223333:parameter/prod/coding-agent/shared",
        "arn:aws:ssm:ap-south-1:111122223333:parameter/prod/coding-agent/shared/*",
        "arn:aws:ssm:ap-south-1:111122223333:parameter/prod/coding-agent/patcher",
        "arn:aws:ssm:ap-south-1:111122223333:parameter/prod/coding-agent/patcher/*"
      ]
    },
    {
      "Sid": "DecryptOnlyTheseParameters",
      "Effect": "Allow",
      "Action": "kms:Decrypt",
      "Resource": "arn:aws:kms:ap-south-1:111122223333:key/KEY_ID",
      "Condition": {
        "StringLike": {
          "kms:EncryptionContext:PARAMETER_ARN": [
            "arn:aws:ssm:ap-south-1:111122223333:parameter/prod/coding-agent/shared/*",
            "arn:aws:ssm:ap-south-1:111122223333:parameter/prod/coding-agent/patcher/*"
          ]
        }
      }
    }
  ]
}
The second statement uses the encryption context that Parameter Store sends to KMS (PARAMETER_ARN), so the role can decrypt SecureString values only under its own paths, even though the key may be shared. If you use the AWS managed key aws/ssm instead of your own key, you can’t edit its key policy, which is one reason to prefer a customer managed key for agent config.

Step 3: Load, merge and cache configuration

GetParametersByPath returns at most 10 parameters per call, so always paginate. Every page counts against the throughput limit, which by default is 40 TPS shared across GetParameter, GetParameters and GetParametersByPath for the whole account and Region. A fleet of 50 containers that each reload 30 parameters every few seconds will be throttled. Load once, cache with a TTL, and fail closed when a required key is missing:
python
import random
import time

import boto3

ssm = boto3.client("ssm")
REQUIRED = {"model_id", "max_tool_hops", "prompt_version", "repos_allowlist"}


def params_under(path: str) -> dict[str, str]:
    out = {}
    paginator = ssm.get_paginator("get_parameters_by_path")
    for page in paginator.paginate(Path=path, Recursive=True, WithDecryption=True):
        for p in page["Parameters"]:
            out[p["Name"][len(path):].lstrip("/")] = p["Value"]
    return out


class AgentConfig:
    def __init__(self, env: str, role: str, tenant: str | None = None, ttl: int = 300):
        base = f"/{env}/coding-agent"
        self.paths = [f"{base}/shared", f"{base}/{role}"]
        if tenant:
            self.paths.append(f"{base}/tenants/{tenant}")
        self.ttl = ttl
        self.values: dict[str, str] = {}
        self.expires = 0.0

    def refresh(self) -> None:
        merged: dict[str, str] = {}
        for path in self.paths:  # later paths override earlier ones
            merged.update(params_under(path))
        missing = REQUIRED - merged.keys()
        if missing:
            raise RuntimeError(f"missing required config: {sorted(missing)}")
        self.values = merged
        # Jitter spreads refreshes so a fleet doesn't reload in lockstep.
        self.expires = time.time() + self.ttl * random.uniform(0.8, 1.0)

    def get(self, key: str) -> str:
        if time.time() >= self.expires:
            try:
                self.refresh()
            except Exception:
                if not self.values:
                    raise  # no config at all: refuse to start
                # keep serving the last good config; log and alert here
        return self.values[key]


cfg = AgentConfig("prod", "patcher")
max_hops = int(cfg.get("max_tool_hops"))
allowed_repos = set(cfg.get("repos_allowlist").split(","))
If you outgrow the default, you can turn on higher throughput for the account and Region (up to 10,000 TPS for GetParameter, 1,000 for GetParameters and 100 for GetParametersByPath). That setting has a charge and is unrelated to the advanced tier; caching is usually the better first step.

Step 4: Lambda-based agents can use the extension cache

For agents that run in Lambda, the AWS Parameters and Secrets Lambda Extension keeps a local cache inside the execution environment. Your code calls localhost:2773 instead of the SSM API. The cache TTL (SSM_PARAMETER_STORE_TTL) can be 0 to 300 seconds and defaults to 300, and the extension doesn’t refresh a value before its TTL expires, so pick a shorter TTL for knobs you change often.
python
import json
import os
import urllib.parse
import urllib.request


def get_param(name: str) -> str:
    url = "http://localhost:2773/systemsmanager/parameters/get?name=" + urllib.parse.quote(name, safe="")
    req = urllib.request.Request(url)
    req.add_header("X-Aws-Parameters-Secrets-Token", os.environ["AWS_SESSION_TOKEN"])
    with urllib.request.urlopen(req, timeout=2) as resp:
        return json.loads(resp.read())["Parameter"]["Value"]


def handler(event, context):
    model_id = get_param("/prod/coding-agent/reviewer/model_id")
    return {"model": model_id}

Step 5: Promote and roll back with labels

A label is a movable alias for a parameter version. Each version can carry up to 10 labels, and a label can point to only one version of a parameter at a time. Agents read name:label, so promotion and rollback are just moving the label:
bash
# Write the new value (creates, say, version 7), then point "live" at it
aws ssm put-parameter --name /prod/coding-agent/patcher/max_tool_hops \
  --type String --value "10" --overwrite
aws ssm label-parameter-version --name /prod/coding-agent/patcher/max_tool_hops \
  --parameter-version 7 --labels live

# Agents read the labelled version
aws ssm get-parameter --name /prod/coding-agent/patcher/max_tool_hops:live

# Roll back: move the label to the previous version
aws ssm label-parameter-version --name /prod/coding-agent/patcher/max_tool_hops \
  --parameter-version 6 --labels live
Plain path reads return the latest version of each parameter. To read labelled versions under a path, add the filter --parameter-filters Key=Label,Values=live,Option=Equals to get-parameters-by-path. Parameters without that label are left out of the result, so either label every key you load this way or let the loader’s required-key check catch the gap. Finally, alert on changes. Parameter Store sends Parameter Store Change events to EventBridge for create, update, delete and label operations:
json
{
  "source": ["aws.ssm"],
  "detail-type": ["Parameter Store Change"],
  "detail": {
    "name": [{"prefix": "/prod/coding-agent/"}],
    "operation": ["Create", "Update", "Delete", "LabelParameterVersion"]
  }
}

Worked example: raising tool hops for one tenant

  1. Tenant acme has a large monorepo, and its patch tasks keep stopping at 8 tool hops.
  2. An engineer opens a pull request that adds /prod/coding-agent/tenants/acme/max_tool_hops = 12. Reviewers see one line of config, not a code change.
  3. The pipeline applies it. EventBridge posts the change to the team channel, and CloudTrail records who made the PutParameter call.
  4. Within one cache TTL, acme’s agents load shared, then patcher, then tenants/acme, and the tenant value wins. Other tenants keep 8.
  5. Your audit records include the effective config values, so the next day’s Athena query compares success rates for acme before and after the change. If the results get worse, the PR is reverted and the old value returns.

Troubleshooting

SymptomLikely causeFix
ThrottlingException on startup bursts40 TPS default shared by all Get* callsCache with jittered TTL; enable higher throughput only if needed
A role can read another tenant’s configIt was granted a parent pathGrant leaf paths only; parent grants expose all levels
AccessDeniedException from KMS on SecureStringMissing kms:Decrypt or encryption context mismatchCheck the key policy and the PARAMETER_ARN condition
Only 10 parameters returnedNo paginationUse the paginator or loop on NextToken
Value too largeOver 4 KB on standard tierSplit the value, move to advanced (8 KB), or store a document in S3 or AppConfig
New version created, agents still use the old valueCache TTL, or agents read a label you didn’t moveWait one TTL or move the label
Can’t move a parameter back to standardAdvanced parameters can’t be downgradedCreate advanced parameters only when you need their features

Production checklist

  • ☐ Hierarchy documented: /<env>/coding-agent/{shared,<role>,tenants/<id>}
  • ☐ IAM grants leaf paths only; no role can read a parent path
  • ☐ SecureString uses a customer managed key with a PARAMETER_ARN condition
  • ☐ Secrets that rotate live in Secrets Manager; Parameter Store holds only their names
  • ☐ Client cache with jittered TTL; startup fails closed on missing keys
  • ☐ Changes applied through IaC, alerted through EventBridge, audited in CloudTrail
  • ☐ Labels for promotion and rollback where a fast revert matters
  • ☐ Kill switches in AppConfig, durable knobs in Parameter Store

FAQ

Standard or advanced tier?

Start with standard: up to 10,000 parameters per account and Region, 4 KB values, no extra charge. Use advanced (100,000 parameters, 8 KB values, parameter policies, cross-account sharing) when you need those features. Advanced parameters are charged and can’t be turned back into standard ones.

Can Parameter Store replace AppConfig?

Not for flags that must change safely. AppConfig adds validators, gradual deployment strategies and automatic rollback. Many teams use both: Parameter Store for durable knobs, AppConfig for kill switches.

How do Batch or Fargate agents authenticate?

Through the task or job role with the path-scoped policy above. No long-lived keys.

Should the model see the configuration?

Generally no. Pass configuration to your tool layer and orchestration code. Putting limits in the prompt invites the model to argue with them.

Official documentation

Related on CheatCoders: Bedrock Prompt Management · Athena for agent audit logs Last updated on October 6, 2026

Watch: 100 Days of AI on YouTube

Short videos from CheatCoders, one AI engineering topic per day.

Open the playlist on YouTube · CheatCoders on YouTube

Deep-dive PDF

Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.

Already subscribed? or open the subscribe page.


Discover more from CheatCoders

Subscribe to get the latest posts sent to your email.

Comments

No comments yet. Why don’t you start the discussion?

Leave a comment

No account needed. Name and email are optional.