Day 14: Guardrails Before Clever Prompts

Prompt cleverness is not a control boundary. Day 14 puts guardrails first: Bedrock Guardrails (or equivalent policy filters), tool allowlists, output filters, and automated injection tests against the hostile text you already store — tickets, emails, README jokes.

⚡ TL;DR: Policy filters on input and output. Allowlist tools per role. Test injections in CI. Treat ticket bodies as hostile. Guardrails complement prompt contracts (Day 7); they do not replace server-side authz.

Defense in depth

1. Authn/Authz — who may call which tools (IAM-ish).
2. Input guardrails — prompt attack, PII, topic blocks.
3. Prompt contracts — layered roles.
4. Tool validators — schema + semantic gates.
5. Output guardrails — secrets, banned topics.
6. Human gates — for prod mutations (Day 17).

# ✅ Fail closed when guardrail intervenes
def moderated_generate(req):
    pre = bedrock.apply_guardrail(req.text, source="INPUT")
    if pre.intervened:
        return refuse(pre.reasons)
    out = generate(req)
    post = bedrock.apply_guardrail(out, source="OUTPUT")
    if post.intervened:
        return refuse(post.reasons)
    return out

❌ “We told the model to be safe” as the only line of defense in a SOC review.

Allowlists beat capability soup

Expose apply_patch, read_file, run_tests — not bash(cmd: string). For AWS changes, prefer typed intents (add_alarm) over raw CloudControl with admin credentials.

Map roles → tool sets: intern bot ≠ platform admin bot.

Injection tests on real corpora

Pull anonymized ticket text into a corpus. Append classic jailbreaks and “ignore previous instructions.” Assert refuse. Run in CI on every policy change. Track false positives — noisy guardrails train humans to bypass them (Day 15 cost is not only tokens; it is trust).

Closing checklist

• [ ] Input and output guardrails enabled in non-dev stages
• [ ] Tool allowlists per role
• [ ] Injection corpus in CI
• [ ] Metrics for intervene rate and false positives
• [ ] No raw shell tool in default coding agents
• [ ] Document residual risk honestly

Worked example: ticket text as weapon

ServiceNow description includes a fake system prompt. With guardrails + delimited user content + refuse eval, the chat returns a safe clarification. Without them, it may echo secrets from earlier context. Add this case before the marketing demo.

Failure modes to watch

• Guardrails only in prod (dev becomes the leak environment).
• Blocking legitimate IaC keywords without an allow path.
• Logging full blocked prompts including secrets (Day 18).
• Alert fatigue from un-tuned false positives.

Field notes from production

Tune guardrails with a precision/recall mindset. Log intervened categories. If ‘prompt attack’ false positives block Terraform deny statements, add allowlisted patterns carefully — document each exception. Pair with output secret scanning before messages leave the VPC.

Implementation sketch

# Implementation sketch: allowlist check
ALLOWED = {"reader": {"read_file"}, "dev": {"read_file","apply_patch","run_tests"}}
def authorize(role, tool):
    if tool not in ALLOWED.get(role, ()): raise AuthzDenied()

Operator addendum

Version guardrail configs in git. Console-only clicks will diverge across accounts. Diff configs in CI like Terraform plans.

False positive management

Create a weekly report of top blocked prompts (redacted). Classify: true positive, false positive, unclear. Feed false positives into allowlist PRs with expiry dates. Guardrails that nobody trusts will be disabled in an incident — keep them trustworthy instead. Align categories with security’s language (prompt injection, PII, malicious URL) so reports map to existing risk registers.

Extended discussion

Return to the core angle for Day 14: Bedrock Guardrails, allowlists, and injection tests on ticket text. That sentence is the acceptance lens for every design review this week. If a proposed change does not make this angle easier to measure or enforce, it is a distraction.

Write down three metrics you will look at after shipping Day 14 ideas, schedule a 45-minute readout, and archive the notes next to the eval artifacts. Architecture without a readout becomes slideshow archaeology.

Pair this day with the adjacent lessons in the series navigation below. Forward links exist so you can keep momentum; backward links exist so you can repair foundations when a later lab fails for boring earlier reasons.

Practically, allocate half a day to implement the smallest vertical slice, half a day to wire measurement, and refuse to polish UI until both are done. This ordering is how bootcamp projects stay honest under time pressure.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 14 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 14 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 14 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 14 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 14 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 14 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Series navigation

← Day 13 · Day 15 →

Last updated September 11, 2026