PII Redaction Before Embeddings: Scrub Wikis Before They Reach Your Vector Index

Your team wants the coding assistant to answer “how do we fail over the billing export?” from the internal wiki. The ingestion job reads every page, chunks it, embeds it and writes the vectors and chunk text to the index. Among those pages: an on-call roster with phone numbers, an incident write-up quoting a customer’s email, and a runbook with an access key someone pasted “temporarily” in 2023.

Embeddings are not encryption. The chunk text is usually stored next to the vector so it can be shown and cited, and anything in it can come back in a retrieved answer to anyone who can query the index. Filtering at answer time is too late: the data is already copied into a new system with its own access rules and backups. The fix is to scrub before chunking and embedding. This guide builds that step on AWS with Amazon Comprehend and a few hard rules, and merges our earlier bootcamp lesson on PII redaction before embeddings into it.

⚡ TL;DR

• Order matters: detect → redact or quarantine → chunk → embed → index. Raw text never goes to the embedding model or the vector store.
• Quarantine pages with secrets (access keys, private keys, bearer tokens). Don’t partially index them; rotate the secret.
• Use Amazon Comprehend DetectPiiEntities (English or Spanish, up to 100 KB of UTF-8 text per call) for names, emails, phones, addresses, ID numbers.
• Replace PII with keyed placeholders (HMAC, not a plain hash) and keep the placeholder-to-value map in a separate, tightly controlled store.
• Version the redaction policy and include it in the index key, so tightening the policy re-embeds affected documents.

Who this is for: engineers building RAG over internal wikis, tickets or chat for coding assistants on AWS. Code is Python with boto3; the pipeline was tested with a stubbed Comprehend client so the offsets and replacement logic are exercised exactly as written.

What leaks through embeddings, and why answer-time filtering isn’t enough

A RAG index typically stores three things per chunk: the vector, the chunk text and metadata. The chunk text is what gets returned and pasted into the model’s prompt, so anything in it can appear in an answer. The vector is derived from the text, so it isn’t a safe container either. And the index is a new copy of the data with its own permissions, backups and retention, often more widely readable than the original wiki space.

Two different kinds of sensitive content need different handling:

Step 1: Know what Comprehend detects

Comprehend’s PII detection works on English or Spanish text. The real-time DetectPiiEntities call accepts up to 100 KB of UTF-8 text and returns each entity’s type, a confidence score and character offsets. ContainsPiiEntities returns only the labels, which is useful as a cheap classifier. For bulk work, an asynchronous job can write redacted copies of documents in S3.

Universal entity types include NAME, EMAIL, PHONE, ADDRESS, AGE, USERNAME, PASSWORD, IP_ADDRESS, URL, DATE_TIME, card and bank numbers, and AWS_ACCESS_KEY / AWS_SECRET_KEY. There are also country-specific types, for example SSN for the US, UK_NATIONAL_INSURANCE_NUMBER, CA_SOCIAL_INSURANCE_NUMBER, and IN_AADHAAR and IN_PERMANENT_ACCOUNT_NUMBER for India. Choose deliberately: redacting DATE_TIME or URL everywhere would strip the content that makes runbooks useful.

Step 2: The scrub step

The function below runs once per page before chunking. It applies the space deny list, quarantines pages matching secret patterns or Comprehend secret types, splits long pages into pieces under the 100 KB limit (keeping track of each piece’s offset), and replaces accepted entities from the end of the text backwards so earlier offsets stay valid.

"""Scrub wiki pages before they are chunked and embedded. Raw text never reaches the embedding call."""
import hashlib
import hmac
import json
import re
from dataclasses import dataclass, field

POLICY_VERSION = "pii-2026-10"
MIN_SCORE = 0.80                      # tune on your own labeled sample
REDACT_TYPES = {"NAME", "EMAIL", "PHONE", "ADDRESS", "SSN", "CREDIT_DEBIT_NUMBER", "BANK_ACCOUNT_NUMBER",
                "PASSPORT_NUMBER", "DRIVER_ID", "IP_ADDRESS", "USERNAME", "IN_AADHAAR",
                "IN_PERMANENT_ACCOUNT_NUMBER"}
QUARANTINE_TYPES = {"AWS_ACCESS_KEY", "AWS_SECRET_KEY", "PASSWORD"}   # secrets: never index, rotate
SECRET_PATTERNS = [re.compile(r"\b(?:AKIA|ASIA)[0-9A-Z]{16}\b"),
                   re.compile(r"-----BEGIN (?:[A-Z]+ )?PRIVATE KEY-----"),
                   re.compile(r"\bBearer\s+[A-Za-z0-9._~+/-]{20,}=*")]
DENY_SPACES = {"HR", "PAYROLL", "LEGAL-PRIVATE"}
MAX_BYTES = 100_000                   # DetectPiiEntities limit: 100 KB of UTF-8 text per call


@dataclass
class Result:
    status: str                       # "indexed" | "quarantined" | "denied"
    text: str = ""
    pii_map: dict = field(default_factory=dict)   # placeholder -> original; stored apart from the index
    findings: list = field(default_factory=list)


def placeholder(kind: str, value: str, key: bytes) -> str:
    # Keyed HMAC, not a plain hash: emails and phone numbers are guessable, so an unkeyed
    # hash could be reversed by hashing candidate values.
    digest = hmac.new(key, f"{kind}:{value.lower()}".encode(), hashlib.sha256).hexdigest()[:10]
    return f"[{kind}_{digest}]"


def chunks_under_limit(text: str, limit: int = MAX_BYTES):
    """Yield (char_offset, piece) with each piece under the byte limit, split on paragraph breaks."""
    start, buf = 0, ""
    for para in re.split(r"(\n\s*\n)", text):
        if len((buf + para).encode()) > limit and buf:
            yield start, buf
            start, buf = start + len(buf), ""
        buf += para
        while len(buf.encode()) > limit:          # one huge paragraph: hard split
            cut = limit // 4                       # safe char count for any UTF-8 width
            yield start, buf[:cut]
            start, buf = start + cut, buf[cut:]
    if buf:
        yield start, buf


def scrub_page(text: str, space: str, comprehend, key: bytes, lang: str = "en") -> Result:
    if space.upper() in DENY_SPACES:
        return Result("denied")
    if any(p.search(text) for p in SECRET_PATTERNS):
        return Result("quarantined", findings=["secret_pattern"])

    spans = []
    for offset, piece in chunks_under_limit(text):
        resp = comprehend.detect_pii_entities(Text=piece, LanguageCode=lang)
        for e in resp["Entities"]:
            if e["Score"] >= MIN_SCORE:
                spans.append((offset + e["BeginOffset"], offset + e["EndOffset"], e["Type"]))

    if any(kind in QUARANTINE_TYPES for _, _, kind in spans):
        return Result("quarantined", findings=sorted({k for _, _, k in spans}))

    out, pii_map = text, {}
    for begin, end, kind in sorted(spans, reverse=True):       # replace from the end: offsets stay valid
        if kind not in REDACT_TYPES:
            continue
        value = text[begin:end]
        token = placeholder(kind, value, key)
        pii_map[token] = value
        out = out[:begin] + token + out[end:]
    return Result("indexed", out, pii_map, sorted({k for _, _, k in spans}))


def index_key(doc_id: str, scrubbed: str) -> str:
    """Re-embed when the scrubbed text OR the policy changes."""
    return hashlib.sha256(f"{POLICY_VERSION}\n{doc_id}\n{scrubbed}".encode()).hexdigest()


def embed(bedrock_runtime, scrubbed: str) -> list:
    body = json.dumps({"inputText": scrubbed, "dimensions": 1024, "normalize": True})
    resp = bedrock_runtime.invoke_model(modelId="amazon.titan-embed-text-v2:0", body=body)
    return json.loads(resp["body"].read())["embedding"]

Three design choices are worth calling out:

• Keyed placeholders. The same email always becomes the same token, so the model can still reason that “the person to page” in two chunks is the same. But a plain SHA-256 of an email can be reversed by hashing candidate emails, so the token uses HMAC with a key kept in AWS Secrets Manager, not in code.
• A separate pii_map. Some workflows need the real value, for example a human looking up who to page. Store the map in its own encrypted table that the retrieval path and the model can’t read, and expose it only through an authorized lookup for people.
• A score threshold. Low-confidence spans (like “billing” mis-tagged as a name in the test below) are left alone. Pick the threshold from a labeled sample of your own pages rather than copying ours.

Comprehend returns character offsets. Python indexes strings by code point; check on a sample of your non-ASCII pages (names with accents, emoji, CJK text) that text[begin:end] matches what Comprehend found before trusting the replacement in production.

Step 3: Test it before it touches real pages

botocore’s Stubber lets you test the pipeline without calling AWS, using the exact response shape Comprehend documents:

import boto3
from botocore.stub import Stubber

from scrub import index_key, scrub_page

PAGE = ("Escalations: page Priya Raman (priya.raman@example.com, +1 206 555 0100) "
        "if the billing export fails.\n\nRunbook: rerun the export job from the console.")
comprehend = boto3.client("comprehend", region_name="us-east-1",
                          aws_access_key_id="test", aws_secret_access_key="test")
stub = Stubber(comprehend)


def span(value, kind, score=0.99):
    i = PAGE.index(value)
    return {"Type": kind, "Score": score, "BeginOffset": i, "EndOffset": i + len(value)}


stub.add_response("detect_pii_entities",
                  {"Entities": [span("Priya Raman", "NAME"), span("priya.raman@example.com", "EMAIL"),
                                span("+1 206 555 0100", "PHONE"), span("billing", "NAME", score=0.31)]},
                  {"Text": PAGE, "LanguageCode": "en"})
KEY = b"from-secrets-manager-not-source"
with stub:
    r = scrub_page(PAGE, "PLATFORM", comprehend, KEY)

print(r.status, r.findings)
print(r.text)
print(len(r.pii_map), "values held in the separate pii_map")
assert "priya" not in r.text.lower() and "555" not in r.text and "billing export" in r.text
print("index key:", index_key("wiki-123", r.text)[:16])

print(scrub_page("key AKIA" + "IOSFODNN7EXAMPLE in runbook", "PLATFORM", comprehend, KEY).status)
print(scrub_page("salary bands", "PAYROLL", comprehend, KEY).status)

indexed ['EMAIL', 'NAME', 'PHONE']
Escalations: page [NAME_bb8fbf9a98] ([EMAIL_85904943ad], [PHONE_f91ef52376]) if the billing export fails.

Runbook: rerun the export job from the console.
3 values held in the separate pii_map
index key: 7a9057f8ee2f54cd
quarantined
denied

The name, email and phone are replaced; the low-score “billing” span is kept; a page with an access key is quarantined; and the payroll space never reaches Comprehend at all.

Step 4: Embed only the scrubbed text, and re-embed when policy changes

embed() sends the scrubbed text to Amazon Titan Text Embeddings V2, which accepts 1,024 (the default), 512 or 256 output dimensions. Chunk the scrubbed text first if pages are long. Use index_key() as the upsert key: it changes when the scrubbed text changes or when you bump POLICY_VERSION, so adding a new entity type to the policy forces affected documents to be re-scrubbed and re-embedded instead of leaving old, less-redacted vectors in place. Delete vectors for documents that become quarantined.

Step 5: Backfill an existing export

For a one-off cleanup of a large export, Comprehend’s asynchronous PII job can write redacted copies to S3. ONLY_REDACTION mode with REPLACE_WITH_PII_ENTITY_TYPE replaces each entity with its type name:

# One-off backfill of an exported wiki: Comprehend writes a redacted copy of each document to S3.
aws comprehend start-pii-entities-detection-job \
  --job-name wiki-backfill-2026-10 \
  --language-code en \
  --mode ONLY_REDACTION \
  --redaction-config '{"PiiEntityTypes":["NAME","EMAIL","PHONE","ADDRESS","SSN","CREDIT_DEBIT_NUMBER"],"MaskMode":"REPLACE_WITH_PII_ENTITY_TYPE"}' \
  --input-data-config 'S3Uri=s3://acme-wiki-raw/export-2026-10/,InputFormat=ONE_DOC_PER_FILE' \
  --output-data-config 'S3Uri=s3://acme-wiki-redacted/export-2026-10/' \
  --data-access-role-arn arn:aws:iam::111122223333:role/ComprehendWikiRedaction

Async jobs accept documents up to 1 MB each. Type-name replacement loses the “same person” consistency of keyed placeholders, so use it for backfills and the real-time path for ongoing ingestion. Secret quarantine still has to run separately: a redacted access key in a runbook still means the key needs rotating. Amazon Macie can help find sensitive data in the S3 buckets that hold raw exports.

Worked example: the on-call wiki

1. The ingest job reads 4,000 pages from the platform wiki (an illustrative figure). The HR space is skipped by the deny list.
2. Pages with access keys or private keys go to a quarantine bucket; the security team rotates the keys and fixes the pages at the source.
3. The rest go through Comprehend; names, emails and phone numbers become keyed placeholders, and the values go into the pii_map table.
4. Chunks of scrubbed text are embedded and indexed. The assistant can answer “who owns the billing export failover?” with “the on-call engineer listed as [NAME_bb8fbf9a98]; see the roster”, and a human can resolve that through the authorized lookup.
5. Later, the team adds IP_ADDRESS to the policy and bumps POLICY_VERSION; the next run re-embeds every page whose index key changed.

Troubleshooting

Production checklist

• ☐ Scrub before chunking and embedding; the embedding role can’t read raw exports
• ☐ Space deny list and secret quarantine with a rotation hand-off
• ☐ Comprehend PII detection with a type policy and a threshold tuned on your data
• ☐ Keyed HMAC placeholders with the key in Secrets Manager
• ☐ pii_map in a separate encrypted store, readable only by an authorized human lookup
• ☐ Policy version in the index key; quarantined documents deleted from the index
• ☐ Regular sampling of indexed chunks with a second detector

FAQ

Can’t I just use Bedrock Guardrails at answer time?

Guardrails’ sensitive information filters can mask PII in prompts and responses, which is a good second layer. But the data would still be stored in your index, and AWS notes that the filter evaluates text content only: for example, PII the model writes into tool-call arguments is neither blocked nor masked. Scrub at ingest and use guardrails as a backstop.

Why quarantine pages with secrets instead of redacting them?

Because the secret is already exposed in the wiki. Redacting it in the index hides the symptom; the key still works. Quarantine forces rotation and a fix at the source.

Does Comprehend support languages other than English?

PII detection supports English and Spanish. Route other languages to a different detector or to review.

Is the placeholder reversible?

Not without the HMAC key or the pii_map. That’s the point: the index alone reveals nothing.

Official documentation

• Amazon Comprehend: detecting PII entities
• PII real-time analysis (API)
• Comprehend guidelines and quotas
• Amazon Titan Text Embeddings parameters
• Bedrock Guardrails: sensitive information filters

Related on CheatCoders: Secret-aware context filters · OpenSearch vs pgvector for code RAG

Last updated on October 10, 2026