PII Redaction Before Embeddings: Scrub Wikis Prior to Vector Indexes

Embeddings are not encryption. If a Confluence page contains a customer email, an API token pasted in a “temporary” runbook, or a screenshot OCR’d into text, that residual signal can survive in vectors and in retrieved chunks forever. The senior control is boring and non-negotiable: scrub in the batch path (Glue / EMR / Step Functions) before any Bedrock embedding call, and refuse to index documents that fail classification gates.

⚡ TL;DR: Detect → redact → hash-stable placeholders → embed. Run Macie / Comprehend / custom detectors in Glue. Never embed raw wiki HTML. Re-scrub on reindex. Audit samples weekly. Pair with Secret-aware context filters and AI coding in VPC.

Put redaction on the write path, not the chat path

# glue/scrub_wiki.py
import re
from dataclasses import dataclass

EMAIL = re.compile(r"\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b", re.I)
AWS_KEY = re.compile(r"\bAKIA[0-9A-Z]{16}\b")
BEARER = re.compile(r"\bBearer\s+[A-Za-z0-9._\-|=]+\b")

@dataclass
class ScrubResult:
    text: str
    findings: list[str]

def scrub(text: str) -> ScrubResult:
    findings = []
    def sub(pat, label, s):
        nonlocal findings
        if pat.search(s):
            findings.append(label)
        return pat.sub(f"[{label}]", s)
    text = sub(AWS_KEY, "AWS_ACCESS_KEY", text)
    text = sub(BEARER, "BEARER_TOKEN", text)
    text = sub(EMAIL, "EMAIL", text)
    # ✅ Stable placeholders keep nearby semantics without leaking values
    return ScrubResult(text=text, findings=findings)

❌ Embedding first and “filtering PII at answer time” — retrieval still stores the secret in the index.

Glue job skeleton before Bedrock

# glue/job_embed_safe.py
def process_page(raw_html: str, doc_id: str) -> dict | None:
    text = html_to_text(raw_html)
    result = scrub(text)
    if "AWS_ACCESS_KEY" in result.findings or "BEARER_TOKEN" in result.findings:
        quarantine(doc_id, result.findings)  # ✅ human review bucket
        return None
    # Optional: Comprehend PII entity detection for names/phones
    entities = comprehend_detect_pii(result.text)
    cleaned = replace_entities(result.text, entities)
    vectors = bedrock_embed(cleaned)  # only scrubbed text leaves the account boundary
    return {"id": doc_id, "text": cleaned, "vector": vectors, "pii_tags": result.findings + entities}

Classification gates and quarantine

Store scrubbed text and vectors under CMKs; keep raw wiki exports in a tighter IAM vault that embedding roles cannot read.

Reindex and drift

When detectors improve, re-scrub the corpus — do not assume old vectors are clean. Content hashes of scrubbed text should drive upserts; if scrubbing changes, re-embed. Sample 1% of indexed chunks monthly with a second detector model and page security if findings reappear.

Closing checklist

✅ Dos
– ✅ Scrub in Glue/batch before any embedding API
– ✅ Quarantine hard secrets; do not “partially” embed them
– ✅ Use stable placeholders for continuity
– ✅ Deny-list sensitive wiki spaces
– ✅ Re-scrub when detectors or policies change

❌ Don’ts
– ❌ Don’t rely on chat-time redaction alone
– ❌ Don’t give embed roles read access to raw exports
– ❌ Don’t index OCR’d screenshots without PII OCR checks
– ❌ Don’t log raw findings with full secret values
– ❌ Don’t skip monthly audit samples

Related reading

• Secret-Aware Context Filters: Stop AI Editors From Shipping Keys
• AI Coding in VPC: Private Bedrock Endpoints and Secret Hygiene
• Bedrock Retrieval Filters: Tenant Isolation for Multi-SaaS Code RAG
• Glue Plus Bedrock: Offline Doc Enrichment Before Online Code RAG

Last updated on September 11, 2026