Day 23: Metadata Filters and Tenant Isolation

The worst RAG bug is not a wrong answer — it is another tenant’s chunk in the context window. Day 23 hardens metadata filters and tenant isolation: query filters, IAM/resource policies, index partitioning strategies, and tests that fail when isolation breaks.

⚡ TL;DR: Every retrieve path requires tenant (and finer) predicates inside the engine. Defense in depth with IAM and separate indexes/collections when risk demands. Add isolation tests that attempt cross-tenant reads. Logs must not print foreign chunk bodies.

Filter is not optional

# ✅ Tenant always required — no default
def retrieve(query: str, tenant_id: str, **filters):
    if not tenant_id:
        raise AuthzError("tenant_required")
    return engine.search(query, filter={"tenant_id": tenant_id, **filters})

❌ tenant_id optional “for admin debugging” without a break-glass role and audit.

Apply the same filter to BM25, vector, and symbol channels (Day 21). One unfiltered channel voids the design.

Isolation strategies

Many B2B apps start with shared+filter and move hot/enterprise tenants to dedicated indexes. Document the threat model.

On AWS, pair OpenSearch/pgvector filters with IAM that prevents the app role from reading foreign raw S3 source objects even if a vector ID is guessed.

Tests you must automate

def test_no_cross_tenant(client):
    client.login(tenant="A")
    hits = client.search("unique_secret_string_only_in_B")
    assert hits == []

Also test empty tenant, SQL injection-ish filter values, and admin impersonation paths.

Field notes from production

Support engineers will ask for a “view as tenant” tool. Build it with JIT access, full audit, and red banner in UI — do not reuse the production retrieve without a ticket ID. Leaks often happen in staging clones of prod data; scrub or isolate staging similarly.

Implementation sketch

-- Row Level Security sketch for pgvector deployments
ALTER TABLE chunks ENABLE ROW LEVEL SECURITY;
CREATE POLICY tenant_iso ON chunks
  USING (tenant_id = current_setting('app.tenant_id'));

Closing checklist

• [ ] Tenant predicate mandatory in all channels
• [ ] Isolation automated tests in CI
• [ ] IAM denies cross-tenant source reads
• [ ] Break-glass documented and audited
• [ ] Staging data isolation policy
• [ ] Logs avoid foreign chunk body exfiltration

Red team isolation

Run scheduled jobs that attempt cross-tenant retrieval with stolen vector IDs, missing filters, and admin tokens. Page on any hit. Include these in chaos/game days. Isolation is a security property — treat filter regressions like auth bypasses in your severity matrix, not like mild quality bugs.

Extended discussion

Return to the core angle for Day 23: KB filters, IAM, and the failure mode of leaking another customer’s snippets. That sentence is the acceptance lens for every design review this week. If a proposed change does not make this angle easier to measure or enforce, it is a distraction.

Write down three metrics you will look at after shipping Day 23 ideas, schedule a 45-minute readout, and archive the notes next to the eval artifacts. Architecture without a readout becomes slideshow archaeology.

Pair this day with the adjacent lessons in the series navigation below. Forward links exist so you can keep momentum; backward links exist so you can repair foundations when a later lab fails for boring earlier reasons.

Practically, allocate half a day to implement the smallest vertical slice, half a day to wire measurement, and refuse to polish UI until both are done. This ordering is how bootcamp projects stay honest under time pressure.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 23 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 23 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 23 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 23 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 23 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 23 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 23 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 23 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Series navigation

← Day 22 · Day 24 →

Last updated September 11, 2026