Day 24: Query Rewriting and HyDE Without Hallucinated APIs

Users ask messy questions. Query rewriting and HyDE (Hypothetical Document Embeddings) can lift recall — or invent methods that do not exist and retrieve garbage with confidence. Day 24 uses rewriting safely: expand known aliases and symbols from a catalog; constrain HyDE; never let the rewriter mint APIs.

⚡ TL;DR: Maintain an alias/symbol dictionary from your repo and services. Rewrite with allowlisted expansions. If using HyDE, generate hypothetical docs grounded in known terms, then retrieve — validate that rewritten queries do not introduce unknown identifiers.

Safe rewrite patterns

# ✅ Dictionary-backed expansion
ALIASES = {"asg": "auto scaling group", "alb 403": "listener rule denied"}

def rewrite(q: str) -> str:
    out = q
    for k, v in ALIASES.items():
        if k in q.lower():
            out += f" {v}"
    # attach known symbols via searcher, not LLM invention
    syms = symbol_suggest(q, limit=5)  # from extracted index
    return out + " " + " ".join(syms)

❌ “Rewrite the query to be more precise” unconstrained — models invent Client.forceAuthRefresh() because it sounds right.

HyDE with seatbelts

HyDE writes a hypothetical answer/doc and embeds that for retrieval. Useful for short conceptual queries. Seatbelts:

1. Provide a glossary of allowed service names.
2. Run a validator: every CamelCase / snake_case token must exist in the symbol index or glossary; else strip it.
3. Compare Recall vs plain query; keep HyDE only where it wins.

def hyde_query(q, glossary):
    hypo = llm.generate_doc(q, only_use=glossary)
    hypo = strip_unknown_identifiers(hypo, symbol_index)
    return embed(hypo)

Multi-query retrieval

Generate 2–3 safe alternates (alias expand, symbol expand, original), retrieve per branch, fuse with RRF (Day 21). Cap branches for cost (Day 15).

Field notes from production

Log original vs rewritten queries. When on-call sees a bad answer, the rewrite is often the villain. Feature-flag HyDE; it is not free latency or tokens.

Implementation sketch

def retrieve_smart(q, tenant):
    variants = [q, rewrite(q)]
    if feature("hyde") and is_conceptual(q):
        variants.append(hyde_query(q, glossary))
    return rrf([hybrid(v, tenant) for v in variants])

Closing checklist

• [ ] Alias dictionary generated from corp systems
• [ ] Symbol suggest from real index, not LLM memory
• [ ] Unknown-identifier strip on HyDE output
• [ ] Eval comparing rewrite strategies
• [ ] Feature flag + logs for rewritten queries
• [ ] Cost cap on multi-query branches

Glossary generation pipeline

Extract service names from the catalog, public method names from the symbol index, and common aliases from search click logs. Publish glossary.json as an artifact. The rewriter and HyDE validator both consume it. Stale glossaries invent less than LLMs but still rot — refresh on catalog changes in CI.

Extended discussion

Return to the core angle for Day 24: Expand symbols and package aliases; never invent methods. That sentence is the acceptance lens for every design review this week. If a proposed change does not make this angle easier to measure or enforce, it is a distraction.

Write down three metrics you will look at after shipping Day 24 ideas, schedule a 45-minute readout, and archive the notes next to the eval artifacts. Architecture without a readout becomes slideshow archaeology.

Pair this day with the adjacent lessons in the series navigation below. Forward links exist so you can keep momentum; backward links exist so you can repair foundations when a later lab fails for boring earlier reasons.

Practically, allocate half a day to implement the smallest vertical slice, half a day to wire measurement, and refuse to polish UI until both are done. This ordering is how bootcamp projects stay honest under time pressure.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 24 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 24 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 24 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 24 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 24 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 24 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 24 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 24 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Series navigation

← Day 23 · Day 25 →

Last updated September 11, 2026