Day 43: Distillation for Edge and Lambda

You do not need frontier latency for “complete this enum in our SDK.” Distill a teacher on private examples into a student that fits Lambda memory and your p95 budget — then refuse to ship if quality floors crack. Day 43 sets budgets before training.

⚡ TL;DR: Write latency/size/quality budgets first. Generate teacher labels on curated prompts; validate; train student; gate on domain+general packs. Serve behind the Day 47 router with instant rollback to teacher.

Budgets first

Example: p95 ≤ 120ms for autocomplete chunk; artifact fits Lambda; quality ≥ 95% of teacher on internal SDK suite.

for prompt in curated_prompts:
    teacher_out = teacher.generate(prompt, temperature=0.2)
    if validator(prompt, teacher_out):
        yield {"prompt": prompt, "completion": teacher_out}

Never ship on latency alone. Keep teacher comparison on the Day 49 set and Day 45 general pack.

Production checklist

• [ ] Latency + size budgets documented
• [ ] Teacher outputs validated
• [ ] Student vs teacher delta reported
• [ ] Rollback to teacher wired
• [ ] Cost per 1k requests compared
• [ ] Safety/citation probes still pass

Series navigation

← Day 42 · Day 44 →

Last updated September 11, 2026