Day 50: Project: Internal SDK Autocomplete Model

Capstone for the customization arc: build a private SDK autocomplete student, distill or adapt from a teacher, serve behind Day 47’s router, and block merges if CI evals regress.

⚡ TL;DR: Curate license-clean SDK examples (Day 46/48). Train adapter/student. Gate on domain + general packs (Day 45). Blind smoke on tickets (Day 49). Ship via router with teacher fallback and 72h shadow.

Lab / project goals

1. Dataset from packages/sdk + examples
2. Train LoRA/student (JumpStart or Bedrock customization)
3. Offline suite: exact API match @k, compile rate
4. Route sdk_autocomplete → student with teacher fallback
5. CI gate on training-data or serving-config changes

{"prompt": "// complete AuthClient from '@acme/sdk'\nconst client = new AuthClient({\n", "completion": "  region: 'us-east-1',\n  credentials: fromEnv(),\n});"}

- run: python eval/run_suite.py --model $CANDIDATE --pack domain_v4
- run: python eval/run_suite.py --model $CANDIDATE --pack general_v2
- run: python eval/gate.py --budgets budgets.yaml

Project idea: IDE plugin stub

Thin client calling /v1/complete with task=sdk_autocomplete. Log accept/reject as preference seeds (Day 42) — secondary signal only, never sole reward.

Deliverables checklist

• [ ] Manifest + legal approval
• [ ] Student artifact versioned
• [ ] Router flag + rollback tested
• [ ] Domain/general gates green
• [ ] Shadow vs teacher ≥72h before 100%
• [ ] Runbook for emergency teacher-only mode

Series navigation

← Day 49 · Day 51 →

Last updated September 11, 2026