Posted inAI Machine Learning
Day 46: Synthetic Data That Doesn’t Clone GitHub Noise
Generate synthetic training data from internal ADRs and failing tests — not scraped GitHub noise that poisons style and license posture.
Posted inAI Machine Learning
Day 45: Catastrophic Forgetting Checks
Hold out general coding tasks so domain fine-tunes do not destroy baseline capabilities — gate deploys on regressions.
Posted inAI Machine Learning
Day 19: Offline vs Online Evaluation
Offline evals gate merges; online evals catch drift — canary tickets in CI plus sampled production traces.
Posted inAI System Design
Day 18: Logging Prompts Without Leaking Secrets
Log prompts for debugging without leaking secrets — redaction, hashed user IDs, and retention legal can defend.
Posted inAI Machine Learning
Day 44: Continued Pretrain vs Instruct Tune vs Adapters
Pick the cheapest customization lever — continued pretrain, instruct tune, or adapters — that actually moves your eval.
Posted inAI AWS System Design
Day 17: Human-in-the-Loop Gates That Don’t Stall
Human-in-the-loop gates that do not stall the architecture — dual control via queues and EventBridge, not blocking modals.
Posted inAI AWS Machine Learning
Day 43: Distillation for Edge and Lambda
Distill larger teachers into Lambda-friendly students with latency budgets and hard quality floors — not blind compression.
Posted inAI AWS System Design
Day 16: Streaming UX Without Lying About Completeness
Stream tokens without lying about completeness — partial output, tool pauses, and backpressure on API Gateway/Lambda.
Posted inAI Machine Learning
Day 42: Preference Data Without Vanity Accept Rates
Build DPO/RLHF-style preference labels from real tickets and patches — not thumbs-up spam that optimizes for sycophancy.
Posted inAI AWS Machine Learning
Day 41: When Fine-Tunes Beat Prompting
Decide when SageMaker JumpStart or Bedrock customization beats another prompt — based on data you actually have and evals that move.