21 views
This guide focuses on orjson and msgspec for production systems, with practical trade-offs for reliability, security, and cost.
json.dumps on large response graphs quietly owns CPU on FastAPI fleets. Microbenchmarks lie; the unfair advantage is swapping in orjson or msgspec and measuring p99 + worker CPU under production-shaped payloads with the same Pydantic models you ship.
⚡ TL;DR: Installorjsonresponse class ormsgspecstructs for the hottest routes; keep Pydantic for inbound validation; benchmark p99 under realistic nested payloads; watch unicode/datetime edge cases. Pair with Pydantic v2 Validators and Bytes vs str Boundaries.
orjson and msgspec: production guidance
python
# ✅ Drop-in ORJSONResponse
from fastapi import FastAPI
from fastapi.responses import ORJSONResponse
from pydantic import BaseModel
app = FastAPI(default_response_class=ORJSONResponse)
class Item(BaseModel):
id: str
score: float
tags: list[str]
@app.get("/items/{item_id}", response_model=Item)
async def get_item(item_id: str) -> Item:
return Item(id=item_id, score=0.9, tags=["a", "b"])
python
# ✅ msgspec for the absolute hottest encoder path
import msgspec
class ItemStruct(msgspec.Struct):
id: str
score: float
tags: list[str]
encoder = msgspec.json.Encoder()
def encode_item(item: ItemStruct) -> bytes:
return encoder.encode(item) # returns bytes — perfect for Response
What to measure
| Metric | Why |
|---|---|
| p99 latency | Serialization sits on the request path |
| Worker CPU % | Often the real capacity unlock |
| RSS | Large dumps can spike temps |
| Error rate | datetime/UUID/NaN edge cases |
python
# ❌ Declaring victory from dumps of {"a": 1} in a notebook
# ✅ Replay production response fixtures (PII-redacted) in CI bench
python
import time
import orjson
import json
def bench(fn, payload, n=2000):
t0 = time.perf_counter()
for _ in range(n):
fn(payload)
return (time.perf_counter() - t0) / n * 1000
payload = {"items": [{"id": str(i), "tags": ["x"] * 20} for i in range(200)]}
print("stdlib", bench(lambda p: json.dumps(p).encode(), payload))
print("orjson", bench(orjson.dumps, payload))
Gotchas seniors hit
orjsonrejects naive non-UTC datetimes depending on options — setOPT_NAIVE_UTCdeliberately.msgspecstructs are not Pydantic models; convert at the boundary or dual-define.- Numpy / Decimals need explicit options or pre-normalization.
Closing checklist
✅ Dos – ✅ UseORJSONResponse (or msgspec) on hot read paths
– ✅ Bench with production-shaped fixtures
– ✅ Keep inbound validation on Pydantic v2
– ✅ Return bytes bodies to skip a final encode
– ✅ Document datetime/UUID encoding options in ADRs
❌ Don’ts
– ❌ Don’t trust microbenchmarks on tiny dicts
– ❌ Don’t mix three JSON libs in one service without boundaries
– ❌ Don’t forget custom types (Decimal, set)
– ❌ Don’t enable pretty-print in production responses
Related reading
- Pydantic v2 Validators
- Bytes vs str Boundaries
- Structured Clone vs JSON in Node
- CPython Specialize Stats
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Python String Methods: Every str Method With Real Production Examples
- Css Animation Performance: Performance-Optimized Motion for Modern UIs
Newly added
- AWS Systems Manager Parameter Store: Hierarchical Runtime Config for Coding Agents
- Amazon Cognito Identity Pools: Federate Coding-Agent Tool Callers Without Long-Lived Keys
- Amazon Athena for Coding Agents: Query Tool Audit Logs and Eval Traces on S3 Without a Warehouse
- AWS Batch for Coding Agents: Parallel Overnight PR Reviews and Long-Running Jobs Without Lambda Timeouts
- Amazon Bedrock Prompt Management: Version and A/B Test Coding-Agent System Prompts
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.
