10 views
Rewriting a service in Rust to fix one hot loop is how platforms lose a year. PyO3 lets you extract a measured bottleneck into a Rust extension, keep Python for I/O and product logic, and prove parity with golden tests before flipping traffic. The senior move is surgical: profile → isolate → golden → ship wheel → feature-flag.
⚡ TL;DR: Port only functions that dominate CPU in flamegraphs. Wrap with PyO3, mirror Python API, differential-test on production traces. Ship manylinux wheels; keep pure-Python fallback. Related mindset: N-API addons for Node.
Python PyO3 Extensions: production guidance
python
# python/scoring.py — keep orchestration here
from typing import Sequence
try:
from scorers_rs import score_batch as _score_batch_rs
_IMPL = "rust"
except ImportError:
_IMPL = "python"
def _score_batch_rs(xs: Sequence[float], w: Sequence[float]) -> list[float]:
return [sum(a * b for a, b in zip(row, w)) for row in xs] # fallback
def score_batch(rows: Sequence[Sequence[float]], weights: Sequence[float]) -> list[float]:
# ✅ Single chokepoint — swap impl without call-site churn
return list(_score_batch_rs(rows, weights))
rust
// src/lib.rs — illustrative PyO3 surface
use pyo3::prelude::*;
#[pyfunction]
fn score_batch(rows: Vec<Vec<f64>>, weights: Vec<f64>) -> PyResult<Vec<f64>> {
Ok(rows
.into_iter()
.map(|row| row.iter().zip(&weights).map(|(a, b)| a * b).sum())
.collect())
}
#[pymodule]
fn scorers_rs(m: &Bound<'_>) -> PyResult<()> {
m.add_function(wrap_pyfunction!(score_batch, m)?)?;
Ok(())
}
Golden differential tests from production traces
python
Capture fixtures from the Python path before enabling Rust in prod. Fail CI if Rust and Python disagree beyond tolerance.
# tests/test_score_parity.py
import json
from pathlib import Path
from scoring import score_batch
def test_parity_against_fixtures():
for line in Path("testdata/score_cases.jsonl").read_text().splitlines():
case = json.loads(line)
out = score_batch(case["rows"], case["weights"])
assert out == pytest.approx(case["expected"], rel=1e-9, abs=1e-9)
Packaging and fallbacks
| Concern | Practice |
|---|---|
| Wheels | maturin manylinux / musllinux for ECS AMIs |
| Import fail | Pure-Python fallback path |
| ABI | Pin Python minor in image; rebuild on 3.x bumps |
| Release | Feature flag % traffic; watch error + p99 |
toml
# Cargo.toml (excerpt)
[dependencies]
pyo3 = { version = "0.22", features = ["extension-module"] }
Closing checklist
✅ Dos – ✅ Prove the hotspot with profiles before writing Rust – ✅ Keep a thin Python API with Rust behind it – ✅ Differential-test on real fixtures – ✅ Ship wheels for every prod platform tag – ✅ Feature-flag rollout with p99 and error budgets ❌ Don’ts – ❌ Don’t rewrite the whole service to fix one loop – ❌ Don’t ship without a pure-Python fallback – ❌ Don’t pass giant Python objects chatty across the boundary – ❌ Don’t ignore musllinux vs manylinux AMI skew – ❌ Don’t skip golden updates when the algorithm intentionally changesRelated reading
- N-API Addons for Node: Move Hot Loops Off the Event Loop
- mypyc Compilation in Production: Speedups That Survive Real Imports
- Lambda Rust Runtime: When Node Stops Being the Right Default
- V8 Maglev vs TurboFan: When Tiering Ups Silently Hurt p99
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Python String Methods: Every str Method With Real Production Examples
- Css Animation Performance: Performance-Optimized Motion for Modern UIs
Newly added
- AWS Systems Manager Parameter Store: Hierarchical Runtime Config for Coding Agents
- Amazon Cognito Identity Pools: Federate Coding-Agent Tool Callers Without Long-Lived Keys
- Amazon Athena for Coding Agents: Query Tool Audit Logs and Eval Traces on S3 Without a Warehouse
- AWS Batch for Coding Agents: Parallel Overnight PR Reviews and Long-Running Jobs Without Lambda Timeouts
- Amazon Bedrock Prompt Management: Version and A/B Test Coding-Agent System Prompts
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.

Pingback: CPython Specialize Stats: Why Hot Functions Stop Adaptive Instructions - CheatCoders