Systems Engineering
6 pieces on Systems Engineering.
News & Analysis
Infrastructure Optimization: Find the Bottleneck Before Changing Models
Prefix routing, Habitat's storage rewrite, cuDNN plans and PAIR address different bottlenecks. Their measurements do not rank infrastructure against model work.
cuDNN Graph API: Fusion, Autotuning, and Plan Reuse Explained
NVIDIA's cuDNN Frontend Graph API makes kernel fusion, engine selection, and compilation timing explicit — here's how each technique works.
Perplexity's Ivy, Tulip and ROSE Serve pplx-embed Without a Separate Engine
Perplexity reuses its LLM prefill/decode kernels for embedding inference, with 512 tokens saturating a sub-1B model on Hopper and Blackwell hardware.
Chunked Prefill Beats Disaggregation Below 1,000 GPUs
Every major inference framework now ships prefill-decode disaggregation — but for most teams it's the wrong default, and chunked prefill is the safer fix.
Risk-Scored Routing Cuts Human Review to High-Signal Queries Only
A text-to-SQL team replaced blanket approval gates with a four-signal risk router, sending only genuinely ambiguous actions to human reviewers.
Four AI Systems Reports: Compare the Baseline Before the Speedup
Custom silicon, quantization, speculative decoding and RDMA solve different problems. Their reported gains need separate baselines and deployment checks.