Part 1 of a ten-part series on serving engines from the inside. Pinned to vLLM 0.32.0 (5df63a7) and SGLang 0.5.22 (baa9bef). Both projects ship every couple of weeks, so every file reference below ...

You hit a four-fold performance drop in a production LLM pipeline. The profiler points straight at a closed-source binary blob: libtpu.so, a proprietary MLIR accelerator plugin, or a closed routine...

These two Rust loops differ by one element. // 1. `0..n` visits 0, 1, ..., n - 1. pub fn excl(n: u64) -> u64 { let mut s = 0u64; for i in 0..n { s = s.wrapping_add(i.wrapping_mu...

This program has no unsafe block. It compiles on stable Rust 1.95, and it prints a number from the wrong vector. pub fn main() { let mut a: Vec<u64> = vec![1]; let mut b: Vec<u64&...

serde is one of the foundational crates in Rust. Its serde_core/Cargo.toml contains a dependency that can never be compiled. # This cfg cannot be enabled, but it still forces Cargo to keep serde_d...

Windows 1.0 shipped in 1985 and asked for 256 KB of memory and two floppy drives.1 Windows 11 asks for 4 GB of memory and 64 GB of storage. That is a factor of sixteen thousand in memory and someth...

Software engineers operate under a comforting fiction: reading a value is free. In C++, we decorate pointers with const to assure the compiler that inspection produces no side effects. In function...

Pull up the die photo of any high-performance CPU from the last fifteen years and point at the parts that do arithmetic. They are small. The integer ALUs and the vector units are a modest slice of ...

I keep noticing the same pattern working with models on real code. The Rust I get back is better than the C++ I get back, and the Lean is better than either. The Rust compiles and does what I asked...

At Hot Chips this week, OpenAI presented Jalapeño, an inference ASIC co-designed with Broadcom.1 The architecture slide circulating since the presentation frames a request pipeline split three ways...