This line of JAX has three outcomes on jax 0.11.0, depending on the machine it runs on: import jax, jax.numpy as jnp from jax import lax # 1. Both operands are literals, so the division happens i...

The XLA review said that when you disagree with XLA’s fusion decisions, your recourse is to restructure your source or leave through custom_call. That is true of fusion. It isn’t true of the pipeli...

Here is the first data movement call most GPU programmers learn:1 // 1. Two untyped pointers. One is in GPU memory, one is in host memory, // and the declaration cannot say which. __host__ cuda...

Part 2 of a ten-part series on serving engines from the inside. Pinned to vLLM 0.32.0 (5df63a7). Part 1 named the five layers and where vLLM and SGLang cut them into processes. This part follows on...

Part 1 of a ten-part series on serving engines from the inside. Pinned to vLLM 0.32.0 (5df63a7) and SGLang 0.5.22 (baa9bef). Both projects ship every couple of weeks, so every file reference below ...

You hit a four-fold performance drop in a production LLM pipeline. The profiler points straight at a closed-source binary blob: libtpu.so, a proprietary MLIR accelerator plugin, or a closed routine...

These two Rust loops differ by one element. // 1. `0..n` visits 0, 1, ..., n - 1. pub fn excl(n: u64) -> u64 { let mut s = 0u64; for i in 0..n { s = s.wrapping_add(i.wrapping_mu...

This program has no unsafe block. It compiles on stable Rust 1.95, and it prints a number from the wrong vector. pub fn main() { let mut a: Vec<u64> = vec![1]; let mut b: Vec<u64&...

serde is one of the foundational crates in Rust. Its serde_core/Cargo.toml contains a dependency that can never be compiled. # This cfg cannot be enabled, but it still forces Cargo to keep serde_d...

Windows 1.0 shipped in 1985 and asked for 256 KB of memory and two floppy drives.1 Windows 11 asks for 4 GB of memory and 64 GB of storage. That is a factor of sixteen thousand in memory and someth...