NVIDIA's CUDA Rust Brings Compile-Time Kernel Safety via Two Crates

NVIDIA's NVlabs has released two open-source Rust crates that make Rust a first-class language for writing GPU kernels: cuda-oxide for the SIMT programming model and cutile-rs for the newer Tile model. Rust code could already orchestrate CUDA kernel launches, but the kernel body itself had to be authored elsewhere. Both crates use Rust's ownership and borrow-checking rules to reject buffer aliasing at compile time — before any GPU work executes — which is the concrete safety advance worth understanding here.

For ML infrastructure engineers writing custom operators or inference kernels, the stakes are direct. The class of bugs these crates block — a kernel reading and writing the same buffer simultaneously — has historically been a runtime problem, invisible until a silent memory corruption surfaces in a production inference run. As software-layer extraction increasingly outpaces hardware acquisition at the systems frontier, compile-time kernel safety is one of the few mechanisms that directly reduces the debugging surface of that layer.

Two Compilation Pipelines, Two Safety Models

The SIMT track, cuda-oxide, is a custom rustc codegen backend. Functions tagged #[kernel] inside a #[cuda_module] block pass through Rust MIR, then to the community Pliron IR framework (where NVIDIA has added GPU dialects), then to LLVM IR, and finally to PTX. The host binary embeds that device bundle and launches it through a checked contract.

The Tile track, cutile-rs, works differently: a #[cutile::module] proc macro captures the kernel AST and embeds it directly in the host binary, deferring actual compilation to a JIT step through CUDA Tile IR on first launch. The resulting GPU cubin is cached per tile-width specialisation — a new value of the const B triggers a new build.

The safety argument differs between tracks. In cuda-oxide, the output buffer is typed as DisjointSlice rather than &mut [f32]. Because a plain mutable slice would require every thread to hold the same mutable borrow simultaneously — which Rust refuses — the type system forces the exclusive-access invariant. c.get_mut(idx) returns Option, converting out-of-bounds access into a handled branch. A #[launch_contract] attribute declares the block shape, and the generated prepare_vecadd method validates the launch configuration before the safe launch proceeds. A write/read alias on the same buffer produces error[E0502]: cannot borrow c_dev as mutable because it is also borrowed as immutable.

On the Tile side, cutile-rs's .partition([128]) call does three things simultaneously: it gives each tile exclusive ownership of its 128-element chunk, fixes the grid at 1,024 ÷ 128 = 8 tiles, and supplies the const tile width B. The generated launcher takes ownership of all tensors and returns them when the GPU finishes. Nothing executes until .sync_on(&stream). An aliasing attempt produces error[E0382]: use of moved value: z, because the launcher consumed ownership of z when it was partitioned. NVIDIA describes this ownership-follows-tensor-across-launch-boundary guarantee as the stronger of the two — Tile exposes no shared memory or thread indexing to misuse. SIMT retains that lower-level control, but shared memory in cuda-oxide currently requires unsafe.

Requirements and Current Deployability

cutile-rs is the operationally closer option: it installs from crates.io with a single cargo add, runs on stable Rust 1.89+, requires CUDA 13.3, and is already embedded in Hugging Face's Grout inference engine and in mistral.rs. cuda-oxide requires the pinned nightly toolchain, CUDA 12.x or newer, clang with libclang, and a git-sourced install. Its cargo oxide doctor subcommand validates the environment; cargo oxide new scaffolds a vector-addition project with host and device code in a single file. Neither project has been confirmed production-ready by NVIDIA.

Where This Fits

The GPU kernel was the remaining gap in an otherwise Rust-native NVIDIA stack: the Nova Linux driver is written in Rust, NVIDIA Dynamo has a Rust core, and NVTX ships Rust bindings. Planned inter-language interop will prevent Rust kernel authorship from locking developers out of C++ or Python kernel code in the same project. The two tracks map directly to the two models CUDA C++ and Python already support — SIMT for explicit thread and memory control, Tile for higher-level data-parallel description — so no third programming model is introduced.

For teams primarily concerned with safe tensor arithmetic at inference time, cutile-rs on stable Rust is evaluable today. For those who need explicit warp-level control, cuda-oxide demands a locked toolchain but offers a compile-time safety boundary that C++ CUDA never provided.