Microsoft released ONNX Runtime 1.30.0 with new CUDA and WebGPU paths for generative AI inference, expanded CPU kernels and Go bindings, giving developers more options for running attention, decoding and quantized workloads across GPUs, browsers and processor architectures. The release covers changes since ONNX Runtime 1.29.1.
CUDA gains new decoding and cache options
The CUDA execution provider now supports variable-length causal convolution through VarlenCausalConvWithState, designed for continuous batching and compact state updates. Paged XQA adds speculative decoding, query-to-KV head group size 6, head size 256 with FP16 caches, and native block tables for 128-token pages. Split-KV is also available for paged FlashAttention decoding.
PagedAttention gains INT4 paged KV caches with per-channel scales and an is_causal attribute. The release also introduces a compact GatedDeltaNet operator with BFloat16 support, while DeepSeek Engram support arrives through the EngramGate and NGramHashMapping contrib operators. Kernel coverage was expanded for Qwen-3.5 operators.
Mixture-of-experts and quantized matrix operations receive several targeted changes. FP4 QMoE kernels are enabled by default in CUDA builds, including on Windows, although source builders can disable them with -Donnxruntime_USE_FP4_QMOE=OFF. A separate FP8 DeepGEMM MoE decode path is opt-in, limited to supported fixed-shape QMoE workloads on Hopper GPUs, disabled by default and unavailable on Windows.
CUDA fpA-intB builds now use a compact default kernel set covering FP16 activations, INT4 or INT8 weights, scale-only quantization and a block size of 32. Source builders that require BF16, zero points, bias, larger block sizes or native Hopper variants must enable the full set with -Donnxruntime_USE_FPA_INTB_GEMM_FULL=ON. Other CUDA work bounds QMoE workspace use, tunes NVFP4 and FP8 GEMV execution, extends speculative-decode GEMVs to 64 rows and adds an optional split-K path for small-N FP16 MatMul shapes.
WebGPU and other providers broaden coverage
WebGPU receives PagedAttention metadata, GPT-OSS support and INT8 KV-cache block quantization. It also adds INT64 Gather, optimizes MatMulNBits wide tiles, and extends convolution fusion with eight additional activations. Pointwise convolution can reuse subgroup-matrix MatMul, while the im2col-matmul path gains convolution-weight prepacking.
Additional WebGPU work enables GELU, BiasGELU and LayerNorm fusion, with the LayerNorm change addressing FP16 inference correctness. Subgroup-size control selects size 32 for subgroup-matrix MatMul and Gemm, and that path is now enabled in WASM builds. Pipeline-compilation workers scale with CPU count, while fixes cover user-supplied GPUDevice synchronization, MatMul cache keys, shader dispatch and out-of-bounds subgroup-matrix loads. Misuse of copy_tensors now reports an error instead of terminating the process.
WebGPU plugin execution-provider packaging now supports Linux AArch64, with the WebGPU and CUDA plugin versions advanced to 0.4.0 and 0.2 respectively. Plugin providers also gain LoRA adapter support and allocator-lifetime fixes. Elsewhere, WebNN adds SkipLayerNormalization, TensorRT corrects shape-value handling, and DirectML validates kernel-shape and output-padding lengths.
CPU, APIs and language bindings
CPU inference gains fused LinearAttention kernels for AVX-512, Arm64 NEON and SVE, plus AVX2 LayerNorm and RMSNorm kernels. The CPU provider also registers BFloat16 LayerNorm and RMSNorm, adds Arm SVE i8mm INT8 QGEMM, improves convolution thread use and optimizes INT4 weight prepacking.
A compatibility change gates CPU FP16 Gemm and MatMul execution on hardware acceleration. FP16 nodes assigned to the CPU without a matching accelerated kernel fall back to FP32. The release also fixes FP16 QuantizeLinear rounding, Arm64 SymmQgemm overflow and edge cases involving empty tensors, reductions and extreme Trilu diagonals.
New Go bindings expose the ONNX Runtime C API. Core runtime additions include importing memory through host pointers, accessing preallocated outputs with KernelContext::GetPreallocatedOutput, and obtaining packed-attention workspace recipes and estimates. Python and C# asynchronous execution receive resource-lifetime fixes; Java provider-option cleanup, Rust tensor validation and string extraction, Node.js cleanup hooks, JavaScript WebGPU Conv3D FP16 support and WinML image dimensions also receive corrections.
Reliability and build hardening
Model loading now limits nested graph depth and canonicalizes external-data locations. Validation was strengthened across operators including Split, Scan, GatherND, ScatterND, convolution and pooling, as well as generation subgraphs, attention inputs, LoRA adapter data types and quantization indexes. Graph optimizers now reject more invalid model-supplied indices and fusion inputs.
GPU reliability work addresses indexing, buffer-size arithmetic and overflow across attention, quantization, reduction, Softmax and other CUDA operations. Resource-lifetime fixes cover MultiHeadAttention scratch storage and CUDA graph replay. WebGPU now zero-initializes writable device buffers and rejects foreign GPU handles during built-in transfers.
Supply-chain and packaging changes include Protobuf 33.6, refreshed JavaScript and Python documentation dependencies, full-length commit SHA pins for GitHub Actions, authenticated package feeds and NPM network isolation. Build updates move to CUTLASS 4.7 and cuDNN Frontend 1.27, fix multiple Windows and CUDA configurations, and expand CUDA PagedAttention and WebGPU CI coverage. Microsoft credits 57 human contributors and notes that the release highlights were prepared with AI assistance.
Source: ONNX Runtime v1.30.0 release notes.
Definition. ONNX Runtime 1.30 is a release that expands attention, decoding, quantization, provider, API and language-binding support across GPUs, browsers and processor architectures.
| Execution area | ONNX Runtime 1.30 changes |
|---|---|
| CUDA | New decoding, cache, attention, MoE and quantized matrix paths; some capabilities are hardware-specific, disabled by default or build-time options. |
| WebGPU | Paged-attention metadata, GPT-OSS, INT8 KV-cache quantization, expanded fusion, WASM subgroup-matrix support and Linux AArch64 plugin packaging. |
| CPU | Fused LinearAttention and normalization kernels, quantization improvements and acceleration-gated FP16 Gemm and MatMul. |
| Other providers | WebNN adds SkipLayerNormalization, TensorRT fixes shape-value handling and DirectML adds validation. |
| Languages and APIs | New Go bindings plus runtime API additions and fixes across Python, C#, Java, Rust, Node.js, JavaScript and WinML. |
Key takeaways
- CUDA adds variable-length causal convolution, speculative decoding, new paged-attention capabilities and expanded quantized workload support.
- FP4 QMoE kernels are enabled by default in CUDA builds, while the FP8 DeepGEMM MoE decode path remains opt-in, Hopper-specific and unavailable on Windows.
- WebGPU gains paged-attention metadata, GPT-OSS support, INT8 KV-cache block quantization and broader fusion and packaging support.
- CPU inference adds architecture-specific attention, normalization and quantized matrix kernels.
- CPU FP16 Gemm and MatMul now require matching hardware acceleration; unsupported nodes fall back to FP32.
- The release adds Go bindings and strengthens validation, resource management, build security and packaging reliability.
FAQ
What does ONNX Runtime 1.30 add for CUDA inference?
It adds variable-length causal convolution, expanded paged attention and speculative decoding, INT4 KV caches, new mixture-of-experts paths and targeted GEMM and GEMV improvements.
Which CUDA features remain restricted or opt-in?
The FP8 DeepGEMM MoE decode path is disabled by default, limited to supported fixed-shape QMoE workloads on Hopper GPUs and unavailable on Windows. Some fpA-intB capabilities also require enabling the full kernel set at build time.
What changed for WebGPU?
WebGPU adds paged-attention metadata, GPT-OSS support, INT8 KV-cache block quantization, INT64 Gather, more convolution and normalization fusion, Linux AArch64 plugin packaging and multiple correctness and reliability fixes.
How does ONNX Runtime 1.30 handle CPU FP16 operations?
CPU FP16 Gemm and MatMul execute only when matching hardware acceleration is available; otherwise, FP16 nodes assigned to the CPU fall back to FP32.
Does ONNX Runtime 1.30 include Go support?
Yes. New Go bindings expose the ONNX Runtime C API.
What reliability improvements are included?
The release strengthens model and operator validation, addresses GPU indexing and overflow issues, improves resource lifetimes, hardens external-data handling and updates supply-chain and build controls.
