Rust CUDA 0.3
Top: upcoming MPM engine that runs on CPU and GPU using rust-cuda, Bottom: toy path tracer that can run on CPU, GPU, and GPU (hardware raytracing) using recent experiments with OptiX
Today marks an exciting milestone for the Rust CUDA Project, over the past couple of months, we have made significant advancements in supporting many of the fundamental CUDA ecosystem libraries. The main changes in this release are the changes to cust to make future library support possible, but we will also be highlighting some of the WIP experiments we have been conducting.
Cust changes
This release is likely to be the biggest and most breaking change to cust ever, we had to fundamentally rework how many things work to:
- Fix some unsoundness.
- Remove some outdated and inconsistent things.
- Rework how contexts work to be interoperable with the runtime API.
Therefore this release is guaranteed to break your code, however, the changes should not break too much unless you did a lot of lower-level work with device memory constructs.
Cust 0.3 changes
TLDR
This release is gigantic, so here are the main things you need to worry about:
Context::create_and_push(FLAGS, device) -> Context::new(device).
Module::from_str(PTX) -> Module::from_ptx(PTX, &[]).
Context handling overhaul
The way that contexts are handled in cust has been completely overhauled, it now uses primary context handling instead of the normal driver API context APIs. This is aimed at future-proofing cust for libraries such as cuBLAS and cuFFT, as well as overall simplifying the context handling APIs. This does mean that the API changed a bit:
create_and_pushis nownewand it only takes a device, not a device and flags.set_flagsis now used for setting context flags.ContextStack,UnownedContext, and other legacy APIs are gone.
The old context handling is fully present in cust::context::legacy for anyone who needs it for specific reasons. If you use quick_init you don't need to worry about
any breaking changes, the API is the same.
cust_core
DeviceCopy has now been split into its own crate, cust_core. The crate is #![no_std], which allows you to
pull in cust_core in GPU crates for deriving DeviceCopy without cfg shenanigans.
Removed
DeviceBox::wrap, useDeviceBox::from_raw.DeviceSlice::as_ptrandDeviceSlice::as_mut_ptr. UseDeviceSlice::as_device_ptrthenDevicePointer::as_(mut)_ptr.DeviceSlice::chunksand consequentlyDeviceChunks.DeviceSlice::chunks_mutand consequentlyDeviceChunksMut.DeviceSlice::from_sliceandDeviceSlice::from_slice_mutbecause it was unsound.DevicePointer::as_raw_mut(useDevicePointer::as_mut_ptr).DevicePointer::wrap(useDevicePointer::from_raw).DeviceSliceno longer implementsIndexandIndexMut, switching away from[T]made this impossible to implement. Instead you can now useDeviceSlice::indexwhich behaves the same.
Line tables
The libnvvm codegen can now generate line tables while optimizing (previously it could generate debug info but not optimize), which allows you to debug and profile kernels much better in tools like Nsight Compute. You can enable debug info creation using .debug(DebugInfo::LineTables) with cuda_builder.
OptiX
Using the generous work of @anderslanglands, we were able to get rust-cuda to target hardware raytracing completely in rust (both for the host and the device). The toy path tracer example has been ported to be able to use hardware rt as a backend, however, optix and optix_device are not published on crates.io yet since they are still highly experimental.
using hardware rt to render a simple mesh
cuBLAS
Work on supporting cuBLAS through a high-level wrapper library has started, a lot of work needed to be done in cust to interop with cuBLAS which is a runtime API based library. This required some changes with how cust handles contexts to avoid dropping context resources cuBLAS was using. The library is not yet published but eventually will be once it is more complete. cuBLAS is a big piece of neural network training on the GPU so it is critical to support it.
cuDNN
@frjnn has been generously working on wrapping the cuDNN library. cuDNN is the primary tool used to train neural networks on the GPU, and the primary tool used by pytorch and tensorflow. High level bindings to cuDNN are a major step to making Machine Learning in Rust a viable option. This work is still very in-progress so it is not published yet, it will be published once it is usable and will likely first be used in neuronika for GPU neural network training.
Atomics
Work on supporting GPU-side atomics in cuda_std has started, some preliminary work is already published in cuda_std, however, it is still very in-progress and subject to change. Atomics are a difficult issue due to the vast amount of options available for GPU atomics, including:
- Different atomic scopes, device, system, or block.
- Specialized instructions or emulated depending on the compute capability target.
- Hardware float atomics (which core does not have)
You can read more about it here.