Microsoft has announced that neural rendering capabilities are coming to DirectX soon. Cooperative vector support, as it’s called, will lead to “cross-platform enablement of neural rendering techniques,” according to Microsoft, and it will usher in “a new paradigm in 3D graphics programming.”
It sounds buzzy, but that’s not without reason. This past week, Nvidia announced its new range of RTX 50-series graphics cards, and along with them, it revealed a slate of neural rendering features. Neural shaders, as Nvidia calls them, allow developers to execute small neural networks from shader code, running them on the dedicated AI hardware available on Nvidia, AMD, Intel, and Qualcomm GPUs. Microsoft is saying that it will enable these features on all GPUs, not just those sold by Nvidia, through the DirectX API.
Microsoft is doing so through cooperative vector support, which it says “directly improves the performance of neural rendering techniques.” It enables matrix multiplication to be run directly from shader code — the programs that your GPU executes — allowing neural networks to function regardless of your GPU brand. At this point, all major GPU brands have dedicated AI hardware available, and cooperative vectors will allow developers to tap into that hardware through shader code.
Cross-vendor support for neural shaders is a big deal. In generations past, Nvidia maintained a closed system of middleware dubbed GameWorks that would only work on Nvidia graphics cards. The suite included features like PhysX in games like Batman: Arkham City and Mirror’s Edge, as well as HairWorks in games like The Witcher 3 and Final Fantasy XV.
With broad neural rendering support through DirectX, it will hopefully encourage developers to include neural shaders without fear of locking out certain players. Nvidia has already demonstrated the capabilities of neural rendering with features such as the Neural Radiance Cache, which uses a neural network to infer light bounces in a path-traced scene, enhancing both the visual quality and performance of games that feature path tracing.
Although the DirectX announcement is recent, we won’t see neural rendering features in games for quite some time. Nvidia and Microsoft are just now laying the foundation for these new rendering techniques, so it could be several years before we see a game running small neural networks through shader code. The potential applications are vast, though, and these techniques could bring about a new wave of rendering capabilities now that they’ll be supported on GPUs from all vendors.
