Skip to main content
  1. Home
  2. Computing
  3. News

Windows 11 will soon harness your GPU for generative AI

Kunal Khullar
By

Following the introduction of Copilot, its latest smart assistant for Windows 11, Microsoft is yet again advancing the integration of generative AI with Windows. At the ongoing Ignite 2023 developer conference in Seattle, the company announced a partnership with Nvidia on TensorRT-LLM that promises to elevate user experiences on Windows desktops and laptops with RTX GPUs.

The new release is set to introduce support for new large language models, making demanding AI workloads more accessible. Particularly noteworthy is its compatibility with OpenAI’s Chat API, which enables local execution (rather than the cloud) on PCs and workstations with RTX GPUs starting at 8GB of VRAM.

Recommended Videos

Nvidia’s TensorRT-LLM library was released just last month and is said to help improve the performance of large language models (LLMs) using the Tensor Cores on RTX graphics cards. It provides developers with a Python API to define LLMs and build TensorRT engines faster without deep knowledge of C++ or CUDA.

Related

With the release of TensorRT-LLM v0.6.0, navigating the complexities of custom generative AI projects will be simplified thanks to the introduction of AI Workbench. This is a unified toolkit facilitating the quick creation, testing, and customization of pretrained generative AI models and LLMs. The platform is also expected to enable developers to streamline collaboration and deployment, ensuring efficient and scalable model development.

A graph showing TensorRT-LLM inference performance on Windows 11.
Nvidia

Recognizing the importance of supporting AI developers, Nvidia and Microsoft are also releasing DirectML enhancements. These optimizations accelerate foundational AI models like Llama 2 and Stable Diffusion, providing developers with increased options for cross-vendor deployment and setting new standards for performance.

The new TensorRT-LLM library update also promises a substantial improvement in inference performance, with speeds up to five times faster. This update also expands support for additional popular LLMs, including Mistral 7B and Nemotron-3 8B, and extends the capabilities of fast and accurate local LLMs to a broader range of portable Windows devices.

The integration of TensorRT-LLM for Windows with OpenAI’s Chat API through a new wrapper will allow hundreds of AI-powered projects and applications to run locally on RTX-equipped PCs. This will potentially eliminate the need to rely on cloud services and ensure the security of private and proprietary data on Windows 11 PCs.

The future of AI on Windows 11 PCs still has a long way to go. With AI models becoming increasingly available and developers continuing to innovate, harnessing the power of Nvidia’s RTX GPUs could be a game-changer. However, it is too early to say whether this will be the final piece of the puzzle that Microsoft desperately needs to fully unlock the capabilities of AI on Windows PCs.

Editors' Recommendations

Topics
Kunal Khullar
Kunal Khullar
Computing Writer
A PC hardware enthusiast and casual gamer, Kunal has been in the tech industry for almost a decade contributing to names like…
Newegg wants your old GPU — here’s how much you could get
Three graphics cards on a gray background.

Upgrading to a new graphics card can be a hassle, and it has been even more difficult ever since the GPU shortage. Today, there are way too many models to choose from, and keeping track of prices is not easy. In an effort to make things a bit simpler, Newegg has announced a new trade-in program. The online retailer is offering customers a deal in which they send in their existing eligible GPU and receive a trade-in credit amount toward the purchase of a new qualifying graphics card.

According to Amir Asadibagheri, product manager of customer experience for Newegg, “the benefit of our trade-in program is the ease to send a used graphics card and buy a new one all within the same platform and avoiding the hassle of selling through a secondary market.” Newegg has given a list of all Nvidia and AMD graphics cards that are eligible, along with an estimated trade-in value. Notably, the trade-in is limited to Nvidia’s RTX series and AMD’s Radeon 5000 series and beyond.

Read more
Windows 11 is turning into Windows 12 right in front of us
Windows 11 set up on a computer.

Windows 12 is already upon us. Microsoft hasn't released the new operating system yet, or even formally announced it, but Microsoft's September 2023 event made it clear that recent updates to Windows 11 are laying the foundation for the next iteration of the OS.

A significant portion of the presentation focused on the upcoming AI assistant that is set to "revolutionize" the way you use Windows. A new update is now available for Windows 11 that adds Copilot to the OS. Copilot is said to be an everyday AI companion that provides assistance by uniquely integrating information from the internet, your work-related data, and your current PC tasks. It offers a range of AI features to numerous default apps by collecting data from your calendar, email, documents, and more to assist in quick text and email composition.

Read more
I saw the AI future of Windows 11, and it blew me away
Microsoft Copilot allows you to ask an AI assistant questions within Office apps.

I don't blame you if you're fed up with all of the AI talk. It started with ChatGPT, it moved to Bing Chat, and now you can't open up Apple News without a deluge of AI news. It's overwhelming. And now, Microsoft wants you to use AI as an assistant throughout all of Windows 11. No thanks.

Call me a cynic, but I pushed back. Bing Chat following me around Windows 11 didn't sound like a good time, but I swallowed my pride, decided to do my job as a tech journalist, and kindly asked what it could do at Microsoft's September 2023 event. And Windows Copilot blew me away.

Read more