Skip to main content

Digital Trends may earn a commission when you buy through links on our site. Why trust us?

Nvidia’s new Guardrails tool fixes the biggest problem with AI chatbots

Nvidia is introducing its new NeMo Guardrails tool for AI developers, and it promises to make AI chatbots like ChatGPT just a little less insane. The open-source software is available to developers now, and it focuses on three areas to make AI chatbots more useful and less unsettling.

The tool sits between the user and the Large Language Model (LLM) they’re interacting with. It’s a safety for chatbots, intercepting responses before they ever reach the language model to either stop the model from responding or to give it specific instructions about how to respond.

Bing Chat saying it wants to be human.
Jacob Roach / Digital Trends

Nvidia says NeMo Guardrails is focused on topical, safety, and security boundaries. The topical focus seems to be the most useful, as it forces the LLM to stay in a particular range of responses. Nvidia demoed Guardrails by showing a chatbot trained on the company’s HR database. When asked a question about Nvidia’s finances, it gave a canned response that was programmed with NeMo Guardrails.

This is important due to the many so-called hallucinations we’ve seen out of AI chatbots. Microsoft’s Bing Chat, for example, provided us with several bizarre and factually incorrect responses in our first demo. When faced with a question the LLM doesn’t understand, it will often make up a response in an attempt to satisfy the query. NeMo Guardrails aims to put a stop to those made-up responses.

The safety and security tenets focus on filtering out unwanted responses from the LLM and preventing it from being toyed with by users. As we’ve already seen, you can jailbreak ChatGPT and other AI chatbots. NeMo Guardrails will take those queries and block them from ever reaching the LLM.

A diagram of Nvidia's NeMo Guardrails tool.

Although NeMo Guardrails to built to keep chatbots on-topic and accurate, it isn’t a catch-all solution. Nvidia says it works best as a second line of defense, and that companies developing and deploying chatbots should still train the model on a set of safeguards.

Developers need to customize the tool to fit their applications, too. This allows NeoMo Guardrails to sit on top of middleware that AI models already use, such as LangChain, which already provides a framework for how AI chatbots are supposed to interact with users.

In addition to being open-source, Nvidia is also offering NeMo Guardrails as part of its AI Foundations service. This package provides several pre-trained models and frameworks for companies that don’t have the time or resources to train and maintain their own models.

Editors' Recommendations

Jacob Roach
Senior Staff Writer, Computing
Jacob Roach is a writer covering computing and gaming at Digital Trends. After realizing Crysis wouldn't run on a laptop, he…
This new Photoshop tool could bring AI magic to your images
A mountainous landscape at night with the Northern Lights in the sky, a lake in the foreground, and a person standing under a rock archway on the right. This image was made with Adobe Photoshop's Generative Fill tool.

These days, it seems like everyone and their dog is working artificial intelligence (AI) into their tech products, from ChatGPT in your web browser to click-and-drag image editing. The latest example is Adobe Photoshop, but this isn’t just another cookie-cutter quick fix -- no, it could have a profound effect on imagery and image creators.

Photoshop’s newest feature is called Generative Fill, and it lets you use text prompts to automatically adjust areas of an image you are working on. This might let you add new features, adjust existing elements, or remove unwanted sections of the picture by typing your request into the app.

Read more
These 2 new Edge features are making Chrome look outdated
Copilot in Windows being used in the side panel.

Microsoft has announced a host of updates that will soon be available for its Edge browser, including the Microsoft 365 Copilot feature and Sidebar app support for developers.

The company is showcasing the new features during its annual Build developer conference, which is currently taking place from May 23 through May 25.

Read more
Nvidia GPUs see massive price hike and huge demand from AI
Nvidia RTX 2060 Super and RTX 2070 Super review

It feels like we’ve only just emerged from the debilitating graphics card shortage of the last few years, but a new report suggests we can’t breathe easy just yet. Could a new GPU shortage be on the horizon, or are consumers safe from a return to another nightmare scenario?

According to DigiTimes (via Wccftech), Nvidia is seeing a huge surge in demand for its chips due to the explosion in artificial intelligence (AI) tools like ChatGPT. Nvidia offers a range of graphics cards that excel at AI tasks, including the A100 and H100, and the company is reportedly struggling to keep up in the wake of such massive demand.

Read more