Skip to main content

AI is now being trained by AI to become a better AI

An OpenAI graphic for ChatGPT-4.

OpenAI has developed an AI assistant, dubbed CriticGPT, to help its crowd-sourced trainers further refine the GPT-4 model. It spots subtle coding errors that humans might otherwise miss.

After a large language model like GPT-4 is initially trained, it subsequently undergoes a continual process of refinement, known as Reinforcement Learning from Human Feedback (RLHF). Human trainers interact with the system and annotate the responses to various questions, as well as rate various responses against one another, so that the system learns to return the preferred response and increases the model’s response accuracy.

The problem is that as the system’s performance improves, it can outpace the level of expertise of its trainer, and the process of identifying mistakes and errors becomes increasingly difficult.

These AI trainers aren’t always subject matter experts, mind you. Last year, OpenAI got caught crowd sourcing the effort to Kenyan workers — and paying them less than $2 an hour — to improve its models’ performance.

a criticGPT screenshot

This issue is especially difficult when refining the system’s code generation capabilities, which is where CriticGPT comes in.

“We’ve trained a model, based on GPT-4, called CriticGPT, to catch errors in ChatGPT’s code output,” the company explained in a blog post Thursday. “We found that when people get help from CriticGPT to review ChatGPT code they outperform those without help 60 percent of the time.”

What’s more, the company released a whitepaper on the subject, titled “LLM Critics Help Catch LLM Bugs,” which found that “LLMs catch substantially more inserted bugs than qualified humans paid for code review, and further that model critiques are preferred over human critiques more than 80 percent of the time.”

Interestingly, the study also found that when humans collaborated with CriticGPT, the AI’s rate of hallucinating responses was lower than when CriticGPT did the work alone, but that rate of hallucination was still higher than if a human just did the work by themselves.

Andrew Tarantola
Andrew has spent more than a decade reporting on emerging technologies ranging from robotics and machine learning to space…
This new text-to-video AI looks incredible, and you can try it for free
A woman wearing a yellow dress seated in front of a similarly colored floral background.

Expectant AI enthusiasts flooded the Luma AI website on Wednesday, resulting in multi-hour waits to access the company's new free-to-use, high-definition AI video generator, Dream Machine, Venture Beat reports.

What's all the excitement for? Well, the Andreessen Horowitz-backed startup's model promises video generation of up to 120 frames per second for as long as 120 seconds. And based on some of the examples being shared online so far, it's pretty impressive.

Read more
Prime Day 2-in-1 laptop deals: Dell, HP and more
The Dell Inspiron 14 2-in-1 laptop against a white background.

One of the more versatile options when it comes to the best Prime Day laptop deals are 2-in-1 laptops, and you'll find quite a few to choose from among the Prime Day deals going on right now, ahead of Prime Day which officially starts tomorrow. Brands like Dell, HP, and Lenovo are seeing some of their most popular 2-in-1 laptops discounted for Prime Day, and we've got the beat on the best Prime Day 2-in-1 laptop deals that you can shop right now. You'll find below our choice for the best 2-in-1 laptop Prime Day deal, as well as some other options if it's not the right fit for you. Prime Day deals are coming and going quickly, so if you see a laptop deal that works for you be sure to grab it while you can.
Best 2-in-1 laptop Prime Day deal
Dell Inspiron 14 2-in-1 laptop -- $450, was $700

The Dell Inspiron 14 2-in-1 laptop is a relatively affordable option that offers the versatility of this type of device alongside reliable performance. It's equipped with the AMD Ryzen 5 7530U processor, integrated AMD Radeon Graphics, and 8GB of RAM, which will be more than enough for daily tasks that you need to get done for work or school. The Dell Inspiron 14 2-in-1 laptop also features a 14-inch touchscreen with Full HD+ resolution for sharp details and bright colors, and it's got Windows 11 Home pre-loaded in its 512GB SSD that should provide enough storage space for your important documents.

Read more
Get a Lenovo IdeaPad 1 for just $172 ahead of Prime Day
The 14-inch Lenovo IdeaPad 1 with a shot of the sky on the screen.

We live in a world where the bigger, most expensive option is seen as the best, but that's not necessarily true. Sometimes a cheap thing that gets the job done is where it's at. And that just might be the case for this laptop. A Prime Day deal has started just a little bit early on this laptop, the Lenovo IdeaPad 1. Now $172, usually $300, you can save $128 on this laptop if you act now. We think it's the perfect laptop for taking with you wherever you go or a great distraction-free laptop to finally get some writing done on. Tap the button below to see it for yourself or keep reading to see our take.

Why you should buy the Lenovo IdeaPad 1
The Lenovo IdeaPad 1 is a laptop that will help you browse the web and get to working. It's got a 14-inch HD screen, 4GB of RAM, and 64GB of hard disk storage space. As a result, the usual distractions do not apply. It's actually going to be hard to pull out multiple programs and get lost in games and apps and movies on this computer. For the right kind of customer (you know who you are) this level of tech is truly a feature.

Read more