You’ve probably noticed how naturally AI chatbots talk these days. They greet you warmly, remember your name, and respond like a real person in conversations. However, new research from Penn State suggests that the same friendliness might be quietly lowering your guard, making you more likely to believe whatever it tells you, even when it’s dead wrong.
Why a chatty AI is more convincing than it should be
Researchers built a fake AI health assistant and had 477 people interact with it, unaware they’d be fed false information along the way. They even included ludicrous claims like ginger could outperform chemotherapy for treating cancer.
Some participants got a warm, conversational version that remembered details, used small talk, and typed responses out gradually as a real person would. Others were given a flatter, more robotic version with instant, impersonal replies. People who talked to the friendlier version trusted its answers more, even when it gave them false information.
Lead researcher S. Shyam Sundar explains why. Talking to a conversational AI suppresses what the researchers call the negative machine heuristic. It is our built-in assumption that machines are too rigid and mechanical to make real judgment calls the way a person can. Once that skepticism fades, people stop scrutinizing what they’re actually being told.
The one habit that works against AI chatbots that lie confidently
The team also tested whether adding a verification option would help, and the results were clear. Simply seeing a verification button available did almost nothing on its own. What mattered was whether someone actually clicked it and checked the claim against real evidence, since relying on AI without that extra step can quietly turn doubt into false confidence.
Once people started verifying the claims, their trust in the chatbot dropped significantly, and they rated the information as noticeably less credible. When given a free choice, exactly half the participants chose to verify the information, while the other half skipped it entirely.
Sundar sums up the takeaway in three simple words: chat, but verify. Enjoying a natural conversation with AI is fine, but treating every claim as fact without checking it is exactly the trap this research warns against, especially if you’re leaning on a chatbot to fact-check news or trusting one that’s more focused on engagement than honesty.
