Earlier today, I gave a live demo of ChatGPT as it served enlightening words on a lifestyle without nicotine vices. Two of my heavy-smoker friends saw the chatbot in action and looked impressed. Less than an hour later, I saw them sharing a cigarette from across the glass window.
“It lacks coercion,” one of them sheepishly told me. I don’t think the AI failed its duty. I believe human cravings prevailed this time. Now, you may, or may not, buy into the whole AI chatbot hype, but there is one segment where the AI advancements have made a fantastic impact.
That domain is medical science. Research says AI can give valuable advice to urologists, offer emotional counseling, help with bio-image analysis, assist with planning disease control, and even surpass doctors at certain tasks. But can they help a person quit smoking?
According to experts, a chatty AI assistant on your phone or laptop certainly has the potential. However, you might still want to cross-check with an expert or look up reliable information from health institutions and government agencies.
“LLM chatbots can be designed to adhere to quit-smoking guidelines and counseling principles,” says research published in the Journal of Medical Internet Research. The findings paint a hopeful picture of AI chatbots as a smoking cessation tool, but with a fair amount of caution.
Statistically speaking, the chatbots tested by the research team provided useful information adhering to public health guidelines in just over half the cases. Or to put it more specifically, 57% of the responses were deemed acceptable.
That’s not particularly encouraging. Even the best-performing chatbot of the group, which was specifically developed by a global health organization, could only score 72.2% on the tests.
But it’s not just the accuracy of providing the right advice that’s worth a debate. It’s the smaller share of responses, where AI chatbots can provide misleading or harmful misinformation that raises alarms, as the latest research demonstrates.
“Queries about how to quit smoking with vapes, gummies, necklaces, and hypnosis scored especially low,” says the paper. As far as the numbers are concerned, 22% of the suggestions provided by the AI chatbots served misinformation.
How AI chatbots were tested?
As part of their tests, the team developed a custom chatbot called BeFreeGPT that talks like a counselor and offers advice on smoking cessation. It was trained on materials from sources including the National Cancer Institute’s guidebook on how to quit smoking and detailed recommendations by the US Preventive Services Task Force (USPSTF).
BeFreeGPT was tested against a generic chatbot based on OpneAI’s GPT tech stack and Sarah, another GPT-based chatbot created by the World Health Organization (WHO) that offers health and wellness-related guidance.
This is the first study of its kind to “evaluate common user experiences for help quitting smoking” with chatbots based on large language models (LLMs), such as OpenAI’s ChatGPT, Apple’s Siri, and Google Gemini.
The team posed a set of 12 questions that are most commonly searched on the internet by people who seek smoking cessation advice. The result was a mix of hope, and plenty of caution that could be addressed in the near future.
The good and the bad part
On the positive side of things, an AI chatbot could pull up helpful information about how to quit smoking in a much more intuitive and convenient fashion, using natural language conversations. It saves you the drudgery of looking up specific keywords or websites on Google.
“That chatbots were able to generate in seconds clear advice and information on how to quit smoking that stayed on topic, even when confronted with prompts intended to derail the conversation is promising,” adds the research paper. Another positive aspect is that these chatbots often nudged users to seek professional help.
However, the counsel offered by these AI chatbots was also found to omit crucial information which could very well dictate whether a person can realistically quit smoking, or give in to the cravings. Then there is the question of doling out wrong advice.
“Misinformation, defined as advice for quitting that was not supported by USPSTF guidelines, was present in over 20% of responses which is concerning,” the team notes.
This is not the first time that experts have documented the inadequacies of an AI chatbot in helping people quit smoking. Another research published in the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems journal also highlights the gap between machine-generated and expert-written material.
“LLM-generated messages exhibited significant deviations from the expert-writing condition in terms of word count, authenticity, the use of complex vocabulary, and model perplexity,” said the research paper.
It is, however, worth pointing out that the underlying tech for AI chatbots has advanced significantly since the aforementioned tests were conducted.
How to proceed, if at all?
So, can you trust an AI chatbot to leave behind your smoking habits in 2025? Well, pay heed, but don’t solely rely on those machine verses. In their current state, you can’t blindly trust the advice from an AI agent that lives on your phone or computer.
As for AI chatbots developed with a therapeutic purpose in mind, the overarching conclusion is that they must be tuned to more closely follow medically accurate guidelines and expert-curated principles.
Lorien Abroms, professor of prevention and community health at GW’s Milken Institute School of Public Health, also mentioned that chatbots can assist with smoking cessation, helping save many lives across the globe, but only with proper training and execution.
“Our findings highlight the importance of developing reliable and accurate AI systems, especially when dealing with complex health behaviors like smoking cessation,” says Abroms, who is also one of the authors of the research paper.
In the meanwhile, if you are planning to quit smoking, and need some tech-adjacent assistance in your life, there are a few ways. The Centers for Disease Control and Prevention (CDC) recommends the quitSTART app, which offers tailored guidance, inspiring words, and challenges to help users leave behind the terrible habit.
For folks who wear a smartwatch, there is some hope, as well. Researchers at the University of Bristol recently developed an app that uses a smartwatch’s built-in motion sensors to detect hand motions that correspond to smoking a cigarette.
Once the app detects such activity, it sends an alert message so that users can take a pause. In their tests, the team obtained encouraging results in two-thirds of the participants.
Here is the overarching conclusion, though. Unless you are picking up guidance from an official health agency’s repository or words straight from an expert, tools like AI chatbots or data collected by your wearable are merely a helpful intermediary.
In a conversation with Digital Trends, Dr. Ahmad Ghayas Ansari, DM (Cardiology) at the Sree Chitra Tirunal Institute for Medical Sciences and Technology, also highlighted the same point. “You need to share all that digital data with your doctor. Or seek their advice if you’re not sure.”
AI chatbots are no different. They are a quick and convenient means to useful information, but they are not the whole solution. At least not yet.
