Despite all of the excitement around ChatGPT and similar AI-powered chatbots, the text-based tools still have some serious issues that need to be resolved.
Among them is their tendency to make up stuff and present it as fact when it doesn’t know the answer to an inquiry, a phenomenon that’s come to be known as “hallucinating.” As you can imagine, presenting falsehoods as fact to someone using one of the new wave of powerful chatbots could have serious consequences.
Such trouble was highlighted in a recent incident in which an experienced New York City lawyer cited cases — suggested by ChatGPT — that turned out never to have happened. The lawyer may face sanctions as a result of his action.
Another incident received widespread attention in April when ChatGPT apparently rewrote history by saying that an Australian mayor had been jailed for bribery while working for a bank when in fact he’d been a whistleblower in the case.
To make its chatbot technology more reliable, OpenAI engineers have revealed that they’re currently focusing on improving its software to reduce and hopefully eliminate these problematic occurrences.
In a research paper released on Wednesday and picked up by CNBC, OpenAI said that chatbots “exhibit a tendency to invent facts in moments of uncertainty,” adding: “These hallucinations are particularly problematic in domains that require multi-step reasoning since a single logical error is enough to derail a much larger solution.”
To tackle the chatbot’s missteps, OpenAI engineers are working on ways for its AI models to reward themselves for outputting correct data when moving toward an answer, instead of rewarding themselves only at the point of conclusion. The system could lead to better outcomes as it incorporates more of a human-like chain-of-thought procedure, according to the engineers.
But some experts expressed doubt about the work, telling CNBC it’s of little use until it’s incorporated into ChatGPT, which in the meantime will carry on hallucinating. OpenAI hasn’t said if and when it might incorporate its work into its generative AI tools.
While it’s good to know that OpenAI is working on resolving the issue, it could be a while before we see any improvements. In the meantime, as OpenAI itself says, ChatGPT may occasionally generate incorrect information, so be sure to confirm its responses if they’re part of any important tasks.
