Communications of the ACM of 2025-01 has an article on GPTs and Hallucinations - Why do large language models hallucinate?
This article is similar to Shanahan’s explainer on LLMs from 2022 (a paper not cited in the references). However, it contains additional anecdotes about experiments.
The experiments have been run on Llama, ChatGPT-3.5, ChatGPT-4 and Gemini from March 27, 2024 to April 29, 2024. However, we are not shown details or a tabulation of experimental input & results.
P.S. “Hallucination” is evidently not the right term, it should be “confabulation” as a “hallucination” is a internal impression perceived by (disturbed) consciousness, while a “confabulation” is a publicly visible message generated by an agent (maybe an LLM, maybe a student) that needs to fill in dots, and it does so with made-up information. Will people ever correct their wording?
For Murray Shanahan’s earlier work, see:
(Image of a friendly anthropomorphic Stochastic Parrot in a bar with a beer and a sign saying “ask me anything” generated by ChatGPT, or rather its image-generating backend, which should be DALL-E)
