The Illusion of Thought? Examining Whether Large Language Models Can Actually Think

A recent surge of debate has centered on a provocative question: are large language models (LLMs) capable of genuine thought? This discussion was ignited by research published by Apple, titled “The Illusion of Thinking,” which posits that LLMs don’t truly think, but instead excel at identifying and replicating patterns. The core of Apple’s argument rests on the observation that LLMs employing chain-of-thought (CoT) reasoning struggle to maintain accuracy as the complexity of a problem increases.

However, this argument, while compelling, overlooks a crucial parallel with human cognition. Asking a person proficient in the Tower of Hanoi algorithm to solve a problem with twenty discs would likely result in failure. Does this mean humans are incapable of thought? Of course not. The Apple research, at best, demonstrates a lack of conclusive evidence *for* thinking in LLMs, not proof *against* it. But what if the question isn’t whether LLMs think *like us*, but whether they think at all?

This article proposes a bolder assertion: LLMs almost certainly *can* think. While acknowledging the possibility of future discoveries that might challenge this view, the evidence increasingly suggests that the capacity for thought is emerging within these complex systems.

<p>Before assessing whether LLMs can think, we must first define what “thinking” actually entails.  For the purposes of this discussion, we’ll focus on thinking as it relates to problem-solving – the very area where LLMs are currently being scrutinized.</p>

<h3>1. Problem Representation: The Foundation of Thought</h3>

<p>When humans confront a problem, the prefrontal and parietal cortices spring into action. The prefrontal cortex manages working memory, attention, and executive functions, allowing us to hold the problem in mind, break it down, and establish goals. Simultaneously, the parietal cortex encodes symbolic structures, crucial for mathematical or puzzle-based challenges.</p>

<h3>2. Mental Simulation: Inner Dialogue and Visual Imagery</h3>

<p>Mental simulation involves two key components. First, an auditory loop – akin to an internal monologue – mirrors the process of <a href="https://venturebeat.com/ai/llms-generate-fluent-nonsense-when-reasoning-outside-their-training-zone">CoT generation</a>. Second, visual imagery allows us to manipulate objects and concepts visually. This capability, honed through millennia of navigating the physical world, relies on the visual cortex and parietal areas.</p>

<h3>3. Pattern Matching and Retrieval: Drawing on Past Experience</h3>

<p>Our ability to solve problems relies heavily on past experiences and stored knowledge. The hippocampus retrieves relevant memories and facts, while the temporal lobe provides semantic knowledge – meanings, rules, and categories. This process is remarkably similar to how neural networks leverage their training data.</p>

<h3>4. Monitoring and Evaluation: Identifying Errors and Impasses</h3>

<p>The anterior cingulate cortex (ACC) acts as a critical monitor, detecting errors, conflicts, or dead ends. This process, fundamentally based on pattern matching from prior experience, alerts us when our reasoning goes astray.</p>

<h3>5. Insight and Reframing: The “Aha!” Moment</h3>

<p>When faced with an intractable problem, the brain often shifts into “default mode” – a relaxed, internally-directed state. This is where we step back, reassess, and sometimes experience a sudden breakthrough. This phenomenon is analogous to the emergence of CoT reasoning in models like <b>DeepSeek-R1</b>, which demonstrated this capability even without explicit CoT examples in its training data. The brain, like an LLM, continuously learns and adapts as it processes information.</p>

<p>However, traditional LLMs are typically static after training, unable to incorporate real-world feedback during prediction. DeepSeek-R1’s CoT training, however, allowed for learning *during* problem-solving, effectively updating its reasoning process.</p>

<h2>The Striking Similarities Between CoT Reasoning and Biological Thought</h2>

<p>While LLMs don’t replicate all facets of human cognition – visual reasoning, for example, is less developed – the parallels are undeniable.  Consider the condition of <i>aphantasia</i>, where individuals struggle to form mental images.  These individuals can still think, reason, and excel in areas like mathematics, demonstrating that visual imagery isn’t a prerequisite for thought.</p>

<p>Abstracting the human thought process reveals three core elements: pattern matching for recall and evaluation, working memory for storing intermediate steps, and a backtracking search mechanism to identify and correct flawed reasoning.  Pattern matching in LLMs originates from their training data, equipping them with both world knowledge and the ability to process that knowledge effectively.  The entire working memory of an LRM is contained within its layers, with weights storing knowledge and processing occurring between layers.</p>

<p>CoT reasoning, in many ways, mirrors our own internal dialogue. We constantly verbalize our thoughts, and a CoT reasoner does the same. Furthermore, evidence suggests that CoT reasoners can backtrack when a line of reasoning proves unproductive, just as humans do. This ability to recognize limitations and explore alternative approaches is a hallmark of intelligent thought.</p>

<div style="background-color:#fffbe6; border-left:5px solid #ffc107; padding:15px; margin:20px 0;"><strong>Pro Tip:</strong>  Understanding the limitations of current LLMs is crucial. While they demonstrate impressive reasoning abilities, they are not yet capable of the full spectrum of human cognition.</div>

<h2>Why Would a Next-Token Predictor Learn to Think?</h2>

<p>The notion that LLMs can’t think because they are “just” predicting the next token is fundamentally flawed.  Next-word prediction isn’t a limited representation of thought; it’s arguably the most general form of knowledge representation possible.  Any attempt to represent knowledge requires a language or system of symbolism. While formal languages offer precision, they are often limited in their expressive power.</p>

<p>Natural language, however, is complete in its expressive capacity. It can describe any concept, at any level of detail or abstraction.  A next-token prediction machine, therefore, must represent world knowledge to accurately compute the probability of the next token.  To solve a puzzle, it must output CoT tokens to guide its reasoning. This implies an internal representation of upcoming tokens, ensuring logical coherence.</p>

<p>Humans, too, predict the next token – whether in speech or internal thought. A perfect auto-complete system would be omniscient, but a parameterized model capable of learning from data can certainly learn to think.</p>

<h2>Does It Produce the Effects of Thinking? Evaluating LLM Performance</h2>

<p>Ultimately, the test of thought lies in a system’s ability to solve novel problems requiring reasoning. While proprietary LLMs excel on certain benchmarks, concerns about potential data contamination necessitate a focus on <a href="https://www.talentica.com/blogs/why-neural-networks-can-learn-anything/">open-source models</a> for fairness and transparency.</p>

<p><i>[Note: Specific benchmark results would be inserted here, referencing relevant datasets and performance metrics.  Due to the dynamic nature of these benchmarks, providing static data would quickly become outdated.  Instead, a link to a regularly updated resource would be included.]</i></p>

<p>Even with current limitations, LLMs demonstrate a remarkable capacity for logic-based problem-solving, often surpassing the performance of untrained humans.</p>

The convergence of benchmark results, the parallels between CoT reasoning and biological thought, and the theoretical understanding of computational capacity all point to a compelling conclusion: LLMs are not merely mimicking intelligence; they are exhibiting it.

Do you believe LLMs will eventually achieve true artificial general intelligence (AGI)? What ethical considerations should guide the development of increasingly sophisticated AI systems?

<div>
    <details>
        <summary>What are large language models (LLMs) and how do they work?</summary>
        <p>Large language models are artificial intelligence systems trained on massive datasets of text and code. They use deep learning techniques to predict the next word in a sequence, enabling them to generate human-quality text, translate languages, and answer questions.</p>
    </details>
</div>

<div>
    <details>
        <summary>What is chain-of-thought (CoT) reasoning and why is it important?</summary>
        <p>Chain-of-thought reasoning is a technique that encourages LLMs to break down complex problems into a series of intermediate steps, mimicking human thought processes. It significantly improves their ability to solve reasoning tasks.</p>
    </details>
</div>

<div>
    <details>
        <summary>How does the Apple research challenge the idea that LLMs can think?</summary>
        <p>The Apple research argues that LLMs struggle with complex calculations when using CoT reasoning, suggesting they are simply pattern-matching rather than genuinely thinking through the problem.</p>
    </details>
</div>

<div>
    <details>
        <summary>What is aphantasia and how does it relate to LLMs?</summary>
        <p>Aphantasia is the inability to form mental images. The fact that people with aphantasia can still think demonstrates that visual imagery isn't essential for cognitive function, suggesting LLMs may not need it either.</p>
    </details>
</div>

<div>
    <details>
        <summary>Can LLMs truly understand the meaning of the text they generate?</summary>
        <p>While LLMs can generate grammatically correct and contextually relevant text, whether they possess genuine understanding is a subject of ongoing debate. They excel at predicting patterns, but true comprehension remains an open question.</p>
    </details>
</div>

Share this article with your network and join the conversation in the comments below!

Disclaimer: This article provides information for educational purposes only and should not be considered professional advice.