Nav and Shaw Identify Cognitive Surrender as Users Accept Wrong AI Answers

The 80% failure rate in the Nav and Shaw study

To quantify this surrender, Nav and Shaw conducted a test with 1,372 people. The participants were given a version of a cognitive reflection test, designed to measure whether a person can override a gut instinct to engage in deliberate, analytical reasoning. The researchers used “trick” questions that invite an intuitive but wrong answer.

One such example asks how many minutes it takes for 100 machines to make 100 widgets, given that 5 machines take 5 minutes to make 5 widgets. While the intuitive answer might be 100 minutes, the correct analytical answer is 5 minutes. Gizmodo Japan reports that participants were allowed to use an AI chatbot, which was programmed to occasionally provide incorrect answers.

The results were stark. Participants consulted the AI about half the time. When the AI provided the correct answer, 93% of users accepted it. More alarmingly, when the AI provided a wrong answer, 80% of users still accepted it. This happened despite the fact that the participants did not actually need to use the AI to solve the problems; they simply chose to prioritize the AI’s incorrect advice over their own judgment.

The emergence of a System 3 cognitive framework

The research builds on the work of the late Daniel Kahneman, who famously categorized human thought into two modes: System 1 (fast, intuitive, and emotional) and System 2 (slow, deliberative, and analytical). Nav and Shaw argue that AI dependency has introduced a third layer: System 3.

Our research results show that people are easily incorporating AI-generated output into their own decision-making processes without resistance or skepticism. This seamless engagement with System 3 highlights the potential to enhance everyday cognitive functions by reducing cognitive effort, accelerating decision-making, and complementing or replacing internal cognition with AI-driven insights that have vast externally processed resources. Nav and Shaw, authors of the research paper

Crime Wave in Bhopal, Navi Mumbai, and Nagpur Leads to Major Convictions

This new system isn’t just a tool; it is becoming integrated into the human cognitive process. However, the researchers warn that this integration comes with significant vulnerabilities. Just as System 1 is prone to systematic biases and prejudices, System 3 has its own specific cognitive flaws that pose challenges for individual decision-makers and society at large.

Confidence gaps and the trust trap

One of the most unsettling findings in the study is that AI doesn’t just lead people to the wrong answer—it makes them feel better about being wrong. Participants who used AI rated their level of confidence 11.7% higher than those who did not use AI, even when the AI’s answer was incorrect.

This creates a “trust trap.” In a human context, this is similar to relying on a highly educated, well-read friend. Because the friend is generally reliable, we stop verifying their claims, ignoring the fact that they are not omniscient and can be wrong or moody. AI mimics this dynamic but on a larger scale: it is more knowledgeable than any single friend, consistently confident in its tone, and available at all times.

When the friction of verification is removed, the tendency to surrender cognitively increases. The result is a loop where the user feels more confident in their conclusion precisely because they outsourced the thinking to a machine.

Combatting cognitive surrender in daily AI use

The existence of System 3 is now an undeniable fact of the AI era. The risk is not the use of the tool itself, but the loss of the questioning habit. To avoid total cognitive surrender, users must consciously reintroduce skepticism into their workflow.

The only way to maintain a healthy relationship with these tools is to treat every AI output as a hypothesis rather than a verdict. By consistently asking, Is this answer actually correct? users can leverage the speed of System 3 without sacrificing the analytical rigor of System 2. Without this habit, the seamless integration of AI may lead to a systemic decline in the ability to think independently.