Cognitive Reflection Test - Examples of Reacting vs Checking
4 minute read | Dec 2, 2022
product, management, finance
The Cognitive Reflection Test (CRT) is a simple three question test that measures people’s tendency to answer questions with the first idea that comes to their mind without checking it.
The first and most popular question is the “bat and ball problem” which more than 50% of students at Harvard, MIT and Princeton get wrong.
1. The Bat and Ball Problem
- A bat and a ball cost $1.10.
- The bat costs $1.00 more than the ball.
- How much does the ball cost?
- 10 cents
Nearly all people have a first reaction of 10 cents which is wrong. The correct answer is 5 cents.
In a study of 3,428 university students, 50% of participants across Harvard, MIT and Princeton gave the incorrect intuitive response and did not actively check their response. A failure to check is instructive because it requires such minimal mental effort.
Anyone who would pause and check would recognise that if the ball is 10 cents the bat would need to cost $1.10 and the sum would be $1.20, which is above the target $1.10.
Daniel Kahneman describes this cognitive bias as the distinction between a person’s two processes of thinking:
- System 1: executed quickly with little conscious deliberation; and
- System 2: executed slowly and more reflective
System 1 processes occur instantly and effortlessly for example knowing the answer to 5 x 5 or indicating left when turning the car left.
System 2 processes require more effort, concentration and calculation for example the answer to 25 x 7 or correcting for car slides on icy roads.
2. How to Improve Your CRT Score
Here are three ways you can improve your CRT score and mitigate against biases of System 1 thinking:
- Acknowledge: that you will always have an initial “gut” reaction to an event or problem and jump to solution mode. This is unavoidable and it is ok. Note your initial reaction and solution;
- Pause and check: your first idea before sharing your feedback. Check who the decision would impact and how much effort the solution would require; and
- Ask questions: to find out who really has the problem and where the problem comes from. Check again whether your first idea would answer these questions.
3. Examples of Reacting vs Checking
Here are three examples of how product, customer success and marketing teams may react with their first idea and how they could pause and check.
|Example||React First Idea||Pause & Check|
Watching an end-user struggle to upload a file
|Propose feature solution of drag and drop file upload||Before solution mode:
Unhappy customer feedback that solution was not a fit with their internal system and they may now churn
|Blame sales team for selling to clients with poor system fits and create a new process to screen out incompatible customers||Before solution mode:
Prospect complains angrily they are getting too many marketing emails
|Disable this week’s new marketing campaigns||Before solution mode:
To learn more about how to pause and check, read about the 4 simple questions you can ask to help identify and solve the right problems before jumping to solution mode (How to solve the right problems).
- Shane Frederick (2005). "Cognitive Reflection and Decision Making". Journal of Economic Perspectives.
- Daniel Kahneman (2011). Thinking Fast and Slow. Chapter 3, The Lazy Controller