The Illusion of Confidence

How we become victims of our own certainty

In 1984, a young lady named Jennifer Thompson was raped in her apartment.

The crime was committed by a black man who broke in and it went on for half an hour. During the debacle, she managed to turn the lights on a few times and tried her best to take note of what he looked like.

Fortunately, she managed to escape after a while. Unfortunately, there had been a similar break-in + rape incident on the same night. The MO was very similar, and the police quickly identified that the same person had committed both crimes.

Working with them, she shared everything she could remember in order to identify the rapist. Based on the sketch of the criminal, a few leads came in. One of the suspects was a guy called Ronald Cotton.

Thompson readily identified his picture out of a line-up of suspect photographs and was then told that he had a history of misbehavior. She also identified him in a live line-up, where all suspects stood in front of her and were asked to say some words.

She was confident that Cotton was the man. But no definitive physical evidence was offered in the trial. Moreover, the victim of the other rape could not identify Cotton.

Based on Thompson’s confidence, Cotton was found guilty and charged with a life sentence.

The accused and the victim (Source: The Scribe)

2 years later, he received a new trial when another prisoner, Bobby Poole, bragged to inmates that it was he who had committed the crimes. Interestingly, both men looked similar. But when the trial happened, Thompson couldn’t identify Poole and said: “I have never seen him in my life”. Strong words.

10 years later, Cotton’s lawyers insisted that DNA evidence be considered for this case- and this is when everything changed. The tests proved that the DNA of the rapist did indeed belong to Poole and that Cotton had been innocent all along!

What a disaster!

This, friends, was an illustration of what can happen when we are done over by an illusion that infects us all: the illusion of confidence.

To be fair, I am not blaming Thompson here. Both men looked similar and we can’t even know how the trauma of being raped must have affected her. If anything needs revamping, its the way the legal system arrives at conclusions that can drastically alter people’s lives.

[Side note: the two of them have reconciled since and work towards awareness and legal reform]

However, a broader point here is the fact that we all succumb to bouts of utmost certainty in various situations- only to be proven completely wrong.

Take this. A survey showed that 63% of American men think they’re smarter than average! If you didn’t understand the contradiction: it’s impossible for more than half the people to be above the average- but a much higher % thought they were in the upper half. The number was 70% for Canadians. Clearly, most of us suffer from delusions of being much better at things than we actually are.

The problem is worsened by the fact that it is generally the incompetent who are typically more overconfident. As Darwin observed, “Ignorance more frequently begets confidence than does knowledge”.

That’s bang on. In fact, I have personally observed that the more I read, the less certain I am about things and my understanding of them. More like imposter syndrome on steroids.

Conversely, I think I used to be much more confident about things many years ago, despite a much shallower perspective and knowing a lot less than I do now. We really suffer from the Unskilled-And-Unaware syndrome.

That also explains why thousands upon thousands rush to Indian Idol auditions with genuine confidence, thinking that they were born for this stuff. We all know the fate that befalls most of the overconfident participants.

So why are we all plagued by this tendency to have an exaggerated sense of confidence in situations where, in reality, our knowledge and abilities tend to be quite pathetic? Is there a genetic basis for this cardinal flaw?

Well, while we can’t point at genes to explain away all of it, they do play a role in determining whether someone inherits these traits. Frank Abagnale, the famous man behind ‘Catch Me If You Can’, was the son of a- wait for it- con artist. Not that a solitary example proves a point, but scientists do agree that confidence is heritable.

There’s another reason this must be happening, and it has got to do with evolution and the mental adaptations we developed when we lived in small groups in the jungle millions of years ago.

When you think of life back then, confidence does indeed seem like a very valuable trait. It acted as a decent proxy for people’s knowledge and abilities and it was essential to look that way to increase your chances of mating and survival.

Moreover, since everyone knew everyone in those tiny groups, people would also be able to adjust people’s claims against the data they already had based on past experiences.

For instance, if I knew that Suresh is generally very overconfident across a range of past events, I can use that understanding to arrive at a more balanced view of the claims he’s making in a new situation.

But that’s hard to do today in the modern world where we mostly meet strangers and have no idea about how under or overconfident they are across a variety of situations.

So this adaptation has become a bit problematic today because we continue to mistake people’s confidence for their actual skills- which is not a very good indicator as many experiences have shown.

For instance, the Georgian president was overconfident about going to war and beating the Russians in 2004 [This narrative has been contested, though]. His like-minded politicians shared the sentiment and the echo chamber resulted in them foolishly declaring war, which lasted for barely a week as the Russians punished them and inflicted a humiliating defeat.

They lacked the military experience needed to make it happen and yet were supremely confident of making quick work of Russian posts.

So let’s come to the gazillion-dollar question: what can we do about this illusion?

Researchers have shown that when you take people who are the worst at performing a certain task (basically the bottom percentile) and train them to do it better, their ability increases AND their overconfidence decreases! So improving one’s knowledge and skills certainly goes a long way in conducting a genuine appraisal of one’s capabilities across situations.

I also reckon that meditation could be another solution to chip away at this quandary. The illusion of confidence is a product of our brilliant capacity for self-deception.

It results from a lack of self-awareness: something that can clearly be improved via meditative practice. I have discussed some of the benefits in the past. Ridding yourself of your delusions and being honest with yourself is certainly high on the list.

More than anything, reading about how our wondrous minds work will help a lot. I had a good book on our best and worst behaviors listed in my recent recommendations article.

I hope that you’ll take a few moments to reflect deeply and think about situations where you may have, knowingly or unknowingly, succumbed to the illusion of confidence.

---

Thanks for reading! If you liked today’s piece, you can check out the first two parts of this series too: the Illusion of Attention and the Illusion of Memory.

If this article reminded you of a stupid friend who’s dead sure about her relationship choices only to end up with horrific disappointments, do share this with her. It’ll help.

And if you have any interesting thoughts, ideas or observations, I’d love to hear them. Share them by responding to this email (or commenting on the website).