Illusion of Confidence

Real knowledge is to know the extent of one’s ignorance.

Confucius

All of us have come across situations where less competent people seem to rate their competence higher than what it actually is. We have also seen very competent and knowledgeable people rate themselves lower. This is a result of genuine cognitive bias called the Dunning-Kruger Effect.

Cornell psychologists David Dunning and Justin Kruger proposed in 1999 that people who are incompetent at something are unable to recognize their own incompetence. They not only fail to recognize their incompetence but also feel confident that they actually are competent. The original paper was titled, “Unskilled and Unaware of It,” for which they won an Ig Nobel Prize in 2000.

This cognitive bias is problematical as people who are incompetent not only reach wrong conclusions, but more importantly they also lack the ability to realize their mistakes. Instead of being confused, perplexed, or thoughtful about their erroneous ways, incompetent people insist that their ways are correct. In a sense, they are blessed with an inappropriate confidence, buoyed by something that feels to them like knowledge.

As Charles Darwin said, “Ignorance more frequently begets confidence than does knowledge.”

David Dunning says “If you’re incompetent, you can’t know you’re incompetent… The skills you need to produce a right answer are exactly the skills you need to recognize what a right answer is.”

He elaborates further saying “For poor performers to recognize their ineptitude would require them to possess the very expertise they lack. To know how skilled or unskilled you are at using the rules of grammar, for instance, you must have a good working knowledge of those rules, an impossibility among the incompetent”.

When a 19-year-old thinks that she can win a national singing competition even though the only person who has told her that she is a good singer is her mother, there’s some real “Dunning-Krugering” happening.

Psychologist Steven Sloman, in his book “The Knowledge Illusion: Why We Never Think Alone” describes a series of experiments in which people were asked to assess how much they knew about the way various systems work — from toilets to single-payer health-care systems. People generally rated their knowledge of those systems as high — but then, when asked to explain in detail how those systems actually worked, most simply couldn’t.

In 2013, Professor Bryan A. Garner, an American lawyer, lexicographer and academic who has written more than two dozen books on English usage and style, advocacy, legal drafting and golf, published an article in the American Bar Association Journal entitled ‘Why lawyers can’t write’.

In his article, Professor Garner argues that most lawyers suffer from the Dunning-Kruger effect. He claims that not only are they unable to write, but they are completely delusional about their writing abilities. In other words, they think they are fabulous writers when they are actually hopeless.

As comedian Stephen Fry puts it, “the greatest enemy of knowledge is not ignorance, it is the illusion of knowledge”.

The Dunning-Kruger effect is not just a curiosity of psychology, it touches on a critical aspect of the default mode of human thought, and a major flaw in our thinking. It also applies to everyone – we are all at various places on that curve with respect to different areas of knowledge. You may be an expert in some things, and competent in others, but will also be toward the bottom of the curve in some areas of knowledge. You are as ignorant as the average person in every other area of knowledge in which you are not an expert.

Illusions of superiority are not always so mundane and can have real consequences.

Consider the anti-vaccination movement. A group of people with no medical or scientific qualifications are refusing to vaccinate their children for fear of them developing autism. Even though there is no scientific link between vaccines and autism, their erroneous opinions are so loud and convincing that they have caused the re-emergence of diseases that had been previously eradicated in the United States. Globally, the anti-vaccination movement has caused the resurgence of many treatable diseases. Unfortunately, it is a difficult battle to win given the people we are dealing with.

“It is hard to win an argument with a smart person, but it is damn impossible to win an argument with a stupid person”.   Bill Murray

Again, the key to the Dunning-Kruger Effect is not that people are uninformed; it is that they are often misinformed—their heads filled with false data, facts and theories that can lead to misguided conclusions held with tenacious confidence and extreme partisanship. An ignorant mind is precisely not a spotless, empty vessel, but one that’s filled with the clutter of irrelevant or misleading life experiences, theories, facts, intuitions, strategies, algorithms, heuristics, metaphors, and hunches that regrettably have the look and feel of useful and accurate knowledge.

The Dunning-Kruger effect and the knowledge illusion aren’t disorders, but are part and parcel of being human. Some people, however, are much more subject to these than others.

Interestingly, people have an easier time recognizing ignorance in others but fail to recognise their own. Each of us at some point will reach the limits of our expertise and knowledge. All those decisions that we take that require knowledge beyond these boundaries can be classified as decisions based on ignorance and are undetectable to us.

Dunning gives his own example.

“I have asthma, and just the other day I ran across a test on how to appropriately use an inhaler. I took the test just for fun because obviously I knew all the right answers. I’ve been using an inhaler for 15 years!”

But it turned out that he had been using the inhaler wrong for all that time. “I was breathing in heavily really quickly when you’re supposed to take breaths slowly. It was a shock to me!” he said. “I had been depriving myself of oxygen that was there for the taking. I’ve been feeling much better since I began doing it correctly.”

It so happens that the Dunning-Kruger Effect has some unfortunate corollaries. High-ability individuals may underestimate their relative competence and based on this, erroneously assume that tasks which are easy for them are also easy for others.

We need to appreciate that we live in a world of rampant misinformation in environments that cannot be so well controlled. The Internet, news media and social media make it almost impossible to decipher truth from fallacy.

Writing in his classic 1992 treatise The Flanshaw Infants on the potential of the world wide web, futurologist Terence Dobson wrote: “with too much information at their disposal, people (might) choose to take facts as given rather than question sources or open minds to the endless possibilities of knowledge and truth that the internet will provide.”

Another disturbing aspect is that we are not forced to face our own ignorance and ask for help as we can just look up in the internet for the answers immediately. This easy and frequent access to internet makes people to consider knowledge stored online as their own. This in turn gives rise to illusion of knowing with all its consequences.

dunning-kruger-2