Intro
I’d like to start off by painting you a picture.
Imagine you’re an experienced data scientist. You work for a small company and report into a team of directors who lead the company and are responsible for all the decisions made. Only proposals that get their buy-in can be implemented.
You’ve been studying an inefficient data process at work for the last month and you finally have a solution. It’s simple, elegant and costs nothing to implement. You’re confident in your solution, but you run it passed the other data scientists for some necessary sense checks. To your delight, they’re all on-board and now you’re brimming with confidence.
You simply can’t wait to get this in front of the directors; you write up a paper and put it forward to present at the next committee. You’re pleased with your new solution and you feel it should be a no-brainer, "surely they’ll be on board with this?" – you think to yourself.
Committee day comes and you present your paper, the presentation was flawless because you rehearsed it diligently. After you finish, you replay the presentation back to yourself mentally. A part of you feels that nobody in the room really understood the problem or the solution. You could have sworn you laid it out as simply as possible, but you can’t get the blank faces, head scratching and yawns out of your thoughts.
At the end of the meeting, you get an email from one of your directors thanking you for your work. Regrettably, they have decided to not proceed with your solution. You feel disappointed and are struggling to understand what went wrong. You feel you weren’t heard at all.
If this has ever happened to you, you might have found yourself blaming office politics or even yourself. But have you considered that what might be at play here is the Dunning-Kruger effect?
What’s the Dunning-Kruger effect?
The Dunning-Kruger effect is a psychological observation coined by psychologists David Dunning and Justin Kruger. They propose a relationship between experience/knowledge and confidence.
The heights of Mount Stupid: This is a state of blissful ignorance. Here you’re on top of the world and overconfident. In data science term’s you might have just built your first linear regression with some toy data. You are high on confidence and believe in just a few months you can be a top data scientist in your field.
Descending into the Valley of Despair: After a few weeks or days riding high on mount stupid something begins to dawn on you. It hit you when trying to build your first model with real world data, or when you encountered that problem that you hadn’t seen in your MOOC. You begin to realise the limits of your "knowledge". The feeling creeps up on you initially, but you sense that it goes deeper. To address the problem you read more books, take more courses, do more projects. To your despair the well is deeper than you thought.
This is indeed where many people will give up. In the valley of despair, things move slowly, and motivation is low and frustration eats away at you. After all, your confidence is at an all time low because your ego has taken a battering.
You’ve finally seen that you didn’t know how much you didn’t know. And worse, you have a long way to go.
Slope of Enlightenment: After what seems like an eternity wandering around in the darkness, you see a glimmer of light. An idea that you struggled to grasp becomes an intuition to you. You’ve now got a few projects under your belt and can see a solution to most problems you encounter. You don’t know when or how but one day things just started to click. Your persistence is finally paying off, you watch yourself grow in competence every day. Learning becomes fun and exhilarating and you welcome it with open arms.
Wisdom: Now you’re truly competent at what you do. Others in your domain recognise your expertise and you have the portfolio of work to back it up. You can look at problems at a high level and your experience guides you in finding the best solutions with relative ease.
You’re not boastful about the knowledge you’ve acquired and recognise that there is always something new to learn. At this new level of confidence, you have gained something that you didn’t have at the start – respect for the process and the journey.
The Dunning-Kruger effect at work
In the scenario outlined at the start, I asked you to imagine yourself as an experienced data scientist. It’s likely you’ll be at the wisdom end of the scale or at least approaching it.
Being the expert, you’ve been through the entire process. You will meet leaders that exist at all points of the Dunning-Kruger curve and it’s your job to know how to approach each of them as an expert. That’s if you want to get yourself heard and get your proposals taken seriously.
Here are my suggestions:
Don’t approach leaders a-top of Mount Stupid
You have to be realistic, people on mount stupid overestimate their knowledge of a domain. You probably won’t be able to convince them that they’re ignorant. I see three realistic options in this scenario.
Option 1: You can be patient and wait it out.
If you’re dealing with somebody that is willing to learn, eventually they will end up in the valley of despair. Here they’ll probably be more ready to recognise your expertise and maybe you can work with them.
The drawback of this approach is that some people are stubborn and unwilling to learn, you’ll be waiting on a person like that for a long time.
Option 2: Only approach these types of leaders with the elementary problems of your domain expertise.
They might understand these problems enough to give you their ear. You must be careful that the problem really is elementary though as often in business problems are more complex.
The drawback here is that you might get infuriated with only being able to implement basic solutions at work.
I’ve seen this time and time again at work. People that do this often feel their Careers stagnating and find they have a lack of meaningful projects under their belts.
Option 3: Cut your losses and change jobs.
If your leaders are content with remaining on mount stupid, then not much you do will get through to them. Don’t waste your time and energy trying to convince people that don’t want to be convinced.
Approach leaders in the valley of despair intelligently and with caution
A leader in the valley of despair might be overwhelmed, defensive and stressed so you’ll need to navigate skillfully here. You need to approach them with humility showcasing empathy for the position they are in. Sharing with them your experience and helping them to understand that you’ve been through the same process will give them extra reassurance. Your aim isn’t to be-little, but to give them what they need to understand and approve your proposal. This will require patience.
The most difficult thing about leaders in the valley of despair is spotting them. Unlike people on mount stupid, they are not proud or over-confident. They understand they don’t know and sometimes this might mean they hide this because of shame.
One way to spot these kinds of leaders is by listening to the questions they ask. Some questions demonstrate a lack of understanding of the subject that only an expert in it could recognise.
The good news is that leaders at this stage can be very receptive to learning. If you approach with caution, you’ll potentially be a catalyst for their growth in your domain of expertise. They will experience the rewards of the slope of enlightenment and begin to recognise and appreciate you for this.
If you’re fortunate enough to get to this stage, you’ll find you get a lot of buy-in with your proposals (on the condition they’re actually good). Your leadership team will have confidence in you potentially granting you a host of opportunities.
Final Thoughts
In my time working in financial services, I’ve seen this story play out many times. I’ve seen people try to push the same great proposals through the door repeatedly to no avail. Hopefully, this article will guide you to not make the same mistake.
What hasn’t yet been mentioned is self-awareness. It’s of the utmost importance that you recognise where YOU are on the Dunning-Kruger curve in any given domain. Becoming aware of where you are will guide you to act appropriately. For example, you will reign in your "confidence" when you recognise you’re on Mount Stupid.
Self-awareness of where you are on the curve will improve your ability to recognise where your leaders are, and therefore apply strategies to work with them.
Please remember not to be arrogant. Your leaders might not be experts in Data Science but they are likely extremely experienced professionals and at the wisdom end of the curve in other domains.
Finally human psychology is very complex but I’ve found that this simplifying framework has helped me to successfully direct my efforts at work.
John Ade-Ojo – Data Science | Tech | Banking & Finance | LinkedIn
