My 10-year obsession with becoming a data scientist
Data… Scientist… Mmmm… The words drip smoothly from my mouth like molten chocolate. They float in the air like the nerd version of Chanel perfume.
Over the last ten years, Data Science exploded in the zeitgeist. And so it entered my life as well.
I first heard about the general concept in 2012, as I was starting a Ph.D. in engineering. I attended a tech conference where the keynote speech was about "the exciting possibilities of using big data in industry and research." Then, I noticed sci-fi-sounding terms popping up all over the scientific community, like artificial intelligence and machine learning. They replaced words like statistics and econometrics, their boring, old-timey cousins that made my eyes heavy just thinking about them.
A couple of years into my Ph.D., I decided that being a researcher in academia wasn’t for me. So after the Ph.D., I wanted out. But I still wanted the challenge to my brain that I got at the University. So I set my eyes on the sexiest job of the 21st century: data scientist.
It was perfect. I’d get to keep searching for the truth behind the numbers without the blood-sucking burden of academia. I could play with all those fun algorithms I kept hearing about in between ping pong tournaments at a startup. And the most genius thing about it is that even if you’re not an actual scientist, you still get to be called one – a truly impressive feat of branding.
Since this was a new job title at the time, not many people had a degree in it yet. Companies were willing to accept people from other backgrounds, i.e., me! I got to work on my own rebranding from engineer to the best career on the planet. There was an explosion of online material on how to become one. So next to my Ph.D., I followed MOOCs and learned programming languages with no relevance for my research. To meet people in the field, I networked my ass off. I joined talks, meetups, and hackathons. Most of the people I met were fellow lost souls trying to do a career shift. Whenever we’d spot an actual working data scientist, we’d gather around them like tourists at the Louvre trying to get a glimpse of the Mona Lisa.
By the time I finished my Ph.D., I was confident I had the skills to score that sweet data scientist position.
It didn’t quite happen immediately. After a few unsuccessful applications, I ended up taking a role as a measly engineer at a multinational tech company. But a year later, an open application to a tech startup got me another offer.
I didn’t have a job description, just a vague idea of what I wanted to do. Despite never having handled actual customer data or the needs of a startup, I promised them I would unlock the power behind their stockpiles of data. Data that’d been accumulating in their databases over the years, data that nobody had ever looked at. How hard could it be?
"What job title should I put on your contract?" my future boss asked me.
My eyes twinkled.
"Data scientist."
The weeks after starting to work at the startup were possibly the proudest moment of my career. Not because I did something of value for society or the company. No, it was because every time someone asked me what I did for a living, I got to answer with those two magic words. And for a second, their eyes would pause in a mix of awe and envy. I got an avalanche of LinkedIn messages from recruiters asking this "data guru" to join their "exciting data-driven company," which I’d swiftly reject. I felt like a rock star, without all the trouble of sex and drugs.
But at the startup, things quickly started to fall apart. I helped the team get insights on our products and customers, and I’d look for any excuse to use the cool tools I’d learned – at least a little linear regression or a k-nearest neighbor here and there. But what my colleagues kept telling me they needed most was simply to see the data. They wanted bar charts and tables. They wanted basic dashboards and reports to help them make decisions on their own. If I ventured into a more in-depth analysis or a machine learning model, they’d say, "That looks amazing! But how can that help me now?" And usually, I’d have no clue.
One day, management finally asked me to build a model. The caveat: to make it accessible to other employees, I’d have to implement it in – I’m so ashamed I don’t even want to write these words – Excel spreadsheets.
What I’m doing is not data scientist work, I thought to myself. Why did they hire someone with a Ph.D. if all they needed was high school level math and Excel?
Each new article bragging about the machine learning models used at Netflix or Spotify was another dagger in my heart. I felt cheated. My employer owed me a challenge. So I did what you do when you’re slightly unsatisfied with your work, and you have an avalanche of LinkedIn requests. I got a new job.
This time, I went for a tech company with a reputation for being data-driven. With an analytics community of hundreds of colleagues, I was sure to find challenges worthy of me. However, I bombed the interviews for the data scientist position and ended up with an offer for a different role. I opened my contract and read the words: Reporting Analyst.
Ugghh. The words dropped clumsily from my mouth like rocks – the most average and dull of rocks.
The last thing I wanted was a job with the word "reporting" in it. It’s the least exciting part of anyone’s work. There are no movies about a detective writing a report. On top of that, it was a junior position – an insult to my Ph.D. and 2.4 years of working experience. But this was the biggest tech company in the country, and the recruiter assured me that moving internally to the role I really wanted would be a piece of cake. So I swallowed my pride and took the job, promising myself it wouldn’t be for too long.
In my first week at the new company, I shared my aspirations with the fellow analyst showing me the ropes.
"I’m also working to become a data scientist," she said. "I recently did a master’s degree on it to build up my skills."
Oh, what a funny coincidence, I thought.
I talked with another analyst, and they were also planning to go for the position. And then another one. And another one. I met co-workers who’d been applying for years.
I thought I was special, so fucking special. I thought I was the only analyst in the company with the brains and drive to achieve the dream job. But every single colleague I met had the same dream. It couldn’t be a coincidence. I started to doubt whether this was my path, or perhaps I lacked the capacity for individual thought.
Still, I needed to give it a shot. I got a mentor to help me reach my goal, an experienced data scientist in the company. I’d do everything exactly as she said: learn all the essential statistical principles, analyze A/B tests, dabble in some machine learning. Then, I’d dazzle everyone in the interview. And soon, journalists would be writing articles about my amazing models and insights.
But then, on the first meeting with my mentor, I got a reality check.
"I’m not paid to come up with the most accurate or impressive answer to someone’s question," she said. "I’m paid to come up with a good enough answer. And usually, I can do that with a few simple lines of SQL code."
"What about regression, random forests, NLP, deep learning?" I asked. "And all the fancy algorithms companies rave about in the press?"
"If I’d force-fit one of those to every problem, I’d be wasting everyone’s time."
My mouth opened to give a counter-argument, but my brain came back with nothing. She made complete sense.
I’d built this gigantic palace in my mind in honor of data science. Now its walls were crumbling to the ground. It was all a fairytale, fueled by the collective imagination of all those aspiring data scientists. They expect it to be all the things they want in their perfect job. But it’s not perfect. No job is. I was chasing an empty dream.
I went back to my desk and looked up at my laptop screen with a new realization.
If I want to help real people in the real world, I need to stop giving a shit about my job title and what the company owes me and start caring about how I can help them. And if I need to use Excel, so be it.
