DATA SCIENCE AND MACHINE LEARNING IN THE WILD
Interviews
In interviews you’re always told to "ask some questions". If you’re anything like me, you have probably got so caught up in trying to impress your interviewer that you’ve asked some pretty silly questions due to what I call "brain fry".
I once got so caught up I asked, "How long would it take for me to get to the office?" As you can imagine I didn’t land that job. I’ve heard things from candidates I’ve interviewed that are just as clumsy.
Look, there really are no silly questions, but in this game some questions help you more than others. So here are five questions that I bet you haven’t considered asking but you really should be if you’re interviewing for a Data science or machine learning job.
What’s the organisational maturity for machine learning?
This is important; you want to gauge the maturity of an organisation in machine learning. Consider this as a warmup question, you can probe further with this most of the time.
Mature organisations will probably start talking about existing teams, infrastructure and tooling, active projects. If you get a vague answer here, this might be an indication that you are walking into an organisation that is at the early stages of machine learning adoption.
It might be what you want, but you should understand what you’re getting into here.
Your job is almost certainly going to be much tougher if you’re the first one in, and I would raise a caution here if you’re particularly inexperienced. There is also the possibility that the company is recruiting analysts but calling them "data scientists" for marketability. In that case you can ask this follow up question to get more clarity.
Do you currently have any machine learning models in production?
A "Yes" here indicates a company that is mature. Get them to talk you through what they have built.
What infrastructure and tooling do you currently have for machine learning ?
You’ll want to know what you’ll be working with. This seems obvious when you say it out loud, but most people I have interviewed didn’t think to ask. Picture this, you’re interviewing with me for a Data Science job. Would you be happy if I told you that we mainly work with Office 2010 Excel spreadsheets and there was no appetite to change that? I’m guessing you wouldn’t be, so don’t wait until you’re hired to find out.
You can’t be a stellar machine learning engineer or data scientist if you don’t have the right tools available to you. Many of us chose this career path to make an impact, you’ll be limited if the right tools and infrastructure aren’t in place, and probably suffer frustration.
Think bigger than Python.
What you should be looking out for here is something on premises such as Hadoop or Spark, some suitable hardware for computational power or even better a cloud platform up and running.
AWS, Azure, GCP it doesn’t matter so much, any cloud platform makes machine learning a lot more feasible.
You’ll want a comfy place to build prototypes, so a labs environment is a necessity. This could be cloud based like Data Bricks, Amazon Sagemaker or an on-premises solution from the likes of Dataiku, SAS or Data Robot.
If the right tools aren’t there, you should find out if the company has a plan for getting those tools in place and who’s responsible for managing the whole thing. If they say you, ask for a lot more money.
What teams (or people) are in place to support machine learning?
Machine learning is a team sport, a mature company understands this. I would expect there to be a few key players supporting you including; data engineers, cloud platform technicians, business analysts, other data scientists and even a scrum master or project manager in some cases.
If the answer is, "you’re the first hire", get ready to start talking leadership and role expansion, you’ll probably need to request to hire the rest in at some point.
You can’t achieve much as a solo data scientist in most organisations, and it’s unlikely that you’ll have mastery of all the skills required to be Kanye West of ML.
Is there a strategy for ML at board or executive level?
Without support at the top level, you’ll find it hard to gain any credibility once you’re in.
ML has to be the "in-thing", and this is usually driven from the top down.
The worst-case outcome for you would be getting hired into a data science position where there is no support for it across the organisation. In this scenario you might end up being viewed as an "edgy-nerd" that has lots of unrealistic ideas.
If you’re lucky, some of your stakeholders might give you some problem statements that you can work out some prototypes off the back of. But even then, you’ll have a difficult time convincing the organisation to invest money in scaling it. Worst case scenario you are resigned to delivering MI and BAU reporting, which isn’t always bad, but probably not why you wanted to be a data scientist.
Don’t assume that because a company is hiring data scientists, that company is culturally ready for you.
If you’re thinking you’ll change the company culture by yourself (with math and logic Elon Musk style), you won’t…you just won’t. The exception to this is in small companies or start-ups with a very flat structure.
I’ve written about this in more detail here…
What funding is available to me?
If you want to make any impact it costs money. This is a key thing to note, to get machine learning working at scale is expensive and you’ll need funding to make it a reality. Organisations that have a budget allocated to innovation I have found are best. Machine learning is highly experimental, an innovation budget indicates a willingness to test and learn and accept failure (somewhat).
If the interviewer can’t give you a conclusive answer here, it’s probably a sign that the organisation is immature. They probably haven’t figured out how expensive it is yet. Beware here, they might expect you to deliver magic and will be disappointed in your inability to do so even if it’s due to lack of funding.
⭐️Woops I almost forgot, here are some bonus questions to ask. The last is one really a must ask!
- What is expected of me in the first six months?
- Is there a framework for moving from ideation to production? – This is specific to Machine Learning.
- Who will I be reporting into? – Only ask if this hasn’t been explicitly stated.
- Can I hire people?
⭐️ Where is my data coming from?
You can’t do any machine learning without access to data. Don’t assume that you will have access to all the data you need. If data pipelines and infrastructure are already in place great. If the expectation is that you must build these, you better have data engineering skills.
Data engineering is a deep skillset, so don’t naively assume you can do this alone.
You should really consider whether you want to work for a company with little to no data access for your own sanity… But if you do chose the dark path, know what you’re getting into.
I hope this helps you on your journey to the career of your dreams. If you take anything away from this, understand that asking considerate questions makes you look and sound like you know what you’re talking about.
⭐️ I love helping people by sharing my experiences of data science in the wild. If you’re not already a member, consider subscribing to Medium to receive more helpful content from me.
