How to become a Machine Learning Engineer?
Recently I posted a job post for a machine learning engineer position in my team. I got a lot of questions from people who are curious to know more about the position itself, how to prepare for it and what kind of problems does it face ?
Data Science vs Machine Learning Engineering
Let’s start by defining the position itself. What is machine learning engineering? Is it a new thing? Does it differ from the so-called sexiest job in the 21st century, the data scientist?
When people invented the term Data Scientist, they were looking for superheroes who excel at the following :
1- Acquiring the data, cleaning it, identifying problems, and asking the right questions as well as setting the correct measures for different KPIs.
2- Doing necessary statistical analysis to come up with a different hypothesis about the solution
3- Using statistical methods to validate the different suggested solutions.
4- At some point, the solution could be more than a mathematical model (equation), it could include some complexity requiring the usage of machine learning. Thus, developing a suitable ML model through different iterations becomes necessary.
5- Once this model is ready, you need to design an engineering system that will serve this model in addition to other aspects that require careful engineering to ensure the continuous effectiveness of this model.
6- Do the necessary continuous analysis and investigations for the model results as well as the must-have monitoring.
Truly they exist on our planet but they are very rare ;)
Since human beings are limited in their ability to acquire knowledge and to keep up with the non-stop changes in each different sub-field, specialization became a must. We started to see different job profiles working on different areas of the pre-mentioned workflow.
For example, some data engineers/scientists who are more excited about infrastructure, specialized in data engineering. This gives them more bandwidth to absorb the internals of different distributed processing systems (ex: Apache Spark, Apache Hadoop, ..etc) and thus be capable of extending these systems to handle very custom use-cases that may not be easy to handle with the standard APIs
We can list 6 profiles that together compose a capable data team able to push its organization to be data-driven and then AI-driven at a later stage.
- Data Analyst & Data Viz
- Data Scientist
- Data Engineer
- Machine Learning Engineer
- Data Infrastructure Engineer
- Data Ops
Still, it is hard to generalize this to every single organization as the needs and size of each are the deciding factors while considering the boundaries between each profile under the data team umbrella.
So what is a machine learning engineer?
I will quote from the excellent article from Oreilly which describes them as the following :
- They have stronger software engineering skills than typical data scientists.
- Machine learning engineers are able to work with engineers who maintain production systems. They also write clean code.
- They understand software development methodology, agile practices, and the full range of tools that modern software developers use: everything from IDEs like Eclipse and IntelliJ to the components of a continuous deployment pipeline.
- They are involved in software architecture and design
- Because their focus is on making data products work in production, they think holistically and factor in components like the logging or A/B testing infrastructure.
- They are up to speed on issues that are specific to monitoring data products in production. There are many resources on application monitoring, but machine learning has requirements that go even further. Data pipelines and models can go stale and need to be retrained, or they might be attacked by adversaries in ways that don’t make sense for traditional web applications.
How to get the experience?
- Be a solid software engineer
- Get ML experience
- For the theoretical part, you can take any of the existing MOOCs on Coursera, Edx or Udacity. One option is Udacity Machine Learning Engineer Nanodegree.
- Get practical experience through doing real projects on real data. Kaggle is the best source for problems if you don’t have some already :)
- Read, Listen and Watch. There are dozens of awesome articles, podcasts and video sessions that are available on the internet.
For ex., here is a blog discussing my experience building our internal ML Infrastructure platform which shall give you a sneak peek into real world ML engineering experience.
Recommended resources
Blogs
- [Towards Data Science]
- [Learn With Passion] — shameless plug ;)
Make sure to subscribe if you would like to recieve posts about machine learning, data engineering and other topics. - [Machine Learning Mastery]
Podcasts
For theoretical background :
What do interviews look like?
It seems most companies will interview you the same way they interview other software engineers. In addition, you may be interviewed on your machine learning theoretical knowledge, data engineering and ML product design.
You can get more details by reading Xiaohan Zeng interviewing experience as a Machine Learning Engineer.
Where can I find suitable opportunities?
Most of the big companies including the big five have their own machine learning and data teams. It should be easy to find suitable opportunities given that you have the right experience. You can always start with Linkedin.
Wrap Up
I tried to give some answers for the what, where and how questions about machine learning engineering positions. It would be nice if you can share your experience getting into the field. As always thanks for reading!
Did you like what you read?
Follow me on Twitter @_ahmdkamal_, to get interesting updates about about #AI and #tech.
Do you know you can give up to 50 👏 to this post 😉 ?!
Clapping lets other people see it & encourages me to publish more high-quality content on medium!
Originally published at blog.ahmedkamal.me.
