It is a challenging transition and requires a tremendous effort while AI/Machine Learning is still a buzzword to organisations and data team is usually in the first way of layoff¹. On the other hand, the AI world is evolving rapidly, with breakthroughs from AlphaFold, Text-to-Image (Stable Diffusion/Dall-E 2), to Multimodal Machine Learning and AI adoption continues to grow and the benefits remain significant².
I transitioned to a Machine Learning Engineer in two and a half years and have been an industry mentor for Master of Business Analytics candidates since 2020. The traps and tips shared in this post are based on my experience and insights from alumni, colleagues, and speakers across industries.
Hope my story will help inspire your transition and guide your decision.
Prior to career transition
I started my career as a system integration engineer at a multinational telecom company and ended up as an advanced technical project manager before I decided to quit and joined a start-up as a co-founder to build a fintech SaaS solution.
A new journey
My family moved to Australia in 2017 and like many new immigrants, it means both challenges and opportunities to review your career plan. I have been following the AI and Cloud evolution since 2013 and am a believer in the Cloud and AI as they have started to reshape industries across the board and this is just the beginning.
Gaps
The knowledge gap is obviously huge so part-time learning was not considered. I was looking for a transition that is intensive, practical, hardcore, close to business and no more than 1 year.
Knowledge gap, prior to the master of business analytics degree
Full-time Master of Business Analytics at Melbourne Business School (MBS) is the best fit:
- Hardcore and practical courses – statistical learning, predictive analytics, decision-making and optimisation, machine learning, programming and how to apply data analytics in business.
- Intensive and fast-paced, the pace was 3x faster compared to a normal program, pushing you to your limits.
- A 5-weeks team internship that integrates academic learning, and practical challenges in implementing data analytics in an organisation.
- Extraordinary and smart candidates, most of the cohort start their data career in well-known consulting firms, financial institutions or Fortune 500 companies.
The program builds a solid and wide spectrum of data analytics foundations for your career, not to mention a powerful alumni community as many alumni have started to become industry leaders.
Business schools across the globe are offering master of business analytics, from Imperial College Business School, United Kingdom to MIT Solan, United States, [check the latest ranking](http://high academy).
If you are looking for an intensive and high-quality transition and already have business domain knowledge, this path may fit you well as a starting point.
Industry gap, post the master of business analytics degree
I started my data career as a Data Consultant at one of Australia’s leading data consultancies and it did accelerate my career in the industry and bridged the gap between "structured and static data in Jupyter Notebook" and "unstructured and massive data in the cloud".
Apart from the experience gained from client projects, cracking certifications/online courses is an efficient way to gain rich industry knowledge and practices. I cracked 19 certificates/online courses in 2 years and here are some certifications highly recommended:
Cracking a certification doesn’t mean you are a subject matter expert in that domain so don’t overrate its value. However, those certificates did help build a holistic understanding of Modern Data/Machine Learning Solutions. With strong cloud engineering and data analytics skills, I later transited from a Data Consultant to a Machine Learning Engineer.
It is not about certifications, it is about awareness of your knowledge/experience gap in the industry and being a fast learner to narrow it with your own strategy.
Traps:
Notes: Insights shared in this section may be biased.
- Title Inflation: Many Data Scientist titled jobs have nothing to do with modelling, model analysis or feature engineering but are Data Analyst oriented (dashboard reporting, ad-hoc SQL query etc). Candidates with a strong data science background (especially PhD candidates) may feel disappointed and lost in those positions. Being a Data Analyst is a very attractive career if you have strong domain knowledge and are close to the business units. But I assume most graduates from Data Science or PhD would prefer Machine Learning modelling/implementation— Be aware of the job scope in reality and manage your expectation well.
- Machine Learning modelling work is rare, organisations powered by Machine Learning usually implement mature industry-level models/solutions. Data engineering, system design and software development skills are far more important than building your own model (There is a low chance your team’s model can beat a cutting-edge model from big techs who are fuelling an AI arms race). Instead, as a Data Scientist or Machine Learning Engineer, your main responsibilities are: improving data quality (along with the data engineer team); identifying use cases (along with business stakeholders); Designing and implementing a cloud-based Machine Learning end-to-end solution (along with DevOps, Software Engineer) to prove the use case – Moving away from Jupyter Notebook and learn software development practices.
- AI 101 education on stakeholders is essential, not because they are the project sponsors but more importantly, they need to understand the limitations of AI instead of treating it as a magic technology to resolve everything – In many cases, you don’t need a fancy machine learning model and businesses need explainable AI.
- The data team is usually in the first wave of layoff especially the Data Analyst, Data Scientist and Machine Learning Engineer while Data Engineer is relatively safer – Data Scientist is probably not "The sexiest job in the 21st century", not for now at least, unfortunately.
Tips:
Notes: Tips shared in this section may be biased.
- One tip to avoid the trap of title inflation is to ask whether this position is Machine Learning productisation related or in-house reporting related during the interview.
- Cloud knowledge (AWS/Azure/GCP) is highly desirable and has become a must-have regardless of your job role while most data science/analytics programs are not touching it. It is a big plus for fresh graduates and juniors.
- Be honest with your own strengths, weaknesses and plan your data career from Day 1.
- Personal branding matters and knowledge sharing is caring – Building your influence start with sharing your knowledge, as early as possible.
- The industry is heading toward the Data-Centric AI approach³, and data engineering and MLOps will be even more critical. Garbage in, garbage out. We recently improved the performance of one model in production by 8% – By simply fixing inconsistent data annotations in the training set.
Below is a list of books/blogs that assisted my transition:
Technical skills
- Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow by Aurelien Geron
- Designing Machine Learning Systems by Chip Huyen
- System Design Interview – An Insider’s Guide: Volume 2 by Alex Xu
- Practical Statistics for Data Scientists by Peter.B, Andre.B & Peter.G
- Designing Data-Intensive Applications: Big Ideas Behind Reliable, Scalable, and Maintainable Systems by Martin Kleppmann
- AWS Solutions Library by AWS
Soft skills
- Storytelling with data by Cole Nussbaumer Knaflic
- The Pyramid Principle by Barbara Minto
- Insights on Artificial Intelligence by Quantum Black
Final thoughts:
Digital transformation and AI adoption are still in their early stage and the high failure rate of AI projects is not new. Unlocking the values of data and enabling a successful ML project requires an investment in strong foundations, especially use case alignment, culture shifts, AI education and process change management which is actually more important than implementing AI technology itself. For people who have business domain knowledge and later transit to a data career, besides your technical skills, building "Analytics translator" skills can make you more competitive and they are more critical to a successful ML/AI project as I have observed.
Enjoying my content? Please consider becoming a member. Your support fuels my work. Thank you!
[1] Reported at IBM, Uber, and Airbnb and Sejuti Das’s analysis "How Data Scientists Are Also Susceptible to the Layoffs Amid Crisis," Analytics India Magazine, May 21, 2020, https://oreil.ly/jobmz
[2] The state of AI in 2021
https://www.mckinsey.com/capabilities/quantumblack/our-insights/global-survey-the-state-of-ai-in-2021
[3] A Chat with Andrew on MLOps: From Model-centric to Data-centric AI