Acing Machine Learning Interviews

Four key sections each machine learning interview has— preparation guide and resources

Image by the author.

Many of my peers from my PhD days are now looking for the next place to move. And knowing all the struggles with finding the first job, I am actively helping them prepare. Over time I have accumulated a large corpus of the interview feedback, and in this article, I summarise relevant sources for machine learning interview preparation.

There are four roles for someone looking for a job in a machine learning field: a Research Scientist, an Applied Scientist, a Research Engineer and a (Development) Engineer. Despite their differences (check out here and here what each of the roles might entail), all these roles have a lot in common when it comes to interview preparation. Generally, interviews for all four positions would have four sections targeting coding, machine learning theory, system understanding and soft skills.

List of useful resources mentioned in the article is summarised at the end!

Coding

Nothing could be easier to practice than coding — solve as many problems as possible, and you will be safe. Generally, there are two types of coding questions — live coding and home task.

Live coding

For ML positions, it is not a general practice to ask about heavy algorithmic problems. It is doubtful that you would be asked to implement a red-black tree… However, general knowledge questions are likely to incorporate some aspects of a day-to-day ML job.

The coding part does not test only your ability to solve problems, but your thinking process and ability to communicate with the interviewer and write clear usable code. So even if you know the solution and can write it straight away, first spend some time explaining your reasoning and put effort into thinking about the structure of the code to make it clean and easy to parse.

If the problem is not straightforward, ask clarification questions and start with the brute force solution. The next step is to outline the pseudo-code for the solution — either with function declaration and comments or just comments. This would make it easier to see the bigger picture. You can then go through the comment sections filling in the solution.

Generally the best way to prepare is to solve 50–100 questions on leetcode. Big advantage of leetcode over other platforms are the detailed answers and question stats — question difficulty and what companies asked it. Another helpful resource is Cracking the Coding interview book.

Homework task

It is common to get a homework coding/engineering task that would be directly testing your domain knowledge and skills on a real problem. Typically you would be given a dataset and a set of instructions. You would either be asked to allocate 4–6 hours (the countdown would start the moment you receive the task), or you would be given a bigger time interval of several days to a week to solve the problem.

To prepare for this kind of question, a good resource is kaggle. Where you can both practice data science problems and lookup solutions from others. The same principles apply to live coding — clear, documented and reusable code with justified algorithmic decisions. If there are papers that you rely on, make sure to include them as well.

Tailor your solution to the job requirements — if the company is a high-frequency trading fund, it would be unwise to use slow deep learning algorithms — simple perceptrons would usually suffice. And on the other hand, satellite imaging companies would expect knowledge of deep models for vision.

Homework would typically be discussed during the next interview stage. You will have a chance to justify your decisions and think about how the solution could be improved.

Question examples:

1. Write an image/text classifier.
2. Given a noisy time series of electricity consumption, detect when light in the kitchen was switched off.
3. Given data, extract features and build a clustering model.

Machine Learning Theory

Typically there would be questions on fundamentals — linear algebra, calculus, probability theory and general machine learning knowledge and also domain knowledge in the area of expertise. For research positions, you would likely be asked to present your current research or read and discuss a paper.

Fundamentals

Linear algebra: least squares, eigenvectors, eigenvalues, rank of the matrix, principle component analysis, singular value decomposition, positive definite matrices, orthogonal and orthonormal matrices;
Probability: random variable, variance, expectation, likelihood, posterior, Bayesian Inference, sampling, distribution, probability density function;
Calculus and optimization: convexity, regularization, integration, chain rule for differentiation;
Machine learning: bias-variance tradeoff, support vector machine, perceptrons.

While preparing this article, I found this interview Q/A book for machine learning positions, which has a solid list of questions with answers to them. Another excellent book, Machine Learning Interview preparation book, has a lot of good interview behaviour and salary negotiation sections and technical questions. Also, I found this blog that summarises feeds from glassdoor on machine learning interviews.

Question examples:

1. What is batch normalisation? What are its drawbacks? Describe normalisations that rectify them.
2. Explain bias-variance trade-off.
3. List common generative models and their pros and cons.
4. Why is it difficult to train vanilla GANs? How can you improve the training.
5. How would you prevent a neural network from overfitting?
6. Explain how to apply drop-out. Does it differ for train and test?
7. Give examples of neural-network optimisers, how do they work? What is gradient explosion? How to rectify it with activation functions?
8. Why neural networks might not be suitable for a given task? (check out this article)
9. Explain attention mechanism in transformers (these are now a general framework that can be applied to a wide range of problems).
10. Describe a dimensionality reduction technique of your choice.

Area of expertise

Most often, the job would assume that you specialise in an area. Depending on the position, you would need to know either the seminal papers or the seminal papers and state-of-the-art models that came out a month ago.

I found the methods section of paperswithcode very helpful for revising the core papers when preparing for my interviews. The most helpful feature of the section is summaries of the papers with images and basic explanations.

CV Question examples:

1. Why CNNs are so good when applied to images?
2. How to apply deconvolution?
3. Why transformers are challenging to apply to images?
4. How would you map emotions from one face to another?
5. What is temporal consistency and how to achieve it?
6. What loss functions can you use for a generative model?
7. Why FID score might is not the best realism metric?
8. Describe convolution types and the motivation behind them.
9. What is instance normalisation?
10. Describe loss functions for image reconstruction and their pros and cons.

NLP Question examples:

1. Why do we need positional encoding in transformers?
2. Describe several attention mechanisms, what are advantages and disadvantages?
3. What is the advantages of LSTMs versus vanilla RNNs?
4. What would we expect from a good embedding?
5. How to evaluate translation quality? Why BLEU is not a very good metric?
6. What is contrastive predictive coding and why is it helpful?
7. What techniques for NLP data augmentation do you know?
8. How to learn rare words when training models?
9. Explain how language can be viewed as a graph. What properties would the graph have?

System understanding

System understanding problems would be testing your approach to a typical problem encountered by a company. For example, you might be asked to list steps to design a recommendation system for an advertisment managing company; for an imaging company, you might be asked about the steps to create a new filter; for a company in Computer Graphics, you would be asked about an algorithm that could reconstruct a normal map of an image.

Question examples:

1. How would you tell which one of two models is better?
2. How would you design a video in-painting/face transfer/normal map reconstruction (this is from computer graphics)/spam detector algorithm from scratch?
3. How to deal with data-drift in the deployed model?
4. How to design an algorithm, that given an image of damaged fence would estimate the insurance cost?

Presenting

In the interviews for research positions, you would be asked to prepare a presentation on your research or talk through one of the recent papers from a relevant conference.

In each case, structure the presentation so that it is tailored to the position you are applying to. And in each of the cases, make sure that you are aware of the wider area and know most of the recent publications and follow up works. I have a guide on how to make a strong presentation here.

Soft Skills

Soft skills are tested in each of the stages of the interview process — how you communicate with an interviewer, how you talk through the coding solution, and how you react to notes, criticisms and challenges. However, there is almost always a separate section that would be testing the behavioural part.

Everyone interviewed by Amazon likes clarity about the values they expect in their employees. Overall, I rank their interview preparation guide as one of the most honest and relevant when it comes to behavioural and soft skill questions. You can find examples of questions testing each of the underlying soft skill values here and here.

Several frameworks can be used to answer principle/behaviour based questions. Most of them fall into the Situation Target Action Result (STAR) framework. Always have a few examples at hand.

After interviewing a few dozen candidates myself, I have realised that it is better to hire someone with slightly inferior technical skills but strong communication and team skills. I have summarised some of these skills in this article. You might want to check it before your interview.

Miscellaneous

Knowing your interviewers

Before the interview, you would be typically given the names of your interviewers. Make sure to look them up and read up on their area of expertise. Knowing a few core papers in their field and the perspective with which they look at machine learning would help you target your answers better.

Attitude

From my experience, attitude is as essential as knowledge and skills. I am a huge fan of two books — Thinking Fast and Slow and Outliers. The first helps you understand common traps that your mind can lure you into, and the second sets the right mindset.

Although very hard in the moment, especially if you are pressed with finding a job, treat the interview as a learning experience rather than a test — this would help you be more open and receptive in the process.

Brain freeze

I have not met a single person that has never experienced a brain freeze during the interview — your brain goes numb, hands are shaking, and even familiar questions become challenging.

There is an easy treatment that worked for me multiple times — instead of trying to hide the freeze, admit it. Say aloud to the interviewer that you are just about to catch a brain freeze and worry that even simple things would stop working out. With that trick, you would lift the pressure of trying to hide the brain freeze and give a signal to the interviewer for support. After all, they want you to perform well as well!

Questions

An interview is a two-way street; use the interview time to learn more about the role and the team. You will be given an opportunity to ask questions — do not just ask, “When can I expect to hear back?”.

Summary

Interviewing is a skill which, unfortunately, one acquires through the painful experience of rejection (or in a safer environment of mock interviews which I strongly encourage you to practice). It is not uncommon to send out hundreds of applications and have tens of interviews to get hired. It is thus better to leave companies that you like the most towards the end so that you are well prepared for the interview. Below I leave a summary of the links I have mentioned in the text:

Soft skills: Amazon interview preparation guide, principles Amazon expects in their employees, Amazon principles explained, Situation Task Action Result technique, soft skills from a machine learning PhD;
Coding: coding interview preparation leetcode, Cracking the Coding interview book, practicing machine learning problems;
Machine learning theory: Machine Learning QA book 1, Machine Learning QA book 2, summary from glassdoor, when not to use machine learning, methods section of paperswithcode, github repo with an excellent set of interview questions

If you liked this article share it with a friend! To read more on machine learning and image processing topics press subscribe!

Liked the author? Stay connected!

Have I missed anything? Do not hesitate to leave a note, comment or message me directly on LinkedIn or Twitter!

Nine tools I wish I mastered before my PhD in Machine Learning

Three soft skills every PhD student gets for free