Office Hours

A technical takehome in my experience usually happens after a couple of rounds of interviews or after the interview with the hiring manager. The motive is different for every company, but it is an obstacle that will keep you from advancing to the next round of the interview if not performed to the company’s standards. As a data scientist, usually, the takehome is building a model or analyzing some data visually, statistically. You are also expected to provide recommendations based on the data you analyzed and/or the model you built. They are very different than Software Engineer technical takehomes since they are more data-focused versus building an application or explaining software fundamentals.
This is an opinion piece based on my personal experience. It is not a research study where I perform a regression analysis to infer how useful a technical takehome is for the technical hiring process for a company or the interviewee. Now having been on both sides; I can understand both of the viewpoints better. I have taken over a dozen technical takehomes over the course of the last five years and also reviewed over a dozen technical homework myself when I was hiring.
There is no golden standard on how we should measure and test Data Science knowledge. From what previous similar standard testing has shown us is, standardized testing can be biased towards gender and income.
However, even if there is no standardized testing in Data Science, companies should be cautious when creating coding/knowledge tests for everyone so that outcomes are not rooted from bias. The only way to mitigate that, is to be open to hire at all levels.
I do believe there should be an involved vetting process for those mid-level jobs to discern the potential and one’s capacity to perform the work; however, it is not necessary to be as intense as some currently are. Those intense and lengthy interview technical homework can be a big headache for both parties and in the end, might be filtering out some good candidates.
Some pros to the company for crafting technical interviews/takehomes would be :
- Better understanding the proficiency of someone’s coding capabilities.
- Seeing how they solve a problem / ask for help or clarifications.
- Weed out the imposters. (Although I am not a fan of those job listings that big companies tend to publish to cast a wide net, since it’s been shown most women do not apply if they don’t fit all qualifications). I am mostly referring to those who said they used the tool or performed the analysis and when it came to explaining it, they fell short which I have been a witness to.
Now the pros for the interviewee are:
- If it’s a synthetic sample of the company’s data, it gives you a better understanding of the data you would be working with and what are some of the issues.
- Ability to show off your knowledge and problem-solving skills.
However, there are a lot of cons when it comes to these types of take-home technical homework for the interviewee:
- They are exorbitant tasks. I remember one startup asked me to connect to their database, explore a trend series data over 2M rows, and create a business case study considering churn. They said it would only take 4 hours. It took me over 20 hours…
- It feels like free unpaid labor sometimes. Piggybacking on the previous example. I asked their Data Scientist what are some of the types of problems they work on, and they said exactly this technical homework assignment. I had a limited time but they had a long period of time to sort through.
- They are exhausting. I remember when I was interviewing, I had 1–2 of these to do every couple of weeks for three months and it was exhausting. It felt like I would take 5x the recommended time they said it would take because I did a thorough investigation to the data, transformed the data, performed visualizations, and built a model. I can see how someone who is a parent might not be able to devote all that time to these technical takehomes without help on other fronts.
The cons for the company is:
- Takes time away from your team to evaluate the homework on top of interviewing the candidates
- You don’t know if people copy code and do not understand what it’s doing. I am not against using stack overflow for debugging errors or articles for inspiration, but I remember myself early in my data science career wanting to land a job I was not ready for, tweaking code I did not fully understand.
- The company might be hiring the same type of person. Depending on how extensive your interview is, you could be winding up with the same type of candidate. I went through a very lengthy infamous interview process at a well-known credit card company and what followed after the recruiter, hiring manager, technical takehome, was a 5-hour all-day dissecting case study. It made me think, what are the type of people they are hiring because with a process that contained so many obstacles acts like filters, further and further filtering.
So the advice to mitigate this would be:
- If you have to have a coding exercise. Instead of having the person send the code over, have them explain and walk through their work with you. This will kill two birds one stone. One, it will provide the interviewee with some feedback on the work provided which is a huge gap in the interview process. As I said, some of these technical takehomes take a lot of effort from the interviewee and there is little validation or constructive feedback even despite moving on to the next round. I can’t explain how many times, I did not move on to an interview process after the technical takehome and I would beat myself up about it when in reality they might not have moved me to the next round for a different reason. Companies stray from giving a reason on feedback for legal reasons when they send that email for not advancing further with your application, but if you are having the person walk through their code and explain, I know personally I would have gauged if I did a good or bad job more accurately. Second, if you are having someone walk through their technical takehome, you save time trying to figure out what they were doing or running it yourself, it will take 30 minutes to max one hour during the interview time, and the interviewer will have a pretty good idea if they would like to move on with the candidate or not.
- Instead of having technical homework, ask them to explain concepts or a process in depth. Ask particular questions. Not questions like "What is Linear Regression", but "Explain how you built your last regression model and what were your findings and how you validated it was an appropriate model for your data". Or "Talk through a scenario where you had to transform the data to fit your model or how you vetted models appropriate for your data?"
- Have them walk you through some previous code if they have maybe on their Github. Many candidates have Github and have published some of the work they have done there. If their repo is digestible, it means they have a good Read ME and their code is well commented that you have a good idea at a high level of what they were trying to do.
So in a nutshell, takehomes for interviews can be extensive, daunting, and feel like a one-way street with little to zero feedback. They are also time-consuming for companies to grade. A process my coworkers and I who were hiring implemented at our company was a small exercise that should only take the time we specify by trying it ourselves and after we give the candidate a chance to walk us through their code. It has been very helpful and that way we save time for ourselves to debug since they run it in front of us, we gauge their communication abilities, and we can provide direct feedback on the code they worked on.
Would love to know your experiences doing technical homework for Data Science interviews and if any of this resonates! I just wish the process was a bit more enjoyable for both parties, more inclusive, a two-way street of feedback, and a fairer judge of someone’s capacity to do the work.