The world’s leading publication for data science, AI, and ML professionals.

4 Things I Didn’t Know About Being a Team Lead in Data Science

What I learned as I transitioned from individual contributor to team lead

Office Hours

In 2019, I began working as a senior data scientist, and by 2020 I became a team lead for my group, Tools, and Machine Learning Platforms. The transition to becoming a team lead was an exciting one, but many unexpected changes came. Before this position, I was not aware of what it meant to be a team lead or a product owner.

1. There is less time to focus on development work.

Transitioning from an individual contributor to a team lead and product owner came with many other tasks that need to get done. These tasks can range from planning out work and road mapping to helping team members with issues as they arise. As I made the transition, I realized it is not an easy shift away from development work. Instead, I try to focus my time on 50% development and 50% other, but it doesn’t always work that way. Some weeks I can efficiently concentrate on my projects, while others I am pulled away to deal with problems that have arisen on the team.

Since my time to focus on development work is less than others on the team, I chose my projects and tasks wisely. I focus on high-value work for my team or stakeholders that will allow me to learn specific skills or be apart of particular projects. My work tends to be research, bug fixes, or minimal viable products (MVPs). If I have done an initial MVP, then I shift my focus to how I can leverage others on my team to progress the work and plan the work alongside the roadmap.

2. Questions, problems, and blockers will arise that you need to help with.

The things that have caused me to pull away from my development work often tend to be questions, problems, and blockers from the team.

Questions These tend to be the easiest to deal with. Most questions I can answer by directing a person to documentation, some code, or another person working on a similar project. These questions tend to be quick to respond to, but they can come up often throughout the day by different people. Sometimes the questions will come from others outside my team looking for something we have done or looking for a status update on ongoing work. This means I need to be informed of most work on the team to answer those questions as they arise.

Problems – All too often, issues arise, bugs appear, or conflicts start. Depending on the situation, I take in work and determine the best next steps to proceed forward. At least once a sprint, something tends to pop up that needs to be addressed. If it is work that can wait, it will get planned. If it cannot wait, either myself or someone else on the team will take it on. Conflicts? We will get to those.

Blockers – If it is stopping work from happening, it is a blocker. These tend to get brought up in our daily meetings. If the blocker is something I can help with, then I will help push the work along. I often get blockers in which people cannot access data or need a subject matter expert on the call to clarify some things. These calls I can set up and facilitate as needed to help resolve issues.

3. Handling conflicts and disputes can be complicated.

Oh, conflicts. They are something that can often arise, no matter the team. Some disputes can be quick to resolve, while others will take time. When I started as a team lead, I was not prepared for the level of conflict and dispute that I would see. Often the disagreements tend to be over code style, visualization preferences, or the "correct" method of approaching a problem. After dealing with my first significant dispute, I developed a set of working agreements that outline the steps to go through when working on a project with another member, what to do when you alter someone’s work, and conflict resolution steps. Luckily, since that dispute, it has been the only one I needed to escalate to higher levels.

Another conflict that I was not prepared for when I started as a team lead was the team dynamic between Data Science and data engineering. The relationship was broken. I wrote an article recently in more detail about having collaboration between data science and engineering. All in all, that conflict taught me we needed a strong and trusted relationship to work together. I spent a year promoting a culture of collaboration and effective communication between the teams as we worked towards a common goal during 2020. As a result, our teams have come out stronger and are working closely together on many projects. It has become much easier to collaborate when we know what the others are working on.

These conflicts and disputes were not easy challenges to face, but they can teach a person a lot about building skills in trust, leadership, and communication.

4. You will work with stakeholders and other leads to develop the roadmap and determine the priority of tasks.

The other 50% of the time I spend planning. I refine the backlog, sprint plan and update the roadmap. I work as a product owner for my team, gathering input from our stakeholders on work that needs to be added to the backlog, priority of work, and feedback.

When I first began to take in work from stakeholders, I noticed issues with the expectations of the work and priorities that were being set. One particular project a stakeholder had expected to start sooner than it did. Six months later, when the work began, the stakeholder exploded in a meeting about why we were so behind. After listening to everyone’s thoughts, I paused the meeting and shared what I knew to be the work’s expectation and the current status. When we determined what was going on, we reflected on the situation and decided what to do next.

The issue? Work was not being filtered through to me to add into the backlog and prioritize with the stakeholder. Instead, they had gone through another team lead in a different group who did not prioritize the work. Therefore, my team had not started the project as the stakeholder did not fully inform the correct team on the expectations and requirements. The stakeholders had become used to bringing all requests to only one team lead. When the two other team leads, myself and another, joined the larger group, stakeholders had not begun to bring requests to us but continued to use the other lead. This led to confusion on work being requested, timelines, and priorities.

The main thing I learned here was how important it is to communicate with your stakeholders and other team leads on the roadmap and priorities. The slightest misunderstanding can lead to a disaster in which both parties are under different assumptions.


Final Thoughts

Transitioning into a Leadership role can come with many unexpected changes and many great learning opportunities as well. As I come into 2021, I look forward to how the position will continue to evolve. To recap:

  1. Focus on work that will provide value to your team or stakeholders.
  2. Understand your team and their work enough to answer most questions related to them and the work being done.
  3. Conflict happens. Have a plan to remedy it when it does and build strong, trusted relationships with your colleagues.
  4. Communication is critical as you plan and prioritize your team’s backlog of work with stakeholders and other leads.

What challenges have you faced as a team lead in data science? How have you overcome them?


If you would like to read more, check out some of my other articles below!

Top 3 Challenges with Starting out as a Data Scientist

Why You Need a Data Science Mentor in 2021

Top 5 Lessons Learned after Hundreds of Job Applications


Related Articles