How to Manage Data Projects and Teams Effectively
It’s never a bad idea to reevaluate the way we approach our work. This is true whether you’re a student working on a portfolio project, a seasoned ML engineer managing an end-to-end pipeline, or an executive responsible for the success of an entire data team.
It could be a superfluous step that you finally nix. Maybe you decide to revise the format of your weekly team call, or to implement one last quality check that takes minutes but occasionally saves hours. Your mileage might (and likely will) vary across teams and disciplines, but the idea is the same: your workflows can almost certainly benefit from some tweaking and streamlining.
To help you start this journey with a few concrete ideas, we’ve selected several recent standouts at the intersection of data science, leadership, and project management. Enjoy!
- How to troubleshoot your data science project. Outliers, missing values, imbalanced datasets: sooner or later, you’re bound to encounter these at the worst possible time. Jason Chong saves the day with a primer on some of the most common issues you’re likely to run into as a data scientist, and proposes “a framework on how to properly deal with [them] as well as their respective trade-offs.”
- The challenges and rewards of a new leadership role. CJ Sullivan reflects on a major career transition: from tech to the ski industry, and from working as an individual contributor to becoming a Director of Data Science. Her post unpacks the lessons such a change can teach us about leading others, and about adjusting the way we communicate the value of our work to non-technical stakeholders.
- What’s in a roadmap? Figuring out how to meet the goals we’d set for ourselves can require a long process of trial and error, but landing on the right goals in the first place is even trickier. Marie Lefevre spells out the benefits of a well-defined roadmap for a data team, and shares a framework for creating one that will give you the space to “think strategically rather than operationally.”
- The importance of building a robust data platform. Closing the gap between data’s potential value for a business and the actual impact it makes often boils down to putting the right tools in the hands of the right people. Mahdi Karabiben explores the (many) limitations data catalogs currently place on stakeholders, and argues for fewer UIs, more APIs, and a push towards greater data accessibility.
- To make the right decisions, you need to find the right metrics. “How do you rigorously, scientifically study concepts that you can’t easily define?” Before you collect data and analyze it, Cassie Kozyrkov draws our attention to the difficult task of coming up with a clear, actionable idea of the phenomena we aim to measure.
- Why data projects flourish through iteration and empathy. Data scientists are problem-solvers; as Taylor Jensen explains, a better understanding of their internal customers’ goals is at least as important (if not more so) than a strong grasp of algorithms and statistics. Taylor suggests that borrowing the principles of design thinking—from empathizing to prototyping—can be a powerful move for data teams.
While you let these ideas around team and project management percolate, we hope you might also spend some time with a few more excellent articles we published recently.
- We welcomed Anna Rogers’ first TDS contribution—a thought-provoking reflection on originality and attribution in the context of generative-AI tools.
- AI-generated art was also top of mind for Danie Theron, who studied gender, skin tone, and intersectional biases in the visual outputs of Stable Diffusion images.
- For a comprehensive, one-stop resource on regular expressions and how to use them in Python, don’t miss Susan Maina’s latest post.
- If your data science career is at its early stages, Arunn Thevapalan’s debut TDS post provides a helpful roadmap for getting your foot in the door.
- Furcy Pin’s detailed history of the Hadoop ecosystem is a useful reminder of just how recent (relatively speaking) big data actually is.
- What does the recent wave of layoffs in the tech sector mean for the future of AI? Wouter van Heeswijk, PhD wonders if an AI winter might be just around the corner.
Thank you for supporting the work we publish. If you’d like to make the biggest impact, consider becoming a Medium member.
Until the next Variable,
TDS Editors
