Communicating Data Science Insights to Non-Technical Teams

Make yourself heard by understanding and catering to your audience

Published in

Towards Data Science

10 min readAug 9, 2022

F1 and AUC are as meaningful to business people as terms like “customer acquisition” or “revenue growth” are to you (hint: not meaningful at all). Simply put,

Non-technical stakeholders do not care about your metrics if it’s perceived to not be affecting them.

You understand the results. You know it affects them. You think they should care. But they don’t. How do we force them to care?

“Analytics teams focused on detecting meaningful business insights may overlook the need to effectively communicate those insights to their cross-functional partners who can use those recommendations to improve the business.” [1]

The key word here is effectively communicate. Handing off your metrics and graphs in their original form is not enough.

“The quantities that data scientists are trained to optimize, the metrics they use to gauge progress on their data science models, are fundamentally useless to and disconnected from business stakeholders without heavy translation.” [2]

To effectively communicate data science insights, you need to learn to speak their language. For that, first, we’ll need to learn the unique languages that each stakeholder is speaking.

This article is an exercise in understanding a stakeholder’s role in an organization, and understanding their unique point-of-view from a data science perspective. We’ll dive into how to communicate key aspects of the data science process to non-technical stakeholders.

Stakeholders

“A stakeholder is a party that has an interest in a company and can either affect or be affected by the business.” [3]

In the below diagram, we can see the various stakeholders in a data science project on a spectrum according to how knowledgeable they are about the type of work the data science team is responsible for.

The spectrum of knowledge of the data science team across stakeholders (Image by Author)

Note that these are generalizations according to my past experiences. This structure may differ from organization to organization, as well as by people within each team.

In general, we can see three different groupings from the above chart:

Product-Focused: Engineering (& Product), Project Manager
User-Facing: Customer Success, Marketing, Sales
Business-Focused: Executive Team

We now want to understand different aspects of these stakeholder groups. We’ll draw upon concepts from UX design: user personas.

“User personas are archetypical users whose goals and characteristics represent the needs of a larger group of users.” [4]

In our case, we’ll use similar methods to try to understand our major stakeholder groups. We’ll use this framework to understand the goals & priorities and requirements from the data of each group.

1. Product-Focused

Product-focused teams are included for completeness but are not relevant to the discussion of communicating with non-technical stakeholders.

2. User-Facing

These teams are the face the user sees of the product and organization.

Goals & Priorities: This may seem quite obvious (eg. “Sales: sell the product”), but this is a naive view. Engineering could be summarized as “make the product” but this wouldn’t allow outsiders to truly learn what engineering does, and how it’s relevant to them. Hence, we’ll provide better-defined goals that inform the team’s day-to-day activities:

Customer Success: “To work with the customer to help them get better use out of the product or service. In turn, they often help to reduce churn by increasing user adoption and NPS, and assist with things like upselling and even account expansion” [5]. This team generally has the best understanding of customers.
Marketing: This team's primary goals include: increasing brand awareness, generating leads, increasing customer value, improving SEO, growing a media presence, and increasing conversion rates [6].
Sales: Setting and achieving SMART objectives such as increasing: annual sales and profit, customer numbers, upsells and cross-sells, conversion rates, customer retention, sales rep productivity, and outreach to qualified leads.

In summary, the user-facing teams draw in leads (Marketing), convert leads to customers (Sales), and increase customer utilization and usefulness of the product (Customer Success). They must be able to convince users that your product performs well and fulfills their needs. Users “…will likely prioritize the performance of the product, as they would be a bit unconvinced if the product has some flaws” [7]. Users also tend not to like what they feel are “unnecessary changes” to the product, especially if it makes their perceived experience worse.

Requirements from Data: This group cares about users, and what data science can provide that lets them understand and better communicate with users. Topics such as improving user experience, reducing customer churn, identifying high-value customers, and personalizing the user experience are of great value to them.

3. Business-Focused

Goals & Priorities: The executive team is “…responsible for creating strategic approaches and operational methods for a company, and supporting employees so they’re able to succeed in their roles” [8]. Priorities include “developing and executing long-term and short-term objectives, making financial decisions and maintaining the organization’s budgets, and collaborating to ensure all departments focus on company-wide goals” [8]. “[Their] main interest in data science projects will always be increasing the value of the company (directly or indirectly). If they are part of the meeting (planning, updates, etc.), they will be looking for the return on investment the project would bring” [7].

Requirements from Data: This group cares about the overall business and how to use data to help answer questions originating from their priorities. However, too much information can also be a burden,

“The first thing about information systems that strikes me is that one gets too much information. The information explosion crosses and criss-crosses executive desks with a great deal of data. Much of this is only partly digested and much of it is irrelevant…” [9]

So the requirements for this group are accurate, succinct, actionable insights key to answering their business questions and increasing the value of the company.

Communication

Now that we understand the goals, priorities, and requirements of our stakeholder groups, how can we utilize that understanding to properly communicate information?

The questions to always keep in mind when communicating:

Why should they care? Am I communicating information that they should care about?
How does the information I’m communicating affect them?

Discover Biweekly

Let’s start with an example that will be used to show how we communicate with each of the stakeholders. Suppose you’re a data scientist at a popular music streaming service. An important component of the service is music recommendation, a feature which we’ll completely originally call “Discover Biweekly”. The service recommends users a new playlist of music every two weeks based on their listening habits. Users like it because it allows them to discover new music they wouldn’t have otherwise heard. The business likes it because it keeps users on the service, listening to more music, which in turn makes them more money.

Suppose our overall business objectives, and objectives for this feature, include:

maximizing a user’s time using the service,
maximizing a user’s time using this particular feature,
maximizing the relevance of the recommendations served to the user.

Now suppose we translated this set of business objectives into a set of quantified objectives (which we’ll use to assess our models). The new model must consider the impacts on:

Time the user spends using the service in a specified period,
Time the user spends using the recommendation feature (listening to their Discover Biweekly),
MAP@K, used as a proxy for the overall relevance of recommendations,
Dissimilarity to determine the personalization of a playlist to a particular user,
Intra-list Similarity to determine if recommendations are sufficiently diverse.

Now to you, data scientist, a cleaned and processed DataFrame containing the time series data collected from a subset of users may look something like this:

DataFrame containing time series data of metrics (Image by Author)

You may also generate some statistics from the collected metrics from both the old and the new model to compare performance:

Statistics generated from collected metrics averaged over time for simplicity (Image by Author)

And maybe generate some plots to help you understand the statistics better:

Plots of the mean and standard deviation of time-averaged data (Image by Author)

And from these visuals, you can draw conclusions about your new model. The problem is that these metrics and graphs are inherently useless to the non-technical stakeholder. If you can convince marketing to be excited because you improved the average MAP@K by 10% — please tell me your secrets.

For future sections, keep the previously-mentioned questions in mind when reading:

Why should they care? Am I communicating information that they should care about? How does the information I’m communicating affect them?

User-Facing

Metrics should be analyzed and presented in a way of “how does this affect users.” These are the teams communicating the how and why of your work to the users.

Using the above example, the improvement of MAP@K and increase in intra-list similarity don’t mean anything to the user-facing teams on their own. Instead, we can break down this analysis, for example, into a visualization of how user experience changes:

Visualization of the change in user experience (Image by Author)

The left graph illustrates the type of change to the user experience we can expect overall, derived from MAP@K. The right graph shows two sample users and how their playlists change, which we can derive from intra-list similarity.

We can clearly see how the information presented in this way is immensely more valuable to user-facing teams than the previous metrics. It directly gives them insight into how actual users are affected and helps them understand the trade-offs of the new model. The answer to “why should they care” is presented clearly in the chart: it will affect users, and here’s how. It allows them to accurately assess the change, determine if it will be beneficial to users, and provides a way for them to communicate the change to the users. This is just one simple example, but it illustrates the concept quite well.

Business-Focused

Similar to user-facing teams, we need to break down and present metrics in a way that’s catered to the business-focused teams. The metrics should be succinct, actionable, and directly relate to how the company will be impacted.

We should focus on illustrating how these changes tie back to the overall business objectives. A visualization like the following may be appropriate:

Visualization of usage over time, new model deployed at Week 0 (Image by Author)

In the above graph, we show a visualization of how the implementation of the new model will impact total time and feature time, two of the key business metrics. This will allow the business teams to quickly judge if these changes are in line with overall business objectives. If metrics are not presented properly, certain characteristics could be overlooked or their importance over-inflated. Your data visualization should tell the truest and most accurate story possible. In this case, your job isn’t to make decisions, it’s to power the decision-makers with actionable insights.

Consider if we just presented the time-averaged mean and standard deviation. The business team would see a drop in total time, immediately assume your new model is worse, and reject your changes. In reality, the above visualization could imply that users are just adjusting to the change, as we see total users on an uptrend, bouncing back to previous levels. To summarize,

“Many analytics teams that don’t emphasize communication let insights slip through the cracks when executives don’t understand recommendations or their business impact.” [1]

In General

Besides considering how to communicate depending on the party we’re communicating to, there are also some general best practices when engaging in data science communication [1]:

KISS: Keep your data visualizations simple. Dense visualizations are confusing and can distract from the key insight.
TL;DR: At the beginning of each analysis, highlight key insights and actionable items before diving into the deeper explanations. Summarize the key insight and why it matters.
Avoid Trivia: “Any insight which is not actionable is trivia. Knowing trivia is fun but can easily turn into a distraction and fog up the general message and recommendations that should be delivered.” [1]