It’s Time to Talk About Organizational Bias in Data Use

Organizational bias in data use has long flown under-the-radar. However, its impact on businesses and their bottom-line can be destructive. This article aims to start the discussion on organizational bias, including what it is and why you need to be vigilant in limiting its impact around the business.

Data use is always susceptible to bias. It can never be completely objective. This is because all stages of the data life-cycle from collection to analysis and onto decision-making involves humans. We design how to collect the data. We decide the way data is stored and how actions are categorized or quantified. And we conduct the analysis or interpret the data signals.

The fact that humans are both creators and users of data means there is an opportunity for bias to creep into the data life-cycle. For this reason, we need to be acutely aware of all types of bias and the ways it can taint data use in decision making.

There are two broad categories of bias in data use:

1. Individual bias
2. Organizational bias

This article focuses on organizational bias.

Organization Bias vs Individual Bias

Between 2011 and 2014, I conducted doctoral research that studied how business leaders used data in their decision-making process. Specifically, I looked at factors that influenced the way leaders selected and used data.

I found that the largest influence and biggest threat to objective data use was organizational bias, not individual bias.

Individual bias is arguably more well-known and more widely discussed. Individual bias is also more clear-cut and easily identifiable. It includes the biases and prejudices employed by the individual or small group of individuals.

Individual bias can be as simple as color selection on a chart that emphasizes one data insight over another to support a preconceived argument. Or it can be more subtle, such as a researcher selecting familiar cities for a research study and not adequately considering all areas in their research design. Two more commonly discussed types of individual bias you may have heard of include confirmation bias and selection bias.

Organizational bias, on the other hand, is a topic that’s less prominent in industry articles and publications. However, its influence can be particularly devastating because it usually has a wider reach than individual bias.

Organizational bias occurs when factors such as culture, senior leadership, strategic focus and team organization influence data selection and data use to a point where selection is no longer based on merit.

Organizational bias can be harder to identify and isolate than individual bias. This is because organizational bias embeds itself in everything from the language used, choice of projects that are proposed and impacts the ways of working throughout the business. The widespread nature of organizational bias and the fact that it is found in so many core business functions means that it is more likely to be convert, systemic and wider-reaching than individual biases. All of which, provide the potential for organizational bias to be particularly devastating if not diagnosed.

The Source of Organizational Bias

Every organization is unique, and therefore the source of organizational bias can come from many different factors and many combinations of factors.

The source factors may include things like internal politics, culture, leadership, industry sector of the business, history of the organization, products, and team setup. Organizational bias is something that every business needs to monitor.

Here are some sources of organizational bias uncovered during my research:

1. Senior leaders explicitly and implicitly telling their teams that one data source is ‘more accurate’ than another (vs selecting data sources based on their merit)
2. Teams being structured in a way that gives them more/quicker access to one data source over another (vs equal access to holistic insights and learning)
3. An enforced strategy based on one specific data-insight or data source (vs a strategy driven by cross-section of holistic insights)
4. Data silos created through poor data insights management, inefficient data sharing practices and little investment in learning centers (vs well-established sharing and organizational learning development)
5. Enthusiastic adoption of new measurement technologies and force-fitting these to projects (vs selecting the right measurement tools based on merit)

Tips to Limit Organizational Bias

Limiting organizational bias is the key to making objective data-driven decisions.

Here are four systems needed to help limit organizational bias and ensure senior leaders are making decisions on the most holistic data available.

1. Use decision making frameworks. Decision frameworks are guidelines that train and support business leaders in their decision-making process. The role of the decision making framework is to encourage leaders to remove emotion. There are many types of decision making frameworks that can be used. In my research, I designed a framework specifically for encouraging data-driven decision making. It is called the ICSAR framework. Read more about how this model can help you make better business decisions. The ICSAR framework is one model that can be used but there are others out there to choose from.
2. Provide democratic access to data insights. One of the key differences between more mature data-driven organizations and less mature organizations is often in how data is share. Less mature organizations control and constrict information flow. Data is controlled by a small subset for reasons like security, power, influence and ownership. Mature organizations invest heavily in sharing data insights and ensuring democratic access to all data regardless of team. They focus less on controlling data and expend energy instead of the best ways to use the information. One litmus test on data maturity can be found in meetings. Meetings among senior leaders where significant amounts of time is spent debating the accuracy of data insights and less on the best ways to use data is a clear sign of organization that has immature data use practices.
3. Well-structured measurement, learning and development mechanisms. The biggest data challenge now facing many businesses is not about gathering data insights, but actually managing and synthesizing the deluge of insights available to them. Leading-edge organizations are investing and developing ways to manage the insights in the way a library manages collections of books. They are developing teams, specialists and the infrastructure needed to catalogue learnings as well plan future pieces of research and analysis to plug knowledge gaps. Less advanced businesses often have no structures in place to manage their insights and data learnings. This results in knowledge silos, duplicated bits of analysis, decisions made without holistic information and wasted investment.
4. External evaluation and monitoring of bias. While there is a lot of work that can be done internally to limit organizational bias, businesses also need external evaluation and monitoring. This is because organizational bias is often widespread and covert. It can live in the language and prejudices of the most senior leaders in the organization, so external support is needed to help those leaders identify any biases. W Edward Deming wrote about management: “A system cannot understand itself. The transformation requires a view from outside.” The same is true here because organizational bias is systemic, businesses need an objective, external evaluation to help them understand where organizational bias might be impacting their decision making and support them through recommendations for development opportunities.

Conclusion

In 2018, businesses have the opportunity to not only capitalize on data but to be ever-smarter in the way they manage and efficient bring together the tsunami of insights available to them.

It’s not enough to simply collect data or to have a data science and insights function. We’ve moved on from that. The potential of data has been well documented.

The critical way for businesses to differentiate and lead in the next few years is to effectively synthesize and coordinate many different learning’s into effective decisions and unique market strategies. None of which will happen if businesses don’t manage and account for organizational bias.

Managing organizational bias won’t happen overnight. It will need committed investment in using decision frameworks, tools to democratize data sharing, building structures to keep developing learning across the business and external support in monitoring organizational bias.

However, the end-result is a combination of more objective, data-driven decision making. More holistic understanding of data and insights. More effective use of the investment already made in the data infrastructure.