The world’s leading publication for data science, AI, and ML professionals.

A Type of Data That Companies Need to Collect Today

Why collecting decision-making data inside a business is necessary.

BUSINESS INTELLIGENCE

Photo by Elia Pellegrini on Unsplash
Photo by Elia Pellegrini on Unsplash

In the era of digital transformation, companies are collecting almost every type of data. Sensors, financial, and logistic data are just a few examples. However, at least one important type of data that most businesses miss is internal "Decision-Making Data."

Decision-Making Data

The "Decision-Making Data" is probably the most important, expensive, and complex data that every organization produces internally but rarely collected. To understand this type of data, let’s start with a straightforward example. Many of you have played chess. Some of you are familiar with PGN or Portable Game Notation. PGN records chess games in a computer-processable format. PGN records chess players’ movements during the game using a series of notations. It is not obvious, but actually, it is recording a valuable "Decision-Making Data." With this kind of data, computers can learn from humans how to play chess.

Playing chess and PGN is a good example to start understanding the concept of decision-making data. In this example, because of a limited number of movements, constant rules of the game, low risk of making a wrong decision, and the possibility of trial and error, computers can learn the game without PGN or any help from an expert.

In real-world tasks, unlike chess, the environment changes constantly. Besides, most if not all of the tasks are riskier than chess playing, and trial and error (i.e., exploration) is not an option. The field of the autonomous car is one of them. We cannot just let an AI driver with a car drives in the streets and learn by itself. The AI driver needs to get access to some kinds of decision-making data and learns from the data. The decision-making data is simply a series of purposeful actions by an expert (in this case, a human driver) along with environmental data (in this case, data from car sensors, cameras, and weather stations). The value of this data for companies like Tesla is as valuable as all its tangible assets together.

An Example from the Oil and Gas Industry

Speaking of decision-making data with examples such as playing a chess game or driving a Tesla car is easy and straightforward. But, how does it look like in an industry or a business? Let me show it by an example from the oil and gas industry.

To answer this question, we need to classify the Decision-Making Data into two levels: Low-Level Decision-Making Data (let’s call it LLDD) and High-Level Decision-Making Data (let’s call it HLDD). Interestingly, the oil and gas industry is traditionally very good at collecting LLDD. When a geologist interprets a series of maps, well logs, and cores, he/she is producing LLDD (sometimes called Soft Data). I hope it helps you figure out the other kinds of LLCD that your industry or business is collecting. We call this type of data "low-level" decision-making data because, like the chess game, the rules around these tasks do not change too much. In fact, you are making a decision in a limited dynamic system. Most companies are already recording low-level decision-making data in the form of structured or unstructured data (e.g., technical reports).

Most of the time, when we are talking about Decision-Making Data, we are talking about High-Level Decision-Making Data (HLDD). Suppose a team of engineers and managers are working on an upcoming oil and gas field development project. Among thousands of different possibilities, they will come up with one optimum plan. Unlike low-level Decision Making, here, we need to incorporate too much data, uncertainties, and considerations to make a single decision. In other words, the system is highly dynamic, and it is why we call it high-level decision making.

Now, how much do we know about this long process of thinking and decision making? If a field development planning like a chess game, how can we collect data from this game? Is there any way to record data of good and bad decision making in our company or business?

AI and Decision-Making Data

Thanks to digital transformation, emerging machine learning technologies, and big data, businesses and companies have made significant advances in predictive analytics in the last few years. But, when it comes to prescriptive analytics (e.g., finding the most optimum business solutions) and cognitive analytics (e.g., making the final business decision and act), AI loses its credibility. One of the reasons is the lack of high-level decision-making data to train an AI system for those purposes. If we need an AI-assisted decision support system for the future, we need to collect internal decision-making data today.


Follow me on Medium and Twitter for the latest stories.


Related Articles