The Third Wave CDO Blueprint for Career Success
How modern CDOs navigate the complex challenges of today’s data environments
It’s never been harder to be a Chief Data Officer. On the one hand, demand for CDOs is higher than ever, with more than two-thirds of enterprises appointing a CDO, up from less than one in eight in 2012. On the other hand, job security is lacking, with the average CDO lasting just 2.5 years. That is shorter than all other C-level executives, and half the overall average C-suite tenure of 5 years.
Part of this is due to the rapid expansion of CDO duties over the past decade. First wave CDOs, focused on data management, in particular setting up data marts, and warehouses. They also added data governance to these data warehouses. They created and enforced processes to make sure data was used efficiently and safely, protected from cybertheft, privacy risk, degradation, and other tasks. In other words, they primarily played the role of bad cop, policing how employees used data.
A purely defensive role was demoralizing for both workers and the CDO, though, which is what spawned the era of the second wave of CDOs. And CDOs know data better than anyone. They understood the power of applying analytics and creating data pipelines and data applications. So second wave CDOs became internal innovation leaders, championing the transformation of how their companies view and use data, from a passive archive like books in a dusty library to the lifeblood of a digital enterprise.
Today we are seeing the emergence of third wave of CDOs. I’ll explain.
The Role of the Wave 3 CDO
Convinced of the value of analytics, the third wave of CDOs are looking to incorporate more data sources such as real-time event streams. Wave 3 CDOs want to build operational dashboards and make data available regularly to their businesses. They’re seeking machine learning-generated predictive analytics for better decision-making. And they’re clamoring for AI-based workflows to automate processes for better efficiency, agility, and cost savings. Wave 3 CDOs are not just being asked to ideate, plan, and champion. They are being asked to execute these transformations. And to do so, they are being handed dedicated data operations teams, oversight over data technologies and domains, and responsibility for overall data reliability and data delivery.
Expectations of the Modern CDO
If CDOs are so important, then why are they getting fired so quickly, so often?
One reason is that CDOs who are naive about business risk often try to modernize their data infrastructure while cutting the wrong costs. Today’s third wave of CDOs have partial or total responsibility for a diverse, multi-cloud setup that includes ERP systems, Salesforce instances, traditional databases, data lakes and other big data deployments, and cloud-native data warehouses.
To deliver more value from their data pipelines and data repositories, they are constantly tinkering and upgrading their infrastructure. Bringing data into the cloud is the most common upgrade. Due to the ease of switching and scaling in the cloud, such migrations can look deceptively easy.
But let’s not trivialize the complexity and work involved to make these migrations successful. Moving data from an on-premises data warehouse to a cloud instance, whether it is a simple lift-and-shift or a total refactoring, will require close monitoring and certification that the data was migrated with all of the datasets, schemas, and dependencies intact. Validating and reconciling data pre- and post-migration is labor-intensive work that a time-pressured CDO and his team may feel they don’t have the bandwidth for.
We see too many CDOs settling for letting their operational teams quickly eyeball the migrated data for any data errors or compromised data reliability. Doing that is a big risk for any company, but a massive one for companies using data in ways that support sales, business operations, or anything else mission-critical. In such scenarios, data errors and broken data pipelines inevitably emerge. The worst part is that without strong oversight during the actual migration, these problems will continue to crop up for a long time, and at the worst times. Failed data migrations are a huge reason why CDOs lose their jobs.
Business Demands More Data
Reason number two is the inability to support the business’s ravenous appetite for new data workflows. Even if they are not actively migrating data from clusters to the cloud, most CDOs are still constantly adding new data sources. There are real-time customer clickstreams, Change Data Capture (CDC) synchronizations from internal repositories and third-party data marts, IoT sensor data that is ingested first by your ERP systems before being shared for wider analytics, and more.
This data does not conform to a single structure. Moreover, the modern way to treat data is no longer schema on write but mostly schema on read. This is a more flexible strategy that makes it easier to store a diversity of unstructured and semi-structured data types in large data lakes. But when it comes time for machine learning and analytics — especially the subtle, hard-to-detect anomalies and the bold sweeping trends that data scientists live for — petabytes of data coming from different sources must be harmonized before they can be processed and queried.
All of this is heavy, complicated work that data scientists outright cannot handle. And it presents plenty of potential drudgery for CDOs and well-trained data engineers if they lack the necessary tools. But because of the high priority given to many ML/AI projects today, CDOs that cannot quickly build these data pipelines are at risk of looking like blockers to the business.
In addition to their duties on offense, the third wave CDO is still responsible for playing defense with the data — data governance, security, and access control. The big shift that has happened, though, is that all these areas are now operational in nature and have to be conducted in real time. Data governance is no longer a one-time annual audit, but must be performed constantly and accomplished to perfection.
Navigating Modern Data Environments
CDOs are busy and consumed with balancing long-term optimization projects and fire drills. Some are common, daily tasks, and some are critical for long-term business success. The people in these roles need to always be assessing what data they should be evaluating. They have to know if the data is available. There has to be awareness about whether or not compute systems are working, and are the business processes well-managed? When CDOs start with those questions and are willing to address the answers to those questions, it gives them a better understanding of the possibilities inherent in their enterprise’s data investments.
In my experience leading data teams, it’s clear that the foundation for operational success is data reliability and validity. Before your data can have much of an impact, you have to know you can trust it and that it is of the highest quality. CDOs need a level of visibility that not only gives them insights into where problems exist, but should allow them to make sense of data activity.
Over time, it became clear to me that to ensure I could have a meaningful impact on my organization, as a data leader, I had to know much more than just how to address an alert. My value came from being capable of making informed decisions about things like spend forecasting, scaling and performance issues, and the myriad of other issues that make up the data landscape. As I sought to have an always-on sense of data operational intelligence that allowed me to interpret the state of the supply chain of data, I had a better handle on the state of data operations.
This is how I’ve arrived at the idea of the wave 3 CDO. It’s someone who thinks of an internal data landscape in terms of data pipelines/platforms and data applications. And not just those as elements of the environment, but the interplay among them as integrated components. Awareness of that activity and behavior is the foundation for making a CDO successful. Ultimately, with visibility and a strategic map, CDOs can successfully navigate the new data landscape and thrive for years to come.
