Oh, You Meant “Manage Change”?
Different perspectives on change in a data organisation
Different brews of change
[Scene: A modern office break room. The whirring of the coffee machine is the only sound, with the aroma of freshly brewed coffee wafting in the air. Alex, the CDO, stands by the coffee machine, pouring herself a cup. Jamie, a data engineer, walks in, looking a tad weary.]
- Jamie: “Another day, another challenge. You know, Alex, managing change is starting to wear me out.”
- Alex (nodding): “Absolutely, Jamie. Change management is my biggest priority right now. We have to ensure we’re adapting and staying ahead.”
- Jamie (raising an eyebrow): “Staying ahead? I’m just trying to keep things from falling apart every time something shifts.”
- Alex: “Exactly. It’s about anticipating those shifts and staying ahead of the curve. We’ve got to keep our team motivated and aligned.”
- Jamie (confused, but trying to agree): “Yeah, aligned and… not falling behind. Got it.”
[The conversation flows to other topics, but the disconnect in their views on ‘change management’ remains unsaid and unacknowledged.]
So, let’s break down what just happened between Alex and Jamie. They both threw around the term change management, but they might as well have been speaking different languages.
Alex, our CDO, thinks big. She’s monitoring market shifts, emerging technologies, and envisioning where the company should be in a few years. However, the strategy is the easy part, what is complicated is to get everyone on the same page.
Introducing a new tool? She’s got to be ready for the eye-rolls and the “not another software to learn” groans. A new process? Brace for the “but we’ve always done it this way” chorus. For Alex, change management is a tightrope walk — balancing where the company needs to go with making sure everyone’s on board and not panicking about their job security.
Then there’s Jamie. His change management isn’t about years down the road; it’s about, well, right now. That pipeline that just broke? His problem. A discrepancy in the sales report? His responsibility. The hardest part is not always the technicalities but often the human element. Like when someone forgets to tell him about a tiny “insignificant” data change that sends everything into a spiral. Or when a task goes wrong, and the blame game starts. For Jamie, change management is about keeping things running smoothly today and dealing with whatever curveballs come his way.
The Strategic View: Aspirations of a CDO
I often interact with CDOs, and the diversity of conversations is an aspect of my job I truly appreciate. Every conversation is different, bringing its unique perspectives. Yet, for some reason — maybe because these topics are ones I care deeply about, or perhaps this is where the real action is — certain common themes inevitably surface.
First and foremost, there’s this passionate emphasis on driving business value. It’s not just about collecting data or implementing the fanciest new tech; it’s about turning that data into actionable insights. It’s about ensuring that every data-driven initiative ties back to the company’s goals, be it increasing sales, improving customer satisfaction, or optimising operations.
Next up is the push for efficiency. CDOs frequently face the task of improving operations, cutting out overlaps, and making sure data reaches where it’s needed promptly. This isn’t a walk in the park; it involves dismantling old barriers, encouraging teamwork, and staying updated with new tech solutions.
One of the more ambitious directions many CDOs are leaning towards is the concept of decentralisation or a data mesh. It’s a significant shift from the traditional centralised data teams to a model where domain teams own, produce, and serve their data as products. The thought process here is simple yet revolutionary: those who know the data best should package and maintain it. This not only ensures better data quality but also fosters a culture of self-consumption, giving more autonomy to different parts of the organisation.
Reaching these goals is tough. Each strategic aim presents change management issues that CDOs like Alex must directly address.
Take the business value first agenda, for instance. For data professionals who have spent years, if not decades, entrenched in technical tasks, shifting the focus towards business outcomes can be jarring. They’ve been hardwired to think in terms of data accuracy, system integrations, and code optimizations. Asking them to “think in terms of business value” is often met with a puzzled look!
Then there is the move towards decentralisation, arguably a great idea on paper: empower teams, let them take ownership, and the organisation becomes more agile and efficient. In practice, this means a lot of change and this needs to be managed. With decentralisation comes the challenge of clearly defining roles and responsibilities. When everyone is a stakeholder, it is easy for tasks to fall through the cracks. Who is responsible for data quality? Who ensures that data is accessible to those who need it? Without clear delineations, balls get dropped, and the blame game begins.
In essence, for each strategic pivot, there’s an underlying web of change management intricacies. It’s not just about charting the course, but ensuring that everyone on board understands their role, is equipped to play their part, and is committed to the journey ahead.
Ground Realities: The Day-to-Day Challenges
While Alex’s role as a CDO is largely about the bigger picture, navigating through broad, unpredictable scenarios, there’s another side to change management. It’s found in the daily, detailed challenges faced by data engineers like Jamie. In their realm, change management isn’t about long-term strategy or overarching business aims. Instead, it focuses on the ongoing, every-moment hurdles of ensuring data remains consistent and accessible in a constantly shifting backdrop.
Primarily, the bulk of data in organisations emerges as a byproduct. As various business activities unfold, data naturally accumulates, much like exhaust from a machine. However, while this data might be a mere byproduct for those generating it, for data teams and their internal and external customers downstream, it becomes central to their daily operations. Ironically, at the source, there’s often little regard for this ‘exhaust’, even though it’s indispensable for these stakeholders further along the chain.
Think of it as trying to set the foundation for a towering building on unstable ground. The earth below is always moving, but you’re tasked with ensuring the immense structure above stays steady. This is the world for many data engineers and BI analysts. They stand at the front, handling the erratic behaviour of data every single day.
A significant issue they face is the complex network of dependencies that weave through the data world. Data moves from one platform to another, undergoes transformation, merges with other datasets, and finally reaches its intended place. Every stage of this process holds the potential for glitches. A minor adjustment in one platform can ripple and cause disruptions elsewhere. And the most challenging aspect? Often those making these changes are oblivious to the potential chain reactions they might set off.
Facing day-to-day problems with changing data chains led thought leaders to devise new concepts. They first introduced data products, which packaged data for ease of consumption, akin to how a shop owner presents goods to customers. But as more people started using these data products, there arose a need for formal commitments — a way to ensure data product owners reliably served their users. This realisation spurred the development of data contracts to ensure these obligations were met.
Data contracts act as a bridge between the people who package data for consumption and the consumers they serve, documenting and enforcing clear commitments: immutability of the data schemas, standards of quality and of availability, and so on. They address quite elegantly the question of managing change in chains of dependencies in data.
Bridging the Divide: Unifying Perspectives on Change
The challenges of strategic transformation and managing volatile dependencies might seem worlds apart at first glance. A CDO like Alex grapples with guiding the overarching strategy and aligning the entire organisation towards a shared vision. Meanwhile, Jamie is the firefighter dealing with day-to-day data challenges. Yet, at the core of both these perspectives, lie some unifying principles that can bridge the divide.
Transparency is paramount. Whether it is Alex communicating the broader goals of a strategic initiative or Jamie flagging a potential downstream impact of a schema change, clear and open communication can prevent many issues.
Collaboration ensures alignment. Everyone in the data organisation needs to be in sync. On a daily level, this means communicating effectively to prevent unforeseen hiccups. Strategically, it’s about ensuring everyone is clear on the broader goals, ensuring both daily tasks and overarching plans push in the same direction
Standardisation offers stability. Introducing practices like data contracts not only addresses the nitty-gritty challenges Jamie faces but also fortifies the foundation upon which Alex’s strategic vision is built. By establishing clear standards, we remove ambiguities and make it easier for both the big-picture thinkers and the detail-oriented doers to cohesively move in the same direction.
The final irony is that in order to address the day-to-day problems of Jamie (the unintended consequences of changes in opaque chains of dependency), you need to make this topic a strategic priority.
And if you want to be successful, given the amount of process, human and technological change, you need to apply all the good principles of change management as advocated by Alex. Of course, Jamie has a critical role to play here, he is the one closest to the problem and its consequences, so he can become a change agent, getting his colleagues and his management on board.
So what started as a quid pro quo could actually be the beginning of a strategic transformation: a clear reason to act, a roadmap, the right leaders and the right tools.
Even to manage change, you need change management!
