Over the last decade, businesses have seen a fundamental shift in the way decisions are made. Winning organisations have moved from an environment, in which personal experience and intuition is king, to a data driven world where decisions have support from quantifiable historic evidence. We now sit on the precipice of the next transformation, the movement from simple to advanced Analytics. This change, like any other, requires thought provoking questions to be asked on both a macro and micro level. A key question transformation leaders should be asking is, "Does the movement from simple to advanced analytics make sense, or is there another route we should consider?"
With the proliferation of online courses, highly publicised successes and general media hype, advanced analytics (marketed in varying forms) has taken centre stage as business’s next evolutionary step. The hype has positives, as businesses that saw little value in their data before are beginning to place trust in it, and are reaping the benefits. However, the other side of the coin is not as shiny. The hype is also creating expectations that are leaving businesses frustrated and many employees demotivated.
In the attempt to increase sales and trust, businesses have jumped on the bandwagon of ‘being on the cutting edge of AI and analytics’. Whether this strategy is successful for boosting sales is unclear, however it does create expectations for new, analytically gifted resources joining the Business. For the sake of simplicity I will refer to these resources as ‘Lukes’. Many Lukes join AI driven businesses wanting to "stretch their legs" and build the latest and greatest predictive models (which is understandable as these models are technically interesting and increase your street cred). Unfortunately, Lukes are unaware of the training that needs to be done and the seeming impracticability of their core technical skills in business. This causes frustration and demotivation within the Luke faction. Frustrations are exacerbated by the decrease in technical understanding with the increase in business seniority.
Regardless how businesses market their analytical capabilities, there is one common truth amongst them. Businesses want to get actionable and believable insights from their data. Little value is placed on the methodologies or techniques used. Instead, true value is gained from analytical capabilities when the insights extracted from data allows for superior solutions (especially in competitive markets). When considering businesses operating in a successful data-driven environment, you will find several business leaders who are particularly good at extracting these insights at minimal levels of complexity. I shall refer to these leaders as ‘Yodas’ from here on out.
In order to curb frustrations, Lukes are provided significant problem solving freedom, with business leaders expecting limited value from their solutions. This approach does reduce frustration in the short-term, however, results in a large portion of analytics work no longer being ‘effective’ and usable by Yodas or business.
Effective Analytics
In order to balance the tradeoff between the usefulness of analytics work for Yodas and frustration among Lukes, the conceptual framework of ‘Effective Analytics’ is proposed.
To introduce the idea of Effective Analytics imagine that for every problem there are an infinite number of possible analytical solutions. Further, this plain of solutions ranges from simple to advanced/complex.

The solution of every problem will fall somewhere on this plain, with the exact point depending on the relationship between the Yoda and the Luke. Yoda naturally prefers simpler, more understandable solutions, whereas Luke favours more complex, advanced solutions. The precise location of where the solution lands will depend on the ‘strength of force’ exhibited by each party. The ‘strength of force’ for each party will depend on their respective credibility, as well as, the business importance of the problem.

Unfortunately, there is no framework that brings Lukes and Yodas into the same frame of reference. Instead, a compromise is made by either Yoda or Luke, leading to a loss being experienced by at least one party.
The Effective Analytics framework attempts to bring both Lukes and Yodas into the same frame of reference. This is done by adding another dimension to the solution plain. The new axis is responsible for measuring the effectiveness of solutions as they move from simple to advanced.

Using this approach, and stealing some theory from other research fields, efficient frontier solution curves can be created for each problem.

Every solution on the resulting curve will have a level of analytical effectiveness associated with it. Further, the group of effective analytics solutions can easily be identified and agreed upon between Yodas and Lukes before the detailed solution is constructed. This transparency will allow Lukes to develop an understanding of the business needs, curbing their frustrations. For Yodas, this approach will allow them to end up with usable and trustworthy solutions and, over time, very powerful Lukes.
How does a business create a culture of effective analytics?
This is a tricky question to answer, mainly because each business is unique and there is no single solution.
Before getting into the detail, there is an important distinction that needs to be made. A culture of Effective Analytics is the next step towards letting data lead the way, it is not the first step. If you are reading this article, it is assumed that your business already has a data-driven mindset, and the value of data already realised. Taking the first step to a data-driven mindset is another kettle of fish and will not be discussed here.
The key in allowing an environment of Effective Analytics to take hold is aligned with Ray Dalio’s concept of radical openness and transparency.
In most businesses, even those with flat hierarchies, there are clear divides between Yodas and Lukes. Operating in these independent spheres leads to misalignment between the subjective reality of Lukes and that of Yodas. By Yodas cultivating true business transparency, the Lukes are better able to understand that most business environments are significantly misaligned with the academic environments in which they have been trained. This requires Lukes to adjust their focus and value placement, with the key adjustment being the skill to differentiate between what is flashy and what adds real value to the business.
Further, cultivating this transparency encourages senior leaders (including both Yodas and the rest of the republic) to develop themselves in the realm of advanced analytics. A conceptual understanding of advanced analytics allows Yodas to better visualise the ‘effective solution frontier’, know where the proposed solution exists on the frontier and provide more effective guidance throughout this space.
Yodas that believe this realm should remain separate from their personal skill set run the risks of a) falling behind Yodas that embrace the next step in the Data Driven world, and b) not attracting or retaining Lukes as their particular skills are not rewarded through growth opportunities.
Understanding the Way of the Force
Getting into the detail, below are some tangible points that the Lukes and the Yodas need to understand/learn to achieve alignment and achieve a culture of effective analytics.
The Lukes must come to learn that:
- The coolest/most accurate models/techniques are not always the best solution to the problem. This will not make sense to them initially, there may even be significant resistance to this, but give them time and they will see that sometimes a spreadsheet is the best tool for the job.
- The problem you are solving should be used as a guide to which skills/techniques are relevant. Don’t take the deep learning hammer to every problem.
- Your time costs money. Use this to calculate the cost of the solution you are building, and then ask yourself the question, "Would I spend that much on what I am doing?" If not, then you need to rethink your approach.
From the Yodas’ perspective, it is important to know:
- The whole analytical landscape. Yes, it looks technical and scary, but get into it, if you truly believe this is the way your business is going to develop. If not, then it is worth rethinking your reasoning behind employing Lukes.
- Each step of the data driven evolution is larger than the last. This suggests that Effective Analytics is not a movement that businesses can take lightly, or to hedge their bets. Commitment issues in this space will likely result in little value gain and self-fulfilment of failure. For value to be added, trust and belief needs to be placed in this direction.
- Do not give in to the AI craze. If you are employing Lukes to solve your problems by "AI-ing" things, you are likely wasting both your and their time.
- Ensure you have an environment in which Lukes can work. Don’t hire Lukes and expect them to work miracles. You will be disappointed, and may even end up with a Vader (a really good Luke that has become negative).
- Lukes do not know the way of the force (i.e. how business works), so give them guidance. If you don’t do this, you will likely end up with really cool solutions but no way to implement or use them.
Conclusion
Well done! You have made it to the end of the idea (apparently people don’t normally last this long). I leave you with this piece of advice. If you are a Luke looking for a Yoda, I trust you take the above into account. If you are a Yoda that already has a Luke, please take this seriously and seek alignment in your business. If you are a Yoda with a Vader… well… May the odds be ever in your favour…