Recently, I sat and watched in horror as a control room performance engineer eyeballed thousands of power curves from wind turbines in order to identify under performance. Her tableau dashboard was the best she had and undoubtedly she was able to make /some/ impact, but a 30×30 pixel plot multiplied by thousands just blends into irrelevance and likely a lot of missed opportunities.
Another time, a solar performance engineer had worked with a data scientist to develop an availability analytic for the inverters. It was a "black box" machine learning based analytic that literally resulted in every inverter having an issue. A British friend described it as a "Bloody Red Dashboard" which I am sure was meant, to an American, as "crazy red dashboard" where everything was alerting; I took it as dripping in blood, and reeking of lost opportunity.
These two anecdotes represent about 80% of what I see day to day from asset managers, operator and performance engineers trying to tackle creating value with their data.
Five things to improve your Energy data analysis
I would like to share five things that I have learned while working with energy sector clients:
- Most insights in energy production are derived from a surprisingly small number of simple time series data sources, including: status, alarm and sensor data. Most are alarmingly bad at creating any value.
- Often, insights are developed in silos within each of these sources.
- With little effort, more comprehensive insights can be developed by breaking down these silos of data.
- By including additional structured and unstructured data created by human activity associated with the asset, insight value soars beyond that created through simple sensor observations.
- Finally, understanding what tribal knowledge is and how to apply it will enhance insights from even the most basic analytic.
Let’s go a little deeper into my thoughts behind each of these.
Insights from simple time series data sources are alarmingly lacking in value
Over the past five years, I have been directly exposed, time and again, to naive insights developed from limited time series datasets coming from sensor observations on energy producing assets.
We typically see SCADA data, or sensor observations, along with state and operational data treated similarly to what I described in the opening. These three well structured and generally clean data types can offer a wealth of insights when properly analyzed. But how do we do that?
Start with identifying what matters to the organization; it is obvious, right? You would be surprised how often the value is not clearn to those implementing these analytics. The financial impact to an organization can be measured in lost energy production due to chronic underperformance, lack of visibility into availability losses and finally the complete lack of awareness of asset health and maintenance needs.
By developing analytics that focus in on these three specific areas and then correlating that to specific causes, you can turn that "Bloody Red Dashboard" into an explicit list of actionable insights which often have a hard value attached to them.
Even using alarm and state data you can, applying a simple Pareto approach, you can easily identify the things that are causing the organziation the most chaos operationally and financially. Yet, more often than not, operators cannot tell you what types of events on their assets are impacting that asset’s value to the organization; this lack of visibility results in analytic development efforts in areas that offer very little value. A double lost opportunity.
I have found that by identifying the 5 things that are having the most impact on an organization and then working backward from that to develop analytics and insights result in radical adoption – and value.
Recognize data silos and their impact on insights
A data silo exists when two or more data sources for a single asset exist and they are not analyzed together.
Using a single time series data source to develop an analysis has become a common occurrence in the energy space. Data scientist get so focused on creating models from years of depth in this data that they often forget to do discovery beyond the single source they are working with. Or, just as likely, they are not even aware of other available sources.
The resulting silo’d analysis will typically be less actionable and therefore will be lacking value.
Imagine taking a performance analysis on a wind turbine where you are considering environmental factors as well as generation and maybe a few other sensor observations from the turbine; with that, you can develop an interesting model for that turbines performance. Indeed, there are even documented IEC standards on how best to do this. In the end, however, it will only tell you that you have a performance issue, which you likely already even knew – or at least had an eyeball on like the engineering I described earlier.
Similarly, you may have performed a Pareto analysis on the faults occurring across your fleet of turbines and understand where you are loosing the most availability due to those faults.
With those two simple analytics, you end up in a place where you are aware of underperformance as well as the impact of specific alarms on availability; this is not a bad place for starting, but, imagine if you combined those two analytics?
Break down silos to form more complete insights from data
Breaking down data silos will immediately enrich the insights you are producing and result in more value created from the data.
What does mean? Go beyond the single data source approach and include whatever additional time series data that is available. Merge those sensor observations with state and alarm data.
Keeping the analysis simple while expanding the data horizontally is what enables you to quickly iterate while developing richer insights, so stay with a simple stats based analysis when starting out. Machine learning has it’s place, just avoid it starting out.
Take the example from the section above; you have an IEC based power curve analysis running and along side it a Pareto analysis on fault data. In combining these two approaches and data sources, you can immediately assign lost value to alarms and warnings. Also, you can start classifying different types of underperformance; some you probably cannot impact through maintenance while others you can.
That "Bloody Red Dashboard" instantly turned to an off shade of pink as you narrow your insights to the things you can actually have an impact on to create value.
By extending this example to include additional sensor data, you will also be able to identify asset data patterns that hint towards a specific fault. This enables you to assign value to missed maintenance opportunities and even availability losses resulting from those.
Break down your data silos and break free of the OEM’s ‘warnings’ and fancy software upgrades; both will hold back your ability to create real and likely transformational value with your time series data sources.
In a future post, I will be sharing actual examples of data and Python demonstrating what I have described. Remember, this is a pure stats based approach and you do not need a data scientist or machine learning to accomplish this.
Start including data created by human activity associated with assets
Data created by human activity generally falls into one of the following types: work order, work log or inspection data. It is time series, as well, and fits amazing well with the sensor, state and fault data generated by your assets.
A challenge in using this data in your analysis generally lies with IT departments not wanting to make the data available outside of the applications used to generate it. Just as it is with the OEM’s, I have found that IT teams often feel threatened by allowing data they manage to be used outside of their organization. It is a sad fact that that data hoarding and protectionism happens.
However, once you have liberated this data, the value that it can add back to the organziation is immense.
Taking the example from earlier, you likely already have identified your top 5 events that impact the organization’s bottom line. With that, you can identify when those events are occurring and on what assets; start correlating that to specific human activity.
Now you can start to identify value opportunities through improved maintenance scheduling; additionally, you can flush out the reverse of that, where maintenance is subpar. Imagine being able to identify repeat visits to solve a single persistent issue and then build a better maintenance run book for that issue? You would even be able to better hold them OEM or contract maintenance teams more accountable for their work.
Tribal knowledge inclusion in an energy data analysis is the holy grail in the development of true insights
"When I see this happen, I typically do that and the problem goes away."
No amount of process flow diagramming can truly capture the logic maintainers use in resolving an issue.
I recall being a young Airman working as a Crew Chief on the USAF KC-10A many times where the "technical order" or "process" document lacked the detail needed to successfully complete a repair in an efficient way. Once, I was sealing a light covering, following step by step instructions in the document. I had managed to get this mil-spec sealant all over me and the aircraft, creating a hell of a mess. It took hours to clean up from a 15 minute task. Afterwards, a greybeard crew chief pulled me aside and explained his easier and cleaner approach to completing the task. His approach followed the process but included some hard won learnings. The next time I had to perform this task, I followed his advice and was in and out in about 15 minutes.
That is tribal knowledge. Typically learned through experience in completing a task and passed on verbally or through work logs.
Through advanced natural language processing (NLP), a skilled analytic developer working along side a subject matter expert can develop the necessary code and logic to extract these tribal insights.
How is that useful? You have already identified the top 5 events causing the most lost value for the organziation and now you can add this NLP value multiplier to that by idenitfying specific things in the previous work logs and including that in your existing time series data analysis. This can evolve into specific data patterns needing specific tools or parts for a preemptive dispatch or for a more efficient repair during already scheduled down time.
I really can not do NLP tribal knowledge extraction justice in this general post; in the near future I will return to this and include some actual data and working python that does what I’ve described.
I have spent much of my professional engineering career developing applications that help people get more value out of their data and content.
This experience ranges from building "hurricane proof" newsroom applications for TBO.com in Florida to "photo contest" applications for National Geographic and, now, these past five years, creating tools that allow operators to improve performance and impact availability of assets in the energy sector.
import this