Many utilities companies are contemplating how they can become more data-driven by using data to create an intelligent, highly personalized service - with trend analysis capabilities, smart optimizations, and reduced operating costs.

It’s true that data management has a direct impact on organizational performance. However, whether you’re in electric, water or gas utilities or in another sector completely, the question remains: how do you manage it?

Data mesh is the latest trend in organizational data management, and it can massively help with data management. This article explains how it differs from traditional methods, and why utilities companies in particular need to move to a distributed data mesh to up their data management game.

How Do Organizations Currently Deal with Data?

Utilities must prioritize collecting and interpreting the wealth of information at their disposal.Investing in a next-generation data lake is one way to gather business insights based on data. The problem is, data lake architecture tends to have common failure modes.

To prevent this, organizations need to move from the centralized idea of a lake or data warehouse, and towards something that is closer to modern distributed architecture: putting domains first, and treating data as a product.

From Warehouses to Clouds: Enterprise Data Platform Architecture So Far

The first generation: data warehouses are solutions with large price tags that only a small group of specialized people understand.

The second generation: big data ecosystems, with long-running batch jobs managed by data engineers. The result is data lake monsters that offer little in the way of analytics, which has left many companies feeling let-down.

Third/current generation: the addition of streaming for real-time data availability, which unifies batch and stream processing for data transformation. It also allows businesses to fully embrace cloud-based managed services for storage, machine learning platforms, and data pipeline execution engines. The downside: it still suffers from many of the underlying characteristics that led to the failures of the previous generations. This is where data meshes can help.

Innovative utility companies are moving forward in the same direction: From big legacy towards microservices, from process-centric to event-driven, from monolithic data storage to a data mesh.

What is Data Mesh?

Data meshes address many of the failure modes common with a traditional centralized data lake or warehouse. The data mesh paradigm shift comes from a modern distributed architecture approach.

A data lake can be seen as a data monolith. With a data mesh, that mass of data is broken down into separate streams. It can be transformed to create a joint aggregate view of the business domain. Data streams are owned by independent teams or users, who are, in turn, attached to business experts who analyze the data for insights.

Each domain generates a huge amount of data as a byproduct. This data can be used by many different groups in the utilities industry:

• Data engineers: They need all domain data to generate modular and OLAP cube base data. It is also necessary for testing before starting transformations.
• Innovators: They require an overview of energy usage patterns to expand their data-driven services to specific clients.
• Data scientists: They build recommended systems and need up-to-date data for training new systems.
• Management: They want an overview of company growth.

The data mesh is applicable to all industries, but it has a unique set of benefits for utilities companies.

Why Utilities Need a Distributed Data Mesh

The future of utilities is analytics-driven. Therefore, energy companies will need an architecture that handles huge amounts of data; an architecture that can deal with this data in a reliable way and process this information in real-time.

DevOps style agility is essential so utilities can react to changes or deliver new services. A distributed data mesh can provide this. By taking this approach to data management, utilities can extract the true value of their data.

That said, shifting to a data mesh is not easy. Time, costs, and feasibility must be considered to eliminate risks. This is why we built Utilihive. Our Energy Service Mesh is fast emerging as the go-to architecture and platform for the digital utilities industry. It enables utilities to accelerate their energy transition and make the process smoother.

Utilihive is an integration platform designed to be the linking layer between systems of record (SORs) and systems of innovation. The platform includes out-of-the-box integrations to seamlessly link the various technologies used in utilities, including energy management systems. Our platform also monitors and analyzes data around the clock to give utilities the insights they need to make informed business decisions.

Fast integrations improve flexibility and agility, and help utilities thrive now and in the future.

About Greenbird:
Greenbird offers out-of-the-box system integration for utilities. We are a true DevOps company, delivering unique time-to-market and reliability. We were named a Gartner ‘Cool Vendor’ in 2018 because of our domain-specific and flexible integration capabilities, crucial for creating easy-to-consume integrated solutions. Utilihive empowers utilities to manage their data flow faster and smoother than traditional system integration models while accelerating the journey towards the energy revolution. Greenbird is based in Oslo, Norway.