Big data integration: Performance and Scalability Matter
![](https://content.greenbird.com/Social-share-Big-Data_2023-03-06-084428_bplx.png)
More data, more variation in data, higher and different resolutions of data, scheduled or event-triggered data and the need to process large amounts of data in as close to real-time as possible: these are the real-life data challenges that utilities around the globe are facing as they optimize their grid and satisfy regulatory requirements with all kinds of new data sources and decision support systems.
Greenbird’s Utilihive platform is designed to handle big data. And lots of it!
Utilihive enables real-time data flows within Advanced Metering Infrastructure (AMI), Smart Grid, Distributed Energy Resource Management (DERM) and other areas such as our Utilihive Datalake.
Performance and scalability are key.
Here are some examples, both from test bench scenarios and real customer cases.
Example 1: Rapid Processing of Smart Metering Data
Ideally, Advanced Metering Infrastructure moves data quickly and securely from one or more Headend Systems (HES) to a Meter Data Management system. Sometimes, data critical for grid operations such as outage events, needs to go into ADMS or other operational solutions. Transporting and transforming the data into the required formats is a core task for our Utilihive platform.
Here’s how this works in practice:
Utilihive processed, transformed, and handed off all data in under 15 minutes.
For full transparency:
Example 2: Processing Data From Multiple Power Grid Sources
When they implemented the same report using Utilihive Datalake, it was generated in just under 10 minutes.
If you find that hard to believe, so did our client. They told us that they had re-run the report several times, because they thought the results were impossible.
How Can Utilihive Achieve Results Like These?
Utilihive is different from other integration platforms or ESBs. How? It is built as network of reactive microservices (service mesh), following the actor model for highly concurrent applications and the principle of event-driven architecture. This allows for high performance, dynamic scalability and ensures high availability by design. In addition, we can often compress data to below 5% of the original data size, further boosting data processing performance.
In the future, utilities will need to handle growing quantities of data from sensors, meters, local producers and much more. We say: Bring it on!
ESB (Enterprise Service Bus) and iPaaS (Integration Platform as a Service) have similar kinds of role to play; they integrate en
Read moreEvent Driven Architecture is reactive and responsive. It enables utilities to handle large quantities of data and adapt to a rap
Read moreUncertainty has become a part of life for utilities. To build resilience in this environment, utilities must leverage data.
Read more