Analyzing and Planning in Real Time
To help accelerate the energy transition toward a fully renewable power system the company developed a set of algorithms that allow it to accurately predict the volume of generated power and the energy price, for the Day Ahead, Intraday, and Imbalance markets.
Founded in 2017, Dexter Energy opted for Google Cloud from the beginning to ensure it had a scalable infrastructure and innovative solutions for implementing its AI-based algorithms and very detailed market analysis. The data comes from everywhere: customers, the energy market, weather, and producers. A large part is in motion: the data is processed in just-in-time flow (data streaming).
AI-Based Algorithms and Detailed Market Analysis
The data comes from anywhere: customers, the energy market, the weather, and producers. A lot of it is dynamic: data is processed on a just-in-time basis (data streaming).
"Overall, we store over 200 terabytes, and process over 2,000 workloads and 250 gigabytes of data daily," explains Danny Bennett, DevOps and MLOps Engineer at Dexter Energy. "The analysis and planning are done in real-time. In other words, the slightest latency or malfunction could be catastrophic for our business. So we need reliability, a large processing capacity, and also scalability to cope with possible peaks in traffic."
This infrastructure scalability is all the more important as Dexter Energy has big ambitions when it comes to future growth. Founded in the Netherlands, the company is currently active in The Netherlands, Belgium, and Germany, currently expanding to Italy. In the near future, it is looking to expand to the UK and some other European countries. "And that's just the start!" says Fons de Leeuw.
Developing Quickly, Easily, and in a Scalable Way
Opting for a cloud provider with the same, evident commitment to sustainable development was a no-brainer for Dexter Energy. But this was not the only reason for the company's decision. Among its selection criteria, technological synergy clearly played an important role, as Danny Bennett points out.
"We share Google Cloud's approach to AI. Google's research in this area is clearly in line with what we are looking to implement. What's more, Google Cloud solutions are known for just how easy they are to familiarize yourself with, especially since they're well-documented. These solutions are based on open source software, which is a very important factor for us because it means they rely on standards and tools we enjoy using while allowing us to remain in control of what we build."
So it's no surprise Google Kubernetes Engine (GKE) became one of the technological building blocks that caught Dexter's attention. Goal: to build an intrinsically scalable architecture by capitalizing on GKE's containers and features to simplify and optimize management. The company also uses serverless technologies and event-driven programming to reduce administration requirements and maximize costs.
"But if we had to pick just one reason, BigQuery was essentially the 'killer app' for us," says Danny Bennett.
"BigQuery allowed us—and continues to allow us—to store and query large amounts of data quickly and easily. In addition, given the complex balance we're working with, it's clearly a big plus to be able to capitalize on Google Cloud's integrated approach, whether it's storage, analytics, ML, AI, or the acceleration of processing queries in real-time for all types of data, without forgetting the scalability and serverless computing that save us time and money, and provides flexibility."
"Another example of open source technology working well with Google Cloud is our use of the TileDB library on top of Google Cloud Storage, this allows us to scale storage of large amounts of historical data with a different use case from BigQuery," adds Fons de Leeuw.
