Big-data company Cloudera Inc. is getting into the observability game with the launch of a new tool that provides insights into the data, applications and infrastructure they operate on-premises and across public clouds.

Cloudera Observability is available now for all Cloudera Data Platform users, the company said, enabling them to optimize costs and improve the performance of their infrastructure and services.

Cloudera said the new tool comes in response to the reality that virtually every enterprise today is operating hybrid and multicloud environments. With enterprise data stored across multiple public clouds and also on-premises, it has become almost impossible for enterprises to keep track of cloud consumption across various business departments. As a result, they have no easy way to manage their cloud spending budgets and troubleshoot problems across different infrastructures.

To change this state of affairs, Cloudera Observability is designed to help companies monitor, understand and optimize their CDP deployments, no matter where the platform is deployed. The tool is available to every CDP user at no extra cost.

It can help users to optimize their use of big data engineering tools like Apache Hive, Apache Impala and Apache Spark, with features such as customizable automatic actions and prebuilt actions to create alerts, proactively avoid issues and optimize workload performance. Users can also pay for Cloudera Observability Premium, which adds higher-value capabilities such as deeper insights and richer, automated troubleshooting, the company said.

Cloudera Chief Executive Rob Bearden (pictured) said Cloudera Observability helps companies with multicloud environments achieve a global view of their spending on cloud-based infrastructure and services. “With Cloudera Observability, customers get unprecedented visibility into workload and resource utilization to better control and automatically manage budget overruns, and improve performance,” he said.

One notable advantage of Cloudera Observability is its interoperability with Apache Iceberg, which is one of the key building blocks of open-source data lakehouses, or repositories companies use to store all of their corporate data in one place. Data lakehouses have become popular with enterprises today, as they make it possible to store both structured and unstructured data together in one place and run quick analytics on it, thereby eliminating data silos and increasing collaboration between teams.

Cloudera said early adopters have already realized some significant savings with its observability tool. In one case, it said a customer was able to identify rogue users who had started sending hundreds of millions of unnecessary queries, severely impacting the performance of other essential workloads. Cloudera Observability eliminated the need to gather information on what was causing this, instantly delivering the insights required to identify the root cause and resolve it in the first attempt.

Everest Group analyst Nitish Mittal said Cloudera Observability should prove useful, simply because so many companies struggle to rein in their cloud costs. “Cloudera has employed metadata in a unique way to give CDP users a way to manage their cloud costs, including identifying rogue users, coordinating workloads for optimal cost and flagging potential resource overruns,” he said.

In addition to price comparisons, cloud costs can vary depending on what resources being rented and when they’re needed: Think about the price paid for an Uber in the moment versus reserving a car in advance. All of the cloud providers have complex pricing rules that provide breaks on these “reserve instances” and spot pricing breaks when they have spare capacity.

The cloud costing business has been in retreat for several years: A decade ago there were more than a dozen competitors. Since then a few have remained, including Cloudorado, another free service that has an attractive dashboard and compares rates across more than a dozen providers. There is also Flexera Inc.’s One, which has a range of monitoring, observability and costing tools for different cloud platforms, while Cloudkeeper.ai is just for Amazon Web Services.

With reporting from David Strom

Photo: SiliconANGLE