UPDATED 08:00 EDT / JUNE 15 2023

BIG DATA

Dremio leverages generative AI for data discovery with Text-to-SQL and more new features

Data lakehouse company Dremio Corp. is jumping on the generative artificial intelligence bandwagon, launching a new Text-to-SQL capability today and announcing additional features to come later.

The company said the updates are all about empowering customers with more innovative tools for data exploration, analysis and discovery across both cloud and on-premises systems.

Dremio is a startup backed by more than $410 million in funding, offering a data lakehouse platform that enables companies to access information that’s stored across various clouds and on-premises servers from one location, without moving it. In that way, its customers can combine and analyze massive amounts of data, no matter where it’s stored, to try to discover helpful business insights.

The new Text-to-SQL Experience refers to a tool that enables users to specify what they’re trying to query in natural language. The tool will then translate those queries into Structured Query Language code. In other words, it’s making data discovery more accessible, as users without SQL skills will be able to create queries themselves – something that wasn’t possible before. Dremio said Text-to-SQL translation is based on a semantic understanding of metadata and data, and enables extremely accurate SQL generation.

Analyst Doug Henschen of Constellation Research Inc. said Text-to-SQL is the most obvious use case for generative AI with Dremio. “It promises to broaden querying capabilities to people who don’t know how to write SQL while also speeding the development of SQL queries, whether the user can write SQL or not,” he said.

In addition to Text-to-SQL, Dremio said it’s working on an AI-powered tool that automatically correct SQL queries that will be made available soon.

A second, upcoming feature called Autonomous Semantic Layer is designed to aid in data discovery and documentation, the company said. It will work by automatically learning the intricate details of users’ data before producing descriptions of datasets, columns and relationships. The idea is that this will eliminate the need for manual cataloging, Dremio said. In addition, the tool will also be able to learn workloads autonomously and create reflections to accelerate data processing, giving users an AI-powered semantic layer for generating data insights.

“Dremio is applying generative AI to semantic modeling, letting it discover and learn about data relationships,” Henschen explained. “It’s a good idea that should help speed and improve data discovery and promote consistency, reliability and reuse.”

Dremio is also working on Vector Lakehouse Capabilities, which will make it easier for customers to store and search vector embeddings. Vector embeddings can be thought of as data representations that carry semantic information. They’re a byproduct of AI models trained on large sets of input data. Dremio said the capability can serve as the foundation for customers to build machine learning applications such as recommendation systems and anomaly detection tools.

“In search, for example, vector embeddings can be used to find similar documents or web pages,” Henschen explained. “In generative AI, vector embeddings can be used to generate new text that is semantically similar to the input text.”

“By delivering vector database capabilities directly in the lakehouse, we will enable companies to build AI-powered applications without creating additional data silos in the organization,” Dremio co-founder and Chief Product Officer Tomer Shiran said. “Generative AI will transform data engineering, data science and analytics over the coming years, and we are excited to provide our users with the industry’s most powerful tools to uncover the true potential of their data.”

Image: Dremio

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One-click below supports our mission to provide free, deep and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU