Comet ML Inc., a startup that provides tools for building artificial intelligence models, is partnering with Snowflake Inc. to ease the work of enterprise machine learning teams.

The partnership was announced this morning. As part of the collaboration, Comet has developed an integration for Snowflake’s Snowpark tool. According to the startup, its new integration will simplify AI projects that focus on using data stored in Snowflake to train neural networks.

New York-based Comet is backed by $69.8 million in venture funding. It provides a platform that helps software teams manage the neural networks, training datasets and other files they use in AI development projects. It also promises to ease related tasks, such as monitoring models deployed in production for technical issues.

Snowflake, in turn, provides a popular cloud data platform that enterprises use to store and process business information. Some customers use the information they store in Snowflake to train their AI models. A retailer’s developers could, for example, train a revenue forecasting model on store sales records.

Developers often require the ability to check which of the datasets their company stores in Snowflake was used to train an AI model. That’s necessary to verify the dataset used in a machine learning project didn’t contain any errors. Finding the records on which a model was trained is also necessary for other tasks, such as troubleshooting technical issues.

Comet’s new integration will make finding the training dataset associated with a neural network easier. Thanks to the integration, the platform can automatically display which Snowflake dataset was used in what AI project. It also surfaces related details: If there are multiple versions of a given file, it can point out which specific version was used for AI training.

“With the integration of Comet and Snowflake, we’re enabling customers to better track and version their Snowflake queries and datasets, ultimately improving their understanding of how their models were developed and how changes made to the data affect model performance,” said Comet co-founder and Chief Executive Officer Gideon Mendels.

The feature is powered by a Snowflake tool called Snowpark that the latter company debuted in 2020. It’s often used by companies for AI development. Snowpark makes it possible to run workloads, such as AI training code, directly on Snowflake’s platform instead of having to deploy the code on separate infrastructure.

Comet’s integration links AI models to training datasets by logging the queries that go through Snowpark. Using the collected information, it can determine which datasets were used in what machine learning project. It also logs the queries that were used to retrieve those datasets, as well as related metadata.

Image: Comet