Teradata embraces open table formats boosting AI & data management capabilities

Teradata has announced its support for open table formats (OTFs) Apache Iceberg and Linux Foundation Delta Lake. This pioneering development is a distinct strategy towards adopting open-source technologies, thereby providing its clientele with exceptional choice and versatility in data management. This will enhance Teradata VantageCloud Lake, the next-gen, cloud-native analytics and data platform for AI, and Teradata AI Unlimited, the on-demand and cloud-native AI/ML engine, which will have public previews on AWS and Azure Marketplaces beginning in Q2 2024. Teradata's strategic outlook is primed to be future-ready, enabling corporates to adopt a contemporary data strategy offering flexibility and agility to implement Trusted AI at large scale.

Open_table formats symbolise a significant shift from proprietary data storage to flexible storage available across platforms. This development aims to enhance interoperability, cost efficiency, and choice. Teradata's support for open table formats and open cataloguing will enable a universal approach for reading multiple catalogues with consistent execution. It promises to deliver optimal performance in the market by leveraging unique methods of parallel processing, workload management, and query optimisation of shared data. Teradata retains its ability to offer a broad choice for data management without compromising its distinct capability.

David Menninger, Executive Director at Ventana Research, part of ISG, noted, "In today's data landscape, we're seeing wide adoption of open table formats with 51% of organisations actively adopting Delta tables and 27% adopting Apache Iceberg. This reflects the industry's focus on a single source of data and the ability to leverage multiple engines against that data. Teradata's commitment empowers customers to combine low-cost storage with enterprise performance and capabilities."

Teradata's fully open platform provides first-party services for Apache Iceberg and Linux Foundation Delta Lake OTFs. This includes full support for reading and writing data stored across various OTFs. Capabilities extend to AWS Glue, Unity, and Apache Hive catalogues and function in multi-cloud and multi-data lake environments. This connectivity aims to offer a unified data experience to Teradata clients without the necessity to relocate or transform data.

Highlighting the company's direction, Hillary Ashton, Chief Product Officer at Teradata, stated, "The future is open. Teradata has cultivated a thriving ecosystem for many years now, and our embrace of OTFs, as well as the expansion of our on-demand AI/ML offering, shows how committed we are to providing organisations with the most open and connected ecosystem for cloud analytics and data."

OTF support further bolsters Teradata's lakehouse capabilities, providing a flexible, cost-efficient, and user-friendly storage abstraction layer. Users can utilise AI Unlimited and other analytical engines against their own datasets, stored in their cloud object stores. AI Unlimited, designed as a fully connected AI/ML engine with OTF support, will soon be available in public preview on AWS and Azure Marketplaces. This is envisaged to accelerate AI innovation and make it more cost-effective. The company is also integrating with third-party tools – Airbyte Cloud, Apache Airflow, and dbt – as part of the modern data stack to speed up the building of data pipelines and increase productivity.

The company, on an innovative spree, has ensured OTF support will be available for VantageCloud Lake and AI Unlimited on AWS and Azure in June 2024. AI Unlimited will also be purchasable in a public preview on the AWS and Azure Marketplaces in Q2 2024. Integrations with Airbyte Cloud, Apache Airflow, and dbt are presently available.