Fivetran unveils new capabilities supporting Delta Lake on Amazon S3

Data integration company, Fivetran, has recently announced new capabilities that include support for Delta Lake on Amazon Simple Storage Service (Amazon S3), thereby enhancing its services for data lake users.

Fivetran's data platform is designed to automatically convert customer data into Delta Lake format, anonymise personally identifiable information (PII), as well as cleansing and normalising the said data.

This approach aims to address complexities associated with data lakes and issues concerning governability, opening opportunities for advanced initiatives like generative Artificial Intelligence (AI) and Machine Learning (ML).

With hundreds of thousands of data lakes operating on Amazon S3, a scalable, secure, and high-performing object storage service from Amazon Web Services (AWS), this update means Fivetran customers can easily access their Delta Lake tables once data has been stored in Amazon S3.

Data lakes are ideally suited for handling vast amounts of unstructured and semi-structured data due to their flexibility and scalability. Fivetran's automation transforms these data lakes from traditionally ungoverned repositories of data into organised, easy-to-access, and governed data stores.

This then enables organisations to build various use cases quickly, such as predictive analytics, AI applications, ML models, and Large Language Models (LLMs).

The announcement follows Fivetran's April proclamation supporting Amazon S3 with Apache Iceberg, another leading high-performance format.

Fraser Harris, VP of Product at Fivetran, expressed his excitement, saying, "We are thrilled to enable our customers to seamlessly leverage Delta Lake on Amazon S3. Data lakes have proven to be the ideal foundation for machine learning, AI and generative AI projects. This enhancement represents a significant step forward in simplifying data management for such initiatives."

Fivetran's no-code platform offers a simple and flexible way for enterprises to move data from almost any data source to any destination while ensuring industry-leading security and compliance, cost efficiency, and ease of use.

Their data platform automatically converts customer data into Delta Lake format and guarantees data quality by anonymising PII, cleansing and normalising the data. This way, customers can avoid the complexity and lack of governability that usually hinder data lake adoption at scale.

With more than 400 pre-built connectors and a fully managed data pipeline platform, Fivetran supports on-premises and cloud databases, data warehouses, SaaS applications, events, and files. Fivetran also provides the option to create custom connectors. This portfolio of source compatibility enables customers to unify their data in the data lake, regardless of where it currently resides.

With 99.9% uptime and self-healing pipelines, Fivetran enables brands across the globe, including Autodesk, Condé Nast, JetBlue, Lufthansa, Morgan Stanley and Pitney Bowes, to accelerate data-driven decisions and drive business growth. Fivetran is headquartered in Oakland, California, with offices around the world.