On-Demand Recordings from PrestoCon’s, Webinars, Meetups, and more
While using the Presto Iceberg connector, the in-heap cache in Presto is likely overloaded. In this talk, Beinan and Chunxu will share the design, implementation, and optimization of the off-heap cache to address the scalability challenges. You will learn how to cache Iceberg data and metadata for the Presto Iceberg connector, followed by future work on improving table scans using Apache Arrow.
Connector based architecture is one of the powerful features in Presto for extensibility. While we have a solid pack of many connectors, the ability to reuse an existing external snippet to fetch data and access through Presto will make it enormously helpful. For example, consider accessing mainframe code through Presto using simple SQL which is quite cumbersome to handle by creating a connector paradigm. Ravishankar explores how he implemented this feature using a protocol server and a protocol connector which eventually helped him to achieve a patent on the concept.
Apache Iceberg is an open table format for huge analytic datasets. The Presto Iceberg connector consolidates the SQL engine and the table format, to empower high-performant data analytics. Here, Beinan and Chunxu would like to discuss and share the architectural design of the Presto Iceberg connector, advanced Iceberg feature support (such as native iceberg connector, row-level deletion, and iceberg v2 support), and the future roadmap.
Apache Iceberg is an open table format for huge analytic datasets. At Twitter, engineers are working on the Presto-Iceberg connector, aiming to bring high-performance data analytics on Iceberg to the Presto ecosystem. Here, Chunxu would like to share what they have learned during the development, hoping to shed light on the future work of interactive queries.