Ending DAG Distress: Building Self-Orchestrating Pipelines for Presto – Roy Hasson, Upsolver
Ending DAG Distress: Building Self-Orchestrating Pipelines for Presto – Roy Hasson, Upsolver dbt and Airflow is a popular combination for creating and scheduling batch data modeling and transformation jobs that execute in a data warehouse like Snowflake. Presto users querying the data lake need a similar solution that is simple to use and makes it easy to ingest, model, transform and maintain datasets, without having to write or manage complex DAGs. In this session you will learn how Upsolver built a tool that allows engineers, developers and analysts to write data pipelines using SQL. Pipelines are automatically orchestrated, are data-aware and maintain a consistent data contract between each stage of the pipeline. You will also learn how to introduce the idea of data products into your company to enable more self-service for your Presto users.
The Past, Present, and Future of Presto – Philip Bell, Meta
PrestoDB recently underwent major architectural updates as the Presto Foundation grows membership and is looking to vastly grow the number of new commits and forks. Achieving this desired end state required successful refactoring and improving of Presto’s already impressive speed, efficiency, reliability, and extensibility. Establishing PrestoDB as a premier Open Source project required a major commitment of time and resources from Meta to ensure the community can benefit from this project for years to come, as well as positioning PrestoDB to evolve beyond what Meta alone could create. Members of the Presto Foundation need more of you to be involved in this major evolution in Presto’s history and core components, and bring your own inventive ideas to the mix.
The Data Lake House: Powering Open Architecture in the Cloud – Ori Rafael, Upsolver
Ori Rafael, Co-founder and CEO of Upsolver, will present the Cloud Lake House as the foundation of an open data lake architecture built on Apache Parquet. Ori will explain how this architecture supports diverse analytic consumers and use cases, from open-source Presto to proprietary data warehouses.