Videos Archive - PrestoDB

Ending DAG Distress: Building Self-Orchestrating Pipelines for Presto – Roy Hasson, Upsolver

Ending DAG Distress: Building Self-Orchestrating Pipelines for Presto – Roy Hasson, Upsolver dbt and Airflow is a popular combination for creating and scheduling batch data modeling and transformation jobs that execute in a data warehouse like Snowflake. Presto users querying the data lake need a similar solution that is simple to use and makes it easy to ingest, model, transform and maintain datasets, without having to write or manage complex DAGs. In this session you will learn how Upsolver built a tool that allows engineers, developers and analysts to write data pipelines using SQL. Pipelines are automatically orchestrated, are data-aware and maintain a consistent data contract between each stage of the pipeline. You will also learn how to introduce the idea of data products into your company to enable more self-service for your Presto users.

Querying streaming data with Presto, Amazon Athena and Upsolver

In this session, Yoni will present on querying streaming data with Presto and Amazon Athena including performance, data partitioning and compaction. In addition, we will demo using the Upsolver platform with Amazon Athena. In addition, he will share what they are working on with Prestodb.

Benchmarking Continuous Data Processing on Snowflake with Upsolver – Sean Spediacci, Upsolver

In this talk, we provide a glimpse at the results of our latest benchmark test which compares the speed and cost of processing data inside Snowflake (ELT) vs. processing and serving prepared live tables from a data lake using Upsolver (ETL).

Prism: Presto Gateway Service at Uber – Hitarth Trivedi, Uber

Prism is a gateway service for all Presto queries at Uber. It addresses Uber specific needs in four main areas – resource management, query gating, monitoring, and security. It is responsible for proxying over three million weekly queries from 6000+ weekly active users across all of Uber. Presto has variable execution times due to high multi-tenancy at Uber. Prism helps in overcoming those challenges using features like query routing, load balancing, query gating, session parameter checks, failover clusters which helps in maintaining a 99.9% availability and reliability SLA for Presto at Uber. Functionality – Query Execution: 1. Async execution API returns data stream 2. Async execution API returns File Descriptor – Routing – Prism can route queries to different clusters based on client sources. Other functionalities: Load Balancing, Query Gating, Failover, Session Properties, Security

The Data Lake House: Powering Open Architecture in the Cloud – Ori Rafael, Upsolver

Ori Rafael, Co-founder and CEO of Upsolver, will present the Cloud Lake House as the foundation of an open data lake architecture built on Apache Parquet. Ori will explain how this architecture supports diverse analytic consumers and use cases, from open-source Presto to proprietary data warehouses.