Headless BI Architecture and Trade-offs – Pavel Tiunov, Cube Dev

Headless BI Architecture and Trade-offs – Pavel Tiunov, Cube Dev

There has been a proliferation of tools in different categories of the modern data stack. This talk will focus on the Headless BI category and Cube’s implementation of Headless BI. Headless BI injects a component between data warehouses and other data sources and tools on the other side of the stack (e.g. CDP, data exploration tools, custom data apps, etc.). This new component encapsulates several critical functions like data modeling, access control, and aggregate awareness while deliberately omitting others, like data visualization and presentation. We’ll explore: – Keeping data models separate from data sources and not substituting data modeling with mere data transformation. – Managing access control centrally, aggregate awareness, and caching in a separate layer upstack from data consumers. – Removing data presentation features and embracing data accessibility via a set of APIs.

Query Execution Optimization for Broadcast Join using Replicated-Reads Strategy – George Wang, Ahana

Query Execution Optimization for Broadcast Join using Replicated-Reads Strategy – George Wang, Ahana

Today presto supports broadcast join by having a worker to fetch data from a small data source to build a hash table and then sending the entire data over the network to all other workers for hash lookup probed by large data source. This can be optimized by a new query execution strategy as source data from small tables is pulled directly by all workers which is known as replicated reads from dimension tables. This feature comes with a nice caching property given that all worker nodes N are now participating in scanning the data from remote sources. The table scan operation for dimension tables is cacheable per all worker nodes. In addition, there will be better resource utilization because the presto scheduler can now reduce the number plan fragment to execute as the same workers run tasks in parallel within a single stage to reduce data shuffles.