Updating the Presto Helm Chart to Support Presto C++

While not mandatory for deploying Presto, Helm offers a streamlined and customizable method of deploying Presto in kubernetes, simplifying what could otherwise be a complicated process. With just a few properties specified in a values.yaml file, the user is able to configure and deploy a fully fledged version of Presto at the sizing of their choosing.

As highlighted recently, Presto is evolving to incorporate Velox, a highly efficient C++ library for data processing into its execution workers. Recently, we began taking a look at the official Presto helm charts to understand the implications required to include support for the new C++ workers. This blog explores this process and details the new functionality.

Updating the Helm Charts

The Presto Helm Charts greatly ease the process of deploying Presto. However, they had been created prior to the advent of Presto C++. Due to this, they were incompatible with a C++ deployment and presented a number of challenges for utilizing the charts with the new execution engine. To resolve the compatibility issues, we added a new prestoCpp section to the values.yaml. Here we can see new properties added by this section:

# Enables Presto to use C++ based workers  
prestoCpp: 
  enabled: false
  # Image details for Presto C++ workers (overrides common image details)
  image:
    repository: prestodb/presto-native
    pullPolicy: IfNotPresent
    tag: ~

The addition of this section resolves the following unique challenges that the adoption of Presto C++ presents:

1. Making Presto C++ optional

While C++ is intended to be the default execution engine of Presto soon, there are still many deployments using the Java based execution engine. To avoid breaking compatibility for any deployment, the charts should be able to support not only C++ deployments, but also traditional Java deployments. To allow for this, the section includes a prestoCpp.enabled determining if the prestoCpp options should be used or not. If set to false, all other PrestoCpp options will be ignored by the templates and the charts will continue to function as they did previously. This means that anyone wishing to deploy a Java based version of Presto will be unaffected by these changes and will continue to be able to use the Presto Helm Charts without any negative affects.

2. Multiple Image Support

One notable aspect of Presto C++ deployments is that they still utilize a Java based coordinator. Therefore, to run Presto C++, two images are required: one for the workers and another for the coordinator. Prior versions of the Presto Helm charts only supported a single Presto image which would then be utilized for the deployment of both the coordinator and the workers. To make this compatible with Presto C++, we added a prestoCpp.image section to the values.yaml. Here the user can specify a repository, pull policy, and tag to be used in C++ scenarios. These settings, when enabled, will supersede the general image settings in the case of only the worker images. In this way, the image used for the workers can be decoupled from the one used for the coordinator and resource manager. By default, the Presto C++ image is set to be pulled from prestodb/presto-native, but this can be modified if desired.

3. Allowing Coordinator and Worker Config.properties Files to be Customized Separately

One aspect of using different images for the coordinator and the workers is that it increases the likelihood that someone would want to add separate configuration properties to each. Even when the same image is used for each, it could happen that the workers and coordinator have different configuration needs. Prior versions of the charts include some basic configuration differences between the coordinator and workers into their configmaps, but even still, more differences could be needed. For example, properties for kerberos authentication are only relevant to the coordinator regardless of the worker image.

To tackle this situation and make it useful for both Presto C++ and fully Java-based Presto scenarios, we made two adjustments. First, if prestoCpp.enabled is true, some basic properties necessary for the healthy functioning of Presto C++ are added to the config.properties of the coordinator. These are:

native-execution-enabled=true
optimizer.optimize-hash-generation=false
regex-library=RE2J
use-alternative-function-signatures=true
experimental.table-writer-merge-operator-enabled=false

Beyond this, this update adds coordinator.config, worker.config, and resourceManager.config properties to the values.yaml, each optional and overwriting the common root level config options. In this way, the user is given fully customizability of the configuration properties of each module of the cluster.

How to Use

Let’s look at an example of how we can use the updated chart to convert a fully Java Presto cluster to Presto C++. First, without making any adjustments to the values.yaml, we can deploy a basic cluster with Java-based workers:

Describing the worker pod, we can confirm it is running the Java based image:

Now, we open the values.yaml and scroll down to the new prestoCpp section. In this case, we make just one change, that is to set prestoCpp.enabled to true: