Iceberg Branches and Tags with Presto

    By: 

    Reetika Agrawal, Software Engineer - IBM

    Modern data lakehouses increasingly require versioned data access, auditability, and safe experimentation without affecting production systems. Apache Iceberg allows you to maintain multiple concurrent timelines of a table through Branches and capture static historical points using Tags. This mechanism is heavily inspired by Git but operates on underlying table snapshots.

    In this blog, we are going to see the support provided by PrestoDB for Apache Iceberg branches and tags, which includes:

    • Creating and dropping branches and tags
    • Querying branches and tags
    • Mutations on branches
    • Running workloads without impacting the production data
    • Support for Java workers and Prestissimo (Presto C++)

    All the mentioned Iceberg branch and tag functionalities are available in PrestoDB version ≥ 0.297

    What are Iceberg Branches and Tags?

    Branch: A branch is a mutable reference to a snapshot. Write operations can be performed on it independently.

    Use cases:

    • Data validation pipelines
    • Audit workflows
    • Experimentation
    • CI/CD for data

    Tag: A tag is an immutable reference to a snapshot.

    Use case: Compliance snapshots

    Creating Iceberg Branches Or Tags in PrestoDB

    PrestoDB provides SQL syntax to create tags and branches directly.

    Create a Branch Or a Tag

    presto> ALTER TABLE iceberg.default.mytable CREATE TAG 'audit-tag';
    presto> ALTER TABLE iceberg.default.mytable CREATE BRANCH 'audit-branch';

    Create a Branch Or a Tag for a Specific Snapshot

    presto> ALTER TABLE iceberg.default.mytable
    CREATE TAG 'audit-tag'
    FOR SYSTEM_VERSION AS OF 3;
    presto> ALTER TABLE iceberg.default.mytable
CREATE BRANCH 'audit-branch'
FOR SYSTEM_VERSION AS OF 3;

    Create a Branch Or a Tag Using Timestamp

    presto> ALTER TABLE iceberg.default.mytable
CREATE TAG 'audit-tag'
FOR SYSTEM_TIME AS OF TIMESTAMP
'2024-03-02 13:29:46.822 America/Los_Angeles';
    presto> ALTER TABLE iceberg.default.mytable
CREATE BRANCH 'audit-branch'
FOR SYSTEM_TIME AS OF TIMESTAMP
'2024-03-02 13:29:46.822 America/Los_Angeles';

    Create a Branch Or a Tag with a Retention Policy

    presto> ALTER TABLE iceberg.default.mytable
CREATE TAG 'audit-tag'
FOR SYSTEM_VERSION AS OF 3
RETAIN 7 DAYS;
    presto> ALTER TABLE iceberg.default.mytable
CREATE BRANCH 'audit-branch'
FOR SYSTEM_VERSION AS OF 3
RETAIN 7 DAYS;

    Retention allows automatic cleanup of old metadata references.

    Querying Iceberg Branches and Tags in PrestoDB

    Querying a Branch

    PrestoDB supports two syntaxes for querying branches.

    • Querying the branch using FOR SYSTEM_VERSION AS OF syntax:
    presto> SELECT * FROM table_name FOR SYSTEM_VERSION AS OF 'branch_name';
    • Querying the branch using dot notation syntax:
    presto> SELECT * FROM "table_name.branch_branchName";

    Here, branch_ is the keyword required along with the branchName we are trying to query

    Querying a Tag

    presto> SELECT * FROM table_name FOR SYSTEM_VERSION AS OF 'tag_name';

    Dropping Branches and Tags

    Dropping references is straightforward. Dropping a branch or tag does not delete underlying data. It simply removes the reference to the snapshot.

    presto> ALTER TABLE users DROP BRANCH 'branch1';
    presto> ALTER TABLE users DROP TAG 'tag1';

    Mutating Iceberg Branches from PrestoDB

    One of the most powerful capabilities is isolated mutations on a branch. All the operations below modify only the branch snapshot lineage, leaving the main branch unaffected.

    This enables powerful workflows such as:

    • Staging transformations
    • Data quality validation
    • Safe backfills
    • Pipeline experimentation

    Example:

    presto> ALTER TABLE orders CREATE BRANCH 'audit_branch';

    All mutations can now be directed to that branch.

    Insert into a Branch

    presto> INSERT INTO "orders.branch_audit_branch"
    VALUES (1, 'Product A', 100.00);

    Update Data in a Branch

    presto> UPDATE "orders.branch_audit_branch" SET price = 120.00 WHERE id = 1;

    Delete Rows from a Branch

    presto> DELETE FROM "orders.branch_audit_branch" WHERE id = 2;

    Merge into a Branch

    presto> MERGE INTO "orders.branch_audit_branch" t
    USING source_table s
    ON t.id = s.id
    WHEN MATCHED THEN UPDATE SET price = s.price
    WHEN NOT MATCHED THEN INSERT (id, product, price)
    VALUES (s.id, s.product, s.price);

    Truncate a Branch

    presto> TRUNCATE TABLE "orders.branch_audit_branch";

    Support in Prestissimo (Presto C++)

    If you are running PrestoDB with Prestissimo (C++ workers), most of the above branch and tag functionality works seamlessly. In Prestissimo clusters, branch mutations are currently limited to:INSERT TRUNCATE

    Support for additional mutation operations may be added in future releases.

    Final Thoughts

    With the help of iceberg branches and tags, version control can be implemented in data lakes using Git version control systems. PrestoDB allows access to these using simple SQL constructs. With the help of such features, PrestoDB allows safe experimentation, auditing, and reproducibility in modern data systems.

    If you’re building lakehouse architectures with Presto and Iceberg, branches and tags should become a fundamental part of your data workflow design.

    Follow Us