Skip to main content
Version: 8.8 (unreleased)

Camunda back up and restore

note

The Camunda 8.8 release introduces breaking changes for Operate and Tasklist.

note

If the Camunda application(s) cannot access Elasticsearch with cluster-level privileges, it is possible to run the backup of Operate and Tasklist indices (steps 2 and 4 from the backup procedure below) as a standalone application separate from the main application (see standalone backup application).

Use the backup feature to back up and restore your Camunda 8 Self-Managed components and cluster.

About this guide​

This guide covers how to back up and restore your Camunda 8 Self-Managed components and cluster. You should automate the procedures in this guide, choosing tools that fulfil the requirements of your organization.

  • Regularly back up the state of your Zeebe, Web Applications (Operate, Tasklist), and Optimize components without any downtime. You can also back up and restore Web Modeler data.

  • Restore a cluster from a backup if any failures occur that cause data loss.

Prerequisites icon

Prerequisites

Set up a snapshot repository in the secondary datastore and configure component backup storage.

Create a backup icon

Create a backup

Create a backup. This involves backing up the WebApps and the Zeebe Cluster.

Restore a backup icon

Restore a backup

Restore a backup. This involves restoring Elasticsearch/OpenSearch and the Zeebe Cluster.

note
  • The examples in this guide are based on using the following tools: curl, jq, and kubectl.

Why you should use backup and restore​

  1. Soft pause exporting in Zeebe. See the Zeebe management API.
  2. Trigger a backup x of the web applications. See how to take a web application backup.
  3. Trigger a backup x of Optimize. See how to take an Optimize backup
  4. Wait until the backup x of web applications is complete. See how to monitor a web application backup.
  5. Wait until the backup x of Optimize is complete. See how to monitor an Optimize backup.
  6. Take a backup x of the exported Zeebe records in Elasticsearch using the Elasticsearch Snapshots API.

For example, using Elasticsearch / OpenSearch’s native snapshot capabilities directly does not produce a coherent backup. This is because Operate, Tasklist, and Optimize each manage their data across multiple indices, which cannot be reliably captured together without involvement from the components that understand their structure. For this reason, backups must be initiated through each component individually, using their built-in backup functionality.

The same principle applies to Zeebe. Backups must be scheduled through Zeebe to ensure a consistent snapshot of all partition data. Simply taking a disk-level snapshot of each Zeebe broker is not enough, as the brokers operate independently and data may not be aligned across them at the time of the snapshot. Since disk-level backups are not synchronized, this can lead to inconsistencies and invalid recovery points.

A complete backup of a Camunda 8 cluster includes:

  • Backups of Web Applications (Operate, Tasklist), and Optimize (triggered through their APIs).
  • Backup of indices from Elasticsearch/OpenSearch containing exported Zeebe records.
  • A Zeebe broker partition backup (triggered through its API).
  1. Wait until the backup x of the exported Zeebe records is complete before proceeding.
  2. Take a backup x of Zeebe. See how to take a Zeebe backup.
  3. Wait until the backup x of Zeebe is completed before proceeding. See how to monitor a Zeebe backup.
  4. Resume exporting in Zeebe. See Zeebe management API. Because the data across these systems is interdependent, all components must be backed up as part of the same backup window. Backups taken independently at different times may not align and could result in an unreliable restore point.
warning

To ensure a consistent backup, you must follow the process outlined in this guide. Deviating from it can result in undetected data loss, as there is no reliable method to verify cross-component data integrity after backup.

Following the documented procedure results in a hot backup, meaning that:

  • Zeebe continues to process and export data.
  • Web Applications (Operate, Tasklist), and Optimize remain fully operational during the backup process.

This ensures high availability while preserving the integrity of the data snapshot.

Prerequisites​

The following prerequisites are required before you can create and restore backups:

PrerequisiteDescription
Set up a snapshot repository in the secondary datastore.

Depending on the choice of secondary datastore, you must configure the following on the datastore itself:

Note: For Elasticsearch configuration with the Camunda Helm chart on AWS EKS using IRSA, see configuration example.

Configure component backup storage.

Configure the backup storage for the components. This is also important for restoring a backup.

note

You should keep the backup storage of the components configured at all times to ease the backup and restore process and avoid unnecessary restarts.

tip

You can use the same backup storage location for both Elasticsearch / OpenSearch snapshots and Zeebe partition backups, as long as different paths are configured:

  • Set the basePath for Zeebe.
  • Set the base_path for Elasticsearch / OpenSearch.
  1. Start Zeebe, Operate, Tasklist, and Optimize. (To ensure templates/aliases etc. are created)
  2. Confirm proper configuration (such as shards, replicas count, etc.)
  3. Stop Operate, Tasklist, and Optimize.
  4. Delete all indices.
  5. Restore the state of the web applications and Optimize.
  6. Restore zeebe-records* indices from Elasticsearch snapshot.
  7. Delete the data directory of all Zeebe brokers
  8. Restore Zeebe.
  9. Start Zeebe, Operate, Tasklist, and Optimize.