Managing secondary storage data

Important best practices for secondary storage management to ensure data integrity and performance optimization.

Modifying secondary storage data

To ensure the reliability, security, and proper operation of your Camunda system, you should follow essential strict guidelines when considering any changes to data stored in secondary storage.

caution

Camunda advises that you do not modify any data in secondary storage unless you have received explicit instructions from the Camunda Support Team during an active support case.

Risks

Making changes to secondary storage data without proper guidance exposes your business to several risks:

Risk	Description
Data Loss	Unintended or manual modifications can result in missing, overwritten, or deleted information that cannot be recovered without backups.
Data Corruption	Changing data structures or values can leave the system in an inconsistent or corrupted state, possibly leading to application errors or downtime.
Unsupported System States	When data is changed outside prescribed processes, your system may enter a state that is no longer supported by Camunda or compatible with upgrades, patches, or new features.
Troubleshooting Challenges	Untracked modifications make it difficult or impossible for support engineers to identify and resolve issues. This can lead to extended downtime and unresolved problems.
Security Vulnerabilities	Unauthorized changes can inadvertently expose sensitive data, weaken access controls, or create exploitable system weaknesses.
Compliance Issues	Many organizations operate under strict regulatory requirements for data integrity. Unauthorized edits can violate internal or external compliance standards.

Configuring shards and replicas

Proper configuration of shards and replicas in Elasticsearch/OpenSearch is essential for the reliability, performance, and resilience of your Camunda system.

Understanding when and how to deploy these features helps ensure you gain the intended benefits, such as fault tolerance and scalability, while avoiding common issues.

There is no one-size-fits-all approach, but here are some general guidelines you should consider:

Shards

The number of primary shards should be determined based on your data size and anticipated growth.

• A common starting point is to use 1-5 primary shards per index.
• Partitioning data into multiple shards can improve scalability.
• However, increasing shard count should be based on system throughput needs and underlying hardware capabilities.
• Avoid over-sharding, as unnecessary complexity can arise and degrade performance.

Replicas

• Single Node Cluster: Do not configure replicas when your Camunda system is running on a single node cluster. Replicas provide redundancy only when distributed across multiple nodes. On a single node, the replicas will not be assigned, in some cases preventing your node ever reporting as healthy.

• Multiple Node Cluster: In a multi-node cluster, it is advisable to configure at least one replica for each index. This ensures that if one node fails, the data remains accessible from another node, enhancing fault tolerance and availability.

Backups

Regular backups of your secondary storage data are crucial for disaster recovery and data integrity:

• Always follow the official Camunda backup procedure, step-by-step, without omission or variation.
• Regularly schedule backups according to your business needs and system volume.
• Periodically test restoration from backups to ensure your procedures are effective and reliable.

Index templates

Camunda uses Index Templates to define settings and mappings for indices.

To ensure that your indices are created with the correct configurations, it is important to avoid the following:

• Using custom Index Templates that may conflict with Camunda's default templates. Templates with a higher priority than those set by Camunda can lead to unexpected behavior, such as creating indices with incorrect mappings.

• Deleting or altering existing Index Templates without guidance from Camunda Support.

If your Elasticsearch/OpenSearch provider includes a predefined wild card index template, it is advised that you set a high priority for the Camunda Index Templates to ensure they do not get overridden.

Monitor secondary storage health

Use the Data Layer Dashboard to monitor the health and performance of your Camunda Orchestration Cluster and its interaction with secondary storage.

This allows you to proactively identify and address potential issues before they impact your system's performance.