Please refer to supported environments to find out which versions of Elasticsearch are supported in a Camunda 8 Self-Managed setup.
The exporter operates on the idea that it should perform as little as possible on the Zeebe side of things. In other words, you can think of the indexes into which the records are exported as a staging data warehouse. Any enrichment or transformation on the exported data should be performed by your own ETL jobs.
When configured to do so, the exporter will automatically create an index per record value type (see the value type in the Zeebe protocol). Each of these indexes has a corresponding pre-defined mapping to facilitate data ingestion for your own ETL jobs. You can find those as templates in this module's resources folder.
The indexes are created as required, and will not be created twice if they already exist. However, once disabled, they will not be deleted (that is up to the administrator.) Similarly, data is never deleted by the exporter, and must be deleted by the administrator when it is safe to do so. A retention policy can be configured to automatically delete data after a certain number of days.
As the exporter is packaged with Zeebe, it is not necessary to specify a
The exporter can be enabled by configuring it with the
classpath in the broker settings.
# Refer to the table below for the available args options
The exporter can be configured by providing
args. The table below explains all the different
options, and the default values for these options:
|Valid URLs as comma-separated string
|Request timeout (in ms) for the Elasticsearch. client
|Refer to Index for the index configuration options.
|Refer to Bulk for the bulk configuration options.
|Refer to Retention for the retention configuration options
|Refer to Authentication for the authentication configuration options.
In most cases, you will not be interested in exporting every single record produced by a Zeebe cluster, but rather only a subset of them. This can also be configured to limit the kinds of records being exported (e.g. only events, no commands), and the value type of these records (e.g. only job and process values).
|This prefix will be appended to every index created by the exporter; must not contain
true missing indexes will be created automatically.
true command records will be exported
true event records will be exported
true rejection records will be exported
true records related to checkpoints will be exported
true records related to command distributions will be exported
true records related to decisions will be exported
true records related to decision evaluations will be exported
true records related to decisionRequirements will be exported
true records related to deployments will be exported
true records related to deployment distributions will be exported
true records related to errors will be exported
true records related to escalations will be exported
true records related to forms will be exported
true records related to incidents will be exported
true records related to jobs will be exported
true records related to job batches will be exported
true records related to messages will be exported
true records related to message subscriptions will be exported
true records related to message start event subscriptions will be exported
true records related to processes will be exported
true records related to process events will be exported
true records related to process instances will be exported
true records related to process instances batches will be exported
true records related to process instance creations will be exported
true records related to process instance migrations will be exported
true records related to process instance modifications will be exported
true records related to process message subscriptions will be exported
true records related to resource deletions will be exported
true records related to signals will be exported
true records related to signal subscriptions will be exported
true records related to timers will be exported
true records related to user tasks will be exported
true records related to variables will be exported
true records related to variable documents will be exported
To avoid too many expensive requests to the Elasticsearch cluster, the exporter performs batch updates by default. The size of the batch, along with how often it should be flushed (regardless of size) can be controlled by configuration.
|Delay, in seconds, before force flush of the current batch. This ensures that even when we have low traffic of records, we still export every once in a while.
|The amount of records a batch should have before we flush the batch
|The size of the batch, in bytes, before we flush the batch
10485760 (10 MB)
With the default configuration, the exporter will aggregate records and flush them to Elasticsearch:
- When it has aggregated 1000 records.
- When the batch memory size exceeds 10 MB.
- Five seconds have elapsed since the last flush (regardless of how many records were aggregated).
A retention policy can be set up to delete old data.
When enabled, this creates an Index Lifecycle Management (ILM) Policy that deletes the data after the specified
All index templates created by this exporter apply the created ILM Policy.
true the ILM Policy is created and applied to the index templates
|Specifies how old the data must be, before the data is deleted as a duration
|The name of the created and applied ILM policy
The duration can be specified in days
ms, and/or nanoseconds
Providing these authentication options will enable Basic Authentication on the exporter.
|Username used to authenticate
|Password used to authenticate
Here is an example configuration of the exporter:
# Elasticsearch Exporter ----------
# An example configuration for the elasticsearch exporter:
# These setting can also be overridden using the environment variables "ZEEBE_BROKER_EXPORTERS_ELASTICSEARCH_..."
# A comma separated list of URLs pointing to the Elasticsearch instances you wish to export to.
# For example, if you want to connect to multiple nodes for redundancy:
# url: http://localhost:9200,http://localhost:9201