Optimize export filtering

Filter which processes and variables the Elasticsearch and OpenSearch exporters write for Optimize to reduce storage costs and scope analytics data.

note

Exporter-side filters require Camunda 8.9 or later. On earlier versions, exporters always write a complete, unfiltered event stream.

How export filtering works

The Elasticsearch and OpenSearch exporters run inside Zeebe brokers and write raw engine events to export indices. Optimize reads those indices and builds its own analytics indices.

Export filters run inside the exporter and permanently drop matching records from the exported stream. Optimize cannot import data that was never exported, and dropped records cannot be recovered later, even if you relax the filters afterward.

These filters affect Optimize only. Operate and Tasklist read through the Camunda Exporter, so their data stays intact regardless of what you configure here.

Process definition filtering

Use process definition filters to include or exclude entire processes from Optimize. Records tied to excluded processes (including process instances, variables, and incidents) are dropped before they reach Optimize's import indices.

This is useful when you have high-volume processes that don't need analytics in Optimize.

Goal	Option
Export only specific processes	bpmnProcessIdInclusion
Exclude specific processes	bpmnProcessIdExclusion

Both options can coexist in the same configuration, with exclusion taking precedence when an ID appears in both lists. Value types without a bpmnProcessId, such as DEPLOYMENT and DECISION, are not affected by these filters.

Example

Elasticsearch

camunda:
  data:
    exporters:
      elasticsearch:
        args:
          index:
            bpmnProcessIdInclusion:
              - orderProcess
            bpmnProcessIdExclusion:
              - debugProcess

OpenSearch

camunda:
  data:
    exporters:
      opensearch:
        args:
          index:
            bpmnProcessIdInclusion:
              - orderProcess
            bpmnProcessIdExclusion:
              - debugProcess

Variable filtering

Use variable filtering to control which variables are exported to Optimize. It is an effective mitigation for Optimize's impact on Elasticsearch or OpenSearch sizing. See impact of Optimize for details.

You can disable variable export entirely, filter by name, or filter by value type.

Disable all variable export

Set variable: false to stop the exporter from writing any variable records to Optimize indices.

Elasticsearch

camunda:
  data:
    exporters:
      elasticsearch:
        args:
          index:
            variable: false

OpenSearch

camunda:
  data:
    exporters:
      opensearch:
        args:
          index:
            variable: false

Filter by variable name

Use name-based inclusion and exclusion lists to export only the variables you need. You can match by exact name, prefix (startsWith), or suffix (endsWith). Exclusion wins over inclusion when both rules match the same variable name.

Goal	Options
Include specific variables	variableNameInclusionExact, variableNameInclusionStartWith, variableNameInclusionEndWith
Exclude specific variables	variableNameExclusionExact, variableNameExclusionStartWith, variableNameExclusionEndWith

Example

Elasticsearch

camunda:
  data:
    exporters:
      elasticsearch:
        args:
          index:
            variableNameInclusionStartWith:
              - business
            variableNameExclusionStartWith:
              - business_debug

OpenSearch

camunda:
  data:
    exporters:
      opensearch:
        args:
          index:
            variableNameInclusionStartWith:
              - business
            variableNameExclusionStartWith:
              - business_debug

Filter by variable type

Use type-based inclusion and exclusion lists to filter variables by their inferred JSON type. Valid types are String, Number, Boolean, Object, and Null.

Goal	Options
Include specific types only	variableValueTypeInclusion
Exclude specific types	variableValueTypeExclusion

Example

Elasticsearch

camunda:
  data:
    exporters:
      elasticsearch:
        args:
          index:
            variableValueTypeInclusion:
              - String
            variableValueTypeExclusion:
              - Object

OpenSearch

camunda:
  data:
    exporters:
      opensearch:
        args:
          index:
            variableValueTypeInclusion:
              - String
            variableValueTypeExclusion:
              - Object

Trade-offs

Filtered data is permanently unavailable in Optimize:

• Process filtering: Excluded processes don't appear in Optimize reports. If you later re-enable a process, Optimize shows a permanent gap for the period when it was excluded, because records from that window were never exported.
• Variable filtering: Filtered variables are unavailable in Optimize reports, including variable filters, variable-based grouping, and raw-data variable columns.

For guidance on non-retroactivity, changing filters on running clusters, and safely applying filters mid-stream, see Camunda 8 system configuration.