Amazon Simple Queue Service Connector
- Outbound
- Inbound
The Amazon SQS Connector is an outbound Connector that allows you to connect your BPMN service with Amazon Simple Queue Service (SQS) to send messages.
Prerequisites
To use the Amazon SQS Connector, you need to have an SQS Queue, IAM key, and secret pair with the sqs:SendMessage
policy relative to your SQS.
Use Camunda secrets to avoid exposing your AWS IAM credentials as plain text. Refer to an appendix entry and the SQS Developer Guide to learn more.
Create an Amazon SQS Connector task
To use the Amazon SQS Connector in your process, either change the type of existing task by clicking on it and using the wrench-shaped Change type context menu icon, or create a new Connector task by using the Append Connector context menu. Follow our guide on using Connectors to learn more.
Make your Amazon SQS Connector for sending messages executable
To make your Amazon SQS Connector for sending messages executable, take the following steps:
- Choose an applicable authentication type from the Authentication dropdown. Learn more about authentication types in the related appendix entry.
- Set the relevant IAM key and secret pair in the Authentication section. For example,
{{secrets.MY_AWS_ACCESS_KEY}}
. The value can be plain text, but this is not recommended due to security concerns. - In the Queue Properties section, set the URL of your SQS queue, its region, and its type.
- In the Input message data section, fill the Message body with the data you would like to submit to the queue. The field requires FEEL input.
- (Optional) In the Input message data section, fill out the field Message attributes to set optional message metadata. This field requires FEEL input. Refer to the relevant appendix section to find out more about this field.
- (FIFO only) If you are using a queue of type FIFO, a Message Group ID must be provided. An optional Message Deduplication ID can be provided as well, depending on how you configured the message deduplication of the queue.
Amazon SQS Connector response
The Amazon SQS Connector returns the SQS message identifier of a newly created message.
The response contains a messageId
variable.
You can use an output mapping to map the response:
- Use Result Variable to store the response in a process variable. For example,
myResultVariable
. - Use Result Expression to map fields from the response into process variables. For example:
= {
"createdMessageID": response.messageId
}
Appendix & FAQ
What are the message attributes and how can I set them?
Amazon SQS lets you include structured metadata (such as timestamps, geospatial data, signatures, and identifiers) with messages using message attributes. The Amazon SQS Connector allows you to include non-binary message attributes in the section Input message data. The message attribute value must be composed to be compliant with Amazon SQS message attribute data format.
Example of a valid message attribute as a FEEL value:
= {
"timestamp":{
"StringValue":today(),
"DataType":"String"
},
"messageSubmittedBy":{
"StringValue":"user12345",
"DataType":"String"
}
}
How do I store AWS IAM Secrets for my SQS Connector?
Use Camunda secrets to avoid exposing your AWS IAM credentials. Follow our documentation on managing secrets to learn more.
AWS authentication types
There are two options to authenticate the Connector with AWS:
- Choose Credentials in the Authentication dropdown if you have a valid pair of access and secret keys provided by your AWS account administrator. This option is applicable for both SaaS and Self-Managed users.
- Choose Default Credentials Chain (Hybrid/Self-Managed only) in the Authentication dropdown if your system is configured as an implicit authentication mechanism, such as role-based authentication, credentials supplied via environment variables, or files on target host. This option is applicable only for Self-Managed or hybrid distribution. This approach uses the Default Credential Provider Chain to resolve required credentials.
To maintain stable behavior from the Amazon SQS Connector, do not subscribe multiple Amazon SQS Connectors to the same queue.
Successfully consumed messages are removed from the queue, even if they are not correlated.
The Amazon SQS Inbound Connector is an inbound Connector that allows you to start or continue a BPMN process triggered by Amazon Simple Queue Service (SQS).
Prerequisites
Before using the Amazon SQS inbound Connector, ensure you have the following:
- An active SQS Queue in your AWS account.
- IAM credentials with the necessary permissions to receive messages from the SQS Queue. Use Camunda secrets to store your AWS IAM credentials securely. Refer to the Camunda secrets documentation for more details.
Create an SQS inbound Connector task
To receive messages from Amazon SQS in your process, follow these steps:
- Start building your BPMN diagram. You can use the Amazon SNS Inbound Connector with either Start Event or Intermediate Catch Event building blocks.
- Select the appropriate element and change its template to an SQS inbound Connector.
- Fill in all the required properties for the Connector, such as the AWS region, SQS Queue URL, and the visibility timeout.
- Complete your BPMN diagram by adding other necessary elements and connectors.
- Deploy the diagram to activate the SQS Inbound Connector.
Configure the SQS inbound Connector
To configure the SQS inbound Connector and receive messages from your SQS Queue, follow these steps:
- Choose an applicable authentication type from the Authentication dropdown. Learn more about authentication types in the related appendix entry.
- Set the relevant IAM key and secret pair in the Authentication section. For example,
{{secrets.MY_AWS_ACCESS_KEY}}
. The value can be plain text, but this is not recommended due to security concerns. - In the Queue Properties section, set the URL of your SQS Queue and its region.
- In the Message polling properties section, set the polling wait time. This is the duration (in seconds) for which the call waits for a message to arrive in the queue before returning. Refer to the Amazon documentation for more details.
- (Optional) In the Use next attribute names for activation condition section, set an array of Attribute names or Message attribute name (e.g.,
["attributeName1", "attributeName2"]
) to receive messages from the queue with specific metadata. Alternatively, you can leave it empty to get results with all available attributes. Learn more about message metadata here. - (Optional) Configure the Activation Condition. For example, if an external message has the body
{"messageId": 1, "body": "Hi team", "messageAttributes":{"key":{"stringValue":"value"}}...}
, the Activation Condition value might look like=(messageAttributes.key.stringValue="value")
. Leave this field empty to receive all messages every time. - Set Variable mapping. For example, to get only the message body, you can set
{resultBody: body}
in the Result expression. Learn more about Variable mapping here.
When using the Amazon SQS inbound Connector with an Intermediate Catch Event, fill in the Correlation key (process) and Correlation key (payload).
- Correlation key (process) is a FEEL expression that defines the correlation key for the subscription. This corresponds to the Correlation key property of a regular Message Intermediate Catch Event.
- Correlation key (payload) is a FEEL expression used to extract the correlation key from the incoming message. This expression is evaluated in the Connector Runtime and the result is used to correlate the message.
Example for correlation and activation condition properties (correlation by ID in the body and activation condition by message attribute):
SQS message:
{
"messageId": "12345",
"receiptHandle": "ABCDE",
"mD5OfBody": "1c6bb59997376e5182a88a6f582cd92a",
"body": {
"id": 4567,
"value": "Hello world"
},
"attributes": {
"ApproximateReceiveCount": "1",
"SentTimestamp": "1703062074171",
"SenderId": "333293239507",
"ApproximateFirstReceiveTimestamp": "1703062074185"
},
"messageAttributes": {
"messageName": {
"stringValue": "myProcess",
"binaryValue": null,
"stringListValues": [],
"binaryListValues": [],
"dataType": "String"
}
},
"md5OfMessageAttributes": "9de691a346c79e4fda4af06248aa9dfc"
}
- Correlation key (process):
=4567
- Correlation key (payload):
=body.id
- Activation condition:
=messageAttributes.key.stringValue="myProcess"
Learn more about correlation keys in the messages guide.
Activate the SQS inbound Connector
Once you click the Deploy button, your SQS inbound Connector will be activated and publicly available. Whenever the SQS inbound Connector receives a new message, a new BPMN process will be created.
Amazon SQS Connector response
The Amazon SQS Connector provides the SQS message as a response. Utilize output mapping to align this response with process variables:
- Use Result Variable to store the response in a process variable. For example,
myResultVariable
. This approach stores the entire SQS message as a process variable namedmyResultVariable
. - Use Result Expression to map fields from the response into process variables. This approach allows for more granularity. Instead of storing the entire response in one variable, you can extract specific fields from the SQS message and assign them to different process variables. This is particularly useful when you are only interested in certain parts of the message, or when different parts of the message need to be used separately in your process. Example:
SQS message :
{
"messageId": "12345",
"receiptHandle": "ABCDE",
"mD5OfBody": "1c6bb59997376e5182a88a6f582cd92a",
"body": {
"id": 4567,
"value": "Hello world"
},
"attributes": {
"ApproximateReceiveCount": "1",
"SentTimestamp": "1703062074171",
"SenderId": "33333333333",
"ApproximateFirstReceiveTimestamp": "1703062074185"
},
"messageAttributes": {
"messageName": {
"stringValue": "myProcess",
"binaryValue": null,
"stringListValues": [],
"binaryListValues": [],
"dataType": "String"
}
},
"md5OfMessageAttributes": "9de691a346c79e4fda4af06248aa9dfc"
}
To store the entire body in a process variable resultBody
, ID from body to bodyId
, and messageId to messageId
, use:
= `{resultBody:body, bodyId:body.id, messageId: messageId}`
Learn more about Variable mapping here.
Appendix
AWS authentication types
There are two options to authenticate the Connector with AWS:
- Choose Credentials in the Authentication dropdown if you have a valid pair of access and secret keys provided by your AWS account administrator. This option is applicable for both SaaS and Self-Managed users.
- Choose Default Credentials Chain (Hybrid/Self-Managed only) in the Authentication dropdown if your system is configured as an implicit authentication mechanism, such as role-based authentication, credentials supplied via environment variables, or files on target host. This option is applicable only for Self-Managed or hybrid distribution. This approach uses the Default Credential Provider Chain to resolve required credentials.
Next Steps
- Explore more about Amazon Simple Queue Service (SQS) and its capabilities.
- Learn about other Connectors available in Camunda to integrate with different systems and services.
- Learn more about using Connectors.
- Learn more about inbound Connectors.