Introduction
When working on an Enterprise integration Project, it is very likely to have a scenario where the Message would be received in the form of Batches containing multiple messages within a single message. So if processing has to be done on individual messages, how it is done?
The design pattern used here is Splitter, where a Splitter/Debatcher splits the composite/Batched message into individual messages and further processes each message separately.
Now depending on the message size, whether aggregation of the Processing results of each individual message is required, whether fast processing is to be done etc ,above pattern can be adjusted with little variation in approach.
For example , say you want to stop processing of batch messages if any of message is faulty or you want to do some actions only after all messages are processed, then above approach can be used. Here single instance of process/workflow/splitter will split the batch message into individual messages and process them one by one.
So it has control as a whole, as it has state of individual messages with it. Thus if any message processing fails you can stop further message processing and take corrective measures.
Above pattern in BizTalk it is done by using Orchestration - Debatching XML Batch Message in Orchestration
And in Logic app it is implemented by using For each control
The design pattern used here is Splitter, where a Splitter/Debatcher splits the composite/Batched message into individual messages and further processes each message separately.
Now depending on the message size, whether aggregation of the Processing results of each individual message is required, whether fast processing is to be done etc ,above pattern can be adjusted with little variation in approach.
For example , say you want to stop processing of batch messages if any of message is faulty or you want to do some actions only after all messages are processed, then above approach can be used. Here single instance of process/workflow/splitter will split the batch message into individual messages and process them one by one.
So it has control as a whole, as it has state of individual messages with it. Thus if any message processing fails you can stop further message processing and take corrective measures.
Above pattern in BizTalk it is done by using Orchestration - Debatching XML Batch Message in Orchestration
And in Logic app it is implemented by using For each control
But if that is not the case, i.e. if it is Fire and Forget scenario (no need to maintain state) then each splitted messages can be processed by a dedicated instance of process, it surely makes processing faster.
Above pattern in BizTalk it is implemented at Receive Port using Pipeline - Debatching XML Batch Message at Receive Port
And in Logic App it is implemented by using SplitOn property on trigger.
In both the above approaches Xpath plays an important role, using which we tell which repeating node is to be looked for and then splitting/debatching is to be done.
XPath is a query language for selecting nodes from an XML document, basically it describes a way to locate and process items in XML documents by using an addressing syntax based on a path through the document's logical structure or hierarchy.
Note: In both the approaches message debatching happens, but in second approach the process/workflow also gets splitted (it scales out based on splitted messages by spinning new instances of itself). However in first approach single instance does all the work.
Create Logic app Instance
Add http trigger, followed by ForEach and Response Action
It is in For each control where we specify the xpath of the repeating node (which is to be splitted)
Read Error related to this -- The template language function 'xpath' expects its first parameter to be an XML object
Note : The default behavior of For each loop is to run in parallel and each iteration in it runs in different
thread, however it can be made to run in sequential form by setting its Degree of Parallelism to 1.
Another example using For each debatching -- Inserting Multiple Records In On Prem SQL Using Logic App
Create Logic app Instance
In Logic app Designer add HTTP trigger followed by send email action (just keeping it simple :) )
We need to set splitOn property on trigger, but no where it is visible -- Not able to understand why Microsoft has not provided.
No problem, we can still do it by going to Logic app code view, and under triggers, add splitOn property with value as xpath expression (same xpath is used as in For each above)
Save it, and now you should be able to see in Designer view as well in trigger Settings
Setting this property forces Logic App engine to create equivalent number of instances as that of number of repeating nodes, so if there are three repeating nodes in message posted to this logic app, then 3 different instance of this logic app will be instantiated.
Note: You can't use SplitOn with a synchronous response pattern , read Error related to this
-- The workflow with 'Response' action type should not have triggers with 'splitOn' property
Using Postman, sent a request having three Purchase node in it to both the logic app
Above pattern in BizTalk it is implemented at Receive Port using Pipeline - Debatching XML Batch Message at Receive Port
And in Logic App it is implemented by using SplitOn property on trigger.
In both the above approaches Xpath plays an important role, using which we tell which repeating node is to be looked for and then splitting/debatching is to be done.
XPath is a query language for selecting nodes from an XML document, basically it describes a way to locate and process items in XML documents by using an addressing syntax based on a path through the document's logical structure or hierarchy.
Note: In both the approaches message debatching happens, but in second approach the process/workflow also gets splitted (it scales out based on splitted messages by spinning new instances of itself). However in first approach single instance does all the work.
Steps to implement Splitter/Debatching Pattern in Logic App
Scenario :
Say we want to debatch/split PurchaseOrders message having multiple Purchase Order in it
Splitting using For each control
Add http trigger, followed by ForEach and Response Action
It is in For each control where we specify the xpath of the repeating node (which is to be splitted)
xpath(xml(triggerBody()),'//*[local-name()="PurchaseOrder" and namespace-uri()
="http://www.adventure-works.com"]')
Using xpath above - we are asking to traverse through the triggerBody and look out of PurchaseOrder node and if found add entry in For each array.(All occurence of PurchaseOrder nodes are added).
Now we add next steps within For each by using Add an action, here I have added Send an email step. So Send an email will be executed based on items in For each array, if there are 3 items then 3 times email will be sent out.
Note : The default behavior of For each loop is to run in parallel and each iteration in it runs in different
thread, however it can be made to run in sequential form by setting its Degree of Parallelism to 1.
Another example using For each debatching -- Inserting Multiple Records In On Prem SQL Using Logic App
Splitting using SplitOn property on trigger
Create Logic app Instance
In Logic app Designer add HTTP trigger followed by send email action (just keeping it simple :) )
We need to set splitOn property on trigger, but no where it is visible -- Not able to understand why Microsoft has not provided.
No problem, we can still do it by going to Logic app code view, and under triggers, add splitOn property with value as xpath expression (same xpath is used as in For each above)
Save it, and now you should be able to see in Designer view as well in trigger Settings
Setting this property forces Logic App engine to create equivalent number of instances as that of number of repeating nodes, so if there are three repeating nodes in message posted to this logic app, then 3 different instance of this logic app will be instantiated.
Note: You can't use SplitOn with a synchronous response pattern , read Error related to this
-- The workflow with 'Response' action type should not have triggers with 'splitOn' property
Testing
Using Postman, sent a request having three Purchase node in it to both the logic app
And three emails were sent by each logic app, however in first logic app single instance
did it whereas in second logic app 3 separate instances did it. It can be checked in Logic
App run history
Logic App run history with Foreach control
Logic App run history with SplitOn property on trigger
Related Post
- Getting Started with Logic Apps - Enterprise Application Integration
- Getting Started with Logic Apps - EDI X12 Fundamentals
- Getting Started with Logic Apps - What happened to the Request?
- Getting Started with Logic Apps - Fundamentals
- Getting Started with Logic Apps - AS2
- Getting Started with Logic Apps - XML to EDI X12
- The provided map definition is not valid
- Required property 'content' expects a value but got null. Path ''.'
- Getting Started with Logic Apps - File Routing
- Installing and Configuring On Premises Data Gateway - By adding user to Active Directory
- Inserting data in On Premises SQL Database using Logic Apps
- Inserting Multiple Records In On Prem SQL Using Logic App
- Check your request parameters to make sure the Path “” exists on your file system
- EDI X12 Documents List
- BizTalk Server : Fetching UNB and UNG Segment Values From Interchange Envelope and Mapping
- BizTalk Server : Fetching UNB and UNG Segment Values From Interchange Envelope and Mapping - Orchestration
- EDIFACTSegmentValueFetcher
- EDI X12 Documents List
- BizTalk Server : Fetching UNB and UNG Segment Values From Interchange Envelope and Mapping
- BizTalk Server : Fetching UNB and UNG Segment Values From Interchange Envelope and Mapping - Orchestration
- EDIFACTSegmentValueFetcher
