Contact us
, , , , , , , ,

B2B on SAP Integration Suite – Cloud Integration – CPI – using Trading Partner Management – Part 10 – Outgoing IDoc Bundling With EDI Bundling

Bhavesh Kantilal

By.

Bhavesh Kantilal

•

min read

Share

Post Views: 1,394

Introduction

In the previous post of this series Part 9 , we saw how Sender IDoc Bundling brings its challenges on Trading Partner Management. We looked at the simplest use case where even if the IDoc is bundled from SAP ERP ( ECC; S/4 and so on), you can always handle them with a plug-in Step 1 Integration Flow. This was a very basic process though.

In a typical Integration Scenarios where IDocs are bundled from ERP, you might also have agreed with your Trading Partner that the EDI Message will be bundled in a single message. While of course, the easier and more pragmatic approach is to inform your Trading Partner that they will receive Split EDI Messages with SAP Integration Suite but in the real-world such battles are not always won, and you need an alternate in SAP Integration Suite to handle Bundled or Packaged IDocs as Bundled EDI Messages. We will look at an approach with limitations to handle this in this post.

Scenario in Scope

For this flow, we will continue to extend the scenario we have covered in Part 4, i.e., an IDoc to AS2 flow over EDI but with IDoc Packaging / Bundling. We will also convert the EDI Message into a bundled EDI Message.

The Solution Overview

A Custom Sender IFlow will process the IDoc from ERP, split IDoc and send to the standard IFlow Step 1b – Write Message to Message queue. All Standard IFlows are processed and instead of sending EDI directly to Trading Partner over AS2, process over to ProcessDirect Receiver that thereby allows you to Bundle the EDI Message

Custom Iflow Custom Iflow to take the Bundled Idoc Capture IDoc count Split individual IDocs Push message to standard Flow Step 1b - Write Message to Message queue Standard Flows All Standard Flows are executed. Step 1b - Write Message to Message queue Step 2 - Interchange Processing Flow V2 Step 3 - Receiver Communication Flow V2 Custom Process Direct Receiver Flow Uses Aggregator pattern to collect EDI Messages using MPL_CorrelationID as the Co-relating field. Checks if all messages are received. If yes uses Custom Groovy to Bundle EDI Messages and send to Trading Partner over AS2 If no, trigger exception

In Part 9, we had use a New IFlow: Step 1 – Sender IDOC Communication Flow V2 Bundled IDocs Process Individual to take the Bundled IDocs and split them into individual IDocs. We will now

  • Create another IFlow where we will take the Bundled IDocs, set some additional headers and then process the EDI Message over.
  • Instead of sending the EDI Message directly to our Trading Partner, over AS2, we will now process this into a PROCESS DIRECT Receiver Adapter and use a Custom Groovy script to Bundle the EDI Message and then send to our Trading Partner.

Note : This approach provides you with an alternate starting point. You would need to handle exceptions etc inline with your Business needs.

Create a new IFlow for IDoc Bundling with Custom Headers

Go to Integration Suite ->Package : Cloud Integration – Trading Partner Management V2, and copy the IFlow: Step 1 – Sender IDOC Communication Flow V2 to another Custom Package with the name Step 1 – Sender IDOC Communication Flow V2 Bundled IDocs ( This is just a sample name, you can use any name you want ). You can download this IFlow here and import it directly into your tenant.

Create a new Iflow to handle Bundled IDocs
  1. IDoc Sender: Change the endpoint to /tpm/b2b/idoc/bundled
  2. Get the Count of IDocs and set them as a header
    • SAP_IDocCount -> XPATH ->count(//IDOC) -> String
    • We use a the header name as SAP_IDocCount as only SAP* headers are allowed in the allowed headers config of the SAP TPM Standard Flows and we need the count of the IDocs in a separate step.
  3. General Splitter: Split at XPATH : //IDOC
Change the required settings in the copied IFlow Step 1 – Sender IDOC Communication Flow V2 Bundled IDocs IDoc Sender: Change the endpoint to /tpm/b2b/idoc/bundled Get the Count of IDocs and set them as a header SAP_IDocCount -> XPATH ->count(//IDOC) -> String We use a the header name as SAP_IDocCount as only SAP* headers are allowed in the allowed headers config of the SAP TPM Standard Flows and we need the count of the IDocs in a separate step. General Splitter: Split at XPATH : //IDOC

Update TPM Agreement to use a Process Direct Receiver Channel

Create a ProcessDirect Receiver for CUSTOMER2

For our Trading Partner in scope, we will create a new ProcessDirect Receiver Communication. Go to Design -> B2B Scenarios ->Trading Partners -> CUSTOMER2 ( if you are extending Part 4) and create a new Communication

  • Name: CUSTOMER2_PD_RECEIVER
  • Alias: CUSTOMER2_PD_RECEIVER
  • Direction: Receiver
  • Adapter: PROCESS_DIRECT
  • Address: /CUSTOMER2_PD_RECEIVER
Create a Process_Direct Receiver for handling EDI Bundling

Update Agreement to use this ProcessDirect Receiver instead of AS2 Receiver

Navigate to your Agreement Company_Self – ERPCLNT200 Sends IDoc To Customer2 over EDI AS2 ( from Part4) and change the receiver channel to this new Process_Direct Receiver Channel. Now all your EDI Messages will be sent to this Process_Direct receiver channel instead of directly processing over AS2.

Update your Agreement to use Process Direct to handle IDoc to EDI Bundling

Save and Update ( Activate) your Agreement.

Create a new IFlow for Process_Direct endpoint

Create a new Integration Flow that will now receive the EDI Messages over Process_Direct instead of AS2. This Iflow will now bundle the EDI Messages using a Aggregator pattern. A Custom Groovy Script will bundle the EDI Messages. When the EDI Message is bundled, the UNB and UNZ Segment are taken from the 1st EDI Message. Additionally UNH Counter is also set dynamically to make it a valid EDI message.

As Integration Suite, Aggregation Pattern relies on the Last Message Expression and does not support the Count of Messages to be aggregated, you would need to set your Completion Timeout accordingly. An additional check is performed to see if all messages are received, if yes, then the EDI is sent over AS2 else, to avoid sending partial EDI messages, the Exception Handler is triggered.

Below is how the Integration Flow will look. You can also download it here.

  • IFlow Name: CUSTOMER2_PD_RECEIVER
  • Process Direct Sender: /CUSTOMER2_PD_RECEIVER
  • Content Modifier: Form XML For Aggregation use a Message Body of below XML format
<row>
	<rawedi><![CDATA[${in.body}]]></rawedi>
	<IDocNumber>${header.SAP_ApplicationID}</IDocNumber>
	<IDocCount>${header.IDocCount}</IDocCount>
	<SAP_MplCorrelationId>${header.SAP_MplCorrelationId}</SAP_MplCorrelationId>
</row>
  • Aggregator
    • Correlation Expression : //SAP_MplCorrelationId
    • Aggregation Strategy
      • Incoming Format : XML Same Format
      • Aggregation Algorithm : Combine
      • Last Message Condition (XPath) : //IDocCount = ‘${property.CamelAggregatedSize}’
      • Completion Timeout (in min) : 1
      • Data Store Name: Aggregator-1
  • Router checks if all Messages are received
    • Yes – ${header.SAP_IDocCount} = ${property.CamelAggregatedSize}
    • Default – Route – No -> Error End
  • Exception Sub Process
    • Add your exception handler as you deem fit.
  • AS2 Receiver channel contains your AS2 Configuration
  • Groovy script to combine your EDI Message
import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;
def Message processData(Message message) {
    //Body
    def body = message.getBody(java.lang.String) as String
    def slurper = new XmlSlurper()
	def xml = slurper.parseText(body)
	def ediMessages = xml.'**'.findAll { it.name() == 'rawedi' }.collect { it.text() }
	// Function to extract segments from an EDI message
	def combinedSegments = []
	int unhCount = 0
	String firstUNBReference = null
	ediMessages.each { message1 ->
		def segments = extractSegments(message1)
		unhCount += segments.count { it.startsWith('UNH') }
		if (firstUNBReference == null) {
			firstUNBReference = segments[0].split('\\+')[5] // Extract reference number from first UNB
		}
    segments = segments[1..-2] // Remove UNB and UNZ segments
    combinedSegments += segments
	}
	// Assuming the UNB segment is the same for all messages, take it from the first message
	String combinedUNB = extractSegments(ediMessages[0])[0]
	// Use the reference number from the first message's UNZ segment
	String unzReference = extractSegments(ediMessages[0]).last().split('\\+')[1]
	// Combine the segments with the UNB segment and add UNZ with the same reference as the first UNB
	combinedSegments = [combinedUNB] + combinedSegments + ["UNZ+${unhCount}+${firstUNBReference}"]
	// Reconstruct the combined EDI message
	String combinedEDIMessage = combinedSegments.join("'") + "'"
	message.setBody(combinedEDIMessage)
    return message;
}

def extractSegments(String ediMessage) {
    return ediMessage.split("'").toList()
}

Test your Flow

Trigger your Flow now with a IDoc package of say 2 IDocs. Ensure your IDoc in ERP is pointing to the new endpoint / IFlow we just configured / developed: /cxf/tpm/b2b/idoc/bundled

Integration Flows that have processed the messages

The message is processed over where the IDoc is sent to the below IFlows:

  1. Step 1 – Sender IDOC Communication Flow V2_BundledIDocs – New Custom IFlow
  2. Step 1b – Write Message to Message queue – Standard IFlow
  3. Step 2 – Interchange Processing Flow V2 – Standard IFlow
  4. Step 3 – Receiver Communication Flow V2 – Standard IFlow
  5. CUSTOMER2_PD_RECEIVER – New Custom Process Direct IFlow where we perform IDoc Bundling.

Combined EDI File

Below is now how the Combined EDI File looks like

UNB+UNOC:3+SELF_1234:30+CUSTOMER2:30+231113:1452+324'
UNH+188617211+DESADV:D:96A:UN:A01051'
BGM+351+28841095'
DTM+137:202309150317:203'
RFF+AAS:28841095'
NAD+CZ+XXXX::92'
NAD+SE+XXXX::92'
NAD+CA+XXXX::92'
NAD+CN+XXXX::92'
LOC+11+XXXX::92'
UNT+10+188617211'
UNH+188617212+DESADV:D:96A:UN:A01051'
BGM+351+28841095'
DTM+137:202309150317:203'
RFF+AAS:28841095'
NAD+CZ+XXXX::92'
NAD+SE+XXXX::92'
NAD+CA+XXXX::92'
NAD+CN+XXXX::92'
LOC+11+XXX::92'
UNT+10+188617212'
UNZ+2+324'
EDI Message in a Editor like EDI Notepad

Message in B2B Monitoring

EDI Message in B2B monitoring in Integration Suite
EDI Message in B2B monitoring in Integration Suite

Limitations of this approach

This approach is just a starting point for you to extend further for your IDoc / EDI Bundling requirements. While this works technically, it is essential to highlight certain limitations of this approach

  • Aggregator pattern in Integration Suite does not support Aggregation on “Message Count”. This means you need to add a suitable Completion Timeout parameter. If one of your B2B message fails ( mapping exception ) in the IDoc Bundle, then you need to determine how you handle the exception. Do you send a partial EDI Message or do you trigger a error and so on.
  • We bundle the EDI message using a custom Groovy Script. This uses the Interchange Control Number of the 1st EDI Message. You need to essentially be aware that your ICN number(s) in B2B Monitoring show the individual messages but you combine them in a another flow and your ICN number is not the same.
  • The Groovy would need to be adapted for ANSI X12.
  • EDI FAs cannot be handled in this approach.

Alternatives

If these limitations are a show stopper, especially the last limitation, you might need to consider not using standard TPM Flows for this specific EDI Processing and look at other alternates as blogged here: SAP EDI/IDoc Communication in CPI Using Bundling and Fewer Mapping Artifacts

Final Thoughts

We have looked at how to handle IDoc Bundling and EDI Bundling with an alternate approach. This allows you to Plug and Play into SAP’s TPM Architecture and allows for future upgrades without having to worry about your customizations stopping upgrade. This approach has its limitations and challenges and hence due consideration is required before implementing this alternate.

The best approach is to “force” your Trading Partners to move away from EDI Bundling but when that does not work or is cost prohibitive, this Alternate or skipping TPM might be the only way ahead till SAP addresses this missing gap in their solution.

Additional Blogs from this Series

Further References

, , , , , ,

Bhavesh Kantilal