Monitoring electronic data interchange transmission and acknowledgement

What to monitor

It's important to track a few key performance indicators (KPIs) to ensure EDI messages are processed correctly.

• Monitoring acknowledgments. Track the receipt of acknowledgment documents (997) within a specified time frame after sending each transaction.
• Transmission success rates. Ensure EDI documents such as POs (850), ASNs (856), and Invoices (810) are successfully transmitted to their respective recipients.
• Transmission errors and retry rates. Look for EDI documents that fail to transmit or are stuck in retry loops. Lost or failed EDI transactions can lead to operational inefficiencies, such as missed orders, delayed shipments, or inventory discrepancies.
• Acknowledgment timing. Monitor delayed acknowledgments.
• Finding repeated failure patterns. Look for patterns of repeated transmission or acknowledgment errors might indicate a systemic issue.

Delays or failures in transmitting or receiving EDI documents can cause significant disruptions in the supply chain, leading to procurement delays, missed deliveries, and potential financial penalties.

Building KPIs in the Splunk platform

This section explains the base for any search to build visual KPIs to track key EDI transmission metrics. As you click into each of linked searches in the section above, you'll see the following components:

First, searching for events with sourcetype="edi:x12" will retrieve all EDI transactions, including both the actual transactions and their corresponding acknowledgments.

index=supply_chain_edi sourcetype="edi:x12"
| table _time edi_ack_status edi_buyer edi_code edi_code_ack edi_cont_num edi_date edi_flag edi_requestor edi_responder edi_seller edi_sequence edi_time edi_tr_id edi_type 

Next, use the eval command to create a field called edi_ack_status_combo, which associates acknowledgment events with the corresponding EDI transactions. The result of this evaluation will look like 997-214-A, which indicates that 997 (the acknowledgment) corresponds to 214 (the Transportation Carrier Shipment Status Message).

Additionally, the edi_code_groupby field is evaluated to group the actual EDI request with its acknowledgment. In the example above, this would be 214. The purpose of this grouping is to combine the request and acknowledgment events, allowing for easier filtering and analysis. For instance, you can quickly search for requests that lack an acknowledgment.

index=supply_chain_edi sourcetype="edi:x12" 
| table _time edi_ack_status edi_buyer edi_code edi_code_ack edi_cont_num edi_date edi_flag edi_requestor edi_responder edi_seller edi_sequence edi_time edi_tr_id edi_type 
| eval edi_ack_status_combo=edi_code+"-"+edi_code_ack+"-"+edi_ack_status
| eval edi_code_groupby=if(isnull(edi_code_ack), edi_code, edi_code_ack)

With the proper evaluation conditions in place, we can now merge EDI requests with their corresponding acknowledgments using the stats command.

index=supply_chain_edi sourcetype="edi:x12" 
| table _time edi_ack_status edi_buyer edi_code edi_code_ack edi_cont_num edi_date edi_flag edi_requestor edi_responder edi_seller edi_sequence edi_time edi_tr_id edi_type 
| eval edi_ack_status_combo=edi_code+"-"+edi_code_ack+"-"+edi_ack_status
| eval edi_code_groupby=if(isnull(edi_code_ack), edi_code, edi_code_ack)
| stats last(_time) AS _time, last(edi_requestor) AS edi_requestor, last(edi_responder) AS edi_responder, first(edi_code) AS edi_type, list(edi_code) AS edi_code, last(edi_ack_status) AS edi_ack_status BY edi_tr_id edi_code_groupby

The result of the stats command provides an excellent starting point for various analyses, particularly in identifying issues related to missing acknowledgments, which could disrupt critical physical processes. For example, by searching through the results of the stats command, you can easily identify processed events that lack an edi_ack_status field, which indicates that the acknowledgment event hasn't been merged with the request event.

Search explanations

Splunk search	Explanation
index=supply_chain_edi sourcetype="edi:x12"	Select EDI X12 data by selecting sourcetype of edi:x12.
| table _time edi_ack_status edi_buyer edi_code edi_code_ack edi_cont_num edi_date edi_flag edi_requestor edi_responder edi_seller edi_sequence edi_time edi_tr_id edi_type	Format the relevant fields into a table.
| eval edi_ack_status_combo=edi_code+"-"+edi_code_ack+"-"+edi_ack_status | eval edi_code_groupby=if(isnull(edi_code_ack), edi_code, edi_code_ack)	Create a new evaluated field edi_ack_status_combo that combines edi_code, edi_code_ack, edi_ack_status field. Use another eval command to create edi_code_groupby field to join all EDI events with EDI 997 acknowledgment events.
| stats last(_time) AS _time, last(edi_requestor) AS edi_requestor, last(edi_responder) AS edi_responder, first(edi_code) AS edi_type, list(edi_code) AS edi_code, last(edi_ack_status) AS edi_ack_status BY edi_tr_id edi_code_groupby	Join all EDI events with EDI 997 (acknowledgment) events into a single event. A successfully joined event will have an EDI edi_ack_status field that indicates there was a matching pair of EDI with EDI acknowledgment. Events with null values in edi_ack_status means that there's no matching pair of EDI acknowledgments. Events with successfully joined EDIs have “A”, “E”, “R” flags indicating their status.