I'm going to try to explain this as simply as possible, but consider the following message flow in BizTalk.

We have several MLLP receive ports/locations setup receiving HL7v2 messages in one application. These ports each receive slightly different message types.

Let's call this one RP1

In another application we have send ports that subscribe to each respective receive port. These send ports each have an outbound map that transforms the messages in HL7v3 and submits it to a WCF (request/response) service.

Let's call this SP1

The WCF service then processes and validates the HL7v3 and sends back an HL7v3 ack message. The SP1 send port has custom send and receive pipeline components. The receive (from the WCF response) just takes the message and promotes certain fields which will be used later for subscriptions.

There are then two more Send ports. SP2 that subscribes to positive ACKs. SP3 to the Negative based on the fields promoted above. The positive ACKs are just consumed and the negative ACKs are emailed off to support staff.

The issue is that in about 10% of messages we are seeing 1 of these 2 error messages popping up:

A response message for two-way receive port "SP.CDX.LAB_MICRO.SubmitCDA.WCFCustom" is being suspended as the messaging engine could not correlate the response to an existing request message. This usually happens when the host process has been recycled. 
 
 MessageId:  {731623F3-995B-4C57-BD21-12865AD78717}
 InstanceID: {084BD473-C857-4C5E-A49B-8A86EA2CAC39}
The following stored procedure call failed: " { call [dbo].[bts_UpdateMsgbox_BizTalkServerReceive]( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}". SQL Server returned error string: "The statement has been terminated.;Cannot insert duplicate key row in object 'dbo.InstancesSuspended' with unique index 'IX_InstancesSuspended_InstanceID'. The duplicate key value is (084bd473-c857-4c5e-a49b-8a86ea2cac39, afa466c7-3bd2-4cde-a293-3df3fb5d8543).".

Usually followed by a suspended service instance in the Group viewer:

The instance completed without consuming all of its messages. The instance and its unconsumed messages have been suspended.

The Service Name of the suspended instance is that of RP1. The Message type of the non-consumed message is that of the ACK from SP1 (so it's the WCF response). This is weird because in my mind RP1 should never be expecting this response message AND there are send ports (SP2, SP3) subscribed to the response message types.

I've seen plenty of posts talking about this happening in Orchestrations, but nothing so far in a strictly subscription based solution. Anyone have any suggestions? This amounts to this error occurring about once every 1 minute or so. So by the end of the day we have quite a few suspended messages.

The other point I forgot to make is, there are 3 Receive Ports like RP1, each with 3 Receive Locations and 3 Send Ports subscribing to their respective Receive Ports.

The BizTalk Server is installed across 2 physical servers sharing 1 BizTalkMgmtDb/Messagebox

Could the sheer number of receive locations be causing any issues? Previous to this we had the same number of messages feeding in, but it was consolidated (at the sending end) to one single Receive Location. The old solution had multiple Orchestrations, but never experienced this issue.

Here's a basic image of what it looks like.

• Edited by Bensonius 13 hours 22 minutes ago

Judge from the error message, this is a Orchestrations issue, it is confused for your scenario

There are no Orchestrations being used in this scenario whatsoever. It's all receive/send ports.