I've based my extended Flat File Disassembler on the example http://www.codeproject.com/Articles/17140/Debatching-Large-Messages-and-Extending-Flatfile-P That uses a MemoryStream to pass on the disassembled data.

msg.Context = ibmParam.Context;
IBaseMessagePart msgPart = pCxt.GetMessageFactory().CreateMessagePart();
System.IO.MemoryStream memStrm = new MemoryStream();
xDoc.Save(memStrm);
memStrm.Position = 0;
memStrm.Seek(0, System.IO.SeekOrigin.Begin);
msgPart.Data = memStrm;
msg.AddPart(ibmParam.BodyPartName, msgPart, true);
outboundMessages.Add(msg);
pCxt.ResourceTracker.AddResource(memStrm);

This works...

However, due to the amount of data I need to process, using the above, I run out of memory. So I'm planning to replace the MemoryStream with a FileStream.

Question.

In the example above, the MemoryStream is created but not closed. So does the FF Disassembler expect the msgPart.Data to be an open IOStream? Or, when I change to a FileStream,  should I call Flush() and Close()?

Thanks

Before anyone can give you specific advice, we need to understand the context of this code. Does it run in .Disassemble or .GetNext?

Some general recommendations:

1. Do not use a FileStream.  For large messages, use VirtualStream which is automatically buffered to disk if necessary.
2. Adding memStrm to the ResourceTracker here is no necessary since it is directly referenced by the Message.
3. Using an XmlDocument or xDocument in a pipeline is rarely a good idea.  Why exactly do you have this?  We can probably recommend something better.

Thanks for the reply.

To answer your questions - recommendations.

This code runs in .GetNext()

"VirtualStream"  - Thanks - I will try that out.  

"ResourceTracker" - That was in the original example - I'll remove it.

"XmlDocument or xDocument" - My system is down at the moment - I'll respond with my exact code when it's back up.

Before anyone can give you specific advice, we need to understand the context of this code. Does it run in .Disassemble or .GetNext?

Some general recommendations:

1. Do not use a FileStream.  For large messages, use VirtualStream which is automatically buffered to disk if necessary.
2. Adding memStrm to the ResourceTracker here is no necessary since it is directly referenced by the Message.
3. Using an XmlDocument or xDocument in a pipeline is rarely a good idea.  Why exactly do you have this?  We can probably recommend something better.

• Proposed as answer by Angie xuMicrosoft contingent staff, Moderator 5 hours 16 minutes ago

Before anyone can give you specific advice, we need to understand the context of this code. Does it run in .Disassemble or .GetNext?

Some general recommendations:

1. Do not use a FileStream.  For large messages, use VirtualStream which is automatically buffered to disk if necessary.
2. Adding memStrm to the ResourceTracker here is no necessary since it is directly referenced by the Message.
3. Using an XmlDocument or xDocument in a pipeline is rarely a good idea.  Why exactly do you have this?  We can probably recommend something better.

• Proposed as answer by Angie xuMicrosoft contingent staff, Moderator Monday, August 03, 2015 1:51 AM
• Marked as answer by TreforE 14 hours 58 minutes ago

Before anyone can give you specific advice, we need to understand the context of this code. Does it run in .Disassemble or .GetNext?

Some general recommendations:

1. Do not use a FileStream.  For large messages, use VirtualStream which is automatically buffered to disk if necessary.
2. Adding memStrm to the ResourceTracker here is no necessary since it is directly referenced by the Message.
3. Using an XmlDocument or xDocument in a pipeline is rarely a good idea.  Why exactly do you have this?  We can probably recommend something better.

• Proposed as answer by Angie xuMicrosoft contingent staff, Moderator Monday, August 03, 2015 1:51 AM
• Marked as answer by TreforE Monday, August 03, 2015 4:06 PM

Before anyone can give you specific advice, we need to understand the context of this code. Does it run in .Disassemble or .GetNext?

Some general recommendations:

1. Do not use a FileStream.  For large messages, use VirtualStream which is automatically buffered to disk if necessary.
2. Adding memStrm to the ResourceTracker here is no necessary since it is directly referenced by the Message.
3. Using an XmlDocument or xDocument in a pipeline is rarely a good idea.  Why exactly do you have this?  We can probably recommend something better.

• Proposed as answer by Angie xuMicrosoft contingent staff, Moderator Monday, August 03, 2015 1:51 AM
• Marked as answer by TreforE Monday, August 03, 2015 4:06 PM

Thanks for the reply.

To answer your questions - recommendations.

This code runs in .GetNext()

"VirtualStream"  - Thanks - I will try that out.  

"ResourceTracker" - That was in the original example - I'll remove it.

"XmlDocument or xDocument" - My system is down at the moment - I'll respond with my exact code when it's back up.

• Marked as answer by TreforE 14 hours 58 minutes ago

Thanks for the reply.

To answer your questions - recommendations.

This code runs in .GetNext()

"VirtualStream"  - Thanks - I will try that out.  

"ResourceTracker" - That was in the original example - I'll remove it.

"XmlDocument or xDocument" - My system is down at the moment - I'll respond with my exact code when it's back up.

• Marked as answer by TreforE Monday, August 03, 2015 4:06 PM

Thanks for the reply.

To answer your questions - recommendations.

This code runs in .GetNext()

"VirtualStream"  - Thanks - I will try that out.  

"ResourceTracker" - That was in the original example - I'll remove it.

"XmlDocument or xDocument" - My system is down at the moment - I'll respond with my exact code when it's back up.

• Marked as answer by TreforE Monday, August 03, 2015 4:06 PM

VirtualStream - that fixed my problem .. Excellent - Thanks