Please share

How to split large xml files into small chunks using stream in custom disassembler comp

First, can you describe why the XmlDisassembler does not meet your needs?

Hi Ravindra,

As mentioned by boatseller, it can be acheived using the deafault XMLReceive pipeline.

Have a look at : http://tech-findings.blogspot.co.uk/2013/07/debatchingsplitting-xml-message-biztalk.html

It might help.

Ravindra, You might refer to my suggestion in this thread

Hi,

Actually we are receiving 2 gb xml file daily and it has nearly 100000 records. so if we debatch using XML disassembler then we need 100000 instances.  So we want to split as 1000 files each has 100 records and storing them into the database.

Then you need the custom pipeline component where you can define a configurable property to set how many records you want to put into a message when debatching. You can see the code I have given in the above thread and instead of creating message for single chunk you can use loop to write as many record as defined by custom property and can then create the message.

Go it.  Then I would wrap the XmlDasm component since it already does all the hard work.

You can add a property for the count. 

Then, using XmlReader/XmlWriter for example, you Write the result GetNext to you 'batch' message n-times, then return.  Repeat.  This is in the overridden GetNext.

Hi,

I think that you still need to play with the schema and the envelop, I don't know if you set the option of max occurs to 100, weither you'll debatch by 100 records instead of 1.

This is a good example that I faced many times, but I never had the time to investigate the 100 max occurs thing?

Thx,