How to implement Custom disassembler pipeline component to split 1 GB xml file into small Chunks?. Please help me

You can use the XMLDisassembler component to do it refer to this blog post to know how to debatch XML message. If you want to create a custom pipleine component because your scenario can't be handled by XMLDisassembler then you can use the following sample code as starting point. I have not tested this code you can modify it as per your requirement:

        System.Collections.Queue qOutputMsgs = new System.Collections.Queue();
        private const int BufferSize = 0x280;
        private const int ThresholdSize = 0x100000;

        #region IDisassemblerComponent Members

        public void Disassemble(IPipelineContext pContext, IBaseMessage pInMsg)
        {
            Stream inputStream = pInMsg.BodyPart.GetOriginalDataStream();

            if (!inputStream.CanSeek)
            {
                ReadOnlySeekableStream seekableStream = new ReadOnlySeekableStream(inputStream);
                pContext.ResourceTracker.AddResource(seekableStream);
                inputStream = seekableStream;
            }


            var outputStream = new VirtualStream(BufferSize, ThresholdSize);

            using (var msgReader = XmlReader.Create(inputStream)
            {
                while (msgReader.Read())
                {
                    msgReader.MoveToContent();
                    //You can implement your logic to debatch here as an example 
                    //this code create new message for each ABCElement
                    if (msgReader.NodeType == XmlNodeType.Element && msgReader.LocalName == "ABCElement")
                    {
                        var fragment = (XElement) XNode.ReadFrom(msgReader);
                        var outputStream = new VirtualStream(BufferSize, ThresholdSize);
                        pContext.ResourceTracker.AddResource(outputStream)
                        using (XmlWriter xmlWriter = XmlWriter.Create(outputStream))
                        {
                            xmlWriter.WriteStartDocument();
                            fragment.WriteTo(xmlWriter);
                        }
                        outputStream.Seek(0, SeekOrigin.Begin);

                        //Generate new message having ABCElement as it's content
                        IBaseMessage outMsg;

                        outMsg = pContext.GetMessageFactory().CreateMessage();
                        outMsg.AddPart("Body",pContext.GetMessageFactory().CreateMessagePart(), true);
                        outMsg.BodyPart.Data = outputStream;
                        outMsg.Context = PipelineUtil.CloneMessageContext(pInMsg.Context);
                        //Ad the generated message to queue
                        qOutputMsgs.Enqueue(outMsg);
                    }
                }
            }
        }

        public IBaseMessage GetNext(IPipelineContext pContext)
        {
            if (qOutputMsgs.Count > 0)
            {
                IBaseMessage msg = (IBaseMessage)qOutputMsgs.Dequeue();
                return msg;
            }
            else
                return null;
        }

        #endregion

Hi Rohit,

Thank for the reply. I will try this one and let you know. 

Why don't you try to use native solutions?

One of it, could be debatching, if it is an xml file you can use envelop to debatch, with the standard XMLReceive pipeline but by specifying the envelop schema in the port -> pipeline -> xml receive ... parameters

Thanks,

Stefan

Hi ,

As stefan mentioned .

Default XMLReceive Pipeline should be able to do it, as 1 GB is not that big for BizTalk.

You can have look at this po