Hi,

Can anyone please assist the below scenario.

On each day, Receiving 500 + flat files from external systems. At End of day, i have to receive all the flat files and merge to a single file and send to another system. it means like copy all the contents and put it in to single file with Adding header/trailer row.

Is it some thing like we need to use singleton orchestration to collect all the files and do merging ?

Thanks,

Vinoth

HI ,

Have a look into below links

Flat file debatching/rebatching

Biztalk 2010 aggregation  and one below link

https://social.msdn.microsoft.com/Forums/en-US/d5b7320a-5a1a-4b8a-a565-495099716214/assemble-multiple-messages-into-one-flat-file-as-output?forum=biztalkgeneral

Thanks

Abhishek

• Marked as answer by Angie xuMicrosoft contingent staff, Moderator 8 hours 30 minutes ago

HI ,

Have a look into below links

Flat file debatching/rebatching

Biztalk 2010 aggregation  and one below link

https://social.msdn.microsoft.com/Forums/en-US/d5b7320a-5a1a-4b8a-a565-495099716214/assemble-multiple-messages-into-one-flat-file-as-output?forum=biztalkgeneral

Thanks

Abhishek

• Marked as answer by Angie xuMicrosoft contingent staff, Moderator Monday, February 09, 2015 2:56 AM

Hi Vinoth,

From experience, I would recommend that you use the following design (always try to use messaging service more than orchestration service when applicable)

1- create a receive port (i.e RP1 ) with a receive location to receive your source files

2- Create a send port that is subscribed to RP1 , and have a flat file assembler pipeline configured on it

3- configure the send port to append the out files to a temp path (i.e. C:\temp\OutputTemp.csv ), the file name should be static so that the aggregation can work

4- create another receive port (RP2) with a receive location to receive the aggregated file from your temp path

5- configure the receive location of RP2 to be scheduled at the time you consider end of day (i.e. 6pm), this means that the files will keep aggregating until 6pm

6- you can create an orchestration that is bound to RP2 which will add the head and the footer to the received file, or you can even create a web page (asp.net) that will do this for you by clicking a button (in case you want it to be a manual process at the end of the day)

this approach is loosely couple which means you will have more control in following up issues (you can archive what you receive , you can change the end of day time anytime you want, avoid singleton issues and zombie messages in the message box , a faster approach ...)

Hi Vinoth,

You can implement Messaging-Database Aggregator Pattern here as well.

You can store messages temporarily in Custom SQL database.

Have One Stored procedure to add the messages to a table in the Custom database, also checks for the completeness criteria. When this criteria is met, the status of the messages is updated to mark them for aggregation.

Second Stored procedure to poll the database for batches ready to be aggregated. When a batch is ready, the data is returned, and the status flag for the messages is updated.

Then you can transform the SQL result set as desired.

Depending on the structure of the Flat Files, have you considered staging them in SQL Server using SSIS?

Then poll out and format for the target system all at once.

Hi,

BizTalk ships along with the sample solution.

Check this path in the machine where BizTalk is installed, or run a search for Aggregator

C:\Program Files (x86)\Microsoft BizTalk Server 2009\SDK\Samples\Pipelines\Aggregator

Hi,

I tries with the Aggregator pattern and it was worked.. Thanks Again..

But now my requirement is little changed..

The Orchestration should be triggered on particular time interval and then need to pick all files and Aggregate it..

So i changed the existing Aggregator pattern like first put a receive shape with activate = true , and then use receive shape with initialize correlation and send port to follow the correlation. but i am getting "you must specify at least one already-initialized correlation set for a non-activation receive that is on a non-selfcorrelating port".. I have tried with few other options. but it is not worked..

Can you please help me how to do that? .. if i need to trigger the orchestration on particular time and then need to pick all the files and aggregate.

Thanks,

Vinoth

Hi,

You can configure the receive port which listens to the message coming from BizTalk working under a timed window. See the image below.