Hi,

I have around 100,000 flat file records (29 MB Size) and I am debatching it into individual records, passing the debatched records into orchestration (as a sequential convoy, I have to use orchestration for some reason), and then passing it out (without doing any mapping or processing), the records are then received by a File dynamic receive port (set to append mode in the orchestration), in the receive port I have a map which formats the individual debatched message, then a pipeline containing flat file disassembler converts the xml to flat file and then a file adapter writes it into a single file (append mode).

However I am receiving BizTalk Throttling,  BizTalk host BizTalkServerApplication throttled because DatabaseSize exceeded the configured throttling limit.

Does anybody have any idea what throttling settings needs to be changed or any other way of implementing the solution.

THANKS

PLEASE HELP

Regards

Vikram

• Edited by Vikram_ Friday, February 06, 2015 10:13 AM

Hi Vikram ,

You are getting correct behaviour because The messages that was received probably required more processing resources than expected, or more messages arrived than expected.Then BizTalk reaches the limits of the available configured resources like memory. This often solves itself butit takes more time. In some scenarios the processing comes to a halt, because of this and BizTalk basically can't continue.If thisis the case you should look at the performance counters which indicates if BizTalk is throttling or not and what the memory consumption is. This MSDN article explains this and guides you which performance counters to monitor:

https://msdn.microsoft.com/en-us/library/aa578302(v=bts.20).aspx

Note : I would suggest not to publish this much are Flat file at the same time as each flat file is creating you 100 of independed threads depending on your debatched message .

Thanks

Abhishek

Hi Vikram ,

You are getting correct behaviour because The messages that was received probably required more processing resources than expected, or more messages arrived than expected.Then BizTalk reaches the limits of the available configured resources like memory. This often solves itself butit takes more time. In some scenarios the processing comes to a halt, because of this and BizTalk basically can't continue.If thisis the case you should look at the performance counters which indicates if BizTalk is throttling or not and what the memory consumption is. This MSDN article explains this and guides you which performance counters to monitor:

https://msdn.microsoft.com/en-us/library/aa578302(v=bts.20).aspx

Note : I would suggest not to publish this much are Flat file at the same time as each flat file is creating you 100 of independed threads depending on your debatched message .

Thanks

Abhishek

• Marked as answer by Angie xuMicrosoft contingent staff, Moderator Thursday, February 12, 2015 8:30 AM
• Unmarked as answer by Vikram_ Thursday, February 12, 2015 9:21 AM

Vikram,

As youre self this question:

I am going to receive a high volume file, need to debatch the 100k records, do some processing and output the record to a single file for these requirements do I really need BizTalk?, Do I really need to use BizTalks message-box or publish-subscription? For this requirement

My option NO. You can very much do this requirement in a standard .NET code with has some file-watcher class implemented into, to poll for any received message in a folder and process the received file and create another flat-file.

If you requirement is for a small file to get debatched and records get appended to single CSV file, then may be I could consider BizTalk still if I really need to use BizTalks publish-subscription pattern. Technically, Yes BizTalk can (also) do this requirement, but when file size becomes too big and involved debtaching and no need to use pub-sub/msgbox db, I would stay away from this ETL process.

I would NOT simply go and tweak the throttling settings, since I have the throttling state at the moment. Can you say whether your BizTalk servers are just going to be used for this above message processing. What if you client/company has got a new business this summer and their message flow/processing is going to triple from existing load. If you already introduce a message processing process which puts your server in throttling, how the future load is going to be handled by you server?

For your requirement where creating a single csv file after debatching a high volume file, I would not use BizTalk. I would do in standard .NET code.

Regards,

M.R.Ashwin Prabhu