I'm looking for guidance on high volume message designs using Biztalk or other Microsoft technologies.  The scenario is simple. the system needs to run email campaigns where one or more campaign could be 50k+ messages.  Each mesage takes time to generate and format the email.  The resulting message size is small.  A web service could generate and format an individual message.  Biztalk would call the web service and send the message upon completion.

Does Biztalk have a design pattern that would allow messages to be built and sent, via SMTP adapter, in asynchronous fashion as described above?  Assume for the moment that the email infrastructure (some linux server, Zirinity) has no need to throttle messages.  What kind of message volume should I expect for a single host application server?  How stable/reliable is this type of solution running on BizTalk?  Are there other technologies/approaches I could consider (for example: MSMQ + WCF services + multi-threading)?

Thanks,

Yes, but this really isn't fit for BizTalk unless the emails are the result of an existing process, order processing for example.

There are many legit commercial e-mail campaign platforms already and if you really need a custom solution, .Net + Windows SMTP is already in the box.

Either way, 50,000 messages of any kind isn't really a lot for a BizTalk app.

It can and has been done. Depends on where you choose to bring BizTalk in... think of a scenario where multiple systems need to generate alerts and you wish to centralize the delivery of alerts. You could look at a scenario where the various systems insert data into a DB Table or Queue. The you could use the "canonical schema" pattern to map from individual sources construct the message within the BizTalk orchestration and use the SMTP adapter to sent it to the SMTP host for dispatch. You can additionally abstract the functionality of storing the message templates in SharePoint Foundation libraries (this will permit the user to manage the template changes w/o affecting the running).

To handle large volumes you would look at multiple systems, processing hosts and separated send handles for SMTP.

However if these messages have attachments then you'd face a bit of a problem in coordinating the attachments with the messages.

If on the other hand you looking at only transiting SMTP messages then BizTalk as an e-mail router is definitely (IMHO, my 2c, FWIW, etc.) a bad idea.

Regards.

I'm less interested in using Biztalk for the SMTP service and more focused on generating the email to be sent.  I suspect the actual code to generate the body of the email will be contained in a web service.  Something needs to be highly reliable and robust to call the web service, receive the result (email body) and then send the email.  Third party platforms would only help send the email.  Generating the email body requires custom code, data, templates per client and we've already written that.

Today this all works via windows service.  The emails for each campaign are created in a big loop and sent out in a serial fashion.  There may be a dozen campaigns queued up waiting to go out because of the size of the one being processed.  So maybe this is just an MSMQ/Web Service solution but we have BizTalk and I know from prior experience that running high volume is possible.