How to run the MAs in Parallel? ----------------------------------- 1. Create a folder structure C:\Jobs\Logs C:\Jobs\Check C:\Jobs\NoCheck C:\Jobs - Put all the scripts here 2. ILM_RunMA.vbs - VBScript to run the Management Agent 3. ILM_RunMA.bat - --------------------------------------------------------------------------- @Echo OFF echo "..." > c:\jobs\%4\%1.log ping -n 2 127.0.0.1 >NUL CScript //Nologo ILM_RunMA.vbs %1 %2 del c:\jobs\%4\%1.log IF %3 EQU "1" ( exit ) ----------------------------------------------------------------------------- 4. ILM_Wait.bat ----------------------------------------------------------------------------- @Echo OFF ping -n 5 127.0.0.1 >NUL :Check SET GO=1 FOR %%M IN ("%dir c:\jobs\%1\*.*%") DO ( ping -n 10 127.0.0.1 >NUL SET GO=0 ) IF %GO% EQU 0 GOTO Check -------------------------------------------------------------------------------- 5. ILM_SetEnv.bat - Set all the env variables here -------------------------------------------------------------------------------- @Echo OFF IF NOT EXIST Check MKDIR Check IF NOT EXIST Logs MKDIR Logs del /Q c:\jobs\Check\*.* ------------------------------------------------------------------------------------- 6. ILM_Your_Jobs_1.bat - Runs the MAs ------------------------------------------------------------------------------------- -- Ex. Run MAs sequentially call ILM_RunMA.bat "MA_Nam1" "Run Profile Name" "0" NoCheck call ILM_RunMA.bat "MA_Nam1" "Run Profile Name" "0" NoCheck call ILM_RunMA.bat "MA_Nam1" "Run Profile Name" "0" NoCheck -- Ex. Run MAs in Parallel. No the last two parameters while calling the batch file call ILM_RunMA.bat "MA_Nam1" "Run Profile Name" "1" Check call ILM_RunMA.bat "MA_Nam1" "Run Profile Name" "1" Check call ILM_RunMA.bat "MA_Nam1" "Run Profile Name" "1" Check call ILM_Wait.bat -> Wait until all the MAs have finished running call ILM_RunMA.bat "MA_Nam1" "Run Profile Name" "0" NoCheck call ILM_RunMA.bat "MA_Nam1" "Run Profile Name" "0" NoCheck call ILM_RunMA.bat "MA_Nam1" "Run Profile Name" "0" NoCheck ------------------------------------------------------------------------------------------- 7. ILM_Your_Jobs.bat - Create a Windows Task to run this. This also creates a Continue.txt file which will make sure that jobs keep running in loop. If you want to stop the schedule, delete Continue.txt file. ------------------------------------------------------------------------------------------- @Echo Off call ILM_SetEnv.bat echo "..." > c:\jobs\Continue.txt :BEGIN CD C:\Jobs\ cls SET LogFileName=%DATE%_%TIME% SET LogFileName=%LogFileName:/=_% SET LogFileName=%LogFileName::=_% SET LogFileName=%LogFileName: =_% SET LogFileName=%LogFileName:~0,-3% CALL ILM_Your_Jobs_1.bat > C:\Jobs\Logs\ILM_Your_Jobs_%LogFileName%.log 2>&1 ping -n 30 127.0.0.1 >NUL IF EXIST Continue.txt ( GOTO BEGIN ) --------------------------------------------------------------------------------------------------------------------------------- We have been using this solution in PRODUCTION for a while now and it is very flexible. - Sachin

Thanks for sharin, Sachin.You should add you VBScript code (ILM_RunMA.vbs) to make this complete.Cheers,MarkusMarkus Vilcinskas, Knowledge Engineer, Microsoft Corporation

ILM_RunMA.vbs ------------------ Option Explicit On Error Resume Next Dim Service Dim ManagementAgent Dim RunStatus Dim cn 'ADO connection object Dim cnCmd 'ADO Command object Dim cnRecordset 'recordset returned from stored procedure Dim cnParam 'ADO parameter passed to stored procedure Dim strCS 'connection string Dim boolError 'boolean is set to TRUE if an error occurs Dim ReturnStr IF wscript.Arguments.Count <> 2 THEN WScript.Echo WScript.Echo "Usage: cscript //Nologo ILM_RunMA.vbs Management_Agent_Name Run_Profile_Name" WScript.Quit(1) END IF Set Service = GetObject("winmgmts:root\MicrosoftIdentityIntegrationServer") Set ManagementAgent = Service.Get("MIIS_ManagementAgent.Name='" & wscript.Arguments(0) & "'") WScript.Echo ReturnStr = Cstr(Now()) + "," + ManagementAgent.name + "," + wscript.Arguments(1) + "," Select Case wscript.Arguments(1) Case "Export" IF 000 = ManagementAgent.numExportAdd() + ManagementAgent.numExportUpdate() + ManagementAgent.numExportDelete() Then WScript.Echo Cstr(Now()) + "," + ManagementAgent.name + ",Export,success," + Cstr(Now()) WScript.Quit(0) WScript.Echo END IF End Select Do RunStatus = ManagementAgent.Execute(wscript.Arguments(1)) Loop While RunStatus = "sql-deadlock" WScript.Echo ReturnStr + RunStatus + "," + Cstr(Now()) Sub ErrorHandler (ErrorMessage) WScript.Echo ErrorMessage WScript.Quit(1) End Sub

hi Sachin; MA.Execute is synchronous method. asynchronous runMA is use the start command and background script (not "start /wait"). and wait background scripts to before the next step. import/export is possible the parallel execute for difference MA. but synchronization is not parallel. because synchronization is the update to metaverse, and access to other MA's Connector Space.

Hi Knoji Yes, that is correct. We run only "Import" and "Export" operation in Parallel and Sync operations sequentially. Cheers Sachin

One thing you have to becareful for is doing too many things in parrallel.If you have too many things going on, locking will occur. Just a warning for anybody setting this up.Make sure your box can handle it before you try to implement it in any production environment. It will require heavy hardware to work, but there are alot of variables also... Just accurately test....You can run multiple sync's at a time, as long as they are not touching the same object. But it is highly recommended to not do this unless you have experience partitioning the environment.HTH,JoeJoe Stepongzi - Identity Management Consultant - ILM MVP - www.microsoftIdM.com,ilmXframework.codeplex.com

Good folks of the ILM/FIM world ... I'm no longer going to beat around the bush on this ... :). Call me obscenely biased if you will ;) ... but you can't go past UNIFY Event Broker for the best way of running your ILM/FIM operations!!! Sure you can write scripts without much trouble, and there's always plenty of examples to find of how to do specific tasks .... but then someone will be coming along behind you one day and you'll no doubt have moved on to bigger and better things. I prefer not to reinvent wheels if I have a good one ... and all the evidence suggests this wheel is pretty good. Event Broker for ILM is designed to handle exactly what Joe is saying by supporting both synchronous and asynchronous run profile execution. Furthermore it is designed by ILM implementers for ILM implementers to be a "one stop shop" for configuring the execution of not only ILM run profiles, but any pre/post processing you may wish to do. A key capability is automating the running of both imports and exports where it makes sense to do so (e.g. when a change occurs in AD, or a pending export is ready to go). The product is in its 3rd iteration, supports MIIS/ILM/FIM, is robust enough for the largest of Enterprise deployments and incurs a very small server footprint. Unlike with scripts there is no risk of memory leaking, has logging built in with configurable log levels, and you can confidently switch all ILM/FIM processing on/off in a central location. The price tag is US$10K and will more than pay for itself at the design/deployment stage - that's before it's smoothly running all aspects of your production ILM/FIM environment. Always happy to give a 30 minute remote desktop presentation - in whatever timezone - just reply here or shoot me an email - but please, no tyre-kickers - others can be salespeople, but I'm a techo interested making changes for the better, and feel the need to "share the love". Until then we at UNIFY will happily implement away ourselves ... but do check out my blog (bobbradley.spaces.live.com/blog/) or www.unifysolutions.net/ourSolutions.cfm?solution=event for more info if you're serious about leaving the alternatives behind ;). Regards, Bob

Adding the PowerShell approach: It is pretty easy with PowerShell to execute management agents in FIM. It is also pretty easy to do it in parallel using PowerShell jobs. The sample below shows how to use the Start-Job cmdlet to run Management Agents in FIM. In this sample I am using Get-WmiObject to get all of the Management Agents, then I apply a Where filter to return just the ADMAs. I could have specified a similar filter using Get-WmiObject, and would have done so if there were a large number of management agents on the server. The Start-Job cmdlet runs the script block in another PowerShell session. If you’re watching Task Manager while the script runs you’ll see another PowerShell.exe process start up. Also, if you’re watching Identity Manager, you’ll see in the Operations pane that all of the ADMAs start at about the same time (cool!). Keep in mind that THIS IS NOT RECOMMENDED FOR SYNCHRONIZATION RUN PROFILES!!! The sync engine does not like this one bit, and will likely grind to a halt dealing with SQL deadlocks. ### ### Run ADMAs - Full Import - In Parallel ### $admas = Get-WmiObject -Class MIIS_ManagementAgent -Namespace root/MicrosoftIdentityIntegrationServer | where {$_.Type -eq 'Active Directory'} foreach ($ma in $admas) { Write-host "Starting Full Import on" $ma.Name"..." Start-Job -ArgumentList $ma.Name -ScriptBlock ` { param($maName)$ma = Get-WmiObject -Class MIIS_ManagementAgent -Namespace root/MicrosoftIdentityIntegrationServer -Filter "name='$maName'"; $result = $ma.Execute('Full Import').ReturnValue; Write-Host ("{0} result: {1}" -f $maName, $result) } | Out-Null } ### Wait for the imports to complete before returning Get-Job | Wait-Job ### Review the output from each job Get-Job | Receive-Job ### Remove the jobs Get-Job | Remove-Job CraigMartin Edgile, Inc. http://identitytrench.com