Hi All

We are on FIM 2010 R2  V 4.1.3496.0

The scenario is,

1. Requestor raises multiple create resource requests
2.Approver selects all the requests in FIM portal and approves them in one go
3. All the requests fail except one (random one)
4. Event log shows following errors

If the approver approves the requests one by one then no issues and they all complete successfully. Any ideas or pointers?

Anyone seen this SQL deadlock errors before?

Error#1

Microsoft.ResourceManagement.ResourceManagementException: 'Resume Request 59fc950c-a7f2-42e2-a671-1dcaef2b6b4b' ---> Microsoft.ResourceManagement.WebServices.Exceptions.UnwillingToPerformException: Other ---> System.Data.SqlClient.SqlException: Reraised Error 50000, Level 13, State 1, Procedure ReRaiseException, Line 37, Message: Reraised Error 50000, Level 13, State 1, Procedure ReRaiseException, Line 37, Message: Reraised Error 1205, Level 13, State 51, Procedure CalculateRequestSetTranstionsMembershipConditionEvaluation, Line 2298, Message: Transaction (Process ID 62) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

Uncommittable transaction is detected at the end of the batch. The transaction is rolled back.

at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)

at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)

at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)

at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()

at System.Data.SqlClient.SqlDataReader.get_MetaData()

at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)

at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)

at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)

at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)

at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)

at System.Data.SqlClient.SqlCommand.ExecuteReader()

at Microsoft.ResourceManagement.Data.DataAccess.ProcessRequest(RequestType request)

--- End of inner exception stack trace ---

at Microsoft.ResourceManagement.Utilities.ExceptionManager.ThrowException(Exception exception)

at Microsoft.ResourceManagement.Data.Exception.DataAccessExceptionManager.ThrowException(SqlException innerException, TransactionAndConnectionScope scope)

at Microsoft.ResourceManagement.Data.DataAccess.ProcessRequest(RequestType request)

at Microsoft.ResourceManagement.ActionProcessor.ActionDispatcher.ProcessInputRequest(RequestType request)

at Microsoft.ResourceManagement.WebServices.RequestDispatcher.ExecuteAction(RequestType request)

at Microsoft.ResourceManagement.WebServices.RequestDispatcher.ExecuteAction[ResponseBodyType](RequestType request)

at Microsoft.ResourceManagement.WebServices.RequestDispatcher.DispatchRequest[ResponseBodyType](RequestType request, Guid requestIdentifier, Object redispatchSingleInstanceKey, Boolean isRedispatch)

at Microsoft.ResourceManagement.WebServices.RequestDispatcher.DispatchRequest[ResponseBodyType](Guid requestIdentifier, Boolean isRedispatch)

at Microsoft.ResourceManagement.WebServices.RequestDispatcher.RedispatchRequest(Object redispatchRequestInfo)

--- End of inner exception stack trace ---

Error#2

Request '59fc950c-a7f2-42e2-a671-1dcaef2b6b4b' failed while trying to commit the changes to the database. Exception: 'Dispatch Request Failure Source: 'Other''.

Error#3

Microsoft.ResourceManagement.WebServices.Exceptions.UnwillingToPerformException: Other ---> System.Data.SqlClient.SqlException: Reraised Error 50000, Level 13, State 1, Procedure ReRaiseException, Line 37, Message: Reraised Error 50000, Level 13, State 1, Procedure ReRaiseException, Line 37, Message: Reraised Error 1205, Level 13, State 51, Procedure CalculateRequestSetTranstionsMembershipConditionEvaluation, Line 2298, Message: Transaction (Process ID 62) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

Uncommittable transaction is detected at the end of the batch. The transaction is rolled back.

at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)

at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)

at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)

at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()

at System.Data.SqlClient.SqlDataReader.get_MetaData()

at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)

at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)

at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)

at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)

at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)

at System.Data.SqlClient.SqlCommand.ExecuteReader()

at Microsoft.ResourceManagement.Data.DataAccess.ProcessRequest(RequestType request)

--- End of inner exception stack trace ---

Error#4

Reraised Error 50000, Level 13, State 1, P

rocedure ReRaiseException, Line 37, Message: Reraised Error 50000, Level 13, State 1, Procedure ReRaiseException, Line 37, Message: Reraised Error 1205, Level 13, State 51, Procedure CalculateRequestSetTranstionsMembershipConditionEvaluation, Line 2298, Message: Transaction (Process ID 62) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

Uncommittable transaction is detected at the end of the batch. The transaction is rolled back.

1 and 3 are the same, usual garbage.  Pretty much saying "cant do it".  2 and 4 don't give much more clues anyways.

Despite that, this is definitely explainable, as there is nothing magic but a SQL Deadlock due to 2 writes happening at the same time to the same resource.

I would need to know more about the nature of these requests and the resources in question.  Do they share anything? I bet they do.

Any chance you can show one request that succeeds and the one that fails.   The detailed attributes created will tell you what