Hi All,

Last year we had deployed FIM 2010 in our client environment. 2 months back, we have migrated the FIM solution to FIM 2010 R2.
Last week we faced an issue in FIM Portal. End user requests for different accesses and permissions on resources from the FIM portal. The requests goes to the user's manager/ resource owner for apporval. Some of the managers/approvers were unable to approve the pending requests in their bin. As per my observation, the issue is explained below in detail:

• Whenever the user (approver) tries to approve the requests from FIM portal (from My Pending Requests tab), he is getting the "Request Failed" popup. the requests is not getting approved and remains in the approvers bin in Pending state.
• There are no failure requests entries generated in the FIM admin portal (when looking into Search requests tab) and the requests remains in Authorizing state which seems like there is a pending approval to complete the request. Because of this, I (as the FIMADMIN) at server end was unable to identify that the requests are not getting approved.
• The requests are stucked. All the workflows in the stucked requests are left in Running state.
• As a process, If the approvers do not approves the request in 5 days, the request gets expired on the 6th day. I have observed that the stuck requests initaited in last week are not getting expired and remains in Authorizing state.
• In the event viewer logs on the FIM Portal machine, we have seen the below error each time the approvers were trying to approve requests and get the "Request Failed" error pop-up.

Request Identifier: 2247a182-3fd1-44f6-a7db-aa6e1130dbb8
Workflow Instance Identifier: fe68df8a-59be-4615-a339-680cf38f0458
Workflow Definition Identifier: 303cac16-d927-41f6-abce-21ccbd883ea2
Workflow Exception: System.IndexOutOfRangeException: Index was outside the bounds of the array. at System.Workflow.ComponentModel.Serialization.ActivitySurrogate.ActivitySerializedRef.System.Runtime.Serialization.IDeserializationCallback.OnDeserialization(Object sender) at System.Runtime.Serialization.DeserializationEventHandler.Invoke(Object sender) at System.Runtime.Serialization.ObjectManager.RaiseDeserializationEvent() at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(HeaderHandler handler, __BinaryParser serParser, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage) at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage) at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream) at System.Workflow.ComponentModel.Activity.Load(Stream stream, Activity outerActivity, IFormatter formatter) at System.Workflow.Runtime.Hosting.WorkflowPersistenceService.RestoreFromDefaultSerializedForm(Byte[] activityBytes, Activity outerActivity) at System.Workflow.Runtime.Hosting.SqlWorkflowPersistenceService.LoadWorkflowInstanceState(Guid id) at Microsoft.ResourceManagement.Workflow.Hosting.ResourceManagementSqlWorkflowPersistenceService.LoadWorkflowInstanceState(Guid Id)

Can anybody help me in identyfying the issue for this? I am looking for the answers to the below queries:

1. Has anybody seen this issue before or have any idea about why the requests are not getting approved?
2. How we can troubleshoot this issue and identify the exact reason for which the requests/workflows got stucked ?

Thanks in Advance.

Thanks,

Sanjog

Sanjog,

What version(build number) of FIM are you using? If you updated to the latest version, 4.1.3451 recently, this might be a known issue and you should contact PSS.

Hi Glenn,

Thanks for the reply.

The FIM version we are using is 4.1.3419.0 (FIM 2010 R2 SP1)

Thanks,

Sanjog

Sanjog,

If you look at the default approval activity in 'workflows', can you look at the XOML of the main activity and put it here? I have seen issue where it makes reference to .NET 4.0 but it should actually be at 3.5. I thought that this was only in the newest update but we should check this anyway.

Hi Glenn,

Thanks for the Reply.

I have checked the default Group Validation Workflow in FIM Portal. the XOML is referencing to .NET version 4.0.2592.0. Below is the XOML for your reference:<o:p></o:p>

 <ns0:SequentialWorkflow RequestId="00000000-0000-0000-0000-000000000000" x:Name="SequentialWorkflow" TargetId="00000000-0000-0000-0000-000000000000" ActorId="00000000-0000-0000-0000-000000000000" WorkflowDefinitionId="00000000-0000-0000-0000-000000000000" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:ns0="clr-namespace:Microsoft.ResourceManagement.Workflow.Activities;Assembly=Microsoft.ResourceManagement, Version=4.0.2592.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">  <ns0:GroupValidationActivity x:Name="groupValidationActivity1" ValidationSemantics="All" /> </ns0:SequentialWorkflow>

 In our environment, we are not using the default approval workflow, we have created a custom workflow. the XOML for that workflow is referencing to .NET version 4.0.3606.2, Below is the XOML for the same:

 <ns0:SequentialWorkflow x:Name="SequentialWorkflow" ActorId="00000000-0000-0000-0000-000000000000" WorkflowDefinitionId="00000000-0000-0000-0000-000000000000" RequestId="00000000-0000-0000-0000-000000000000" TargetId="00000000-0000-0000-0000-000000000000" xmlns:ns1="clr-namespace:CustomNamespace_Request;Assembly= CustomNamespace_Request, Version=1.0.0.0, Culture=neutral, PublicKeyToken=6862ca308c05536d" xmlns:ns2="clr-namespace:ApprovalDelegation2;Assembly=ApprovalDelegation2, Version=1.0.0.0, Culture=neutral, PublicKeyToken=02906e7f2d105bbd" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/workflow" xmlns:ns3="clr-namespace:System.Workflow.Activities;Assembly=System.WorkflowServices, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:ns0="clr-namespace:Microsoft.ResourceManagement.Workflow.Activities;Assembly=Microsoft.ResourceManagement, Version=4.0.3606.2, Culture=neutral, PublicKeyToken=31bf3856ad364e35"> 

Do you believe this can be the issue for the stuck requests in FIM portal? have you seen the exact issue in your case before?

Thanks,

Sanjog

Sanjog,

That is not actually the .NET version but the fact that it shows 4.0.2592 and 4.0.3606 makes this a problem. You stated the version of FIM you are using is SP1, 4.1.3419. Both of the above workflow activities should have same version number.

You said this is a custom WF activity.....it sounds like this was compiled using a different version of the FIM DLLs......................Verify that the dev machine is pointing to the correct version of FIM for compilation.............

Assuming that this fixes it, this won't fix the in-flight workflows retroactively, you should go to those requests and cancel them manually.

BTW, in the second workflow, the line

.......Version = 3.5.0.0, Culture-neutral.....................

That is the line that shows the .NET version and 3.5 is correct............

Hi Glenn,

Thanks for the reply.

As per my understanding, all the workflows are running fine and were stucked only for a set of requests raised in a particular period of time. Can you please help me understand the below queries:

1. Is the mismatch in version number of default (version 4.0.2592) and custom workflow (version 4.0.3606 ) activities can be the reason for this intermittent behavior for the stuck workflows? Please suggest on this.

2. Is there any way to re-evaluate the stuck/failed requests or any approach in which these requests are processed again? Because cancelling the stuck requests will impact large number of users to request again for the resources and the number is huge.

Thanks,

Sanjog

Sanjog,

The above mismatch is not necessarily a concern. you should have binding redirect statements similar to this at the bottom of your service config file:

dependentAssembly>

assemblyIdentityname="Microsoft.ResourceManagement"publicKeyToken="31bf3856ad364e35"culture="neutral"/>

bindingRedirectoldVersion="4.0.0.0-4.65535.65535.65535"newVersion="4.0.3644.2"/>

dependentAssembly>

So custom activites will show newer version but because of statements like the above, both should work normally. You error is probably being caused by something else.

Are you still experiencing this problem?

Sanjog,

Could you please share some details about your topology?

1) Do you have more that one FIMSerivce instance?

2) If yes, then are those instances of the same origin (i.e. both are clean installs or both are the upgrades from previous version of FIMService) or are different? And if different then how?

Thank you,

Denis.