SCOM 2007 R2 - SDK Service Exception with Event ID's 27000, 26371, 26338, 26380, 26319
Greetings, We are building out a SCOM 2007 R2 environment (no CU's). The environment looks like the following: 1 RMS Server (W2K8 R2 Ent) 1 SQL Server 2008 R2 (W2K8 R2 Ent, non-clustered) We have followed the guidance for installing the OpsMgr DB and DW on SQL 2008 R2 using the DBCREATE tool. Overall the system looks pretty healthy. However, when a user tries to connect via remote console, and that user is NOT in the "Operations Manager Adminsitrators" User Role but in any other role such as "Advanced Operators", we are seeing the SDK service exception. The client receives the following error: Application: System Center Operations Manager 2007 R2 Application Version: 6.1.7221.0 Severity: Error Message: Failed to connect to server 'XXXXXMOMD01.xxxxx.net' Microsoft.EnterpriseManagement.Common.ServiceNotRunningException: The sdk service is either not running or not yet initialized. ---> System.ServiceModel.EndpointNotFoundException: Could not connect to net.tcp://xxxrtpmomd01.xxxxxx.net:5724/SdkDataAccessSSL. The connection attempt lasted for a time span of 00:00:01.0029297. TCP error code 10061: No connection could be made because the target machine actively refused it xxx.xxx.xxx.xxx:5724. ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it xxx.xxx.xxx.xxx:5724 at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress) at System.Net.Sockets.Socket.Connect(EndPoint remoteEP) at System.ServiceModel.Channels.SocketConnectionInitiator.Connect(Uri uri, TimeSpan timeout) --- End of inner exception stack trace --- Server stack trace: at System.ServiceModel.Channels.SocketConnectionInitiator.Connect(Uri uri, TimeSpan timeout) at System.ServiceModel.Channels.BufferedConnectionInitiator.Connect(Uri uri, TimeSpan timeout) at System.ServiceModel.Channels.ConnectionPoolHelper.EstablishConnection(TimeSpan timeout) at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) Exception rethrown at [0]: at System.ServiceModel.Security.IssuanceTokenProviderBase`1.DoNegotiation(TimeSpan timeout) at System.ServiceModel.Security.SspiNegotiationTokenProvider.OnOpen(TimeSpan timeout) at System.ServiceModel.Security.WrapperSecurityCommunicationObject.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) at System.ServiceModel.Security.SecurityUtils.OpenCommunicationObject(ICommunicationObject obj, TimeSpan timeout) at System.ServiceModel.Security.SecurityProtocol.OnOpen(TimeSpan timeout) at System.ServiceModel.Security.WrapperSecurityCommunicationObject.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) at System.ServiceModel.Channels.SecurityChannelFactory`1.ClientSecurityChannel`1.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) at System.ServiceModel.Channels.LayeredChannel`1.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.CallOnceManager.CallOnce(TimeSpan timeout, CallOnceManager cascade) at System.ServiceModel.Channels.ServiceChannel.EnsureOpened(TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) Exception rethrown at [1]: at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) at Microsoft.EnterpriseManagement.Common.ISessionManager.Connect(Boolean useCache) at Microsoft.EnterpriseManagement.DataAbstractionLayer.SdkDataAbstractionLayer.CreateChannel(TieredManagementGroupConnectionSettings managementGroupTier) --- End of inner exception stack trace --- at Microsoft.EnterpriseManagement.DataAbstractionLayer.SdkDataAbstractionLayer.HandleIndigoExceptions(Exception ex) at Microsoft.EnterpriseManagement.DataAbstractionLayer.SdkDataAbstractionLayer.CreateChannel(TieredManagementGroupConnectionSettings managementGroupTier) at Microsoft.EnterpriseManagement.ManagementGroupInternal.Reconnect() at Microsoft.EnterpriseManagement.DataAbstractionLayer.SdkDataAbstractionLayer.Connect(ManagementGroupConnectionSettings connectionSettings) at Microsoft.EnterpriseManagement.ManagementGroup..ctor(String serverName) at Microsoft.EnterpriseManagement.ManagementGroup.Connect(String serverName) at Microsoft.EnterpriseManagement.Mom.Internal.UI.Common.ManagementGroupSessionManager.Connect(String server) at Microsoft.EnterpriseManagement.Mom.Internal.UI.Console.ConsoleWindowBase.TryConnectToManagementGroupJob(Object sender, ConsoleJobEventArgs args) System.ServiceModel.EndpointNotFoundException: Could not connect to net.tcp://xxxrtpmomd01.xxxxxx.net:5724/SdkDataAccessSSL. The connection attempt lasted for a time span of 00:00:01.0029297. TCP error code 10061: No connection could be made because the target machine actively refused it xxx.xxx.xxx.xxx:5724. ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it xxx.xxx.xxx.xxx:5724 at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress) at System.Net.Sockets.Socket.Connect(EndPoint remoteEP) at System.ServiceModel.Channels.SocketConnectionInitiator.Connect(Uri uri, TimeSpan timeout) --- End of inner exception stack trace --- Server stack trace: at System.ServiceModel.Channels.SocketConnectionInitiator.Connect(Uri uri, TimeSpan timeout) at System.ServiceModel.Channels.BufferedConnectionInitiator.Connect(Uri uri, TimeSpan timeout) at System.ServiceModel.Channels.ConnectionPoolHelper.EstablishConnection(TimeSpan timeout) at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) Exception rethrown at [0]: at System.ServiceModel.Security.IssuanceTokenProviderBase`1.DoNegotiation(TimeSpan timeout) at System.ServiceModel.Security.SspiNegotiationTokenProvider.OnOpen(TimeSpan timeout) at System.ServiceModel.Security.WrapperSecurityCommunicationObject.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) at System.ServiceModel.Security.SecurityUtils.OpenCommunicationObject(ICommunicationObject obj, TimeSpan timeout) at System.ServiceModel.Security.SecurityProtocol.OnOpen(TimeSpan timeout) at System.ServiceModel.Security.WrapperSecurityCommunicationObject.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) at System.ServiceModel.Channels.SecurityChannelFactory`1.ClientSecurityChannel`1.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) at System.ServiceModel.Channels.LayeredChannel`1.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout) at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.CallOnceManager.CallOnce(TimeSpan timeout, CallOnceManager cascade) at System.ServiceModel.Channels.ServiceChannel.EnsureOpened(TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) ---------------------------------------------------------------------------------------------------- At the same time on the SCOM server in the Operations Manager Event Logs there is in Informational event that shows that user connection (Eid 26328), then the SDK errors ensue: ----------------------------------------------------------------------------------------------------- >>Event ID: 26338 /Source OpsMgr SDK Service ----------------------------------------------------------------------------------------------------- The SDK service is not able to generate auditing events for SDK operations. Verify that the SDK service account has the correct rights to generate security audits. The SDK service will continue without generating auditing events. Details: System.OutOfMemoryException: Insufficient memory to continue the execution of the program. at Microsoft.EnterpriseManagement.Mom.Internal.Auditing.MomAuditLog.HandleError(Int32 error) at Microsoft.EnterpriseManagement.Mom.Internal.Auditing.MomAuditLog.ReportSecurityEvent(Int32 auditId, AuditType auditType, Object[] parameters) at Microsoft.EnterpriseManagement.Mom.Internal.Auditing.MomAuditLog.LogAuditEvent(Int32 eventId, AuditType auditType, String[] formatPlaceholders) at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.Auditing.AuditFailure(WindowsIdentity windowsIdentity, String operationName, MethodBase callingMethod) ----------------------------------------------------------------------------------------------------- >>Event ID: 26319 /Source OpsMgr SDK Service ----------------------------------------------------------------------------------------------------- An exception was thrown while processing GetTypeCache for session id uuid:0a04c129-b406-4f38-8bd4-c228764eed70;id=4. Exception Message: Insufficient memory to continue the execution of the program. Full Exception: System.OutOfMemoryException: Insufficient memory to continue the execution of the program. at Microsoft.EnterpriseManagement.Mom.Internal.Auditing.MomAuditLog.HandleError(Int32 error) at Microsoft.EnterpriseManagement.Mom.Internal.Auditing.MomAuditLog.ReportSecurityEvent(Int32 auditId, AuditType auditType, Object[] parameters) at Microsoft.EnterpriseManagement.Mom.Internal.Auditing.MomAuditLog.LogAuditEvent(Int32 eventId, AuditType auditType, String[] formatPlaceholders) at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.Auditing.AuditFailure(WindowsIdentity windowsIdentity, String operationName, MethodBase callingMethod) at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.AuthManager.CheckResults(AuthContext authContext) at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.AuthManager.AccessCheckOnOperations(AuthContext authContext) at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.AuthManager.AccessCheck(AuthContext authContext) at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.AuthManager.AccessCheck(MethodBase callingMethod) at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess.IsUserInOperation(MethodBase callingMethod) at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess.IsUserAdministrator() at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess.GetTypeCache(String languageCode) at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccessTieringWrapper.GetTypeCache(String languageCode) at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccessExceptionTracingWrapper.GetTypeCache(String languageCode) ----------------------------------------------------------------------------------------------------- >>Event ID: 26380 /Source OpsMgr SDK Service ----------------------------------------------------------------------------------------------------- The System Center Operations Manager SDK Service failed due to an unhandled exception. The service will attempt to restart. Exception: System.OutOfMemoryException: Insufficient memory to continue the execution of the program. at Microsoft.EnterpriseManagement.Mom.Internal.Auditing.MomAuditLog.HandleError(Int32 error) at Microsoft.EnterpriseManagement.Mom.Internal.Auditing.MomAuditLog.ReportSecurityEvent(Int32 auditId, AuditType auditType, Object[] parameters) at Microsoft.EnterpriseManagement.Mom.Internal.Auditing.MomAuditLog.LogAuditEvent(Int32 eventId, AuditType auditType, String[] formatPlaceholders) at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.Auditing.AuditFailure(WindowsIdentity windowsIdentity, String operationName, MethodBase callingMethod) at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.AuthManager.CheckResults(AuthContext authContext) at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.AuthManager.AccessCheckOnOperations(AuthContext authContext) at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.AuthManager.AccessCheck(AuthContext authContext) at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.AuthManager.AccessCheck(MethodBase callingMethod) at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess.IsUserInOperation(MethodBase callingMethod) at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess.IsUserAdministrator() at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess.GetTypeCache(String languageCode) at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccessTieringWrapper.GetTypeCache(String languageCode) at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccessExceptionTracingWrapper.GetTypeCache(String languageCode) at SyncInvokeGetTypeCache(Object , Object[] , Object[] ) at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs) at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc& rpc) at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet) at System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(RequestContext request, Boolean cleanThread, OperationContext currentOperationContext) at System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(RequestContext request, OperationContext currentOperationContext) at System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(IAsyncResult result) at System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result) at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously) at System.ServiceModel.Channels.FramingDuplexSessionChannel.TryReceiveAsyncResult.OnReceive(IAsyncResult result) at System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result) at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously) at System.ServiceModel.Channels.SynchronizedMessageSource.SynchronizedAsyncResult`1.CompleteWithUnlock(Boolean synchronous, Exception exception) at System.ServiceModel.Channels.SynchronizedMessageSource.ReceiveAsyncResult.OnReceiveComplete(Object state) at System.ServiceModel.Channels.SessionConnectionReader.OnAsyncReadComplete(Object state) at System.ServiceModel.Channels.StreamConnection.OnRead(IAsyncResult result) at System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result) at System.Net.LazyAsyncResult.Complete(IntPtr userToken) at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken) at System.Net.Security.NegotiateStream.ProcessFrameBody(Int32 readBytes, Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.NegotiateStream.ReadCallback(AsyncProtocolRequest asyncRequest) at System.Net.FixedSizeReader.CheckCompletionBeforeNextRead(Int32 bytes) at System.Net.FixedSizeReader.ReadCallback(IAsyncResult transportResult) at System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously) at System.ServiceModel.Channels.ConnectionStream.ReadAsyncResult.OnAsyncReadComplete(Object state) at System.ServiceModel.Channels.SocketConnection.FinishRead() at System.ServiceModel.Channels.SocketConnection.AsyncReadCallback(Boolean haveResult, Int32 error, Int32 bytesRead) at System.ServiceModel.Diagnostics.Utility.IOCompletionThunk.UnhandledExceptionFrame(UInt32 error, UInt32 bytesRead, NativeOverlapped* nativeOverlapped) at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP) ----------------------------------------------------------------------------------------------------- >>Event ID: 27000 /Source OpsMgr Client ----------------------------------------------------------------------------------------------------- The OpsMgr SDK Client threw an unknown exception while processing a type cache refresh notification. The cache was not updated successfully: Microsoft.EnterpriseManagement.Common.ServerDisconnectedException: The client has been disconnected from the server. Please call ManagementGroup.Reconnect() to reestablish the connection. ---> System.ServiceModel.CommunicationObjectFaultedException: The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it is in the Faulted state. Server stack trace: at System.ServiceModel.Channels.CommunicationObject.ThrowIfFaulted() at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) Exception rethrown at [0]: at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) at Microsoft.EnterpriseManagement.Common.ITypeSpaceDataAccess.GetTypeCache(String languageCode) at Microsoft.EnterpriseManagement.DataAbstractionLayer.TypeSpaceOperations.GetTypeCache(String languageCode) --- End of inner exception stack trace --- at Microsoft.EnterpriseManagement.DataAbstractionLayer.SdkDataAbstractionLayer.HandleIndigoExceptions(Exception ex) at Microsoft.EnterpriseManagement.DataAbstractionLayer.TypeSpaceOperations.GetTypeCache(String languageCode) at Microsoft.EnterpriseManagement.Configuration.TypeSpaceCache.Refresh(Boolean forceDatabaseSync, ManualResetEvent acquireCacheUpdateLockEvent) at Microsoft.EnterpriseManagement.ManagementGroupInternal.RefreshTypeCacheInternal(Boolean forceDatabaseSync, ManualResetEvent acquireCacheUpdateLockEvent) at Microsoft.EnterpriseManagement.ManagementGroup.RefreshTypeCacheNotForce(Object eventToSignalOnLockAcquisition) ----------------------------------------------------------------------------------------------------- The following troubleshooting steps have been performed: ----------------------------------------------------------------------------------------------------- 1. Restart the SDK Service 2. Restart the SCOM server 3. Verified the service accounts and account / SQL permissions on the DB's including SDK Account, Config Account, RDR & WRTR accounts 4. Verified local permissions for service accounts 5. Had user delete local cache and restart console ----------------------------------------------------------------------------------------------------- What I do not understand is that if I put the same user in a group that is in the Operations Manager Admin role and the condition abates. It is only users that try and connect via a non-admin SCOM roles. Everything that I have sifted through in the body of the events seems to indicate a security issue. I tried searching up and down for "IsUserAdmin" seen throughout the events but to no avail. Many thanks in advance to those who reply! --------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------- Richard Schwartz, SCOM Architect - IBM Client Services MCITP-EA, MCTS - SCCM / SCOM CCNP \EOM
June 30th, 2011 1:29am

Hi, take a look at these links they can help you to resolve your problem: http://support.microsoft.com/kb/2526827 http://social.technet.microsoft.com/Forums/en-US/systemcenterdeployment/thread/d075eac3-0858-4cfe-b467-9922df3dae7a/Oussama Oueslati | System Engineer | vNext Consulting
Free Windows Admin Tool Kit Click here and download it now
July 2nd, 2011 8:06pm

Hi, Based on my research, I would like to suggest the following: 1. Try the method in the following Knowledge Base article: OpsMgr 2007: Operations Console does not open and gives SDK service error http://support.microsoft.com/kb/2002620 2. Check the firewall settings and ensure the required ports can be accessed: Using a Firewall with Operations Manager 2007 http://technet.microsoft.com/en-us/library/cc540431.aspx In addition, please also refer to the following thread: connection cannot be established to scom!! http://social.technet.microsoft.com/Forums/en-US/operationsmanagerextensibility/thread/c6f05902-0d73-411c-a64e-6a01c10adab6/ Hope this helps. Thanks.Nicholas Li - MSFT Please remember to click Mark as Answer on the post that helps you, and to click Unmark as Answer if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.
July 3rd, 2011 4:06am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics