Problems with creating a Cluster (2012R2)

Hey guys,

I'm trying to find a solution for my problem for 3 days now...google can't help me ;)

What we have here:
- 2x HP ML350 G9 
- 1x HP MSA 2040 via SAS
- 2012R2 + HyperV + Failoverclustering installed
- the domain is an old 2011 SBS (which I didn't install)
- I configured a Management + CSV + Livemigration + HyperV network (i can ping each host over each individual adress)
- remote registry works + firewall completely disabled + smb connection works
- I moved the 2 nodes (SRV-VM01 + SRV-VM02) into the "Computer" OU

All the cluster checks are done without a single warning or error.
But the creation of the cluster stops at "creating computer object: CLUSTER01".
It says something like: it takes longer than expected and then it reverts everything.
The CLUSTER01 Computer account is temporarely created in the same OU as the nodes are.

Here is a snip from the last log I created:

000014e0.000022a4::2015/07/15-08:57:35.318 INFO  [CS] Reporting to SCM that cluster service has started.
000014e0.00002214::2015/07/15-08:57:35.443 INFO  [NM] Received request from client address fe80::4d27:97d8:90fa:13e8.
000014e0.00002214::2015/07/15-08:57:35.443 DBG   [API] Authenticated client--Client: TBM\eag Interface: b97db8b2-4c63-11cf-bff6-08002be23f2f Server: srv-vm01$@TBM.BIZ.LOCAL Level: RPC_C_AUTHN_LEVEL_PKT_PRIVACY Service: RPC_C_AUTHN_WINNT Protocol Sequence: ncacn_ip_tcp Client Address: fe80::4d27:97d8:90fa:13e8 Network Option: .
000014e0.00002214::2015/07/15-08:57:35.443 WARN  [VER] Could not read version data from database for node SRV-VM02 (id 2).
000014e0.00002214::2015/07/15-08:57:35.443 DBG   [VER] Calculated cluster versions: highest [Major 8 Minor 9600 Upgrade 3 ClusterVersion 0x00082580], lowest [Major 8 Minor 9600 Upgrade 3 ClusterVersion 0x00082580] with exclude node list: ()
000014e0.00002214::2015/07/15-08:57:36.304 DBG   [JPM] Node 1: contacts size for node SRV-VM02 is 1, current index 0
000014e0.00002214::2015/07/15-08:57:36.304 DBG   [JPM] Node 1: Trying to connect to node SRV-VM02 (IP: 10.1.6.3:~0~)
000014e0.00002214::2015/07/15-08:57:36.304 DBG   [HM] Trying to connect to SRV-VM02 at 10.1.6.3:~3343~
000014e0.00002214::2015/07/15-08:57:36.304 INFO  [CONNECT] 10.1.6.3:~3343~: Established connection to remote endpoint 10.1.6.3:~3343~.
000014e0.00002214::2015/07/15-08:57:36.304 INFO  [NODE] Node 1: New join with n2: stage: 'Attempt Initial Connection'
000014e0.00002214::2015/07/15-08:57:36.304 INFO  [SV] New real route: local (10.1.6.1:~52167~) to remote SRV-VM02 (10.1.6.3:~3343~).
000014e0.00002214::2015/07/15-08:57:36.304 INFO  [SV] Got a new outgoing stream to SRV-VM02 at 10.1.6.3:~3343~
000014e0.00002214::2015/07/15-08:57:36.304 DBG   [SM] Joiner: Initialized with SPN = SRV-VM02, RequiredCtxAttrib = 0, HandShakeTimeout = 40000
000014e0.000021bc::2015/07/15-08:57:36.304 DBG   [SM] Handling auth handshake posted by thread id 8724
000014e0.000021bc::2015/07/15-08:57:36.304 DBG   [SM] Joiner: ISC returned status = 590610 output Blob size 1532, service principal name HOST/SRV-VM02, auth type MSG_AUTH_PACKAGE::KerberosAuth
000014e0.000021bc::2015/07/15-08:57:36.304 DBG   [SM] Joiner: Sending SSPI blob of size 1532 to Sponsor
000014e0.000021bc::2015/07/15-08:57:36.304 DBG   [SM] Joiner: Received SSPI blob from the Sponsor of size 48000, auth type: MSG_AUTH_PACKAGE::CurrentAuth
000014e0.000021bc::2015/07/15-08:57:36.304 DBG   [SM] Joiner: ISC returned status = -2146893048 output Blob size 48000, service principal name HOST/SRV-VM02, auth type MSG_AUTH_PACKAGE::KerberosAuth
000014e0.000021bc::2015/07/15-08:57:36.304 DBG   [SM] Joiner: Sending SSPI blob of size 48000 to Sponsor

ERROR BEGINS HERE I GUESS:

000014e0.00002214::2015/07/15-08:57:36.304 INFO  [NODE] Node 1: New join with n2: stage: 'Authenticate Initial Connection' status HrError(0x80090308) reason: '[SV] Authentication failed'
000014e0.00002214::2015/07/15-08:57:36.304 DBG   [CHANNEL 10.1.6.3:~3343~] Close().
000014e0.00002214::2015/07/15-08:57:36.304 WARN  cxl::ConnectWorker::operator (): HrError(0x80090308)' because of '[SV] Authentication or Authorization Failed'
000014e0.00002214::2015/07/15-08:57:36.304 DBG   [HM] Connection attempt to SRV-VM02 failed with error HrError(0x80090308): Failed to post-process connection to remote endpoint 10.1.6.3:~3343~.
000014e0.00001d64::2015/07/15-08:57:36.320 INFO  [ACCEPT] 0.0.0.0:~3343~: Accepted inbound connection from remote endpoint 10.1.6.3:~51532~.
000014e0.00002214::2015/07/15-08:57:36.320 INFO  [SV] New real route: local (10.1.6.1:~3343~) to remote  (10.1.6.3:~51532~).
000014e0.00002214::2015/07/15-08:57:36.320 INFO  [SV] Got a new incoming stream from 10.1.6.3:~51532~
000014e0.00002214::2015/07/15-08:57:36.320 DBG   [SM] SrvCtxt initialized with MaxTokenSize = 48000, RequiredCtxAttrib = 0, HandShakeTimeout = 40000, ServerName =  SRV-VM01
000014e0.00002144::2015/07/15-08:57:36.320 DBG   [SM] Handling auth handshake posted by thread id 8724
000014e0.00002144::2015/07/15-08:57:36.320 DBG   [SM] Sponsor: Received SSPI blob from the Joiner of size 1532, auth type: MSG_AUTH_PACKAGE::CurrentAuth
000014e0.00002144::2015/07/15-08:57:36.320 DBG   [SM] Sponsor: SSPI ASC returned status = -2146893044
000014e0.00002144::2015/07/15-08:57:36.320 DBG   [SM] Sponsor: Sending SSPI blob of size 48000 to Joiner
000014e0.00002144::2015/07/15-08:57:36.320 DBG   [SM] Sponsor: Authentication handshake final Status -2146893044
000014e0.00002214::2015/07/15-08:57:36.320 DBG   [CHANNEL 10.1.6.3:~51532~] Close().
000014e0.00002214::2015/07/15-08:57:36.320 WARN  mscs::ListenerWorker::operator (): HrError(0x8009030c)' because of '[SV] Authentication or Authorization Failed'
000014e0.00002214::2015/07/15-08:58:16.138 INFO  [NM] Received request from client address SRV-VM01.
000014e0.00002214::2015/07/15-08:58:16.138 DBG   [API] Authenticated client--Client: NT-AUTORITT\SYSTEM Interface: b97db8b2-4c63-11cf-bff6-08002be23f2f Server: (null) Level: RPC_C_AUTHN_LEVEL_PKT_PRIVACY Service: RPC_C_AUTHN_WINNT Protocol Sequence: ncalrpc Client Address: SRV-VM01 Network Option: .
000014e0.00002214::2015/07/15-08:58:16.138 WARN  [VER] Could not read version data from database for node SRV-VM02 (id 2).
000014e0.00002214::2015/07/15-08:58:16.138 DBG   [VER] Calculated cluster versions: highest [Major 8 Minor 9600 Upgrade 3 ClusterVersion 0x00082580], lowest [Major 8 Minor 9600 Upgrade 3 ClusterVersion 0x00082580] with exclude node list: ()
000014e0.00002214::2015/07/15-08:58:36.311 DBG   [JPM] Node 1: contacts size for node SRV-VM02 is 1, current index 0
000014e0.00002214::2015/07/15-08:58:36.311 DBG   [JPM] Node 1: Trying to connect to node SRV-VM02 (IP: 10.1.6.3:~0~)
000014e0.00002214::2015/07/15-08:58:36.311 DBG   [HM] Trying to connect to SRV-VM02 at 10.1.6.3:~3343~
000014e0.00002214::2015/07/15-08:58:36.311 INFO  [CONNECT] 10.1.6.3:~3343~: Established connection to remote endpoint 10.1.6.3:~3343~.
000014e0.00002214::2015/07/15-08:58:36.311 INFO  [SV] New real route: local (10.1.6.1:~52179~) to remote SRV-VM02 (10.1.6.3:~3343~).
000014e0.00002214::2015/07/15-08:58:36.311 INFO  [SV] Got a new outgoing stream to SRV-VM02 at 10.1.6.3:~3343~
000014e0.00002214::2015/07/15-08:58:36.311 DBG   [SM] Joiner: Initialized with SPN = SRV-VM02, RequiredCtxAttrib = 0, HandShakeTimeout = 40000
000014e0.000021bc::2015/07/15-08:58:36.311 DBG   [SM] Handling auth handshake posted by thread id 8724
000014e0.000021bc::2015/07/15-08:58:36.311 DBG   [SM] Joiner: ISC returned status = 590610 output Blob size 1532, service principal name HOST/SRV-VM02, auth type MSG_AUTH_PACKAGE::KerberosAuth
000014e0.000021bc::2015/07/15-08:58:36.311 DBG   [SM] Joiner: Sending SSPI blob of size 1532 to Sponsor
000014e0.000021bc::2015/07/15-08:58:36.311 DBG   [SM] Joiner: Received SSPI blob from the Sponsor of size 48000, auth type: MSG_AUTH_PACKAGE::CurrentAuth
000014e0.000021bc::2015/07/15-08:58:36.311 DBG   [SM] Joiner: ISC returned status = -2146893048 output Blob size 48000, service principal name HOST/SRV-VM02, auth type MSG_AUTH_PACKAGE::KerberosAuth
000014e0.000021bc::2015/07/15-08:58:36.311 DBG   [SM] Joiner: Sending SSPI blob of size 48000 to Sponsor
000014e0.00002214::2015/07/15-08:58:36.311 INFO  [NODE] Node 1: New join with n2: stage: 'Authenticate Initial Connection' status HrError(0x80090308) reason: '[SV] Authentication failed'
000014e0.00002214::2015/07/15-08:58:36.311 DBG   [CHANNEL 10.1.6.3:~3343~] Close().
000014e0.00002214::2015/07/15-08:58:36.311 WARN  cxl::ConnectWorker::operator (): HrError(0x80090308)' because of '[SV] Authentication or Authorization Failed'
000014e0.00002214::2015/07/15-08:58:36.311 DBG   [HM] Connection attempt to SRV-VM02 failed with error HrError(0x80090308): Failed to post-process connection to remote endpoint 10.1.6.3:~3343~.
000014e0.00001d64::2015/07/15-08:58:36.327 INFO  [ACCEPT] 0.0.0.0:~3343~: Accepted inbound connection from remote endpoint 10.1.6.3:~51533~.
000014e0.00002214::2015/07/15-08:58:36.327 INFO  [SV] New real route: local (10.1.6.1:~3343~) to remote  (10.1.6.3:~51533~).
000014e0.00002214::2015/07/15-08:58:36.327 INFO  [SV] Got a new incoming stream from 10.1.6.3:~51533~
000014e0.00002214::2015/07/15-08:58:36.327 DBG   [SM] SrvCtxt initialized with MaxTokenSize = 48000, RequiredCtxAttrib = 0, HandShakeTimeout = 40000, ServerName =  SRV-VM01
000014e0.00002144::2015/07/15-08:58:36.327 DBG   [SM] Handling auth handshake posted by thread id 8724
000014e0.00002144::2015/07/15-08:58:36.327 DBG   [SM] Sponsor: Received SSPI blob from the Joiner of size 1532, auth type: MSG_AUTH_PACKAGE::CurrentAuth
000014e0.00002144::2015/07/15-08:58:36.327 DBG   [SM] Sponsor: SSPI ASC returned status = -2146893044
000014e0.00002144::2015/07/15-08:58:36.327 DBG   [SM] Sponsor: Sending SSPI blob of size 48000 to Joiner
000014e0.00002144::2015/07/15-08:58:36.327 DBG   [SM] Sponsor: Authentication handshake final Status -2146893044
000014e0.00002214::2015/07/15-08:58:36.327 DBG   [CHANNEL 10.1.6.3:~51533~] Close().
000014e0.00002214::2015/07/15-08:58:36.327 WARN  mscs::ListenerWorker::operator (): HrError(0x8009030c)' because of '[SV] Authentication or Authorization Failed'
000014e0.00002214::2015/07/15-08:59:36.318 DBG   [JPM] Node 1: contacts size for node SRV-VM02 is 1, current index 0
000014e0.00002214::2015/07/15-08:59:36.318 DBG   [JPM] Node 1: Trying to connect to node SRV-VM02 (IP: 10.1.6.3:~0~)
000014e0.00002214::2015/07/15-08:59:36.318 DBG   [HM] Trying to connect to SRV-VM02 at 10.1.6.3:~3343~
000014e0.00002214::2015/07/15-08:59:36.318 INFO  [CONNECT] 10.1.6.3:~3343~: Established connection to remote endpoint 10.1.6.3:~3343~.
000014e0.00002214::2015/07/15-08:59:36.318 INFO  [SV] New real route: local (10.1.6.1:~52180~) to remote SRV-VM02 (10.1.6.3:~3343~).
000014e0.00002214::2015/07/15-08:59:36.318 INFO  [SV] Got a new outgoing stream to SRV-VM02 at 10.1.6.3:~3343~
000014e0.00002214::2015/07/15-08:59:36.318 DBG   [SM] Joiner: Initialized with SPN = SRV-VM02, RequiredCtxAttrib = 0, HandShakeTimeout = 40000
000014e0.000021bc::2015/07/15-08:59:36.318 DBG   [SM] Handling auth handshake posted by thread id 8724
000014e0.000021bc::2015/07/15-08:59:36.318 DBG   [SM] Joiner: ISC returned status = 590610 output Blob size 1532, service principal name HOST/SRV-VM02, auth type MSG_AUTH_PACKAGE::KerberosAuth
000014e0.000021bc::2015/07/15-08:59:36.318 DBG   [SM] Joiner: Sending SSPI blob of size 1532 to Sponsor
000014e0.000021bc::2015/07/15-08:59:36.318 DBG   [SM] Joiner: Received SSPI blob from the Sponsor of size 48000, auth type: MSG_AUTH_PACKAGE::CurrentAuth
000014e0.000021bc::2015/07/15-08:59:36.318 DBG   [SM] Joiner: ISC returned status = -2146893048 output Blob size 48000, service principal name HOST/SRV-VM02, auth type MSG_AUTH_PACKAGE::KerberosAuth
000014e0.000021bc::2015/07/15-08:59:36.318 DBG   [SM] Joiner: Sending SSPI blob of size 48000 to Sponsor
000014e0.00002214::2015/07/15-08:59:36.318 INFO  [NODE] Node 1: New join with n2: stage: 'Authenticate Initial Connection' status HrError(0x80090308) reason: '[SV] Authentication failed'
000014e0.00002214::2015/07/15-08:59:36.318 DBG   [CHANNEL 10.1.6.3:~3343~] Close().
000014e0.00002214::2015/07/15-08:59:36.318 WARN  cxl::ConnectWorker::operator (): HrError(0x80090308)' because of '[SV] Authentication or Authorization Failed'
000014e0.00002214::2015/07/15-08:59:36.318 DBG   [HM] Connection attempt to SRV-VM02 failed with error HrError(0x80090308): Failed to post-process connection to remote endpoint 10.1.6.3:~3343~.
000014e0.00001d64::2015/07/15-08:59:37.334 INFO  [ACCEPT] 0.0.0.0:~3343~: Accepted inbound connection from remote endpoint 10.1.6.3:~51541~.
000014e0.00002128::2015/07/15-08:59:37.334 INFO  [SV] New real route: local (10.1.6.1:~3343~) to remote  (10.1.6.3:~51541~).
000014e0.00002128::2015/07/15-08:59:37.334 INFO  [SV] Got a new incoming stream from 10.1.6.3:~51541~
000014e0.00002128::2015/07/15-08:59:37.334 DBG   [SM] SrvCtxt initialized with MaxTokenSize = 48000, RequiredCtxAttrib = 0, HandShakeTimeout = 40000, ServerName =  SRV-VM01
000014e0.00002144::2015/07/15-08:59:37.334 DBG   [SM] Handling auth handshake posted by thread id 8488
000014e0.00002144::2015/07/15-08:59:37.334 DBG   [SM] Sponsor: Received SSPI blob from the Joiner of size 1532, auth type: MSG_AUTH_PACKAGE::CurrentAuth
000014e0.00002144::2015/07/15-08:59:37.334 DBG   [SM] Sponsor: SSPI ASC returned status = -2146893044
000014e0.00002144::2015/07/15-08:59:37.334 DBG   [SM] Sponsor: Sending SSPI blob of size 48000 to Joiner
000014e0.00002144::2015/07/15-08:59:37.334 DBG   [SM] Sponsor: Authentication handshake final Status -2146893044
000014e0.00002128::2015/07/15-08:59:37.334 DBG   [CHANNEL 10.1.6.3:~51541~] Close().
000014e0.00002128::2015/07/15-08:59:37.334 WARN  mscs::ListenerWorker::operator (): HrError(0x8009030c)' because of '[SV] Authentication or Authorization Failed'
000014e0.0000213c::2015/07/15-09:00:16.144 WARN  [VER] Could not read version data from database for node SRV-VM02 (id 2).
000014e0.0000213c::2015/07/15-09:00:16.144 DBG   [VER] Calculated cluster versions: highest [Major 8 Minor 9600 Upgrade 3 ClusterVersion 0x00082580], lowest [Major 8 Minor 9600 Upgrade 3 ClusterVersion 0x00082580] with exclude node list: ()
00001ea4.000000bc::2015/07/15-09:00:35.458 DBG   Cluster node cleanup thread started.
00001ea4.000000bc::2015/07/15-09:00:35.458 DBG   Starting cluster node cleanup...
00001ea4.000000bc::2015/07/15-09:00:35.458 DBG   Disabling the cluster service...
00001ea4.000000bc::2015/07/15-09:00:35.458 DBG   Stopping the cluster service...
000014e0.00001904::2015/07/15-09:00:35.458 DBG   [NODE] Node 1: eating message sent to the dead node 2
000014e0.00001904::2015/07/15-09:00:35.458 INFO  [CS] Service Stopping...
000014e0.00001904::2015/07/15-09:00:35.458 INFO  [CORE] Node quorum state is 'Not yet formed or joined a cluster'. Form/join status with other nodes is as follows:
000014e0.00001904::2015/07/15-09:00:35.458 INFO  [NODE] Node 1: Farthest reported progress joining with node SRV-VM02 (id 2) is: Authenticate Initial Connection at time 2015/07/15-06:59:36.318: status 776 [SV] Authentication failed
000014e0.00001904::2015/07/15-09:00:35.458 INFO  [DCM] UnregisterSwProvider(): CSV providers are not registered
000014e0.00001904::2015/07/15-09:00:35.458 INFO  [RGP] node 1: MergeAndRestart +() -(1)
000014e0.00001904::2015/07/15-09:00:35.458 INFO  [RGP] sending to 64 nodes 1: 001(1) => 101() +() -(1) [()]
000014e0.00001cec::2015/07/15-09:00:35.458 INFO  [CORE] Node 1: Proposed View is <ViewChanged joiners=() downers=(1) newView=101() oldView=001(1) joiner=false form=false/>
000014e0.00001904::2015/07/15-09:00:35.458 DBG   [NODE] Node 1: eating message sent to the dead node 2
000014e0.00001904::2015/07/15-09:00:35.567 INFO  [DM]: Shutting down, so unloading the cluster database.
000014e0.00001904::2015/07/15-09:00:35.567 INFO  [DM] Shutting down, so unloading the cluster database (waitForLock: true).
000014e0.00001904::2015/07/15-09:00:35.567 DBG   [DM] Unloading Hive, Key \Registry\Machine\Cluster, discardCurrentChanges true
000014e0.00001904::2015/07/15-09:00:35.567 INFO  [CS] Service Stopped...
000014e0.00001904::2015/07/15-09:00:35.567 INFO  [CS] About to exit service...

I hope you can give me a hint, what to do ;)

Regards

Dominik


July 15th, 2015 8:24am

Hi Dominik,

It would be easier for us, if you list out any troubshooting on the error you have done.

Found this which talks about same CN user accounts as the Computer CNs

2 node cluster windows 2008 R2 cluster won't form:

https://social.technet.microsoft.com/Forums/windowsserver/en-US/5891d2aa-0dfd-41ed-9dbf-40793c391ee3/2-node-cluster-windows-2008-r2-cluster-wont-form?forum=winserverClustering

Free Windows Admin Tool Kit Click here and download it now
July 15th, 2015 11:17am

Hey, 

thanks for the fast reply.
- I tried your solution already, without success.
- I could verify, that there are no similar CN Computer and CN User accounts.
- I checked, that the RSA keys have the correct permissions
- I tried to give more permissions in the AD OU for the administrator user
- It even fails, when I try to just use one node (SRV-VM01 or SRV-VM02)

Thats why I don't understand the problem. It says something about auth failure and that it couldn't read the database from the second note, but if I just try to add one single node.....how? ;)

July 15th, 2015 11:30am

Hey guys, got the solution myself today.
It was a horror-google-trip for me...

In my case the failure was the AD itsself. The company which installed the SBS2011 Server did some major modifications in terms of GPO's and stuff. I change so many things, that I can't really say, what the root cause was....

BUT the last thing I did was the "Access this computer from the network" permission. There you need to set "everybody, authenticated users, the cluster nodes".

Further I set the permissions in the OU where the nodes are in and the Cluster computer account will be created.

AFTER THAT I had serious problems to get the LiveMigration fixed...wow...ridiculous.

This link fixed it: 

http://de.slideshare.net/AshwinPawar/hv-lmerror0x80090303

AFTER THAT I had problems with the cluster aware update. I couldn't set the permission in the OU.
Okay, this was easy then...

Yeah, nice experience...not ;)

Free Windows Admin Tool Kit Click here and download it now
July 16th, 2015 3:33pm

Hi DominikHeiss123,

Appreciate you coming back and updating the solution to us.

July 17th, 2015 2:45am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics