2008 server reboots after failure of LSASS.exe (ntdll.dll and RPC service)
I have the following scenario, all servers are 2008 servers.
- 1 DC, native mode.
- 2 virtual and 2 fysical 2008 Terminal servers.
- 1 TS Gateway
- 1 Session broker
- A shim made with Application Toolkit 5.5. We have an application that has a HKLM key that we need to redirect per user to another location in the HKCU, we created a shim for that (redirect HKLM\..\..\subkey -> HKCU\software\classes\virtualstore\..\..\..\subkey)
and installed it with "Sdbinst ACT_Application.sdb".
After the install we experienced spontaneous reboots on the Terminal servers. This happens at no specific time and happens quite a lot. I could not find a pattern.
The Windows events are;
Before restarting, I get the following System Log in Event Viewer
===============================
Log Name: System
Source: USER32
Date: 18-5-2009 13:34:27
Event ID: 1074
Task Category: None
Level: Information
Keywords: Classic
User: SYSTEEM
Computer: TS02.domain.loc
Description:
The process wininit.exe has initiated the opnieuw opstarten of computer TS02 on behalf of user for the following reason: Er is geen titel voor deze reden gevonden
Reason Code: 0x50006
Shutdown Type: opnieuw opstarten
Comment: Het systeemproces 'C:\Windows\system32\lsass.exe' is onverwacht afgesloten met statuscode 255. Het systeem wordt afgesloten en opnieuw opgestart.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="USER32" />
<EventID Qualifiers="32768">1074</EventID>
<Level>4</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2009-05-18T11:34:27.000Z" />
<EventRecordID>257763</EventRecordID>
<Channel>System</Channel>
<Computer>TS02.domain.loc</Computer>
<Security UserID="S-1-5-18" />
</System>
<EventData>
<Data>wininit.exe</Data>
<Data>TS02</Data>
<Data>Er is geen titel voor deze reden gevonden</Data>
<Data>0x50006</Data>
<Data>opnieuw opstarten</Data>
<Data>Het systeemproces 'C:\Windows\system32\lsass.exe' is onverwacht afgesloten met statuscode 255. Het systeem wordt afgesloten en opnieuw opgestart.</Data>
<Data>
</Data>
<Binary>06000500</Binary>
</EventData>
</Event>
===============================
There are also these 2 other errors in the Application log in Event Viewer
==========================
Log Name: Application
Source: Application Error
Date: 18-5-2009 9:13:03
Event ID: 1000
Task Category: (100)
Level: Error
Keywords: Classic
User: N/A
Computer: TS02.domain.loc
Description:
Faulting application lsass.exe, version 6.0.6001.18000, time stamp 0x47918d7c, faulting module kerberos.dll, version 6.0.6001.18000, time stamp 0x4791a76c, exception code 0xc0000005,
fault offset 0x00003d12, process id 0x278, application start time 0x01c9d5b2761c0cbf.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Application Error" />
<EventID Qualifiers="0">1000</EventID>
<Level>2</Level>
<Task>100</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2009-05-18T07:13:03.000Z" />
<EventRecordID>8526</EventRecordID>
<Channel>Application</Channel>
<Computer>TS02.domain.loc</Computer>
<Security />
</System>
<EventData>
<Data>lsass.exe</Data>
<Data>6.0.6001.18000</Data>
<Data>47918d7c</Data>
<Data>kerberos.dll</Data>
<Data>6.0.6001.18000</Data>
<Data>4791a76c</Data>
<Data>c0000005</Data>
<Data>00003d12</Data>
<Data>278</Data>
<Data>01c9d5b2761c0cbf</Data>
</EventData>
</Event>
========================
========================
Log Name: Application
Source: Microsoft-Windows-Wininit
Date: 18-5-2009 9:13:07
Event ID: 1015
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: TS02.domain.loc
Description:
A critical system process, C:\Windows\system32\lsass.exe, failed with status code 255. The machine must now be restarted.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Wininit" Guid="{206f6dea-d3c5-4d10-bc72-989f03c8b84b}" EventSourceName="Wininit" />
<EventID Qualifiers="49152">1015</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2009-05-18T07:13:07.000Z" />
<EventRecordID>8527</EventRecordID>
<Correlation />
<Execution ProcessID="0" ThreadID="0" />
<Channel>Application</Channel>
<Computer>TS02.domain.loc</Computer>
<Security />
</System>
<EventData>
<Data>C:\Windows\system32\lsass.exe</Data>
<Data>255</Data>
</EventData>
</Event>
===============================
We dit not find any memory dumps and we expect that the crash of ntdll.dll causes a service crash of the RPC service, which has a tab "Recovery options" that says to reboot on crashingof the service, so the system account (user32 event) starts a reboot cycle. We did find some files "Problem Reports and solutions",
-------------
Product
Local Security Authority Process
Problem
Stopped working
Date
13-5-2009 16:21
Status
Not Reported
Problem signature
Problem Event Name: APPCRASH
Application Name: lsass.exe
Application Version: 6.0.6001.18000
Application Timestamp: 47918d7c
Fault Module Name: StackHash_0e89
Fault Module Version: 6.0.6001.18000
Fault Module Timestamp: 4791a7a6
Exception Code: c0000374
Exception Offset: 000b015d
OS Version: 6.0.6001.2.1.0.16.36
Locale ID: 1043
Additional Information 1: 0e89
Additional Information 2: d96ebd0182612edc086757726eacf7e2
Additional Information 3: 46b4
Additional Information 4: 4ac0abacf80463ad5d81740e44bd5143
Files that help describe the problem
Version.txt
AppCompat.txt
memory.hdmp
minidump.mdmp
We tried;
- patched all the servers
- ran a fullscan with forefront client security on multiple servers.
- disable the Forefront client security.
- uninstalled almost all the software, except for the shim
- removed the printer drivers (there were some printer errors during the reboot).
Regards,
Dennis
May 20th, 2009 11:18am
Hi,
According to the error message, we find it seems to be system crash issue and we need to use ADPlus to create dump file and then analyze it to narrow down the root cause of the issue. Unfortunately, it is not effective for us to debug the crash dump file here in the forum. Therefore, I would like to suggest that you contact Microsoft Customer Service and Support (CSS) via telephone so that a dedicated Support Professional can assist with your request.
To obtain the phone numbers for specific technology request please take a look at the web site listed below:
http://support.microsoft.com/default.aspx?scid=fh;EN-US;OfferProPhone#faq607
Hope the issue will be resolved soon.
In addition, you are warmly welcome to share the resolution when the problem is resolved. Thanks in advance!
Best regards,
Vincent Hu
Free Windows Admin Tool Kit Click here and download it now
May 21st, 2009 12:49pm
Hallo Dennis,Ik ondervind precies dit probleem op mijn terminal server farm.Wellicht hebben jullie een oplossing voor dit probleem en kan jij mij helpen.Alvast bedankt voor je reactie.Groeten Sander Botman
October 7th, 2009 1:25pm