Windows Search Service Terminated Unexpectedly; Faulting application: SearchIndexer.exe
A number of users with laptops have reported that searches in Outlook either don't work or are incomplete. All of these users are using Outlook in cached mode, and almost all of them were imaged recently. I got my hands on one of these machines and discovered that the Windows Search service seems to tank randomly. The 'SearchIndexer.exe' process seems to crash also. It appears to be an issue with the machines as a whole versus the profile since I encountered the same problem when I logged on as myself. (This was the first time I logged onto this machine.) On Friday I logged on as myself for the first time, build the OST & left it on over the weekend to index. I checked the machine this morning and found the System event log full of errors : The Windows Search service terminated unexpectedly. It has done this 5785 time(s). (Event ID: 7034; Source: Service Control Manager; OpCode: Info) The Windows Search service entered the running state. Wash, Rinse & Repeat (more details below) I went through the following to no avail: Logged in as a new user - Problem persists. Ran through the Search & Indexer Troubleshooter. Other than it being 'aware' that the service failed recently and that the service not running it found nothing wrong. When I started the service & ran the troubleshooter again, it mentioned that the service had failed recently but that's all. Explorered numerous potential options here to no avail. Tried Microsoft's 'fix it' as well Set 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Search\SetupCompletedSuccessfully' from 1 to 0 Rebuilt the index Every time it gets up to a point (like 27kish) then breaks and starts the stop/start service process again. Its hard to say what its choking on since I'm not aware of a way to 'parse' the mail file or enable super-verbose logging to see what its having trouble with. The only indexed locations are 'Offline Files' for my user, and 'Microsoft Office Outlook' for my user as well. We're not indexing any drives, libraries or network locations, even via the symlink 'hack'. This is a new profile so there's not much else here in terms of files to index. At the same time of setting up the 'problematic' machine on Friday, I imaged & logged into 3 laptops as myself to build and index the OST. All three machines finished indexing over 57k items without a problem, and there are no issues in the Application or System event logs. This is Windows 7 with just about every update available prior to the release of SP1 and Office Enterprise 2007 with SP2. (Outlook 12.0.6539.5000 SP2 MSO 12.0.6535.5002) Short messing around with the 'Indexing Service' (the Windows Feature), uninstalling/reinstalling WDS reimaging this machine, I don't know what else to try. ------------------------------ APPLICATION LOG Outlook Search has encountered an error and is temporarily disabling indexing for store C:\Users\jgp2750\AppData\Local\Microsoft\Outlook\outlook.ost (error=0x800706ba). - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> - <System> <Provider Name="Outlook" /> <EventID Qualifiers="32768">36</EventID> <Level>3</Level> <Task>0</Task> <Keywords>0x80000000000000</Keywords> <TimeCreated SystemTime="2011-04-18T14:32:45.000000000Z" /> <EventRecordID>218711</EventRecordID> <Channel>Application</Channel> <Computer>FQDN</Computer> <Security /> </System> - <EventData> <Data>C:\Users\jgp2750\AppData\Local\Microsoft\Outlook\outlook.ost</Data> <Data>0x800706ba</Data> </EventData> </Event> Faulting application name: SearchIndexer.exe, version: 7.0.7600.16385, time stamp: 0x4a5bcdd0 Faulting module name: NLSData0001.dll, version: 6.1.7600.16385, time stamp: 0x4a5bda85 Exception code: 0xc0000005 Fault offset: 0x00260a0d Faulting process id: 0xb38 Faulting application start time: 0x01cbfdd577062774 Faulting application path: C:\Windows\system32\SearchIndexer.exe Faulting module path: C:\Windows\System32\NLSData0001.dll Report Id: b967dd6d-69c8-11e0-99bd-0024d60226ee - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> - <System> <Provider Name="Application Error" /> <EventID Qualifiers="0">1000</EventID> <Level>2</Level> <Task>100</Task> <Keywords>0x80000000000000</Keywords> <TimeCreated SystemTime="2011-04-18T14:32:45.000000000Z" /> <EventRecordID>218710</EventRecordID> <Channel>Application</Channel> <Computer>FQDN</Computer> <Security /> </System> - <EventData> <Data>SearchIndexer.exe</Data> <Data>7.0.7600.16385</Data> <Data>4a5bcdd0</Data> <Data>NLSData0001.dll</Data> <Data>6.1.7600.16385</Data> <Data>4a5bda85</Data> <Data>c0000005</Data> <Data>00260a0d</Data> <Data>b38</Data> <Data>01cbfdd577062774</Data> <Data>C:\Windows\system32\SearchIndexer.exe</Data> <Data>C:\Windows\System32\NLSData0001.dll</Data> <Data>b967dd6d-69c8-11e0-99bd-0024d60226ee</Data> </EventData> </Event> Faulting application name: SearchIndexer.exe, version: 7.0.7600.16385, time stamp: 0x4a5bcdd0 Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000 Exception code: 0xc0000005 Fault offset: 0x0000004c Faulting process id: 0x10c8 Faulting application start time: 0x01cbfdd546f303ec Faulting application path: C:\Windows\system32\SearchIndexer.exe Faulting module path: unknown Report Id: 87c5842d-69c8-11e0-99bd-0024d60226ee - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> - <System> <Provider Name="Application Error" /> <EventID Qualifiers="0">1000</EventID> <Level>2</Level> <Task>100</Task> <Keywords>0x80000000000000</Keywords> <TimeCreated SystemTime="2011-04-18T14:31:22.000000000Z" /> <EventRecordID>218698</EventRecordID> <Channel>Application</Channel> <Computer>FQDN</Computer> <Security /> </System> - <EventData> <Data>SearchIndexer.exe</Data> <Data>7.0.7600.16385</Data> <Data>4a5bcdd0</Data> <Data>unknown</Data> <Data>0.0.0.0</Data> <Data>00000000</Data> <Data>c0000005</Data> <Data>0000004c</Data> <Data>10c8</Data> <Data>01cbfdd546f303ec</Data> <Data>C:\Windows\system32\SearchIndexer.exe</Data> <Data>unknown</Data> <Data>87c5842d-69c8-11e0-99bd-0024d60226ee</Data> </EventData> </Event> SYSTEM LOG The Windows Search service terminated unexpectedly. It has done this 5831 time(s). - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> - <System> <Provider Name="Service Control Manager" Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}" EventSourceName="Service Control Manager" /> <EventID Qualifiers="49152">7034</EventID> <Version>0</Version> <Level>2</Level> <Task>0</Task> <Opcode>0</Opcode> <Keywords>0x8080000000000000</Keywords> <TimeCreated SystemTime="2011-04-18T14:34:52.973976600Z" /> <EventRecordID>74749</EventRecordID> <Correlation /> <Execution ProcessID="520" ThreadID="4604" /> <Channel>System</Channel> <Computer>FQDN</Computer> <Security /> </System> - <EventData> <Data Name="param1">Windows Search</Data> <Data Name="param2">5831</Data> </EventData> </Event> The Windows Search service entered the running state. - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> - <System> <Provider Name="Service Control Manager" Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}" EventSourceName="Service Control Manager" /> <EventID Qualifiers="16384">7036</EventID> <Version>0</Version> <Level>4</Level> <Task>0</Task> <Opcode>0</Opcode> <Keywords>0x8080000000000000</Keywords> <TimeCreated SystemTime="2011-04-18T14:35:30.537732600Z" /> <EventRecordID>74750</EventRecordID> <Correlation /> <Execution ProcessID="520" ThreadID="1364" /> <Channel>System</Channel> <Computer>FQDN</Computer> <Security /> </System> - <EventData> <Data Name="param1">Windows Search</Data> <Data Name="param2">running</Data> <Binary>57005300650061007200630068002F0034000000</Binary> </EventData> </Event>
April 18th, 2011 10:44am

I am having essentially the same problem.
Free Windows Admin Tool Kit Click here and download it now
April 18th, 2011 9:06pm

Hi, Thanks for the post! It seems a known issue in Outlook 2007, please refer to the following two KB articles to check if it works: http://support.microsoft.com/default.aspx?scid=kb;EN-US;927676 http://support.microsoft.com/default.aspx?scid=kb;EN-US;930010 Since it's related to the Office issue, I recommend also ask this question in Office forum. Thanks for your understanding and cooperation! Regards, MiyaThis posting is provided "AS IS" with no warranties, and confers no rights. | Please remember to click "Mark as Answer" on the post that helps you, and to click "Unmark as Answer" if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.
April 20th, 2011 3:49am

Morning - thank you for the responses all. Miya Yao: I didn't see any '3036' Event ID's in the Application or System logs, mostly: Event ID 1000 SearchIndexer.exe Event ID 36 - Outlook Search has encountered an error and is temporarily disabling indexing for store C:\Users\%username%\AppData\Local\Microsoft\Outlook\Outlook.ost (error=0x80000ffff) Event ID 7034 The Windows Search service terminated unexpectedly. It has done this 5674 time(s). Event ID 7036 The Windows Search service entered the running state. This is a regular user's machine, not an IT machine so the 'Exchange System Manager tool' (or tools) shouldn't be present on the machine. Despite that however I ran 'fixmapi', noticed my first 3036 Event ID (The content source <mapi://{S-1-5-21-1234567890-987654321-135792468-01928}/> cannot be accessed.) in the Application Event log & rebooted. After a the reboot, I rebuilt the index, noticed another 3036 Event during the process, but it ultimately continues to fail around/after indexing 26k items. I could post in the Office suite, but it seems that there's may be an underlying issue with the Windows Search system. If you think its better to post there, I'll do so. Microsoft Provided a Windows Search Diagnostics tool (v6.1.6511.1) to help troubleshoot. I don't dare paste the log here since its uber long (but I've made it available on pastebin) so here are the highlights: --BEGIN Running Search Version Check:... Search Version Check: Passed Running Setup Check: Verify that Search is installed. Check Search Version. Check if the registry settings are ... Setup Check: Failed C:\ProgramData\Microsoft\Search\Data\ missing or has wrong permissions Running Event Log Check: Scan the event log for errors.... Event Log Check: Passed Running Outlook Check: Check if Outlook is running and verify its configuration.... Outlook Check: Passed Running Indexing Statistics: Run common queries and report the results... Indexing Statistics: Failed Running Indexing Progress: Periodically check the progress of indexed files.... Indexing Progress: Failed Running CPU and Memory Usage: Check the computer's CPU and memory usage.... CPU and Memory Usage: Passed Running Disk Space Check: Check disk space usage of all Search related processes and files.... Disk Space Check: Passed Running Filter Check: Check whether Search filters with known issues are installed... Running Battery Check: Check whether the system is running on battery/AC power... Battery Check: Failed *** Search Validation Filaed *** Please click the log file link below for details --EOF The utility also has options to start iDNA and ETW Traces: iDNA Trace fails (tttracer.exe exits with a status of 1) and it creats a zero byte file. ETW Trace appears to succeed (it launches) and runs until I terminate it (logman.exe exits with a status of 0). It creates a .ETL file, which I can't access; apparently I'm missing a suite of tools that include MSSLogToDatabase, MSSLogToText (among others? I'm sure) to properly extract (or read) the data contained within. I'll likely be sending the logs to them for further analysis but we'll see. Open to ideas from all. Thanks again.
Free Windows Admin Tool Kit Click here and download it now
April 20th, 2011 11:08am

Morning - thank you for the responses all. Miya Yao: I didn't see any '3036' Event ID's in the Application or System logs, mostly: Event ID 1000 SearchIndexer.exe Event ID 36 - Outlook Search has encountered an error and is temporarily disabling indexing for store C:\Users\%username%\AppData\Local\Microsoft\Outlook\Outlook.ost (error=0x80000ffff) Event ID 7034 The Windows Search service terminated unexpectedly. It has done this 5674 time(s). Event ID 7036 The Windows Search service entered the running state. This is a regular user's machine, not an IT machine so the 'Exchange System Manager tool' (or tools) shouldn't be present on the machine. Despite that however I ran 'fixmapi', noticed my first 3036 Event ID (The content source <mapi://{S-1-5-21-1234567890-987654321-135792468-01928}/> cannot be accessed.) in the Application Event log & rebooted. After a the reboot, I rebuilt the index, noticed another 3036 Event during the process, but it ultimately continues to fail around/after indexing 26k items. I could post in the Office suite, but it seems that there's may be an underlying issue with the Windows Search system. If you think its better to post there, I'll do so. Microsoft Provided a Windows Search Diagnostics tool (v6.1.6511.1) to help troubleshoot. I don't dare paste the log here since its uber long (but I've made it available on pastebin) so here are the highlights: --BEGIN Running Search Version Check:... Search Version Check: Passed Running Setup Check: Verify that Search is installed. Check Search Version. Check if the registry settings are ... Setup Check: Failed C:\ProgramData\Microsoft\Search\Data\ missing or has wrong permissions Running Event Log Check: Scan the event log for errors.... Event Log Check: Passed Running Outlook Check: Check if Outlook is running and verify its configuration.... Outlook Check: Passed Running Indexing Statistics: Run common queries and report the results... Indexing Statistics: Failed Running Indexing Progress: Periodically check the progress of indexed files.... Indexing Progress: Failed Running CPU and Memory Usage: Check the computer's CPU and memory usage.... CPU and Memory Usage: Passed Running Disk Space Check: Check disk space usage of all Search related processes and files.... Disk Space Check: Passed Running Filter Check: Check whether Search filters with known issues are installed... Running Battery Check: Check whether the system is running on battery/AC power... Battery Check: Failed *** Search Validation Filaed *** Please click the log file link below for details --EOF The utility also has options to start iDNA and ETW Traces: iDNA Trace fails (tttracer.exe exits with a status of 1) and it creats a zero byte file. ETW Trace appears to succeed (it launches) and runs until I terminate it (logman.exe exits with a status of 0). It creates a .ETL file, which I can't access; apparently I'm missing a suite of tools that include MSSLogToDatabase, MSSLogToText (among others? I'm sure) to properly extract (or read) the data contained within. I'll likely be sending the logs to them for further analysis but we'll see. Open to ideas from all. Thanks again.
April 20th, 2011 11:11am

I've got a positive update for this issue here. Credit goes to my colleage - he's a rockstar! Random nlsdataxxxx.dll files in the system32 directory are getting corrupted in our environment. The corruption causes searchindexer.exe behavior from crashing to cpu “spinning”. A system file scanner check (sfc /scannow) discovered these errors. Replacing the corrupted DLL’s with good DLL’s resolves the indexing problem. For example in corrupted file nldata002a.dll, comparing the good and bad, the bad DLL has bogus data from E000 to F000. It is also disturbing that sfc says the source files are also corrupt. Currently my solution is to run “sfc /scannow” and review the log at c:\windows\logs\cbs\cbs.log to determine the corrupt file. I then replace the corrupt file with a file from a good machine. Note: the corrupt file will have the same size, version, timestamp as the good file. It may be easier just to script the replacement of all the NLSDataxxxx.dll files. Still unknown as to why the files are getting corrupted on certain machines and not others. The CBS.log was too large for pastebin.com so here's a link to it for further analysis: http://www.juli.us/support/forums/technet_IndexSearch_CBS.log For now we're going to take ownership of the NlsData*.dll files, then replace them with good copies, stored on the network, which were taken from a known working machine. The areas highlighted in bold above are big concerns. Its one thing for a file to get corrupt, and another for the source file(s) to get corrupt also. But for a block of data within a file to get corrupt - that's very specific. (We used WinDiff to compare both DLL's and a little ways down you see data, but it's zero'd out! No screen shot at the moment) This is pretty scary. We've got a 'fix' but we don't know the cause, so we'll always have this gray cloud looming over us. I'm open to other ideas, but I think I'm going to close this thread by the end of the week if there isn't any more input.
Free Windows Admin Tool Kit Click here and download it now
May 3rd, 2011 5:13pm

I've got a positive update for this issue here. Credit goes to my colleage - he's a rockstar! Random nlsdataxxxx.dll files in the system32 directory are getting corrupted in our environment. The corruption causes searchindexer.exe behavior from crashing to cpu “spinning”. A system file scanner check (sfc /scannow) discovered these errors. Replacing the corrupted DLL’s with good DLL’s resolves the indexing problem. For example in corrupted file nldata002a.dll, comparing the good and bad, the bad DLL has bogus data from E000 to F000. It is also disturbing that sfc says the source files are also corrupt. Currently my solution is to run “sfc /scannow” and review the log at c:\windows\logs\cbs\cbs.log to determine the corrupt file. I then replace the corrupt file with a file from a good machine. Note: the corrupt file will have the same size, version, timestamp as the good file. It may be easier just to script the replacement of all the NLSDataxxxx.dll files. Still unknown as to why the files are getting corrupted on certain machines and not others. The CBS.log was too large for pastebin.com so here's a link to it for further analysis: http://www.juli.us/support/forums/technet_IndexSearch_CBS.log For now we're going to take ownership of the NlsData*.dll files, then replace them with good copies, stored on the network, which were taken from a known working machine. The areas highlighted in bold above are big concerns. Its one thing for a file to get corrupt, and another for the source file(s) to get corrupt also. But for a block of data within a file to get corrupt - that's very specific. (We used WinDiff to compare both DLL's and a little ways down you see data, but it's zero'd out! No screen shot at the moment) This is pretty scary. We've got a 'fix' but we don't know the cause, so we'll always have this gray cloud looming over us. I'm open to other ideas, but I think I'm going to close this thread by the end of the week if there isn't any more input. Here's a good resource for reading .CBS logs: http://support.microsoft.com/kb/928228
May 3rd, 2011 7:03pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics