SharePoint 2013 Search, High CPU usage

I have a sharepoint 2013 site with 100+ subwebs and about 3.5GB documentations. The site performance is normal if I do not setup any enterprise search service.

But once I setup a enterprise search service, the MSSQL and w3wp progresses will spend near 100% CPU for a long time. After the CPU usage turn down, once I click any link on the web page then the CPU usages will grow up to 100% for 15-20 minutes again.

I made the following step to reduce the resource usages on the server.
1. Reduce SharePoint logging, only log down Error events and Unexpected events.
2. Optimize MS-SQL server configuration, give 2 BG minimun memory to the database server, and boost the SQL Server's priority.
3. In the SharePoint search center, limit the crawl impact to 1 document at a time.

But the problem is still not fixed.

Check with the logs, 99% items are similar to:

12/20/2012 02:33:00.60  w3wp.exe (0x15D4)                        0x146C                                0x6FB7003                      7t61 High     [Forced due to logging gap, cached @ 12/20/2012 02:33:00.53, Original Level: Verbose] {0} 
12/20/2012 02:33:00.60  w3wp.exe (0x15D4)                        0x146C SharePoint Server              Database                       8acb High     [Forced due to logging gap, Original Level: VerboseEx] Reverting to process identity 
12/20/2012 02:33:34.36  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Asp Runtime                    aj1kl High     [Forced due to logging gap, cached @ 12/20/2012 02:33:34.31, Original Level: Verbose] SPRequestModule.PreRequestExecuteAppHandler cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:34.36  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       8acb High     [Forced due to logging gap, Original Level: VerboseEx] Reverting to process identity cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:34.46  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       ahjqp High     [Forced due to logging gap, cached @ 12/20/2012 02:33:34.40, Original Level: Verbose] SQL connection time: 0.0505 cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:34.46  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       8acb High     [Forced due to logging gap, Original Level: VerboseEx] Reverting to process identity cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:34.73  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       ahjqp High     [Forced due to logging gap, cached @ 12/20/2012 02:33:34.46, Original Level: Verbose] SQL connection time: 0.1201 cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:34.73  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Web Controls                   90i3 High     [Forced due to logging gap, Original Level: Verbose] PersonalActions::CreateChildControls() - Was called. cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.44  w3wp.exe (0x13A4)                        0x214C Web Content Management         Publishing                     aftdl High     [Forced due to logging gap, cached @ 12/20/2012 02:33:34.73, Original Level: Verbose] TaxonomyNavigationContext: Checking for target URL query parameters cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.44  w3wp.exe (0x13A4)                        0x214C Web Content Management         Publishing                     8vy8 High     [Forced due to logging gap, Original Level: Medium] CrossListQueryCache::GetSiteData() - Query: {0} Url: {1}. {2} cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.53  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       ahjqp High     [Forced due to logging gap, cached @ 12/20/2012 02:33:35.47, Original Level: Verbose] SQL connection time: 0.0542 cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.53  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       8acb High     [Forced due to logging gap, Original Level: VerboseEx] Reverting to process identity cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.62  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       ahjqp High     [Forced due to logging gap, cached @ 12/20/2012 02:33:35.58, Original Level: Verbose] SQL connection time: 0.0712 cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.62  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       8acb High     [Forced due to logging gap, Original Level: VerboseEx] Reverting to process identity cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.70  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          General                        g3ql High     [Forced due to logging gap, cached @ 12/20/2012 02:33:35.67, Original Level: Verbose] GetUriScheme(/aboutus/Pages/home.aspx) cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.70  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       8acb High     [Forced due to logging gap, Original Level: VerboseEx] Reverting to process identity cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.76  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       ahjqp High     [Forced due to logging gap, cached @ 12/20/2012 02:33:35.75, Original Level: Verbose] SQL connection time: 0.024 cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.76  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       8acb High     [Forced due to logging gap, Original Level: VerboseEx] Reverting to process identity cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.84  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       ahjqp High     [Forced due to logging gap, cached @ 12/20/2012 02:33:35.81, Original Level: Verbose] SQL connection time: 0.0375 cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.84  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       8acb High     [Forced due to logging gap, Original Level: VerboseEx] Reverting to process identity cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.90  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       ahjqp High     [Forced due to logging gap, cached @ 12/20/2012 02:33:35.89, Original Level: Verbose] SQL connection time: 0.0545 cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.90  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          Database                       8acb High     [Forced due to logging gap, Original Level: VerboseEx] Reverting to process identity cea0ec9b-b1d5-600d-f3ef-49eb3271ba12
12/20/2012 02:33:35.97  w3wp.exe (0x13A4)                        0x214C SharePoint Foundation          General                        g3ql High     [Forced due to logging gap, cached @ 12/20/2012 02:33:35.95, Original Level: Verbose] GetUriScheme(/depts/people/Pages/Home.aspx) cea0ec9b-b1d5-600d-f3ef-49eb3271ba12

It is real bad that, when this problem appears, even if I reboot the OS, disable the SharePoint Search server, once I access my web site again, the CPU will group up to 100% quickly. (the problem become worse).

That means I have to restore my content database everytime once I use the enterprise search.

Any idea?

December 20th, 2012 1:56pm

I run the following sql to get the top CPU usage

SELECT TOP 10 qs.execution_count, total_CPU_inSeconds = qs.total_worker_time/1000000, total_elapsed_time_inSeconds = qs.total_elapsed_time/1000000, SUBSTRING(st.text, 0 , 60) as cmd from sys.dm_exec_query_stats as qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) as st cross apply sys.dm_exec_query_plan (qs.plan_handle) as qp ORDER BY qs.total_worker_time desc

The result shows:

Execution_count, Total_CPU_inSeconds, Total_elapsed_time_inSeconds, Cmd
12558 300 301 CREATE PROCEDURE dbo.proc_NavStructDirtyPageDependents( ....
12558 117 118 CREATE PROCEDURE dbo.proc_NavStructDirtyPageDependents( ....
2108 116 117 CREATE PROCEDURE dbo.proc_GetWebNavStruct( @SiteId uni....
12558 56 56 CREATE PROCEDURE dbo.proc_NavStructDirtyPageDependents( ....
945 40 40 CREATE PROCEDURE dbo.proc_NavStructDeleteNodeByIds( @S....
945 18 18 CREATE PROCEDURE dbo.proc_NavStructDeleteNodeByIds( @S....
2108 17 17 CREATE PROCEDURE dbo.proc_GetWebNavAcls( @SiteId uniqu....
588 7 7 CREATE PROCEDURE [dbo].[profilesynch_US_GetChangedLanguageA....
23 3 3 SELECT TOP 10 qs.execution_count, total_CPU_inSec....
7120 3 3 (@LFFP uniqueidentifier,@SITEID uniqueidentifier,@IU int,@L....

Press F5 to refresh the result in each second, the execution_count of "CREATE PROCEDURE dbo.proc_NavStructDirtyPageDependents" increases 30+ in every second.

What happen on SharePoint13? I just have 100+ sites and 300+ pages there. My web site compeletly crashed due to this stupid and unacceptable performance.


Free Windows Admin Tool Kit Click here and download it now
December 22nd, 2012 5:41pm

Hi,

I have exactly the same problem, do you find any solution ?

Kouilb

March 7th, 2013 5:27pm

We've had the same problem on several SP 2013 installations, with both new and upgraded content databases. The only solution we've found is to delete the publishing page that is reported in the logfiles and recreate it. In your case that means deleting "/depts/people/Pages/Home.aspx".

Deleting the page has solved the problem (for now) in our installations. For a upgraded solution we got the problem also on a seccond page a bit later, so we deleted and recreated that aswell.

We couldn't find anything wrong with the pages themselves. They worked as expected when we opened them, the only thing they had in common is that all were publishing pages in different Pages libraries.

ElinK

Free Windows Admin Tool Kit Click here and download it now
March 8th, 2013 4:58pm

Yes, after deleting page recorded in logs the server is OK.

But we don't know why this page break our farm.

Kouilb.

March 8th, 2013 5:40pm

I am seeing the exact same behaviour in a new environment. (WIN2008R2,SP2013, SQL2008R2).

However, I am unable to delete pages referenced in the ULS log as some of the pages are SharePoint System pages (_layouts15/settings.aspx, forms/AllItems.aspx). After a few hours the environment is very slow and eventually it will stop working all together.

Is this a SQL or SharePoint 2013 related issue?

Anyone found a reason for this behaviour or even a solution?

/Nicolaj

Free Windows Admin Tool Kit Click here and download it now
March 12th, 2013 7:56pm

Hello,

We had the same problem for several times now, we had to delete the page to fix the problem, and then restored the page. The pages caused the problem were created by Admin, and edited by web author, and we also used "Add Link" to add the page link to our global navigation.

Anyone know why it happened, is a bug in SharePoint?

June 26th, 2013 6:29pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics