Optimized Index Server Infrastructure
Hi, I have 4 front end server. One index server which performs query role as well.Two database server.25 live sites are running. Every site has it own content DB, Web Application & application pool.Many batch jobs is running in index server to push content sites. Overall data volume more than 100GB. problem is indexing is not working sometimes. Sometimes incremental crawling is hanged.Sometimes it is running too long. So what should be the optimized server infrastructure by which I can solve this problem. Please help.I am using MOSS 2007. My product catalog is dependant on managed proerty so if crawl fails then the whole thing are messed up. Please let me know if you have any query. thanksArup MCTS - SharePoint Play Sudoku Online
December 23rd, 2010 4:30am

Do you run Query role on WFE servers and Index server? If yes this configuration is not supported by SharePoint. You have to run Query role on WFE servers and Index role on Application server or Query and Index just on Application. Read this http://technet.microsoft.com/en-us/library/cc263044(office.12).aspxOleg
Free Windows Admin Tool Kit Click here and download it now
December 23rd, 2010 11:34am

Do you run Query role on WFE servers and Index server? If yes this configuration is not supported by SharePoint. You have to run Query role on WFE servers and Index role on Application server or Query and Index just on Application. Read this http://technet.microsoft.com/en-us/library/cc263044(office.12).aspxOleg
December 23rd, 2010 11:34am

If your Crawl is hanging or taking a long time then I would suspect the problem may be the structure of the content, memory or perhaps a bad iFilter. The incremental crawl I believe runs a off a change log in the content DB so if you have batch jobs which change a lot of content then the incremental crawl will be prolonged, perhaps you can optimize the way that you push content to be more discreet? A common mistake if you are using PDFs is to install the 32bit PDF iFilter which is actually Acrobat 6. For PDF on 32bit servers ensure that you are using the iFilter in the 9 reader or better still foxit (faster). Look at your crawl logs and see if there is any thing in there in terms of errors. If you have folders containing lots of items (i.e. big document libraries or lists) then the crawler will load the entire contents of the list into memory before commencing the crawl and can run out of memory. There are registry settings to increase the amount of memory available to the crawler. 64bit will make life a lot easier here. If it helps I crawled 1Tb of data on a 32bit sharepoint farm with 3 web front servers (query role), 1 index server and crawl on all WFEs enabled. It took a long time but it worked. What also did help was setting up a content source per site collection in case a crawl did crash a full crawl was only required on a single site collection and not the whole thing.
Free Windows Admin Tool Kit Click here and download it now
December 23rd, 2010 1:37pm

If your Crawl is hanging or taking a long time then I would suspect the problem may be the structure of the content, memory or perhaps a bad iFilter. The incremental crawl I believe runs a off a change log in the content DB so if you have batch jobs which change a lot of content then the incremental crawl will be prolonged, perhaps you can optimize the way that you push content to be more discreet? A common mistake if you are using PDFs is to install the 32bit PDF iFilter which is actually Acrobat 6. For PDF on 32bit servers ensure that you are using the iFilter in the 9 reader or better still foxit (faster). Look at your crawl logs and see if there is any thing in there in terms of errors. If you have folders containing lots of items (i.e. big document libraries or lists) then the crawler will load the entire contents of the list into memory before commencing the crawl and can run out of memory. There are registry settings to increase the amount of memory available to the crawler. 64bit will make life a lot easier here. If it helps I crawled 1Tb of data on a 32bit sharepoint farm with 3 web front servers (query role), 1 index server and crawl on all WFEs enabled. It took a long time but it worked. What also did help was setting up a content source per site collection in case a crawl did crash a full crawl was only required on a single site collection and not the whole thing.
December 23rd, 2010 1:37pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics