SharePoint Server Search toasted ... stuck on "crawling full"

Overview

It's the same old story. You wake up one morning and errors are flooding into the web server. This time it is your friend Mr. 10036. And he brought his SQL Server buddy 17310.

Event Type: ErrorEvent
Source: Office Server SearchEvent
Category: Gatherer
Event ID: 10036
Date: 9/15/2008
Time: 1:02:55 AM
User: N/A
Description: A database error occurred.
Source: Microsoft OLE DB Provider for SQL
ServerCode: 0 occurred 257 time(s)
Description: Unspecified error
Context: Application 'SharedServices'


Event Type: ErrorEvent
Source: MSSQLSERVER
Event Category: (2)
Event ID: 17310
Date: 9/15/2008
Time: 5:29:08 AM
User: N/A
Description:A user request from the session with SPID 58 generated a fatal exception. SQL Server is terminating this session. Contact Product Support Services with the dump produced in the log directory.
Data:0000: 9e 43 00 00 14 00 00 00 žC......0008: 08 00 00 00 4d 00 4f 00 ....M.O.0010: 53 00 53 00 44 00 45 00 S.S.D.E.0018: 56 00 00 00 00 00 00 00 V.......


The short version is that the Office SharePoint Search Service is malfunctioning. The content source always believies it is "crawling full" even after web server and database server reboots.


Side note, the User Profile store also utilizes the Search Service to do its updating/querying. It believes that it is constantly "importing" profiles ... which it is not.

Solution

Here are some possible solutions that may work:

• Install SQL Server 2005 SP2 if it is not already installed
• Reset all crawled content (if possible)

Ultimately, we had to recreate the SSP to fix the problem ... this is not as scary as it sounds:

1. Go to Shared Serivices Administration
2. Select New SSP
3. On the New SSP page, you can select you current MySite web application IF you don't care about losing your user profile images.
4. The rest of the information for the New SSP page should be self explainatory ... if it isn't seek help.
5. After the new SSP is created you will need to reenter ALL values for ... everything. Don't worry your old SSP is still there and you can just pull it up in another browser tab and compare back and forth until you have it all filled in.
6. Next change the associations of the new SSP to associate with the port 80 and other sites/apps you have
7. Finally, change the new SSP to your default SSP
8. Run the search and profile import to confirm your issues are gone.