SharePoint 2010 vs. FAST Search for SharePoint (FS4SP) Index

When implementing Enterprise Search using SharePoint 2010 I am often asked whether a company should invest the money to use FAST Search for SharePoint (FS4SP) as the index.  When making the decision regarding which index to use the answer typically comes down to 1 of 2 questions.

Question 1: How many items do you want to index?

Decision Point 1: The SharePoint index can handle 100M items while the FS4SP index can handle a virtually unlimited number of items. (650M in the lab last I heard but don’t quote me)

Answer 1: If you have more than 100M items to index the answer is FS4SP. No other questions need to be asked.

Question 2: How important is it that end users find their search items?

Decision Point 2: SharePoint 2010 has implemented Facetted Navigation (or Refiners) for both indexes but there is a big difference between the two solutions.   The SharePoint Index has the concept of a Shallow Refiner while the FS4SP index has the concept of Deep or Shallow Refiners.

Shallow Refiner:  A shallow refiner uses the statistics from managed properties for the top 100 matching results of a search query.

Deep Refiner: A deep refiner uses the statistics from managed properties for all the matching results of a search query.

What’s does that mean?  With a shallow refiner if the Managed Property is not in the first 100 matching results the Query Refinement option will not be populated and not shown under the refinement options.  If you setup a deep refinement option with FS4SP you will always have a refiner to narrow the results set and more likely to find any result in the search query.

Conclusions: If you have more than 100M items to index then use FS4SP.  If you have less than 100M items either index will be sufficient so answer Question #2.  If it is critical that the end user needs to be able find an item whether its’ relevancy places it as the 10th result or the 10,000^th result use FS4SP.