The new search doesn't work directly on the flyertalk database. Instead it works on a copy of the data that is formatted specifically for searching. By working on this copy it not only makes the searching faster, but also avoids causing contention in the main db (which is what was causing the board to become unusable). Obviously we need to keep that copy in sync with the flyertalk db and the indexer is the utility that does this. The delay should typically be a minute or two, but in this case the indexer ran into a problem and shut down. Its not a load issue. The indexer can handle substantially more than the typical rate of flyertalk (and has when we've had to reindex the entire db).
Kevin