Originally Posted by wbernahl

Bill from E-Commerce here. What we believe is happening with the web site is that unauthorized robots are coming into the site and spidering our special offers and hotel listings. The major robots are thoughtful and spread their traffic out over time. These problematic robots seem to be coming in and just hammer us over a very short period of time doing complex queries. One specific robot consumed >25% of the entire system resources. We've been shutting them off and flagging more pages as off limits for robots (a suggestion to them only). We're also working to add more capacity and modify the code to be more efficient in response to the queries giving putting the high load on the system.

I really appreciate what an inconvenience this is to everyone and I'm even more upset than you are when this happens.