FT crawled to a stop in my browser this morning. In fact, my machine froze. Re-booted, and still got the same problem. Ugh.

Bots....
 

We continue to analyze the speed thing. Came across this on a Board that serves admins of the bigger boards. Given that there is a "search tool" war going on now - all these new search engines - not just Google anymore, it seems this may also be contributing to our problems. Remember, we do have nearly 4 million posts here.

Here's comments from a discussion by owners of other large bulletin boards.:
Some robots are well behaved and gradually cache sites. Others are killers. They jump in and start hammering away as fast as they can, eating up my entire T1 line and basically making my system unavailable to legitimate users. While there are some advantages to letting the robots in, my feeling is that they do more harm than good. So my robots.txt file basically tells 'em to get lost.

I'd keep google... I've seen some new spiders, gigabot, that hammer my site. I'm thinking about blocking some of the less-known spiders such as gigabot and only allowing Yahoo, MSN, Google and AltaVista.

MSN bot is the killer on my forum. Every hour never fails to come.


We'll tinker with this slightly to see what happens perhaps as one Board operator has done and lock out gigabot. With A3 and MSN now really in the search area it does present more problems for us.

^ Thanks!

Too bad you can't restrict the bots to target-rich forums like Miles Buzz or Travel Buzz, letting the bots help FT get exposure from search engines while, at the same time, leaving some access for real members.

What about restricting it from Omni?

I think that's a great idea! @:-) Let's hope the vBulletin package let's them do that.

The idea being, of course, that permanently restricting the bots from accessing a portion of the available ports improves board performance for most members. For example, say FT has 100 ports. That is, 100 people can access it at a time without impacting performance. If more than that number is trying to access the board, performance slows down. What I think Randy's post indicates is that there are bots which are "hammering" all available ports, slowing down board performance every time they are accessing the board. Well, if you restricted the number of ports available to bots to 50, then it wouldn't matter (as much) when bots were accessing the board. Also, it helps FT to have these search engines directing web surfers here.

OK, I just found the "REAL" reason the board is so slow every now and then and have reported my find to the tech staff. In the mean time, if you are a moderator, please check my thread in the moderator forum.

Scott, whatever it is you've found, I hope it's right. We'll have to wait till Monday to see if the FT techs can correct it, but it'll be worth the wait if that fixes the slowness and hanging. Oftentimes it slows to the point of my losing the connection, or totally freezing my pc, causing me to have to reboot.

Hope you've found the problem, and it can be fixed.

Thanks,

bj-21.

intermittant slowness right now.. just took over 60 seconds to load My Flyertalk

Yea I am getting it as well...sooo slow...

Hope FT speeds up soon....this is getting really hard...

What I don't quite understand is if the problem is loading the ads, why does POSTING take so long?

Is it because after a post you are taken right back to the forum, where it has to reload everything?

I've noticed that when you see a delay in posting the post actually has posted, even while you wait for a reload.

Personally, I don't think the it's directly related to the ads. I'll be blazing along ads and all, then suddenly hit a brick wall for a minute or two. Then right back to blazing again. Been happening for several weeks now. Most recently when I just now came here to make this post.

I certainly hope ScottC has a winner. This has been one of the more mysterious and annoying FlyerTalk problems in a long time.

My experience is very much like PremEx's. It's as if FT just freezes for awhile and then gets back into it and everything is blazing again.

I wonder if it is some kind of disc or I/O problem.

Just happened to me again at 7:59:03 - 8:01:45 Eastern time, in case someone wants to check the logs.

I was in this thread at the time:

(NFL discussion, go Patriots!)

http://www.flyertalk.com/forum/showthread.php?t=353812

Yes, that was me, I was making sure my theory was correct as some people thought it was just a coincidence. I've emailed the FT tech staff and will leave it to them...

tip
 

for those on a mac using ie-

this has speeded things up for me but you will lose some functions such as adding smilies, the pop-up incoming message window, slow posting replies and a few others but it sure beats 45 seconds for a page to load using dsl. (mac b&w g3, os 9.2.2, ie 5.17).

edit menu
preferences
web browser
web content
uncheck "enable scripting" (and "show scripting error alerts", if enabled), "enable plugins", and "enable activex".

do this before you access f/t and reset after you log off