barelyelite

I've had this problem for several weeks now... seems to be related to upgrading to FF5.

The feature is not working properly on two office computers running FF5. Works fine on IE7 on those same computers, as well as Chrome on my home computer.

josephstern

I'm also on a Mac with Firefox 5.

I'm sure IB can see how many of us are Mac and how many PC, but I'd wager that there's a pretty decent Mac presence here. Especially with MacBooks for travelling, and the generally affluent nature of this board, I think Macs are probably over-represented compared to general sales numbers.

ftnoob

Sample size of two so far, but I just tested that type of link on my FF 5 / Win 7 machine that has been having this problem, and it worked correctly, in both cases on threads that I had visited today.

Example:redirected to:the last post (at the time) and the only one I had not yet read.

I then clicked the -new-post.html link for the same thread and was taken to:a post that it is three days old, precedes a post I made on the thread, and that is seven posts prior to the one I just read using the old style link!

That suggests the problem is in the communication between vbSEO (or the custom FT code?) and core VB?

Would an option be to revert the new-post links to the old style, but add the nofollow tag? (The nofollow tag might actually be superfluous; a guest with no cookies doesn't see any -new-post.html links anyway, so perhaps the bots never see them either.)

I'm not a real JavaScript coder (I just play one on FT for the benefit of myself and the rest of the RR forum) so I'd be pokey at putting it together...but it seems a user-side fix would be a GreaseMonkey script to rewrite the -new-post.html URLs to the old style. If nothing else, that would help more quickly collect enough data to confirm or disprove the conclusion suggested by my small sample size results.

Could your proximity to the server be a factor?

ftnoob

Here is a user-side fix in bookmarklet form:
I only tested the bookmarklet once and on only one browser, but while slogging through figuring out the required code I did complete several more tests of the old-style URL, so far without the URL failing to function as expected.

As with all bookmarklets, it has to be applied anytime the page it affects changes. Therefore, if it is not already your practice, you should Ctrl-click or shift-click all -new-post.html links.

The bookmarklet uses jQuery (already used by FT), so it will take a bit more work to convert the bookmarklet to a GreaseMonkey script than otherwise, but I'll post that when it is available.

ETA: I don't access FT threads via MyFlyerTalk, so it hasn't been tested there. It should work, but let me know if it doesn't.

Global_Hi_Flyer

OK I got really annoyed at it today & tried something that worked for me. It's really kludgy & we still need to find a better fix, but see if this works until IB can find a better fix:

Clear your browsing cache. In FF, you do this by going to tools>options>advanced>network and punch the "Clear Now" button in the offline storage section (next to the line that reads "Your cache is currently using xxx.x MB of disk space").

I've tried it this afternoon & the results seem repeatable. After you clear cache, the "go to next unread" works correctly. I cleared cache, read the new posts in thread (correctly handled), exited thread, additional new posts were made, went to read new posts without clearing cache & it took me to something I already read..... exited thread, cleared cache, additional new posts were made in thread & accessing the "read new posts" function worked correctly after clearing cache.

If it works for others, then we need to figure out why the cache isn't being cleared like it used to after the database pointer is updated.

sc flier

In FF5.0 on Win7:
If I clear my cache (Tools menu > Options > Advanced Tab > Network Tab > Clear Now button), then the links function correctly for me. Of course, I later need to clear my cache again as new links/pages get cached.

Edited to add: Looks like Global_Hi_Flyer and I were going through this around the same time. Is there something that can be done to tell the browser not to cache these?

ftnoob

Here is the GreaseMonkey script to change the -new-post.html links to the old style:

baliktad

The comments about clearing the cache resolving the issue got me thinking, so I did a little data-gathering.

The process by which a browser navigates to the last unread post has been explained above, but I wanted a closer look at the traffic to see if something funny was occurring. The fact that the newest browsers are affected would seem to indicate that as the browsers are becoming more and more standards-compliant, incorrect or outdated behaviors are being exposed.

First, a little background. When a browser requests a web page, the server can respond all sorts of different ways. If the page doesn't exist, many people are familiar with the "404 Page Not Found" error code. If it exists and everything is fine, the server responds with "200 OK" and the contents of the page. There are actually many such codes which are grouped into series: 200-series indicate success, 500-series indicate server error, etc.

The 300-series response codes are interesting, and as a group, categorized as the redirect responses. This generally means the client (browser) has made a request, the server understands it, but the server wants the client to look elsewhere for the content. FT employs a 300-series request when redirecting the browser from the new-post "magic" link to the actual post. Let's take a look at the raw request and response for a problematic thread:

Starting from my My FlyerTalk page, I click on the First Unread Post icon next to a thread title I'm interested in. In this case, it's the AS Newbie thread, which I generally track whenever I'm online. Here's the request headers my browser sends. Note that this is the new-post "magic" link:

Now here's what flyertalk.com sends back: a pointer to a showthread page identifying a specific post ID. This response basically boils down to "I understand what you want, but it's not here. Please ask for it at this new address."

The client immediately makes a new request for this showthread page. Here's the browser asking flyertalk for the new page it was told to look at (some cookie values redacted for privacy and brevity):

Showthread.php basically takes a post ID and generates a search-engine friendly URL. The server responds with yet another redirect:

The client accepts this new address and makes a 3rd and final request for the page it wants:

Finally, the server responds with the actual page content the user cares about. Only the headers are presented here, but the content is delivered immediately following:

Now some analysis. The reason the 300-series codes are so interesting here is that the different codes have different meanings. Lots of info on Wikipedia, but the one FT uses is 301 Moved Permanently. As the description indicates, 301 Moved Permanently is useful when a page used to be located in one place, but has a new permanent home somewhere else. It's an easy way to tell a client "look somewhere else," but in this case, it's wrong, because this response code really means "look somewhere else, and don't come ask me again, because the answer will always be the same." As defined in the RFC: "This response is cacheable unless indicated otherwise."

All this means that when a user clicks on a new-post "magic" link, FT is responding with a URL to specific post, and specifically indicating to the browser that it CAN and SHOULD save that answer for later. So later, when that thread comes up again in your My FlyerTalk, modern browsers will default to their cached values instead of taking the time to round-trip to the server. This is good design as it allows the browser to save one entire unnecessary round-trip, which can speed up the browsing experience significantly.

So what's the fix? Well, FT has already figured it out. Examining closely the responses above, you can see the other 2 responses from FT (including one other 301 response) both include specific directives to the client to not cache the results. These are the 3 relevant lines:

I would expect that if FT added these no-caching directives to the new-post magic redirector, the issue would be resolved. Alternatively, the new-post magic redirector could be adjusted to use a more appropriate redirect response that does not suggest the client cache the results. 307 Temporary Redirect is probably the suitable option here.

Edit: Here's the post that explains this behavior is new in IE9 that basically confirms my analysis above. The problem is definitely with FT's implementation of 301 response code, and is only just now being exposed by new efficient browsers.

Global_Hi_Flyer

And just like that, the diagnosis is made....

travelmad478

Did the clear cache thing on FF 5.0 and it is still not working (today I got the worst new posts list ever, with "new" posts all the way back to June 22, all of which I have already looked at, of course).

Could someone at IB please fix this damn thing already? Especially now that baliktad has done the work for you?

IB-Dick

I really think that this is the problem. It even explains why I'm not having the issue.... I have caching disabled, since when you work in WebDev, caching is usually a headache.

I'm working to find an elegant solution to this.

PhoenixRev

I am glad I found this thread as this problem was driving me crazy.

For my part, I have FF 5.0 on my desktop and FF4.0 on my laptop and both are having the same problem.

So, this morning, I cleared by cache and cookies and it worked, but when I got home, it took me back to a 7:00 AM post even though there were other posts made after it that I read. So, I cleared the cache and cookies again. It then moved me to the last afternoon posts I read. So far, so good.

Logged off to have dinner and returned, launch the browser, clicked a thread and ... BAH.. back to the 7:00 AM post reference above.

Keeping my fingers crossed that you can resolve this soon.

Thanks!

swag

Thanks!

If the solution still requires us to do a one-time cache clear to get rid of the already cached redirects, can I suggest that once it's implemented, you post an Announcement at the top of the forums to let folks know the required step? This problem seems pretty widespread and yet many users probably never check this sub-forum.

ftnoob

Hey, me too!

I did some looking and discovered things like:Still, at first blush one is inclined to say "it isn't a cache problem, because the newly served page includes information that is not in the cache!" As I was beginning to suspect, that is an outdated view; browsers have evolved. Clearly you agree:
I agree with your analysis except to point out that your explanation implies a communication from the browser to FT (and back) that with modern browsers will not actually occur if there is a cached 301 response. This at last is the answer to something that had me scratching my head these past few days:
Not being a web technologies expert, I found the "pending" status for the initial request puzzling. Now it is becoming clear: the status is pending because the browser intercepted the request, saying "Wait a sec, I already know the new permanent home for that URI; I won't bother the server to ask for it, I'll just substitute in a request for the new permanent home of the URI the user clicked!" (This is the avoided round trip mentioned in your post.)

I might be behind the curve on this one, but I didn't see any clearly erroneous caching instructions sent from the server.

Can you double check this and confirm this specific change is needed?

I have seen both 302 and 307 mentioned, but this is exactly the direction I was going. The -new-post.html (which as previously noted should not normally be seen by a bot) should receive a "temporary redirect" response, then the old-style .php URI should receive the 301 "permanent redirect" response. To my untrained eye, that should take care of the problem. Perhaps even take care of it in an elegant fashion.

Thanks for that link; I'd been looking for discussions of updates to modern browser handling of 301 caching, and hadn't come up with much. I kind-of think I'd seen that msdn blog post, but I need to review it to be sure of what it is saying.

After reading the nearly-simultaneous posts by Global_Hi_Flyer and sc flier, I would have laid big money that such was the case!

ETA: left unexplained by these analyses are the upthread reports of this problem occuring in FF 4.x. I'm inclined to think those are erroneous reports, but I could be mistaken.

baliktad

The caching instruction is implied by the 301 response code. HTTP response codes are defined to have certain behavior in RFC 2616. The definition of response code 301 (Moved Permanently) begins with this text:

In effect, this RFC tells browser-makers that when they get a HTTP 301 response, they needn't ask the server anymore for the original location, since the data will never be there, and more importantly (and perhaps dangerously), the data will ALWAYS be at the new location specified. I would argue that things are rarely permanent on the internet, but fortunately, it seems that modern browser manufacturers have taken "permanent" to mean "until the cache is emptied."

I'm not sure if you're asking me or IB-Dick, but there are several specific changes that are possible here. I suspect the easiest one is to adjust the magic new-post PHP to write some explicit no-cache headers, which is why that's the one I recommended. The ideal solution would be to move away from 301 altogether, as a permanent redirect is never intended here. So rather than giving some instructions and then immediately nullifying them, just start out with the proper instructions. Either HTTP 302 or HTTP 307 seems to be the proper response for this case:

This response code indicates that browsers should re-request the magic new-post page every time the user asks for it to ensure that the latest post is received.

I'm not sure HTTP 301 is ever an appropriate response, since FT has never really moved anything permanently. If posts are deleted or the user changes the number of posts displayed per page, the resulting URL may be different. If the server issued a 301 (and didn't explicitly indicate that caching is not allowed), clients would potentially run into cases where their saved HTTP 301 redirects are pointing to the wrong locations.

For a dynamic site like FT where pages are all generated on the fly, these types of redirects should really be handled with HTTP 302/307 response codes, along with an explicit no-cache directive to ensure that the intent is clear to the browser: these redirects are ephemeral and may change at any time. Do not cache the resulting links. Always ask the server, and you will always get the right behavior.