Flyertalk can't seem to remember last viewed post
#61
No longer with Internet Brands
Join Date: Mar 2011
Location: Los Angeles, CA
Programs: DL DM 1.6MM, Marriott LT Plat
Posts: 5,343
FF5 is not doing anybody any favors -- see this thread about aa.com not working with it:
http://www.flyertalk.com/forum/techn...t-work-me.html
http://www.flyertalk.com/forum/techn...t-work-me.html
#62
Moderator, Southwest Airlines and Choice Privileges
Join Date: Mar 2008
Location: Central Texas
Posts: 3,039
I then guessed: the fact that a new post on an older thread worked correctly could be an important clue...but when I pulled up this thread to report that tidbit, the new-post URL worked correctly here!
BTW, because FF 5 is making a point of noting it is the first browser to support "do not track" across all platforms, my first guess had been that the problem is related to the DNT header. That option is not enabled by default, however, and it already existed in FF 4, so that guess is probably wrong also.
#64
Join Date: Aug 2006
Location: DCA / WAS
Programs: DL 2+ million/PM, YX, Marriott Plt, *wood gold, HHonors, CO Plt, UA, AA EXP, WN, AGR
Posts: 9,388
No change as of today, and even this thread is having the problem.
#67
Ambassador: Alaska Airlines
Original Poster
Join Date: Jul 2009
Location: Seattle
Programs: AS MVP Gold
Posts: 2,732
I know IBobi is only the messenger, but IB's stance on this has been to blame it on anyone but FT from the start. First it was IE9, now it's FF5. There has never been any workaround offered or genuine update confirming a specific problem with a particular configuration or setting. There has been no bug filed with any browser manufacturers or any other visible action taken by IB. We are coming up on 3 months for this issue with no realistic resolution or even timeframe for one.
#68
Join Date: Sep 2005
Location: Lafayette, CO, USA
Programs: SPG Lifetime Plat, AA Gold, UA Gold, DL Silver, HH Gold, Vail Epic
Posts: 9,096
Could someone provide a technical explanation about how the system is supposed to keep track of the last viewed posts so that maybe some users here might get some ideas about how to address it on the browser side?
#69
Ambassador: Alaska Airlines
Original Poster
Join Date: Jul 2009
Location: Seattle
Programs: AS MVP Gold
Posts: 2,732
Last viewed post is tracked entirely server-side (ie, not in the browser). When you visit a page, an entry is made in a database that indicates the thread and presumably the last post or time viewed. Later when you visit the thread again, the server looks up the last viewed post in that thread and redirects the browser to the first new post.
There is little, if anything, the browser contributes to this process other than maintaining and presenting your identity to the server via a cookie. The fact that new threads show up in "My Flyertalk" when there are new posts means that FT is well aware of the last post you read and what the new post should be. But the new post redirection is failing.
The symptoms would seem to indicate that there is something server-side that is failing to redirect properly. Without any knowledge of FT's architecture, I would guess that a front end caching server somewhere is failing to keep up with new writes, but only the site engineers are in a position to confirm or rectify the issue.
There is little, if anything, the browser contributes to this process other than maintaining and presenting your identity to the server via a cookie. The fact that new threads show up in "My Flyertalk" when there are new posts means that FT is well aware of the last post you read and what the new post should be. But the new post redirection is failing.
The symptoms would seem to indicate that there is something server-side that is failing to redirect properly. Without any knowledge of FT's architecture, I would guess that a front end caching server somewhere is failing to keep up with new writes, but only the site engineers are in a position to confirm or rectify the issue.
#70
No longer with Internet Brands
Join Date: Mar 2011
Location: Los Angeles, CA
Programs: DL DM 1.6MM, Marriott LT Plat
Posts: 5,343
Hi all,
We're going to take another look at this this morning and report in the thread. The scope of this issue seems to have dramatically increased, especially with FF5's release. Hope to have an update today and have Dick respond.
I will keep you posted; thank you for being patient and reporting specifics for us to work with.
Paul
We're going to take another look at this this morning and report in the thread. The scope of this issue seems to have dramatically increased, especially with FF5's release. Hope to have an update today and have Dick respond.
I will keep you posted; thank you for being patient and reporting specifics for us to work with.
Paul
#72
Join Date: Sep 2005
Location: Lafayette, CO, USA
Programs: SPG Lifetime Plat, AA Gold, UA Gold, DL Silver, HH Gold, Vail Epic
Posts: 9,096
There is little, if anything, the browser contributes to this process other than maintaining and presenting your identity to the server via a cookie. The fact that new threads show up in "My Flyertalk" when there are new posts means that FT is well aware of the last post you read and what the new post should be. But the new post redirection is failing.
The symptoms would seem to indicate that there is something server-side that is failing to redirect properly. Without any knowledge of FT's architecture, I would guess that a front end caching server somewhere is failing to keep up with new writes, but only the site engineers are in a position to confirm or rectify the issue.
The symptoms would seem to indicate that there is something server-side that is failing to redirect properly. Without any knowledge of FT's architecture, I would guess that a front end caching server somewhere is failing to keep up with new writes, but only the site engineers are in a position to confirm or rectify the issue.
But something is getting written to the database successfully because the threads don't show up in My FlyerTalk again until there's a new post. It's tracking whether or not I've read a thread, but once a new post shows up, then it's failing to direct us to the correct new starting point.
Today, I keep getting reminded of last night's post pointing out that Caroline is not much of a coke head every time I check the new posts in this thread...
http://www.flyertalk.com/forum/omni/...l#post16636288
And maybe that's important. Is the server caching the time for the redirects? So when I return, I'm redirected to the cached value? But then sometimes, the server's cache gets cleared?
#73
Moderator, Southwest Airlines and Choice Privileges
Join Date: Mar 2008
Location: Central Texas
Posts: 3,039
(The original formatting from above perhaps made it difficult for non-techs to follow, so let me add some background and attempt some reformatting.)
Understand that on the thread listing pages, the URL attached to the Go to first new post icon () is what for non-technical consumption I refer to as a "magic url." I call it that because there is no actual page at the location flyertalk.com/..../....-new-post.html, and for someone who doesn't understand what happens behinds the scenes, it feels like magic: the exact same URL can display different content to everyone with a different last read post for the thread.
When you attempt to retrieve a -new-post.html URL, the server uses data stored in your cookies to figure out who you are, then looks in the database for your "last read post" value for the thread in question. It also has to look up your posts-per-page setting (from the database or another cookie) to calculate on which page your first unread post should appear. When the server has all the variables it needs from the database (and/or cookies), it builds the redirect URL, and tells the browser to display the redirected location instead of the URL it requested.
One way you can confirm that this is going on is to change the Firefox network.http.redirection-limit value on your about:config page. The default value is 20:If you set the value to zero and attempt to retrieve one of the "magic" -new-post.html URLs, the redirect process will be shut down by Firefox and you will see this (somewhat misleading) message:If you try it, be sure to set the value back to 20 after your experiment.
Now here are the reformatted examples from an earlier post:
Code:
[1] http://www.flyertalk.com/forum/rr/1229903-southwest-early-bird-boarding-sham-new-post.html [2] http://www.flyertalk.com/forum/rr/1229903-southwest-early-bird-boarding-sham.html#post16620273 [3] http://www.flyertalk.com/forum/rr/1229903-southwest-early-bird-boarding-sham.html#post16623396
[1] Original virtual URL
[2] Incorrect redirect experienced on FF 5
[3] Correct redirect experienced on FF 4
For those who aren't already aware, the #post... part of the URL instructs the browser to scroll to a particular "anchor" in the page, rather than just go to the top of the page.[2] Incorrect redirect experienced on FF 5
[3] Correct redirect experienced on FF 4
This is important to note:
It is also important to note, however, that if you are not currently viewing the page with your first unread post, the View First Unread link is not a real URL, it is simply the -new-post.html "magic" URL.
Conclusion #2: We can slightly reduce the hassle of this bug by setting our posts-per-page to the max (40, IIRC). Then if instead of clicking the first unread link from the thread list page, we:
- first click a link that takes us to our correct first unread page;
- then click the View First Unread link on that page;
There is one point where you went astray, including an unreliable thread link toward the end (I removed the hot link so it can be easily read):
http://www.flyertalk.com/forum/omni/679219-wimbledon-tennis-grand-slam-tournament-new-post.html
On a tangent, it is important to know that the -39 part of the URL normally causes the link to fail for anyone whose posts-per-page setting causes Post 16636288 to appear on some other page. Don't attempt to share post-specific links using -pg.html style links on FT. In FT posts and PMs, wrap post tags around some text to link to a specific post:[ post=16636288]Caroline is not much of a coke head[/post]
Caroline is not much of a coke head
(Remove the extra space from the markup on the first line to create the hotlink in the second line. The link is dynamically generated from the markup every time the page is viewed, based on each person's posts-per-page setting.)Caroline is not much of a coke head
Impossible to say from our side, but we do know that the correct information is getting into the database in at least one place. It seems clear to me the problem has to be in the part of the code that builds the redirect URL for us. Whether due to caching or the data otherwise being stored in multiple places and not updated in all of them, we'll have to leave it to IB to figure out.
#74
No longer with Internet Brands
Join Date: Mar 2011
Location: Los Angeles, CA
Programs: DL DM 1.6MM, Marriott LT Plat
Posts: 5,343
Hi all,
We have begun a test that we're hoping will resolve this; please report any change in behavior (positive or, heaven forbid, negative) over the next hours/days.
Thanks!
Paul
We have begun a test that we're hoping will resolve this; please report any change in behavior (positive or, heaven forbid, negative) over the next hours/days.
Thanks!
Paul
#75
Join Date: Feb 2009
Posts: 959
ftnoob's post is accurate, and I'd like to offer my thanks for writing it.
Just to add a little complexity to this, I want to make a note on how the system marks what you've read. Every time you view a thread, it marks in the database the timestamp of the last post on the page that you looked at. If you look at page 2 of a thread that has 4 pages, then it assumes that you've finished page 2 and will next time direct you to page 3. If you land on page 5 of a thread with 5 pages, but page 5 only has 2 posts on that page, when you return you will return to the third post of page 5 and update the record in the database with the new appropriate value. While this isn't always completely accurate, it does the best job that it can, and it's the most accurate method that the software offers to get users to the correct new post.
Now, if you think about a table that records every viewed thread by a logged in user, you might thing "Wow, that table has to be HUGE." Well, you'd be half-correct there. Since that data model isn't exactly scalable, the forum software makes some assumptions. The biggest assumption is that after a certain period of time, you've read every thread. This configurable setting was previously set to 10 days, but has since been changed to 5 days for an experiment. A scheduled task runs through the table where we keep track of the last read threads and removes all records for that are past that 5 day limit. When you open a thread, it assumes that any post that was made over 5 days ago was read. It checks the table to see if there is a record in there for the specified thread and if so, it directs you to where it assumes you left off. If there isn't a record in there, then it assumes that the first unread post is first post made since 5 days ago.
I can test this by going to a long thread that I've never been to. In this example, I'll use this thread:
http://www.flyertalk.com/forum/commu...-new-post.html
Notice that this is the 'magic' url to use ftnoob's term. Since I've never been to this thread, the first unread post would be post 1 on page 1. That's not the case, because it assumes that every post made before 5 days ago I've already had ample time to read. Instead, I get directed to here, http://www.flyertalk.com/forum/commu...l#post16627511 the 7th post on page 35. I got the 7th post because it's the earliest post in the 5 day window, since the previous post was made on the 22nd (6 days ago). Now, here's the tricky part... I just view the thread and was directed to the 7th post on page 35. I didn't read the full page, but the software doesn't know that. When I go to that magic link again, it assumes that I've read page 35, so it redirects me to last post of the page. If there had been a page 36, it would have directed me to the top of page 36. If then I went to the 'magic' url again, it would redirect me to page 37, you know... if there was one in this example. I should have picked a better one. Feel free to give this a shot yourself... Go to a forum that you don't usually frequent, find a 100 page thread that was started over 5 days ago but still has recent posts, and click on the last unread post button. You'll end up on the last post from 432,000 seconds ago (5 days ago). When you return to the magic -new-post.html page, you'll be semi-intelligently and predictably moved along the thread.
I've walked through the system a number of times since this was reported, and I haven't been able to find any flaws in the system mechanics. Is it funky in the way it works, making incorrect assumptions about what you have viewed and what you haven't? Yup. Can it drop you off, seemingly in the middle of nowhere, and make the system look horribly broken? Yup. Is there a better way? Not really.
We're continuing to replicate the problem. At first glace, it looks like the system is terribly broken. Upon further inspection and understanding of how the system is designed to work, we can't find anything that isn't functioning as designed. Since there is some chance that it's a matter of database scaling (which I don't believe), we turned the threshold for remembering posts from 10 to 5. If the problem is with the database not properly marking data or redirects not being sent accurately, this should help (since we've cut the database size literally in half). If the problem is that the software offers a confusing user experience, this will make it worse. The good news is that if this setting makes things worse, then we at least have a way to make things better. Instead of marking the threads read after 5 days (current setting) or 10 days (old setting), we can try setting this to 15 days and see if things are less confusing.
This isn't completely accurate. When you pasted the 'magic' url into FF 5, the database was updated to show that you've progressed along the thread. When you pasted the 'magic' url into FF 4, you got a later post because of the newly updated record in the db. If you had done these in reverse order, it would look like FF 4 is incorrect and FF 5 was broken.
Kinda. I can also exacerbate the problem, since now if I read the first 20 post of a thread and not the whole page, when I return it can dump me on page 2, causing me to miss 20 posts.
Just to add a little complexity to this, I want to make a note on how the system marks what you've read. Every time you view a thread, it marks in the database the timestamp of the last post on the page that you looked at. If you look at page 2 of a thread that has 4 pages, then it assumes that you've finished page 2 and will next time direct you to page 3. If you land on page 5 of a thread with 5 pages, but page 5 only has 2 posts on that page, when you return you will return to the third post of page 5 and update the record in the database with the new appropriate value. While this isn't always completely accurate, it does the best job that it can, and it's the most accurate method that the software offers to get users to the correct new post.
Now, if you think about a table that records every viewed thread by a logged in user, you might thing "Wow, that table has to be HUGE." Well, you'd be half-correct there. Since that data model isn't exactly scalable, the forum software makes some assumptions. The biggest assumption is that after a certain period of time, you've read every thread. This configurable setting was previously set to 10 days, but has since been changed to 5 days for an experiment. A scheduled task runs through the table where we keep track of the last read threads and removes all records for that are past that 5 day limit. When you open a thread, it assumes that any post that was made over 5 days ago was read. It checks the table to see if there is a record in there for the specified thread and if so, it directs you to where it assumes you left off. If there isn't a record in there, then it assumes that the first unread post is first post made since 5 days ago.
I can test this by going to a long thread that I've never been to. In this example, I'll use this thread:
http://www.flyertalk.com/forum/commu...-new-post.html
Notice that this is the 'magic' url to use ftnoob's term. Since I've never been to this thread, the first unread post would be post 1 on page 1. That's not the case, because it assumes that every post made before 5 days ago I've already had ample time to read. Instead, I get directed to here, http://www.flyertalk.com/forum/commu...l#post16627511 the 7th post on page 35. I got the 7th post because it's the earliest post in the 5 day window, since the previous post was made on the 22nd (6 days ago). Now, here's the tricky part... I just view the thread and was directed to the 7th post on page 35. I didn't read the full page, but the software doesn't know that. When I go to that magic link again, it assumes that I've read page 35, so it redirects me to last post of the page. If there had been a page 36, it would have directed me to the top of page 36. If then I went to the 'magic' url again, it would redirect me to page 37, you know... if there was one in this example. I should have picked a better one. Feel free to give this a shot yourself... Go to a forum that you don't usually frequent, find a 100 page thread that was started over 5 days ago but still has recent posts, and click on the last unread post button. You'll end up on the last post from 432,000 seconds ago (5 days ago). When you return to the magic -new-post.html page, you'll be semi-intelligently and predictably moved along the thread.
I've walked through the system a number of times since this was reported, and I haven't been able to find any flaws in the system mechanics. Is it funky in the way it works, making incorrect assumptions about what you have viewed and what you haven't? Yup. Can it drop you off, seemingly in the middle of nowhere, and make the system look horribly broken? Yup. Is there a better way? Not really.
We're continuing to replicate the problem. At first glace, it looks like the system is terribly broken. Upon further inspection and understanding of how the system is designed to work, we can't find anything that isn't functioning as designed. Since there is some chance that it's a matter of database scaling (which I don't believe), we turned the threshold for remembering posts from 10 to 5. If the problem is with the database not properly marking data or redirects not being sent accurately, this should help (since we've cut the database size literally in half). If the problem is that the software offers a confusing user experience, this will make it worse. The good news is that if this setting makes things worse, then we at least have a way to make things better. Instead of marking the threads read after 5 days (current setting) or 10 days (old setting), we can try setting this to 15 days and see if things are less confusing.
Originally Posted by ftnoob
[1] Original virtual URL
[2] Incorrect redirect experienced on FF 5
[3] Correct redirect experienced on FF 4
[2] Incorrect redirect experienced on FF 5
[3] Correct redirect experienced on FF 4
Originally Posted by ftnoob
By setting the posts-per-page to 40 we reduce the number of mult-page threads as well as the number of times we end up on a page that is before or ofter the one that actually contains our first unread post.