Here's my best guess...
- A very popular business route with a ton of Plats who fly to SF on a regular basis
- People flying BF connecting to/from flights out of EWR and IAH who were unable to get F themselves for the SFO leg (so they are ahead of you on the upgrade list no matter their status)
- CO knows they will sell tix at Y fares and fill the plane even though F is sold out
I fly to SFO 1-2 times a month & typically book 4 weeks out so as not to have my company pay H & Y...at least I get the exit row.
In terms of your hopes for the future, I am sorry to say they aren't good. I just broke an 0 for 60-something upgrade drought flying to/from EWR-SFO this past Saturday and am still 0-fer SFO-IAH...so I guess you'll probably have to get used to it or try flying on a Saturday (and even that is no guarantee). Flying on Monday morning, Thursday afternoon or Friday morning, I am so far down on the upgrade list as a Plat (usually K or V fares) it is laughable...my record low is 26th.