Personally, I find getting to SFO from Palo Alto much easier in the afternoons (during rush hour) than going to SJC. The southbound traffic in San Jose on the 101 can be unbearable.
Sometimes I’ll fly TO SJC, but I’ll usually fly OUT of SFO. The lounge / restaurant selection is also much better out of SFO.