Your plan looks fine to me. You might hit significant traffic on westbound 580 after Livermore, but probably not that bad. By the time you get to the 238 connector, the morning rush hour will be done. Maybe a bit of a backup as you approach the toll plaza at the San Mateo Bridge (Hwy. 92) It's generally true that the morning rush hour tends to be lighter on Fridays before a three-day weekend.

As you mention, the main flow of traffic will be leaving the SF Bay Area, and you're going in.