The traffic through and around Seattle can be very bad. Stop 'n go northbound pretty much all the way through Everett for the PM rush hour, and even heavy beyond, even at 7 pm or later. I imagine the morning northbound rush hour between Olympia/Tacoma and Seattle is pretty bad too. The stretch between Olympia and the Columbia river can be slow as there's only 2 lanes quite a bit of the way and you do get drivers who insist in driving in formation on all lanes under the speed limit.
If you have 2 persons in the car and headed through Seattle northbound after midday, I'd take I-5 as the (reversible) express lanes (separate roadway from the rest of I-5) make for faster travelling than I-405. Otherwise the 2 lanes through Seattle on the normal lanes can clog up traffic.
What's been said about the border is pretty much right. Best time to cross, IMHO, is after 11pm (I get from the SEA parking lot to the middle of the City of Vancouver in 2.5-3 hrs leaving at 9-11p). I'd say if you leave Portland at 7p but before 3-4a, it'd take 5 hours but longer otherwise. You could leave Portland around 9-10 and avoid most of the bad congestion but the border waits then could be anything when you cross in the afternoon.