If the Clipper doesn't work out, there are a couple of other options:

Kenmore Air flies from Lake Union to downtown Victoria. There's good public bus service from the Inner Harbour to U Vic. Kenmore Air is not cheap, but neither is the Clipper, and they sometimes have fare sales that allow them to compete with the Clipper. The view is fantastic.

Renting a car and driving up to Tsawassen might be a better option than renting a car and driving to Port Angeles, if you decide to drive, depending on the lines at the border. The BC Ferries run more often than the Coho and the trip is more scenic. You can also park and walk on in Tsawassen, with okay bus service from Swartz Bay to U Vic (there's a transfer, I think).