It's not surprising that BWI service isn't that good - IAD is a hub, BWI is not (and is much smaller). A similar situation (but perhaps not quite as bad) affects OAK vs. SFO.

There is a single non-stop SFO-BWI (and BWI-SFO) every day... at perhaps inconvenient times, but it's there. Otherwise, expect to connect, but there are plenty of 1-stop SFO-XXX-BWI flights daily (XXX = LAX, ORD, or DEN).

I wouldn't recommend driving from IAD to Baltimore unless you can avoid traffic. Without traffic, it is a little over an hour's drive if you drive like a usual Californian. You could try Amtrak but that has its own hassles (including getting to an Amtrak station from IAD). I would strongly recommend just picking one of the 1-stop flights if the non-stops don't work for you; there are enough options that you can arrive pretty much any time of day.