My vote would be for the downtown Marriott (on West Harbor Drive). It's a better hotel (aren't they renovating too) and in a better location. Closer to many more San Diego sights (zoo, balboa park, etc...) Although the one in La Jolla is closer to La Jolla beaches, you still have to drive to them. I'd be willing to drive a little further if I can get a better hotel. Just avoid rush hour and the drive to la jolla should be fine. One other thing, if you've never been to Torrey pines state park, go! They have pretty easy hikes to take your toddler on and the views are beautiful!
One other thing about the marriott is I believe they have a water taxi at their marina that can take you to the coronado marriott across the harbor which i thought was pretty cool. Double check though.