I'm no expert, but the times I've stayed there I stayed closer to the Guggenheim and would not change that. Casco Viejo is close, and the tram can make it into a fast trip as well.

Don't forget to take the metro up north to visit the Bizkaia moving Bridge in Portugalete and enjoy some good Pintxos there.

I also recommend having lunch at Kafe Antzokia - a cafe situated in a theater.