There are plenty of parking spaces right before the border. Just watch the rates. Closer you are, more expensive it is. It is not necessary to walk across, you can get a bus for very little.
Things to visit are the Rock, of course either by cab or small bus tour. The harbour is nice and otherwise shopping. There are different buildings worth a look at if you like colonial architecture.