It's easy to get around the city (except during rush hour - 6am to 10pm ) by just jumping in a taxi at any hotel and hiring him for as long as you like. If you explain your goal to the concierge, even better. Silverbird taxis are really cheap compared with, say, a euro taxi. I'd think a several hour tour would run you less then $20 or so, maybe even less.

As to what to see, the 'old port' is interesting. Ancol is the name of the amusement park you mentioned, but one large section there is devoted to craft vendors. Since I like to collect teak and mahagony wood carvings, I find it interesting.
I'm sure others can add more. I'm always busy with work there and never really toured around.

The fastest way to get to Yogyakarta is a short flight.