I suppose...if the OP is interested in taking pictures of modern city buildings and cityscapes. I don't know. I just don't find Kaohsiung all that interesting from a cultural perspective. I was operating under the assumption that the OP was looking more for taking pictures of old temples and historical Taiwanese buildings (and yeah, skip Ft. Zeelandia).

I was thinking more in terms of the area around Chikan Tower and Wu Temple across the street (lots of traditional Taiwanese food options in that area as well). Or Shennong St. Then there's the recently restored Hayashi department store and the old city hall for a glimpse of the Japanese colonial era. Also, the Confucius temple (oldest in Taiwan), and across the street is a nice pedestrian alley with lots of handicrafts and food stalls that makes for a nice stroll.