I think you'll find it's easier to just take the Airport Express to Hong Kong Station, stroll over to the Star Ferry and take it across the harbour, stroll around, then head right back to the Star Ferry and cross the harbour again before walking over to Hong Kong Station to catch the Airport Express for your return.
Look at a map and you'll see this is actually quite simple.
Yes, that is a good point - the Kowloon station of the AE is not near much of tourism interest, aside from a mall.