I generally agree with what's been posted here, except that it is a good idea to spend at least some time in Cairns. It's worth visiting Kuranda (preferably by train one way and cable car the other, as both are spectacular forms of transport), which is near Cairns. Also, quite a few tours to the Reef go out from Cairns.

Port Douglas is much nicer, but really restricts things if you only stay there. Best to stay a couple of days in Cairns and do the Cairns-centric things there, then spend the majority of your time in Port Douglas.