The Angkor temples are amazing and certainly worth a visit, but you need to give yourself more time. If you must dash in and out, pay the premium to fly through Siem Reap. If you have more time and less money, go through Phnom Penh. There's really not much to see there (by some standards, but I really like the markets), so use it as a sort of hub. You can get a cheap bus from Phnom Penh to Siem Reap (the roads are good), but I recommend the fast boat. It leaves at dawn and takes a half day to get across the Tonle Sap lake, an adventure in and of itself. You can get your three-day temple ticket that afternoon so you can see the sunset from the temples for free.

What does the rest of your itinerary look like? We may be able to make other suggestions.