Bagan !
I've been to Angkor, Borobudur and Bagan; and I would rank Bagan right at the top alongside Angkor as the most amazing sites in SE Asia.
Borobudur is interesting but theres only one temple and it will keep you occupied for half a day at most (inclusive of travelling from Jogjakarta). Ofcourse theres also Prambanan which is sort of a mini "Angkor" but that will also keep you busy for the other half of the day.
Bagan, on the other hand is magical especially during sunset with the hundreds/thousands of temples glowing under the setting sun . There are so many temples to explore and it can easily keep you occupied for days. You wont see any modern building around (except for the museum and probably your hotel) and you really do feel a sense of isolation from the rest of the world. I cant really describe how special the place is and you have to be there to feel it !