I was in Yogya and Bali last fall. There's a TR in the Trip Report section.
http://www.flyertalk.com/forum/trip-...nh-f-tg-c.html
I loved Yogya. It was HOT when I was there in October. I assume it will be even hotter when you will be there but maybe not by much. So try to get to the sights early in the day. Better light then as well. Personally I think spending only two days would be too few.
Flights are pretty cheap. I understand the train trip to east Java is not very comfortable according to a friend of mine who did it a couple of years ago.
I liked Ubud a lot too. Certainly it is a nicer place to be than Seminyak or Kuta.