You would be waaaay too rushed, even if you took taxis. It just takes time to get around Gyeongju. Even if you take a taxi, it still takes 40 minutes from town to the temple. And then 40 minutes back. You'd be moving nonstop at a pace you couldn't really enjoy anything. Yes you technically have from 9-6 but you have to factor in getting to/from the bus station. Sure, you don't need to be at the station until around 20 minutes before departure, but I just don't think you'd enjoy the rush of a checklist visit.

There are buses every 20 minutes or so allllll day nonstop to and from Busan.