Unfortunately, you are stuck with their times. The only solution I could think of would be to coordinate private driver to the JSA, get on the bus at the JSA, driver waits somewhere for you outside (like Dorasan Train Station), then you get back in with private driver and go back to Seoul. But I bet that'd cost a pretty penny, if even possible.