I think Newbie's comments are on point. I spent a few days hiking around Utoro and Rausu. Unfortunately, I didn't get to climb Mt. Rausu because of poor weather. I think there's some great countryside to explore around there, but I'm not sure how much you'll get from a whistle stop drive through. Maybe you'll see one or two picturesque scenes on the way over the pass.

As Newbie said, Rausu is quiet - the only memories I have of it are the seaweed drying on the shoreline, the very seaweedy miso soup that we had with dinner, the bathhouse in the woods at the back of the minshuku, and the deer that were poking around as I emerged from my bath. I loved it. I have a feeling that evergrn would not. For that reason, I think you might as well cut Rausu from the agenda and give yourself more time for the boat trip and sites around Utoro.