I liked both places very much but for "off the beaten track," I'd probably go with Matsue. .

Matsue and Itsumo are both imminently walkable so a little extra time to see things would be a good idea. Shimane is on the Japan Sea coast so be prepared for possible wet weather - just in case. Plan a nice soba lunch in Itsumo where it is a specialty. I geek out over history so was able to spend a lot of time in Matsue wandering through Lafcadio Hearn's home and work area, samurai homes and the castle. I did not take the castle moat boat tour but a lot of people do and seem to enjoy it very much.

That area might be a bit difficult to get to now since you have to change from the shinkansen to the coast-bound train at Okayama which was hit pretty hard this week during the floods. By November, things should be back to normal but be sure to check.

I hope you will get to Shikoku some time. It is probably one of my two favorite parts of Japan (with Hokkaido) but I think you need more than 4 days. In addition to the places you mentioned, I think Kochi is a must do. The train trip up and and over the mountains, along the river gorge, is worth the trip alone but Kochi is worth a couple of days itself.