Get a Kansai One Pass + Haruka in place of the usual ICOCA, it offers nice discounts to temples, castles, and other places.

After Arashimaya, you can try Nishiki in the afternoon. Not sure why you're including the Ao Matsuri festival, but without that you should be able to do Higashiyama/Gion/Pontocho in a day, especially with extra use of taxis. The next day, from Fushimi Inari you can head to Nara or catch up on anything you've missed.

If you're into Tokyo urban scenes, Harajuku/Omotesando and Shibuya will offer a proper crush on Sunday.