I have done the ex-TPE thing multiple times, and it is no fuss, so easy, and saves so much money!
The published minimum is 1 hour, but is easily doable in 45 minutes. Most flights to Taipei arrive on-time, and there are so many later flights in case you miss it. I have done it in 1 hr before, but you 1 hr 20 min is perfect, as it will give you some time to sample the extra variety in the noodle bar (why can't they extend this to LHR/NRT lounges is beyond me).
But HK can't issue your TPE-HKG boarding pass, so make sure you get to the transfer counter. Ususally not staffed, but just call the number at the counter.