When it comes to HK, more is always better. I was there for 5 days last year and wished I had 5 weeks. There is SO much to see and do, esp. if you want to take your time and enjoy everything without rushing around (perhaps the point of a vacation).
Macau is worth overnighting, I didn't make it there because I didn't have the time.
I think that if you spent 2 days in Macau and the rest in HK, you could easily fill 10 days, esp. if you have a wide variety of interests (shopping, culture, food, lounging...)