Sorry, this may not be what you want to hear, but HK is awesome. I think you should go. Nov is a great time to go to HK... perfect weather, hairy crab, cheap flights. With 3-4 nights, you'll be able to see a lot of HK. Of course, if you have a long list of restaurants to try, want to do some hikes, do Disney, visit Macau, and head to Outlying islands, you'll need a week or more. But for most visitors, 3-4 days will suffice. HK is an amazing place with awesome food, great flair and international vibe. It's very different than Tokyo imo. I've been going to HK just about every year for over a decade. Often we combine HK and Jpn on the same trip. Sometimes, the airfare is cheaper for US-HK-Jpn-US than US-Jpn-US alone. If you're worried about costs, then don't stay at the Pen. Compromise for something a lot cheaper but almost as memorable, like Intercontinental.