I've stayed in both hotels and you were right to move if you have business near the HI Downtown. I used to have meetings near there on some of my trips and would stay there too, although I preferred the Great Wall. I find the HI Downtown a bit cramped.
However, a saving grace is definitely the Indian restaurant there which has one of the best Peshawari Nan's I have ever tasted.