Hong Qiao market, which holds the pearl sellers and other undesirables, is perhaps the worst experience in China. Take the Metro there and, with any luck, you'll get horribly lost and emerge at the Summer Palace, instead.
I think that you mean the Temple of Heaven, not the Summer Palace. Hongqiao market is not as bad as the Silk Market. It is actually somewhat bearable compared to the other one.