Many of those Mandarin-speaking people in "business" work for Chinese-funded mainland firms, which have not yet been internationalized so they are not representative of the international banks that have by far a much more significant influence in Hong Kong finance.
Otherwise, the primary language of business remains English for those working in the likes of ICC and IFC. Even the mainland banks have to switch modes to English when they deal with the outside world.