Depending what TST MTR entrance/exit you are at, the nearest one to Knutsford Terrace is approx. just over 300 meters away and the furthest is approx. just over 700 meters., although an easy HK$24 earned that equals to approx. US$3.06.
The taxi driver COULD have been parked up for 1 min or 30 mins waiting for a customer, the taxi driver should not have refuse a fare but put yourself in his shoes would you have taken the fare?
Fares off the meter is illegal and it is up to you if want to report the taxi driver on either of the offences.

2 articles about HK taxi drivers For and Against.
https://www.scmp.com/news/hong-kong/...ernment-brings
https://www.scmp.com/comment/letters...t-condemnation