There are two trails between Tung Chung and Ngong Ping. The one you saw under the cable car route was built to support the cable car. It's very strenuous, especially if you do it in the summer. And it takes hours.

But there's a much older trail that people had been using for ages, and you can hike from Po Lin Monastery down to the main road in Tung Chung in about 45 minutes. More going up.