I can't be sure what you mean by "big Buddha temple". You need to be far more explicit than that.

Do you mean the Big (outdoor) Buddha at Ngong Ping on Lantau Island? If so, you can't take a single transport from the Peak Tram to there because there is a separate group of taxis dedicated to south Lantau Island, and private cars can't go there without a residents' licence. And there are a couple of monasteries there but not a temple as such.

The fastest way to get from the Peak Tram (I assume you mean the lower terminus) would be to walk to Hong Kong Station (maybe 10 minutes) then get the Tung Chung line of the MTR to Tung Chung. Then get a light blue (Lantau) taxi to take you to the Big Buddha; or, if the line is short, get the Ngong Ping 360 cable car up.

Also be aware that the taxi stand at the lower Peak Tram terminus is notorious for rogue drivers who scam gullible tourists. I would never recommend taking a taxi from there. If you really want to get one then walk a couple of hundred yards away and hail one in the street.