It has little cultural significance. There are lots of "stuff" everywhere in China where some people worship it for one reason or another.
I grew up in HK and visit HK often enough. Might have visited it once as a kid (that tree might have died years ago and not the one people visit it today), never care to visit again, never tell any visitor to visit it.