Technically you could just walk through any of the hotel lobbies and enter into the garden. Who is going to stop you unless you are causing trouble? It's like most hotels in that respect. In my time there, security or personnel weren't bothering people.

However, I am pretty sure that if they see people with a bad appearance (rare in that district of Tokyo) and large groups of non-hotel guests gathering then they will probably ask questions.