At 4pm? Perfectly OK. I haven't been in any hotel where public spaces aren't put into use in the late afternoon. Sometimes it's for group business functions, sometimes it's a wedding.

So no, I don't have a problem with a pool being closed at 4pm - particularly if there is another one I can use. But even if there isn't, it's really not a big deal to me.

I would complain if it were closed for the entire morning or afternoon. 4pm - not so much.