Due to upgrade availability on flights, there are many occasions that myself, wife and son are in different cabins. I have never wandered around to go see them and vice versa. Personally, I think its bad manners and shows lack of respect for other people.
I recently flew AUH-MAN and I was in J. There was an elderly woman sat across the aisle from me who's daughters spent a good 3 hours wandering backwards and forwards from economy and stood next to me/her chatting with their mother. The crew chose to do nothing about it.