I find the etiquette of this a bit confusing, particularly in the US where the current boarding group system started (and has largely been shown to be superior to other approaches and is being adopted everywhere).
Boarding a United Houston Bonaire flight the other day Group 3 was being boarded, but everyone in the waiting area was in a single line. I was in Group 3 and eventually just stood at the back of the line and boarded last (by which points Group 3 and 4 were boarding according to the monitor, though no announcements had been made since Group 2).