I think they should. Especially when flying in J, hearing FAs whining about how they got the Ft. McMurray turn the next day, or talking about how great the Delta St. John's is, or giggling about what they're reading in US weekly is annoying! I'm not saying they have to sit silently for hours, but whisper, or go to the back past the washrooms to talk.
I find that if the FAs have the back row to themselves, they usually read! Alot of my YUL-YEG flights don't have anyone in the last row AC side, so the FAs take breaks here. If they have a comfy seat with good light, they can read, have some food, and relax for a bit. If they're all cooped up in the galley, they'll obviously talk, and usually loudly!