Exactly. F/A's like to cite "security rules" when they tell people "no," but it is a bunch of BS. The actual rule is in regard to "congregating" or "lining up."

I'm still waiting for someone to post the exact federal law or TSA/FAA regulation that bars people from using a bathroom in a different cabin from which they were assigned.

Airlines can't say that, it would ruin their PR. If airlines were legally permitted to segregate the FC LAV to only F PAX, they would install pin code door handles. The combinations could be changed (easily done) after every flight and handed out to FC PAX when they boarded the AC.