They are so immature that might have a laugh at your expense after you are gone, but who cares at that point.

As long as you are not hiding anything within the clothing it's none of their business what the gender of your clothing is.