Makes me wonder: if lots of women wore job-shirts without any underlayer and brassieres would there be more requirements to remove the job-shirts, too?

My take on it is: the short answer is that to minimize TSO guessing which garments must come off and which can remain on and therefore minimize inconsistently requiring passengers to disrobe is to have all outer "coat-/jacket-type" garments be removed and all shirt/sweater type garments remain on. It isn't so much the thickness of the garment as it is the outward appearance of type.