Are there restrictions with wearing masks around US airports?

I'd imagine the TSA's would want to match your face to your ID (maybe), but does anyone else care whether or not you have a mask on?