Agreed, but I don't necessarily assume passport checkers (whether TSA or border patrol) to be reasonable people. The vast majority of them are fine, but I've met specimens that don't quite fit the definition of 'in their right mind'

Basically, if you go this route, I fully support taking off the cover when you hand it over to anyone with any authority.