Just bring the baseball cap and be done with it. So they think you are American? Who cares? As soon as they hear you speak German, especially with a cute American accent, they will love you.

Your behavior is way more important then the way you dress unless you go to an event where the dress matters but as a tourist? (of course in churches, etc. certain rules apply)