Just wanted to add my 2¢. I can't wear a hat - my head is large (no, not grossly big, but enough that hats balance on the top of my head) and my hair is a little on the long side.
I've been in stores around the world where strangers have seen me trying on hats in a shop and burst out laughing and then just shake their heads no. Even my Tilley, which is large, looks pretty silly on me.
I thought it would get better, but at 61 my head is just getting bigger and my hair is getting thicker - and longer. Not complaining, just a comment.