I look for the location of my life vest almost every time I fly overwater (I know, I'm paranoid, but it'd be problematic if, right before landing in the water, you discovered your life vest was missing).

I think there'd be more problems with people inflating the life vests in the cabin. If people did this, there would be mass chaos! Even if I knew how to don my life vest (I'm not sure I could in an emergency) and not to inflate until exiting, I'd estimate about 75% of pax wouldn't know how to don the life vest and almost everyone would inflate it while in the aircraft.