Before the merger on a CO redeye I had a drunk/high dude in F trying to feel up my junk (I'm also a dude) and just generally invading my personal space to a level where I went from "Ok, I'm not going to be comfortable sleeping with him next to me" to "I'm not going to be comfortable awake with him next to me" and I was prepared to spend the night in SFO... So I grabbed my carry on, mentioned my concern to the crew and that one of the two of us was leaving the aircraft. The captain made the decision that he was not fit to fly and a gate agent escorted/dragged him off the flight.

If you have someone clearly unwell you can always bring it to the crew's attention and remove yourself anytime up to the boarding door closing. On a mainline aircraft a single passenger is unlikely to make any kind of significant W&B concern... On express (particularly small express) that's a slightly different story, but I'd still put health and wellbeing above a minor recalculation.