If you were travelling with another male chances are she would have been swapped to sit elsewhere. Perhaps the passenger hadn't mentioned anything to anyone until she got onboard and the crew were dealing with it as best they could in the moment.

It sounds like you two still ended up sitting together, you got an aisle as you prefer. I'm unsure why the objection is that you had "chosen the seats you wanted" since you had done so hoping the middle would remain vacant, it wasn't and some shuffling needed to happen to accomodate the passenger that was there. Was your preferred outcome that she be punted somewhere else and another passenger dropped into the middle seat?