I thought it was the responsibility and by legal regulations of the airline to ensure that a parent is seated with children?
I would probably had said no as well but told the crew about the situation and volunteered for an upgrade
But it's a difficult situation. Somehow I don't think it's up to a pax with an aisle seat preference to sort out seating problems where children aren't seated with at least one parent.