You did a GREAT thing having your infant son sit in your lap. That opened an extra seat for upgrade to someone else. It's especially good if the person who .....ed at you was your seat mate who had to get up three times to let you out.

Two questions:
Did you tell the gate agent/FA your son will be an INFT for proper passenger counting?
Was someone else upgraded into your son's vacated seat?