Also there is a safety aspect to this. The manifest is based on BPs scanned at the gate. If the flight crew needs to declare an emergency ATC will ask for "souls on board" i.e. total of paxs and crew (also remains in hold, but there is a whole other story). In the event of a crash rescue crews may be looking for the last "missing" pax.
BTW if you need to leave a flight just let the GA know and they can offload you. Granted you will not get mileage for the flight.