Even if it was the previous pax who lit up, it was incumbent on the driver to make sure his car was ready to pick up new passengers.
Leave a low rating + comments.
But previous to that, not offending the driver shouldn't be a concern of yours in forcing yourself into taking an unwanted ride.