Actually, I don't blame the OP, who could arguably have expected the driver to leave (although he could also have told the driver to do so). I would blame the driver, and not just for ripping off the OP.
I take shared vans, know that they build in a cushion, and fully understand why. But if a pax is out there ready for a pickup scheduled for 7:00 to 7:15 and has to wait until 7:45, in the meantime the pax won't know (without calling) if the van is a little late, very late, or has forgotten about the pickup altogether. It can be a bit nerve-wracking. Of course, if they originally gave the pax a 45-minute window for the pickup, then the pax is on notice that the cushion's built in there. Maybe they did that here; I don't know.