Four passegers will be fine, they all are licenced for five nowadays. The bags can be spread between three places; the trunk (which is quite small), the space alongside the driver, where you can get a couple of large cases at least, and where the one unused rear-facing jump seat in the passenger compartment is.
It is normal for the driver to huff and puff while organising this; they think they get better tips this way.