I've been on several flights with fewer than 5 passengers (on 100+ seat planes). It isn't an easy decision for an airline to cancel a flight as it needs the airplane and crew at the destination for future flights. So they wind up having to cancel lots more flights, or to fly one empty to get it into position. Also on some routes lots of seats are not pre-reserved, so the seating plan shows many empty seats which will be occupied by passengers already ticketed but without seat assignments (I suspect some consolidator ticket sales prohibit advance seat assignment) -- so your flight may not be as empty as you think.