The airlines all know from historical data roughly how many passengers are likely to be no-shows for a given flight. Based on this, they try to sell enough tickets that they're likely to fill up all the seats exactly. Of course, this is all based on statistics, so they'll always be off by a little, so they figure in the cost of having empty seats and the costs of compensating passengers who don't have seats and try to strike the best balance for maximum profit. (One could argue they should factor in customer goodwill, too, for the cases where they can't find enough volunteers to take a later flight and have to bump unwilling passengers, but I don't know whether that ever factors in.)

There is no such thing as "an oversold passenger"; if the flight is oversold, then there are more confirmed reservations than there are seats, but no passenger has any more of "valid" ticket than anyone else. In that case, it's usually the most frequent fliers who are given "operational upgrades" to make room in the other cabin.