I would have explained it to them, and if the other people had not checked in they probably would have swapped you. If they already gave the other room away, there's little you can do except say thanks.

Going forward, especially at small Inns and B&B's where they are trying to do you a favor, if there's a specific room you want for whatever reason, you need to let them know in advance you want a very specific room, please don't upgrade or whatever.