I agree with your brother, this is quite ridiculous. A middle seat is really not that big of a deal. I wouldn't even drive an extra 15 minutes to a different airport for this. If you have kids, you can always ask someone to switch seats so that you can sit with them. And seats do open up later on. I have successfully got window seats by asking at the gate, even if the website showed only middle seats available.

But, to each his own. If this matters so much to you, then it is perfectly fine to do as you see fit.