I tend to stay in high-end hotels whenever they are available, but sometimes travel elsewhere and have perfectly nice experiences at four star hotels, inns, etc...
Has the OP tried luxury hotels/resorts? The only way to know the answer to his question is to have experienced luxury hotels as a staying guest.
I think that there are luxury hotels for everyone, but obviously not every luxury hotel will suit everyone.
In cities where luxury hotels are overpriced and still not that great, one isn't really missing out on much by not staying. Then again, if lower end hotels also provide terrible value, like in New York, by comparison I still think high-end is the best way to go.