For me Four Seasons and Hay-Adams are vastly superior to the other hotels in town. Capella may also be at the same level, but I have not stayed there so cannot comment on it.
I would rather stay in a room at the above hotels than in a standard suite at PH, MO, or the R-C properties.
One does not need to stay at PH or MO to dine at their very good restaurants, visiting once for dinner will do.