I don't know what your definition of "inexpensive" is... we usually stay at the Palms Hotel which is a friendly little place in a quiet little side street. Its web site is normally at
www.palmshotelkeywest.com but it seems to be down this evening... you can find an archived version
here