I looooved Istanbul, and people telling you that Sultanahmet is the place to stay are correct. I spent six nights in an area a bit higher up the hill in the Sultanahmet area called Canturkaran, which is very ancient and in the vicinity of the Topkapi Palace. The hotel was Yusuf Pasa Konagi, a traditional Ottoman house converted to an inn. It wasn't particularly fancy, but it was very clean, comfortable and the staff was helpful and friendly. They also provided an excellent Turkish buffet breakfast as a part of the room fee.

http://www.yusufpasakonagi.com/