In Phnom Penh, a couple years ago, I stayed at the Hello Guesthouse. It was $3-5 per night with bathroom in room. It was definitely a good bargain. It is just a short walk to the river promenade, the Naga boat casino, and embassies and consulates. Run by a nice family.
Whatever you do, stay away from the Capitol Guesthouse! Rooms were so nasty there. Leave that one for the Lonely Planet freaks.