The best advice would be to search online for　東京銭湯（or 温泉)　ぴあ and do some research from there. Alternatively, once you`re here, any bookstore should have some onsen/sento pias that will list location along with various amenities (in a generally detailed fashion, with pictures). A quick search found the following site:
http://hotspringtokyo.com/