Apparently sentos in Tokyo tend to have their water rather hot though - normal ones are usually 41-44 degrees according to Japanese forum/blog posts. There were quite a few people complaining that 40 was lukewarm. I have no idea how to judge the temperature, to be honest. I'm a shower person (only trying to help out here because of the language).

The Internet connection is now a bit better so I did a quick search again.

This one http://spa.s5.xrea.com/broom/tokyo/rokugo.htm apparently has a 'cold' bath according to the Japanese text. It says it was too cold, so it probably isn't what you are looking for - I assume you're looking for a not-too-hot, rather than cold.

The only definite option I could come up with that was definitely 'cooler' was in Musashimuraya, a bit of a long way to go. It's called Katakuri-no-yu and they have a 37 degrees bath.

Having said all that, I've looked up some studies in Japan and it says that once you are over 8 weeks, you should be OK in a hot bath as long as you do not spend too ages in it - and really, Japanese 'ages' is at least 30 min in the bath, if not 60min+. Personally, it sounds to me like the risk from a short, hot bath is not significant after 8 weeks of gestation?

If a cooler sento/onsen is a serious issue for you, is it an option for you to stay in a hotel so that you can get your own bath, or just to have a shower instead of having a bath?