Ice at any restaurant is made from boiled or purified or bottled water. Won't be straight from the tap. That's one of the reasons why it cost more to have cold drinks at many low-end eateries, as there's a not-insignificant cost to them.

Above posters already answered why hotels may not recommend tap water. I'll put it this way - if all the local hotel workers NEVER drink from their taps, and actually think it's ridiculous or dangerous to even have that thought, then it may be a little bit hypocritical for them to tell the guests it's totally safe, even if it actually is.