Agreed, totally okay. As long as its not married with a sleeveless [insert cheap beer brand here] branded singlet or the aforementioned socks with flip flops - you're good to go.
As mentioned, not in temples, but additionally I would advise not to visit any government offices, immigration, or any appointments in shorts. Plus up market restaurants in evenings etc.
Common sense applies, but I wear shorts pretty much every day except in the situations I stated above.