If the military guy is in your backyard helping you flip burgers on the grill, then call him "Bob". If you are in a board room negotiating a contract with the military, then call him "Colonel Smith". Everything in-between is a judgment call.

Also ... if you are going to use the rank as a salutation, then make sure you know the differences. For instance; if you see someone with a gold oak leaf; is he/she a Major, Lieutenant Colonel, Colonel, Commander or Lieutenant Commander?