The socks are not a problem. Shoes that take 3 minutes to get on and off are a big problem.
You are not obliged to remove shoes in your hotel room, but you will notice that all hotel staff will slip out of their shoes before they enter your room.

Never step on tatami (straw mats) with shoes. Not even slippers in some cases. Look carefully for any clues before you enter anywhere. It could even be where carpet begins or the colour of the floor may change.
Watch out at restrooms, there may be special toilet slippers to wear (the idea is that you change out of your guest slippers and into the toilet slippers at the entrance to the toilet, the trick is to change back and not walk away in the toilet slippers)
You will have a much, much easier time with shoes you can remove and put on easily. I use Ecco walking shoes with Velcro fastenings.

Wearing shoes where you are not supposed to is really one of the worst things you can do in Japan.

Hadn't you been there before?